Timing system prototype for ProtoDUNE SP II

Design overview

Section giving a brief overview of the DUNE/!ProtoDUNE SP || system design Link to master design document

Prototype set-up

Section describing how to set up a timing system prototype

mTCA system

A functioning mTCA system is composed of several mandatory and optional components. These are listed below along with the models chosen for prototyping.

As per the timing system design described above, a custom mTCA Interface Board will sit in one of the MCH slots and distribute clock and data to the 12 AMCs using the mTCA backplane. The AMCs will then fan out these signals to the timing endpoints using optical fibres.

To set up the crate, plug in the MCH module in the secondary MCH slot, which is the most right-hand slot on the front of the crate. The power supplies can either be plugged in the two front slots above the MCH slots (PM1,PM2) or in two of the slots on the back (PM1A,PM2A). The JSM plugs in the back of the crate in the JSM slot. The location the aforementioned slots is shown in the following two images. For more information refer to the crate manual, https://schroff.nvent.com/wcsstore/ExtendedSitesCatalogAssetStore/Attachment/SchroffAttachments/Documents/63972-338.pdf.

mtca_crate_front_view.png mtca_crate_rear_view.png

Once all of the modules have been inserted, with their hot-swap handles fully in, connect the mains cables to the power supplies and flip the on/off switch to on (if not already in the on position). The LEDs of the power supplies and MCH should start blinking and the fans of the crate should start spinning. After a few seconds, several power supply and MCH LEDs should be solid green (see image below), indicating the MCH and power supplies have booted successfully. Any blue LEDs near the hot-swap handle either solidly on or flashing indicate a problem and therefore that particular module will not be receiving payload power.


mTCA Carrier Hub (MCH)

For the initial configuration of the MCH, one can use telnet over USB (a microUSB port is available on the front of the MCH) or network access. The parameters for the telnet session are listed below.
 Baud rate: 19200 Par/Bits: SN1
If the connection has been established successfully, the following prompt should appear:
To verify that the MCH is able to detect the power supples and any connected AMCs, type the command:
The output should be similar to the one below.
FRU Information:
 FRU  Device   State  Name
  0   MCH       M4    NAT-MCH-CM
  4   mcmc2     M4    NAT-MCH-MCMC
 13   AMC9      M4    AFC LOADE2
 40   CU1       M4    Schroff uTCA CU
 41   CU2       M4    Schroff uTCA CU
 50   PM1       M4    NAT-PM-AC600 
 51   PM2       M4    NAT-PM-AC600 
The M4 state (hot-swap blue LED off) indicates that the FRU is receiving payload power.

Network configuration
Configure the network parameters as follows:

  1. Enter ip at the command line to show the current configuration
  2. Delete the current IP address using the backspace key. Enter the new address in dotted quad format, e.g.
  3. Change all other parameters by repeating step 2
  4. Reboot the MCH using the command reboot.

  • If an IP address is configured to, it will be ignored by the MCH
  • If you configure a non-zero gateway IP address the related routing configuration will be performed automatically when the MCH starts up

Current network configuration can be also displayed using the command

Bristol lab network parameters
IP Configuration Setup:
IP Address           :
IP Net Mask          :
IP Broadcast Addr    :
IP Gateway Addr      :
The MCH is accessible via bayban.phy.bris.ac.uk or nova.phy.bris.ac.uk.

Web interface access
Once the MCH has been correctly set up on the network, the web interface can be accessed by pointing a web browser to the IP address of the MCH. The default username and password are "root" and "nat" respectively.

Bristol lab web access
To access the web interface from the general Bristol network, the web traffic needs to be routed via bayban or nova. To set up the port forwarding, set up an ssh connection as follows:
ssh -L 8080: bayban.phy.bris.ac.uk
The web interface should then be available on

SSH access
SSH access is disabled by default. It can enabled by opening the web interface, and navigating to the "Base Configuration" page, and changing the "SSH access" configuration to "enabled". N.B. SSH access may not available immediately, as ssh keys need to be generated.

JTAG Switch Module (JSM)

If the JSM is installed successfully, its configuration should be available from the "JSM" page of the web interface. Through the JSM, one is able to access the backplane JTAG lines going to each AMC, and switch between them. The active AMC can be selected either through the rotary switch on the front-panel or though the JSM MCH web interface.

Rotary switch
A photo of the JSM front panel, with the rotary switch clearly visible, is shown below. It has 12 positions, labeled as 1-12 in hex, representing the 12 AMC slots.


MCH JSM interface
As mentioned above, the JSM web interface is available on the JSM page of the MCH web interface. There, the active AMC settings can be overridden selecting the desired AMC slot, and clicking the "Override" button. If the operation was successful, the override status should change to "Override enabled".

The JSM interface also allows the transmission of svf files, however before doing so, the svf files need to converted to NAT compliant svf files. This is achieved by downloading the tool from the JSM interface and executing as follows.

./svf_to_nsvf <svf source file>
N.B. This step is mandatory.

The newly created .nsvf file can then be sent over JTAG by following the steps outlined in the JSM web interface.

Power supplies

AMC FMC Carrier (AFC)

JTAG configuration
The AFC features a JTAG scan bridge, SCANSTA111, allowing JTAG access to the on-board FPGA as well as to the two FMC connectors. In order to be able to program the on-board FPGA using the Vivado tools, the SCANSTA111 needs to be put into transparent mode. This is achieved by transmitting the following svf script to the SCANSTA111 chip.
SIR 8 TDI (00); ! 00000000 Address ScanBridge
SIR 8 TDI (A0); ! 10100000 Load instruction to enable transparent mode for LSP0
SIR 8 TDI (a5); ! 10100101 Verify SIR
SDR 8 TDI (5a); ! 01011010 Verify SDR
SIR 8 TDI (C3); ! 11000011 Try to load GOTOWAIT in ScanBridge
SDR 8 TDI (5a); ! 01011010 Verify that ScanBridge did not recognize GOTOWAIT
! Now TDIB  lsp0  TDOB
N.B. The script above enables transparent mode for LSP0, which is where the AFC on-board FPGA is connected. More information can be found here: https://www.ti.com/lit/an/snla068c/snla068c.pdf.

Timing system tests

ProtoDUNE SP I timing system tests

Tests involving ProtoDUNE SP I timing system hardware, i.e. AIDA 2020 TLU, timing FMC + Enclustra PM3/AX3

ProtoDUNE SP II timing system tests

Tests with ProtoDUNE SP II/DUNE-like timing system, i.e. mTCA crate, GIB, MIB, FIB

-- StoyanMiroslavovTrilov - 2020-02-18

Topic attachments
I Attachment History Action Size Date Who Comment
JPEGjpg jsm.JPG r1 manage 687.1 K 2020-02-19 - 15:31 StoyanMiroslavovTrilov  
JPEGjpg mch+psu.JPG r2 r1 manage 1685.8 K 2020-02-19 - 15:44 StoyanMiroslavovTrilov  
PNGpng mtca_crate_front_view.png r1 manage 826.8 K 2020-02-18 - 16:42 StoyanMiroslavovTrilov  
PNGpng mtca_crate_rear_view.png r1 manage 531.1 K 2020-02-18 - 16:42 StoyanMiroslavovTrilov  
Edit | Attach | Watch | Print version | History: r5 < r4 < r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r5 - 2020-02-19 - StoyanMiroslavovTrilov
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CENF All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback