Reference manual

Chimaera2 is a readout board designed to receive binary hit data from 128 digital channels through a high density connector. The readout logic is implemented in a Spartan6 FPGA and external connectivity is provided via Ethernet, USB and custom connectors. Physical layout and connector details differ between the v1, v2 and v3 versions of the hardware. the board geometry is optimised for close-packing of Hamamatsu R11265 MaPMTs with each Chimaera2 connected to two MaPMTs. However, the firmware can be adapted to support other applications if desired.

ALERT! The Chimaera2 board exists in three versions: v1, v2 and v3. Be sure to note the sections specific to the hardware version you are using.

ALERT! v1 Chimaera2 boards are considered obsolete and should not be used.

Inventory

Cambridge
Edinburgh
CERN
Ferrara
Genova

Operation

The readout architecture is very similar to the LHCb readout. Events are tagged with a timestamp (BXID) and event ID and the data are sent out as multi-event packets (MEPs). MEP building is done using internally generated readout type (ROT) and MEP destination commands. Readout is controlled with external triggers (TRG). In addition there is an (optional) external start-of-frame (SOF) signal which acts rather like the bunch count reset signal. It resets the timestamp counter. An SOF counter is also sent with the data.

Connections

Ethernet

As of MAY 2016, firmware releases for all boards use 100Mbit Ethernet instead of 1000Mbit.

The Chimaera2 data and configuration interfaces use the same 100baseT connection. 10Mb and 1000Mb operation are not supported. Standard network hardware can be used to connect to a PC.

ALERT! The Chimaera2 data packets must not be routed through a general-purpose network. Use either a PC with a separate NIC for the data and configuration or, if several boards must be used simultaneously, connect them via a dedicated network switch.

A managed switch may be needed because it may be necessary to disable some features such as "storm control" or to create a VLAN.

Power

A single 3.3V/1A power supply is required. A single board draws about 600mA.

2x5 Header [v2]

The function of these pins is defined by the FPGA firmware. There are 4 pairs of signal pins connected directly to FPGA general-purpose IO pins.

The connector layout is summarised in the table. The P and N suffixes refer to the polarity of the signal when programmed as a differential signal.

1 AUX4P 2 AUX4N
3 AUX3P 4 AUX3N
5 AUX2P 6 AUX2N
7 AUX1P 8 AUX1N
9 +3.3V 10 GND

2x5 Header [v3]

The function of these pins is defined by the FPGA firmware. There are 4 pairs of signal pins connected directly to FPGA general-purpose IO pins.

The connector layout is summarised in the table. The P and N suffixes refer to the polarity of the signal when programmed as a differential signal.

1 3V3 2 GND
3 AUX1P 4 AUX1N
5 AUX2P 6 AUX2N
7 AUX3P 8 AUX3N
9 AUX4P 10 AUX4N

Dual LEMO

In normal usage the dual LEMO connector is used to receive TRG50 and SOF50 signals (3V3 LVTTL). Termination to ground through on-board 50 Ohm resistors is provided. TTL outputs of NIM-TTL converters are designed to drive 50 Ohm cables so these can be connected directly to the TRG50 and SOF50 inputs.

The input signals are reflected as differential signals TRGOUT and SOFOUT at the 2x5 header to allow daisy-chaining.

USB [v2]

The C8051 microcontroller implements a USB to SMB (I2C) bridge which can be used to configure the board and read back status information via the FPGA. This functionality may not be available for all versions of the firmware.

USB [v3]

v3 uses the cy7c68013 (FX2LP) USB controller. It is configured to implement an 8bit diagnostic port to the FPGA and a USB to JTAG bridge for firmware updates.

FE SAMTEC connector

Front-end data are received through a 6x30 SAMTEC SEAF connector. This connector also supplies LV power to the front-end. All non-power/ground pins are connected to FPGA general purpose IO pins. In normal usage, two groups of pins are defined: one group of 128 pins for FE data and the remaining IO pins used for front-end configuration and monitoring. The connector layout is tabulated at the end of this document.

FE power [v2]

ALERT!Do not use the v2 board with the MAROC.

For the CLARO elementary cell, 2V5 power is provided through the SAMTEC connector. 2V5 is sourced by a linear regulator. 3V3 is also available through the SAMTEC connector.

FE power [v3]

The MAROC3v2 FE board is powered from a separate 3V5 bench power supply connected to dedicated header pins. The FE power jumper (JP1) must be removed in this configuration to isolate the Chimaaera2 FE regulator. The IO voltage jumpers, JP2, JP3, JP4 should all bridge VCCO_n with 3V3.

For the CLARO elementary cell, 2V5 power is provided through the SAMTEC connector. 2V5 is sourced by a linear regulator. The FE power jumper (JP1) must bridge VDDFE with VDDR (2V5) or removed completely to isolate the power if the EC is powered by another connected Chimaera2. 3V3 is also available through the SAMTEC connector.

Firmware updates [v2]

This requires the use of a programming cable such as the Xilinx Platform Cable USB and Xilinx software. The Xilinx LabTools software is available as a free download but you will need to register on the website. The full ISE installation is not required.

Firmware updates [v3]

No additional hardware is required for the v3 board. Firmware updates are done through the USB connector. The Xilinx LabTools software should be used with the Digilent Adept plugin. Both can be downloaded free from the respective web sites.

Alternatively, a programming tool is available here.

Network environment

ALERT!It is not recommended to connect the Chimaera2 to your general purpose network. The board is capable of sending data packets at very high rates that could overwhelm your network. It is strongly recommended to use a dedicated network interface on a private network.

Chimaera2 supports only IPv4. Ensure that your NIC is configured to use IPv4. You can/should disable IPv6 on your DAQ interface.

DAQ link speed

As of MAY 2016, firmware releases for all boards use 100Mbit Ethernet instead of 1000Mbit.

The current Chimaera2 firmware supports only 100baseT Ethernet. The immediate upstream network port will probably need to be fixed at 100Mbit full-duplex (no auto-negotiate). This can usually be done through the management interface of a switch or using the ethtool program for a NIC in a Linux PC e.g.:

ethtool -s eth1 advertise 0x020

To make this setting persistent across reboots the command could be added in a suitable udev rules file in /etc/udev/rules.d/.

[Note: Usually autonegotiation tries the highest speed first then falls back to the next slowest speed so you might find that things work fine even without doing the above step.]

ARP

Chimaera2 does not implement a compliant MAC controller so it may also be necessary to add a manual entry in the ARP cache on the readout PC for each connected board.

arp -s $FEB_IP_SOURCE $FEB_MAC_SOURCE

This setting is not persistent across reboots.

When using the provided GUI and feb-proxy programs, the ARP cache is updated automatically by feb-proxy when the GUI starts up. In this case it is not necessary to manually add the entries.

Firewall

You may also need to change your firewall configuration to allow the Chimaera2 packets through on the DAQ interface. For example, the following command inserts an ACCEPT rule at the beginning of the INPUT chain for interface eth1 that will accept packets coming from any Chimaera2 on the network 192.168.244.0 and having protocol 242:

iptables -I INPUT -i eth1 -p 242 -s 192.168.244.0/24 -j ACCEPT

This will not be persistent across reboots in all Linux environments.

Java GUIs

The easiest way to get started is to use the Java GUI for 64-bit Linux that can be downloaded at http://www.hep.phy.cam.ac.uk/lhcb/RICHEC/RICHECKIT.tgz

Readout of multiple boards is supported.

Unpack using the command

tar xzf RICHECKIT.tgz

There are three GUIs: JRichEcControl, JRichEcConfigurator and JRichTBTrigger. JRichEcConfigurator manages sets of preferences (network settings, channel configuration) that are used by the DAQ control GUI JRichEcControl. JRichTBTrigger controls the trigger distribution board. JRichEcConfigurator and JRichTBTrigger are now integrated with JRichEcControl. JRichTBTrigger starts automatically (if enabled) when JRichEcControl is started and JRichEcConfigurator can be started from JRichEcControl→Main→Configurator.

JRichEcConfigurator

This manages sets of elementary cell configuration parameters for use in small lab setups or beam tests. The GUI does not perform any run control or hardware configuration functions. It is always used in combination with a run control GUI such as JRichEcControl.

The basic unit of configuration is the Elementary Cell which consists of a 2-by-2 arrangement of MAPMTs labelled A, B, C and D. JRichEcConfigurator simplifies the setting of the configuration parameters by allowing selection of the channels to be configured by dragging and releasing the mouse. The program converts the xy coordinates on the EC of the selected anodes into hardware channel numbers. This mapping is stored in a lookup table and can be changed if necessary but has been set to reflect the hardware numbering of the channels as defined for the 2014 beam test hardware.

The GUI works on Windows or Linux platforms that have the Java runtime environment. However, the data acquisition proxy started by the run control GUI, JRichEcControl, will currently only run on Linux systems. Therefore, since JRichEcControl uses the parameters managed by JRichEcConfigurator, it is usually more convenient to run on Linux systems.

Follow the JRichEcConfigurator topic for more information.

JRichEcControl

JRichEcControl can be run with normal user privileges. However, it starts a subprocess that requires root privileges in order to manipulate raw sockets. The GUI starts this processes automatically using scripts that assume that your system administrator has granted root privilege to run feb-proxy passwordless through sudo. A suitable line similar to the following in /etc/sudoers should grant the lhcbrich user the required permissions:

lhcbrich ALL = NOPASSWD: SETENV: /home/lhcbrich/bin/feb-proxy

An alternative way to grant root privilege is to give the feb-proxy program the setuid attribute and ensure that it is owned by root. If you do this, you should edit the startProxy.sh script to remove sudo. If you have difficulty making this work you may find it helps to look in feb-proxy.log for clues.

You must also tell the GUI the location of the feb-proxy program. Navigate to the menu Settings→Proxy→Proxy path... and enter the full (absolute) path of the program.

To start the GUI, change directory to the location of JRichEcControl.jar and run the command

PATH=/usr/bin:./ java -Djava.library.path=./ -jar JRichEcControl.jar

If you have run the GUI before (from the same user environment), the previous settings will be remembered. You must use JRichEcConfigurator to set up some basic parameters before using JRichEcControl.

Now, to start acquiring data, click the Start button in the Run control section of the main panel and Stop to stop acquisition. You must have set up the system to either use the internal pulser or external pulse generator to generate triggers.

To save the acquired data, the Recording box should be selected. If all goes well, the data will be written into the folder specified on the main panel and tagged with the specified run number. Also, if Recording is selected, the current preferences used for the run are saved as XML files in the same folder and are tagged with the run number.

An automatic acquisition to perform a threshold scan is also provided (JRichEcControl→Settings→Threshold scan). The scan panel allows the range and step size for the scan to be defined. When the scan is started (Threshold Scan button), the GUI automatically acquires data at each step of the threshold. The data for all the steps are stored in the same output file but they contain a tag which encodes the threshold used for a particular trigger. The number of events stored per threshold step is set in the Max events field of the scan panel. If you are using the internal pulser, the Pulse count field should be set to be somewhat larger to ensure that enough triggers are generated.

FE data capture

Chimaera2 receives up to 128 fast digital signals from the FE ASIC - CLARO or MAROC. The outputs of both these ASICs are asynchronous - an output is asserted when the corresponding input is above threshold and deasserted otherwise. Edge-sensitive latches in the Chimaera2 FPGA register the incoming signals on the rising edge of the digital input. The latch is deasserted after a fixed number of internal clock periods referred to as strobe_length. The output of the latch is stored in a pipeline that acts as an internal delay for the data. The pipeline is implemented with a dual-port memory clocked at 40MHz. The difference between the read and write pointers determines the pipeline latency. Each front-end channel has its own independent input latch and pipeline but the strobe_length and latency settings are common to all channels.

At the time of arrival of a trigger, the data at the output of the pipelines are simultaneously and synchronously captured and stored in a buffer. In fact, the trigger is also passed through an adjustable pipeline before being used to capture the data. This allows the trigger to be delayed by an amount referred to as trigger_delay.

The strobe_length[0,7], latency[0,255] and trigger_delay[0,255] settings may all be adjusted using the GUI provided. They are positive integers and the step size is about 20ns but depends on the hardware and firmware version.

Timing

The different time of arrival of the detector signals and the corresponding trigger can be accommodated by the trigger_delay and latency configuration parameters. The normal case would be that the detector signals arrive at Chimaera2 before the trigger. Then the latency parameter can be used to delay the detector signals to bring them into alignment with the trigger. On the other hand, in some set-ups, it may happen that the detector signals arrive later than the trigger. In this case, use the trigger_delay parameter to delay the trigger.

ALERT!To avoid data corruption when reading the latency pipeline it is important that latency and trigger_delay are not both set to values close to zero.

The correct timing is usually found by taking a series of data points at different latency (or trigger_delay) settings while monitoring the number of hits seen in the detector. The correct timing is achieved when the number of hits is maximum. Remember that it is the difference between trigger_delay and latency that is important so only one of these parameters should need to be scanned. A good strategy would be to start by setting both parameters to 16 and adjusting one of them until optimum efficiency is reached.

It may help when finding the correct timing to use a longer than usual strobe_length value. Once the approximate timing is found, strobe_length can be reduced and a mini-scan done to home in. Note however that a longer strobe_length will also integrate more noise so may not help in a noisy environment.

The internal pulser

This may be used as a source of triggers for testing or for applications where the data capture does not need to be synchronised with an external stimulus.

The period of the internal pulser is (pulse_delay+2)*3.2 microseconds. The maximum pulser frequency is therefore about 160kHz. pulse_delay[0,65535] is an integer number that you enter in the GUI main panel. The pulse_count[0,4G] setting determines the number of triggers that are generated at the end of every configuration. To disable these internal pulses, set pulse_count to zero.

Status register map

Status is accessible through the USB interface. The JChimaera2Depp utility can be used to display these internal status registers. The registers are described here.

Configuration register map

Configuration is loaded through the Ethernet interface. The registers are not currently readable. The configuration payload also includes a 4 byte suffix containing the target ID (DNA(31..0)). If the target ID does not match the hardware, the configuration block is rejected. This is also the case if the configuration protocol type (0x11, UDP) does not match the expected value.

The registers are described here.

Set-ups that do not use LHCb TFC commands should use

mep.static-destination=1
mep.no-tfc=1

MEP data format

The MEP data structure is given in the table below. A MEP can contain several events therefore the part after the MEP header may be repeated as many times as there are events in the MEP. In principle there may also be more than one bank per event but Chimaera2 only writes one bank with type=0x09 (RICH) and version=0xc0 (TB2012).

Field Content
  31..24 23..16 15..8 7..0
MEP Event index
MEP Timestamp Event count
MEP Tag
EVT Event length Event ID
BANK Length Magic = 0xCBCB
BANK Source Version = 0xc0 Type = 0x09
ING[0] 0x00 0x01 0x00 0x00
L1 [31..27]=RGFMZ=00101,[26..16]=Length [15..11]=Event ID,[10..0]=FE ID
L1 Timestamp
L1 Frame ID Event ID
DATA[0] Index = 7 FE2[7..0] FE1[7..0]
... ... ... ... ...
DATA[7] Index = 0 FE2[63..56] FE1[63..56]
ING[1] 0x10 0x01 0x00 0x00
DATA[0] MAR1ADC[62] MAR1ADC[63]
... ... ... ... ...
DATA[31] MAR1ADC[0] MAR1ADC[1]
PAD[0]        
... ... ... ... ...
PAD[31]        
ING[2] 0x20 0x01 0x00 0x00
DATA[0] MAR2ADC[62] MAR2ADC[63]
... ... ... ... ...
DATA[31] MAR2ADC[0] MAR2ADC[1]
PAD[0]        
... ... ... ... ...
PAD[31]        

The data part has 1, 2 or 3 sections with header ING[0,1,2]. For the CLARO readout, only the ING[0] section is present. For the MAROC readout, sections ING[1] and ING[2] contain the ADC data for MAROC[1] and MAROC[2] respectively. Either or both sections may be absent if the corresponding ADC is not enabled.

MDF data format

The MDF data format written by feb-mdfwriter is similar to the MEP format except that the MEP data are always separated into individual events and the MEP and MEP event headers are replaced with MDF and MDF event headers. The header part is shown in the table below.

FieldContent
MDF[0]Length
MDF[1]Length
MDF[2]Length
MDF[3]Checksum = 0x00000000
MDF[4][7..0]Compression = 0x00
MDF[4][11..8]Size = 0x7
MDF[4][15..12]Version = 0x3
MDF[4][23..16]Type
MDF[4][31..24]Spare
EVT[0]Mask[0]
EVT[1]Mask[1]
EVT[2]Mask[2]
EVT[3]Mask[3]
EVT[4]Run
EVT[5]Orbit
EVT[6]Bunch

The FE connector

Below is tabulated the pinout of the FE connector.

The numbering of the data busses (D1, D2) is defined by the firmware so the numbering shown in the table is merely indicative. 2V5 signalling is assumed for these.

Similarly the function of the MISC_IO pins is determined by the particular firmware. The LVDS_P and LVDS_N pair are layed out with 100 Ohm differential impedance traces. The FPGA firmware determines the function and whether they are driver or receiver. These two pins can also be used like the other MISC_IO pins if desired. 3V3 signalling is assumed for these.

Pin 1 2 3 4 5 6
1 D2[2] D2[4] D2[5] D1[63] D1[62] D1[61]
7 D2[1] D2[3] D2[6] D1[64] D1[60] D1[59]
13 D2[7] D2[8] GNDFE GNDFE D1[58] D1[57]
19 D2[9] D2[10] MISC_IO MISC_IO D1[56] D1[55]
25 D2[11] D2[12] MISC_IO MISC_IO D1[54] D1[53]
31 D2[13] D2[14] FE2V5 FE2V5 D1[52] D1[51]
37 D2[15] D2[16] MISC_IO MISC_IO D1[50] D1[49]
43 D2[17] D2[18] 3V3 3V3 D1[48] D1[47]
49 D2[19] D2[20] GNDD GNDD D1[46] D1[45]
55 D2[21] D2[22] GNDFE GNDFE D1[44] D1[43]
61 D2[23] D2[24] MISC_IO MISC_IO D1[42] D1[41]
67 D2[25] D2[26] MISC_IO MISC_IO D1[40] D1[39]
73 D2[27] D2[28] MISC_IO MISC_IO D1[38] D1[37]
79 D2[29] D2[30] MISC_IO MISC_IO D1[36] D1[35]
85 D2[31] D2[32] GNDFE GNDFE D1[34] D1[33]
91 D2[34] D2[33] MISC_IO MISC_IO D1[31] D1[32]
97 D2[36] D2[35] MISC_IO MISC_IO D1[29] D1[30]
103 D2[38] D2[37] LVDS_N MISC_IO D1[27] D1[28]
109 D2[40] D2[39] LVDS_P MISC_IO D1[25] D1[26]
115 D2[42] D2[41] GNDFE GNDFE D1[23] D1[24]
121 D2[44] D2[43] FE2V5 FE2V5 D1[21] D1[22]
127 D2[46] D2[45] GNDD GNDD D1[19] D1[20]
133 D2[48] D2[47] 3V3 3V3 D1[17] D1[18]
139 D2[50] D2[49] MISC_IO MISC_IO D1[15] D1[16]
145 D2[52] D2[51] FE2V5 FE2V5 D1[13] D1[14]
151 D2[54] D2[53] MISC_IO MISC_IO D1[11] D1[12]
157 D2[56] D2[55] MISC_IO MISC_IO D1[9] D1[10]
163 D2[58] D2[57] GNDFE GNDFE D1[7] D1[8]
169 D2[64] D2[62] D2[59] D1[1] D1[5] D1[6]
175 D2[63] D2[61] D2[60] D1[2] D1[3] D1[4]

Programming the board

Some users may wish to develop their own program or GUI to operate the board. This requires a deeper understanding of how the board is configured. The most important thing to note from the outset is that all settings and configuration data for the board and any attached FE module are sent as a single IP packet. It is not possible to read or write single registers. Therefore any program must first prepare all the data in a local buffer and send the buffer in a single IP transaction. JRichEcControl is an example of a GUI that does this. When the packet is received by the hardware, a state machine extracts all the configuration data, forwards configuration data to any attached FE module and performs various reset and re-initialisation functions. Finally, the state machine generates a programmable number of internal pulser triggers.

JRichEcControl is a Java program that uses a Java Native Interface (JNI) library to build the configuration data packet and send it to the network. The JNI library is written in C++ and is dynamically linked to the Java VM at run time. Note that LabView uses the same way of extending its functionality - the user must provide a dynamically loadable library of functions that provide the low-level interface to the external hardware. Therefore the JNI library should provide a good starting point to build a suitable LabView DLL if that is the preferred environment.

If instead you prefer a command line approach, you could either write a Java command line program or you could write your own C++ tools. The first has the advantage that the JNI library can be used immediately while the second option would probably require the creation of a new library with the same functionality. Writing your own low-level library of course means that you risk hardware incompatibilities in the event of firmware updates. Another benefit of the Java approach is that you can take advantage of the built-in Java preferences interface to manage configuration settings. The JRichEcConfigurator program manages sets of configuration settings which can then be readily accessed by other Java programs (JRichEcControl for example or a Java command-line program).

Firmware

Different versions of the firmware are used depending on how the Chimaera2 hardware is used (EC-R readout, EC-H readout or trigger board). In addition there are different versions depending on the Chimaera2 board hardware revision. However, nearly all boards now in use are of the Chimaera2v3 type and, for these boards, firmware updates are done through the USB port.

Firmware is distributed through the Cambridge website at this link.

It is recommended to download the firmware to the appropriate subdirectory of richtbuser@lbrichtb:~/Public/cfg preserving the naming scheme of the files on the website. Always copy both the .mcs and the .bit files since scripts depend on their existence even if the .bit file is not normally used.

To flash the new firmware make sure that a USB cable is connected from lbrichtb to the target board. Then cd to the directory containing the downloaded file and type programFlash. The process takes about 3 minutes to complete. Don't interrupt it otherwise you will have to start again.

IDEA! The command should automatically pick up the most recent file in the directory (based on the timestamp in the .bit file name).

In case the Xilinx environment variable is not already defined you will first need to do

. /opt/Xilinx/14.7/Labtools/settings64.sh

JChimaera2Depp utility

A generic diagnostic utility is available that displays selected status and configuration buffers that are implemented in the firmware. The program configures itself using JAXB based on the user-assigned Adept device name (set using the Adept dadutil program). An xml file named <adept-device-name>.xml must be present in the execution directory (case sensitive). When the program is launched with:

java -Djava.library.path=. -jar JChimaera2Depp.jar

the program detects all the connected devices and presents the user with a list of detected devices in the format <adept-device-name>/<connection-name>. When the user selects the device, the program uses the selected device name to load the configuration from the XML file with the same name.

ALERT!It is inadvisable to modify the prepared XML files without a detailed knowledge of the target Chimaera2 firmware.

Edit | Attach | Watch | Print version | History: r17 < r16 < r15 < r14 < r13 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r17 - 2018-10-05 - StephenWotton
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LHCb All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback