16-CBC3 module

This section is dedicated to the 16CBC3 module received from KIT

Set-up

Description

2S Module

Main components: 2 parallel strip sensors (2 x 1016) 1.8mm sensor spacing 5cm x 90μm cell size sensors are read out by the CMS Binary Chip (CBC). Each CBC reads out 254 strips (2 x 127) and performs hits correlation (stubs) and sends the stub data out at each BX. 2 Front-end hybrids (FEH): Right and Left FEH 8 x CBC3 (Right FEH, FeId ="0") + 1 x CIC2 mezzanine card 8 x CBC3 (Left FEH, FeId ="1") + 1 x CIC2 mezzanine card CBCs send data to CIC that performs data sparsification, formats the output data, and sends them to the service hybrid (via GBT transmitter) Service hybrid (SEH): Power distribution (DC-DC converters) Data serialization Opto-electrical conversion VTRx (VersatileTransceiver, operating at up to 10 Gb/s in the upstream and up to 5 Gb/s in the downstream directions) GBT (Giga-Bit Transceiver) GBT-SCA (Slow Control Adapter)

slides

share point

KIT set-up

DESY set-up

DAQ system

MicroTCA crate: backplane which can accommodate up to 12 Advanced Mezzanine Card (AMC) modules along with up to two μTCA Carrier Hub (MCH) modules.

Current configuration:

• FC7 ACM card
  • FPGA, Micro SD card slot, 2 FMC connectors:
    • FMC (FM-S14)
    • DIO5
• MCH:
  • system management and data transfer to the PC (Ethernet)

Powering

Low Voltage

Channel 1 (12V): fan for DC-DC cooling (testbench), Channel 2 (10V): module powering + grounding (black cable)

Power distribution on the SEH (2S module)

High Voltage

Grounding: SHV shielding has to be connected directly with the black cable to the LV Power supply GND

Remote control

* XDAQ application: already installed on sbgat209 machine ( contact Christian Bonnin for details)

• Point your browser to

  http://sbgat209.in2p3.fr:1973

  and click on ChrominiSupervisor
• if it does not respond, login to sbgat209 and do the following

  • cd ~/TriDAS/pixel/PixelRun

  • ./run.sh --xml ~/TriDAS/pixel/PixelRun/telescopeIphc.xml

  • leave the terminal open

* LV: the NGP800 power supply can be controlled via XDAQ web interface

• go to XDAQ web interface(http://sbgat209.in2p3.fr:1973) > Device Under Test > LV control

• go to Device Under Test > HV (control)

• 
  "
• N.B: to unlock do Ctrl+Click (Click only won't work)

Some reference plots from calibration runs.

X-Y Table

Operate the x-y table Operate the x-y table

1. CONNECTION :

   Ouvrir MCS2ServiceTools
      Dans "selectMCS2" mettre l'adresse "network:192.168.7.117"
      Puis connect.
   On doit trouver "MCS2M-D-3C"

   Verifier que pour les modules parameters on est bien pour channel 1 et channel 2 en INVERTED
      et que le Default MAx CLF a pour valeur 1000
    Si OK,on peut déconnecter 

ET passer à la suite: 
   Ouvrir MCS2DemoGUI
   Dans "MCS2 Locator" mettre l'adresse "network:192.168.7.117"
   Selectionner "Movement"
   Choisissez le mode "manual" (cliquant sur "slider")

RAPPEL : X positif vers le mur de la salle. X: canal 0 ("Channel 0")
    y positif vers le haut          Y: canal 1 ("Channel 1")
    z positif

2. POSITIONS :
   A priori, en (X=0,Y=0) le centre du DUT devrait etre dans l'axe du faisceau.
   Limites :   vers le mur :   X = +66 mm 
         vers la salle : X = -55 mm
         vers le haut : Y = 89 mm
           vers le bas :  Y = -7 mm (si X=0, on bute sur le bas du support DUT)
                    (si = -55 mm, on peut descendre plus bas...)
   Si le système perd son 0 :
      aller à fond en X positif et setter X=66 mm
      aller à fond en Y positif et setter Y = 89 mm
       NE PAS TENTER DE DESCENDRE A VIDE : IL FAUT UNE MASSE DE 300 GRAMMES...

   DONC : Positionpour monter le DUT : à fond en haut     (Y = 89 mm)
                   a fond vers le mur (X = 66mm)

Ph2_ACF

Ph2_ACF is the CMS Tracker Phase2 Acquisition & Control Framework.

Get started

1. First of all login to sbgat209 (or sbgat 419, sbgat231 has internet issue) and chose your working directory.

2. Then get the current being used code (at CYRCé):

    git clone -b StrasBugFix https://gitlab.cern.ch/cms-iphc/Ph2_ACF.git
    

For the official code, do

   git clone --recurse-submodules https://gitlab.cern.ch/cms_tk_ph2/Ph2_ACF.git
   

3. Setup (every time you login):

   cd Ph2_ACF
   source setup.sh
   

4. Compile the code:

   cmake3 -S . -B build
   cmake3 --build build -j 4
   

5. Optional: chose a running directory

   mkdir rundir
   cp settings/D19CDescription_iphc*.xml rundir/
   

You will have 2 files: D19CDescription_iphc_calib.xml and D19CDescription_iphc_data.xml.

Parameter setting

You can look at D19CDescription_iphc_calib.xml or D19CDescription_iphc_data.xml**D19CDescription_iphc_calib.xml or D19CDescription_iphc_data.xml

Useful settings

Open D19CDescription_iphc.xml with your favorite editor and have a look

• Address: uri="chtcp-2.0://sbgat209.in2p3.fr:10203?target=192.168.7.170:50001"

• Optical link: <GBT <span class='WYSIWYG_COLOR' style='color:#008000'>phaseTap="7" enable="1"/>

• Events: <Setting <span class='WYSIWYG_COLOR' style='color:#008000'>name="Nevents"> 20
• Config files path: CBC_Files path="./settings/CbcFiles/
  • To be update after calibration
• Clock: <Register <span class='WYSIWYG_COLOR' style='color:#008000'>name="ext_clk_en"> 0
  • 1 -> enable external clock
  • 0 -> disable external clock
• Trigger: <Register <span class='WYSIWYG_COLOR' style='color:#008000'>name="trigger_source"> 3 </verbatim>
  • 1 -> TTC
  • 2 -> Stubs
  • 3 -> Internal trigger (oscillator)
  • 4 -> TLU
  • 5 -> External trigger
  • 6 -> Test Pulse (charge injection)
• DIO5: <Register <span class='WYSIWYG_COLOR' style='color:#008000'>name="dio5_en"> 1
  • 1 -> enabled
  • 0 -> disabled
• Sparsfication: <CIC2 enableBend="1" enableLastLine="0" <span class='WYSIWYG_COLOR' style='color:#008000'>enableSparsification="1" clockFrequency="320"/>
• Hit comparator mode: <Misc analogmux="0b00000" <span class='WYSIWYG_COLOR' style='color:#008000'>pipelogic="0" stublogic="0" or254="1" tpgclock="1" testclock="0" dll="0"/>
  • 0 -> sampled
  • 1 -> OR
  • 2 -> HIP
  • 3 -> latched

https://gitlab.cern.ch/cms_tk_ph2/Ph2_ACF/-/blob/Dev/HWInterface/CbcInterface.cc#L254

xml file testing

Test the parsing of the D19CDescription_iphc.xml file

systemtest -f D19CDescription_iphc.xml

Run the code

At each new terminal enter Ph2_ACF directory and setup the code environment

cd <path-to>/Ph2_ACF
source setup.sh

Then go to your running directory

FC7 configuration

Check if the FC7 is connected

ping -c 3 192.168.7.170

List all available firmware images on the FC7 FPGA

fpgaconfig -c D19CDescription_iphc.xml -l

Add a new firmware image on the FC7 FPGA

fpgaconfig -c D19CDescription_iphc.xml -f <your image> -i <name without .bin> 

"name" you chose must be not too long, otherwise you receive error "TextExceedsSpaceAvailable":

terminate called after throwing an instance of 'uhal::exception::TextExceedsSpaceAvailable'
  what():  Text exceeds space available for it in the MMC (no additional info)
Aborted (core dumped)

Load firmware image at each FC7 reboot (current firmware)

fpgaconfig -c D19CDescription_iphc.xml -i uDTC_2S_CIC2_3Links

*Note*: first run after loading not good

Before data-taking

Pedestal equalization and noise measurement (CBC)

Change trigger settings: <Register <span class='WYSIWYG_COLOR' style='color:#008000'>name="trigger_source"> 3

Important: Make sure both Test Pulse and Sparsification are disabled: <TestPulse <span class='WYSIWYG_COLOR' style='color:#008000'>enable="0" polarity="0" amplitude="0x3F" channelgroup="0" delay="0" groundothers="1"/>

How to run pedestal equalization and noise measurement

feh_2s_test -f D19CDescription_iphc.xml --withCIC -t -m [-a] [-b] --lvps "http://192.168.7.94/scpi_response.txt?request=inst:nsel%202"

• -t : tune offset
• -m: measure noise
• -a: all channels at once
• -b: batch mode

Calibration files are produced in the Results folder. Change the config files path in D19CDescription_iphc.xml in other to use the obtained ones for data taking.

If you do not run with the -lvps "..." option, you will notice that you need to control the power supply to the service hybrid during the power-cycling . At the moment it is done manually (Ph2_ACF pauses to let you do that)

Under Results folder you will find a ROOT file

Noise distribution in the two FEHs

SCurve for a given CBC

Hit and Stub latency scan with Test Pulse

First of all change trigger settings: <Register name="trigger_source"> 6

Make sure Test Pulse is enabled: <TestPulse enable="1" polarity="0" amplitude="0x3F" channelgroup="0" delay="0" groundothers="1"/>

To run Hit latency scan, do:

commission -f D19CDescription_iphc.xml --withCIC -l -m 0 -r 512 --lvps "http://192.168.7.94/scpi_response.txt?request=inst:nsel%202" -b

• -m : minimum value
• -r : range in clock cycles (max=m+r)
• -l : latency (hits)

To run Stub latency scan

1. add the hit latency found previously under <Settings threshold="550" latency="104"/>

2. Then run

   commission -f D19CDescription_iphc.xml --withCIC -s -m 0 -r 512 --lvps "http://192.168.7.94/scpi_response.txt?request=inst:nsel%202" -b
   

• -m : minimum value
• -r : range (max=m+r)
• -s: stub latency

3. After you've found the stub latency, add it to <Register name="common_stubdata_delay"> 5

For test pulse: hit latency = 104 and stub latency = 35.

Hit and Stub latency scan with source or beam

Change trigger settings: <Register name="trigger_source"> 5

Make sure Test Pulse is disabled: <TestPulse enable="0" polarity="0" amplitude="0x3F" channelgroup="0" delay="0" groundothers="1"/>

To run Hit latency scan, do:

commission -f D19CDescription_iphc.xml --withCIC -l -m 0 -r 512 --lvps "http://192.168.7.94/scpi_response.txt?request=inst:nsel%202" -b

• -m : minimum value
• -r : range in clock cycles (max=m+r)
• -l : latency (hits)

To run Stub latency scan

1. add the hit latency found previously under <Settings threshold="550" latency="104"/>

2. Then run

   commission -f D19CDescription_iphc.xml --withCIC -s -m 0 -r 512 --lvps "http://192.168.7.94/scpi_response.txt?request=inst:nsel%202" -b
   

• -m : minimum value
• -r : range (max=m+r)
• -s: stub latency

3. After you've found the stub latency, add it to <Register <span class='WYSIWYG_COLOR' style='color:#008000'>name="common_stubdata_delay"> 5

With CYRCé beam: hit latency = 119 and stub latency = 48.

• to find stub latency faster, you can play with external trigger delay <Register name="ext_trigger_delay_value"> 100 .
• Currently, for external trigger delay = 100, hit latency = 119 and stub latency = 48.

DLL scan with source or beam

At the end of the D19CDescription_iphc.xml file, you will find DLL configuration:

<Setting name="StartVcth"> 590
<Setting name="EndVcth"> 540
<Setting name="StartLatency"> 113
<Setting name="LatencyRange"> 4
<Setting name="DLLStep"> 5

To run DLL scan, do

hit_count_pulse_shape -f D19CDescription_iphc.xml --withCIC -b -o DLL_Scan/

Take data with source or beam

Change trigger settings: <Register name="trigger_source"> 5

Make sure you have updated

• the config files with ones from calibration
• the latency values

To take data and save results in to binary files, run

Using number of events

miniDAQ -f D19CDescription_iphc.xml --withCIC -e 1000 --dqm --daq --lvps "http://192.168.7.94/scpi_response.txt?request=inst:nsel%202" 

Using data taking duration (in seconds)

miniDAQ -f D19CDescription_iphc.xml --withCIC -D 30 --dqm --daq --lvps "http://192.168.7.94/scpi_response.txt?request=inst:nsel%202" 

• -e : number of events
• -D: duration
• --dqm (-q) : create histograms
• --daq (-d) : : save the data into a binary .daq file (another .raw format is also available
• -o: output directory. WARNING: the directory must be created before running miniDAQ (to be fixed). Default: Results/

Reproduce results from binary files

You can still reproduce qdm histograms from the binary file, by running:

miniSLinkDQM -f file.daq --dqm [--tree] - o output

Make sure you create "output" directory first!

Troubleshooting

Troubleshooting Grounding of the mini module

   Starting back-end alignment procedure ....
   terminate called after throwing an instance of 'std::out_of_range'
   what():  vector::_M_range_check
   Abandon (core dumped)
   

   I2C readback failed..  for hybrid 0 register 0xa4 [CIC]
   terminate called after throwing an instance of 'std::runtime_error'
   what():  I2C readback mismatch...
   Abandon (core dumped)
   

   Reading monitoring values on SEH connected to link 0
    .... Reading 1.5V monitor on SEH  : 0 V.
    .... Reading 2.5V monitor on SEH : 0 V.
    .... Reading Vmin monitor on SEH : 0 V.
    .... Reading internal temperature monitor on SEH : 0 [ raw reading is 0].
   According to the Firmware status registers, it was compiled for: 6 hybrid(s), 1 CIC2 chip(s) per hybrid
   Enabling FE hybrid : 0 - link Id 0
   Error in read-back of I2C from CBC.. register which failed is 0xc
   terminate called after throwing an instance of 'std::runtime_error'
   what():  I2C error with CBC on front-end hybrid.. 
   Abandon (core dumped)
   

   ...No data in the readout ... Trigger in counter is 0 asked for 1000 events and have 0 words in the readout... Re-trying point
                             ||I| Failed to readout all events..... Retrying...
                             ||I| Read back 0 from FC7... readout request is 0
   

   D19cFWInterface has received ... 18 ... events from DDR3.. data size is 1440 32 bit words.
   terminate called after throwing an instance of 'std::out_of_range'
   what():  map::at
   Aborted (core dumped)
   

   read back from FC7 with ReadData
                             ||I| Iter#0 5632 words in FIFO.. going to readout data
                             ||I| D19cFWInterface has received ... 387 ... events from DDR3.. data size is 6192 32 bit words.
                             ||I| 387 events read back from FC7 with ReadData
                             ||I| Stopping triggers...
                             ||I| EventType: d19c CIC2
   

   terminate called after throwing an instance of 'uhal::exception::ControlHubTargetTimeout'
   what():  The ControlHub did not receive any response from the target with URI "chtcp-2.0://sbgat209.in2p3.fr:10203?target=192.168.7.170:50001"
   ControlHub returned error code 3 = 'no reply to status packet'
   

 ||I| Want to configure CIC to output patterns from readout chips... 
 ||I| CIC output pattern configured by setting MISC_CTRL to 00001010
 ||I| Re-loading original coonfiguration of fast command block from hardware description file [.xml] 
 ||I| Re-loading original coonfiguration of fast command block from hardware description file [.xml] 
 ||I| Resetting all registers on back-end board 0
terminate called after throwing an instance of 'std::out_of_range'
  what():  vector::_M_range_check
   

References

Document	Link
Workshops	daq-school-2019 , 2S Module, daq-school-2021 (intro), daq-school-2021
2S Module	Dimension drawing, 8CBC3-ModuleCarrier-KIT, Dimension drawing KIT
CBC specifications	CBC3_universal_interface_specification_V1_2
FC7 User Manual	FC7 Manual
Micro TCA	schroff
MTCA.4 Tutorial	indico.cern.ch/event/489996
NAT-MCH	NAT-MCH –User’s Manual
d19c user’s and developer’s manual	d19c_manual
CERN, Development of DC-DC converter at CERN	dcdc project
The VTRx+, an Optical Link Module for DataTransmission at HL-LHC	inspire-prod-file
CBC3 CMSSW	cbc3_CMSSW, Unpacker_and_analysis
CYRCe for CMS Test Beam	Preliminary, OT system test meeting, Tracker Plenary during TK Week
Outer tracker	FEE2016_Kloukinas_Kostas
CHROMIE tutorial for users	indico
2S Module Apps	Gipht GUI, KIRA Pulse Generator
Thesis	De Clercq, Mykyta, Nikkie, Stefan (KIT
Further reading	CMS NOTE -2019/006, H. Robert report 2019