This is the testing manual used at KEK ATLAS Group in Module Testing of RD53A and ITkPix v1

Introduction

In this section, plenty of pictures will be presented and the reference documents will also be presented. For people who would like to use specific function of certain device, or encounter some errors while doing QC, it might be helpful to check some of them.

Basics in Fuji B4

Here are the list for devices one should get familiar with as quick as possible to work on DAQ of any module.

1. ATLASPC9 CENTOS7: With Yarr system installed

2. Peltier + Chiller: Cooling and Heat control

3. Raspberry Pi + Cooling Box: Temperature setup

4. High Voltage Supply: -100V Bias Voltage + Pigtail Connector

5. Low Voltage/Current Supply: 4.6A Current supply 1.6V

6. Canary board: instant RTC module temperature update

7. Xray Source: Bump disconnection check and more

Quick Start

Quickly working through a QC flow might be the best way for beginners to get started. Following each step and if one is lucky then he or she could complete the QC on the module. 1. Physical Conditions Setup

• open the vaccum chamber on the side of the wall and change the’ air flow to the scale of 1.5 to 2.

• turn on the chiller and set up temperature to around 10C first, later when testing at lower temperature is necessary cooling box set up could be change Accordingly. Push both pump on and run button to let chiller runs properly.

2. Module Installation

• Installing the module, always put the seal above the case in avoidance of screwer slides down accidentally and break the modules or wirebondings [either case is bad]. Connect the pigtail connector between power adapter and the module [RTC Temperature+HV+LV]. Change the HV supply to -1V and check the connectivity. Add Spacer between the module case to while HV shows 0 leakage current.

• Connect Module and Osaka Board. If data transmission cable is not connected, please carefully insert one side to the module and the other side to the Osaka board and lift up the lock carefully. [Shown as Lakmin board] in previous diagram.

• Solidify the module position. All necessary wires are connected to the module, now add the aluminum cover and connect the air flow to the module case. Then close the Delrin case to further Solidify the position of the module. Please notice whether HV display shows 0 leakage current or not. If it is 0, then the spacer you added might not be thick enough.

3. Set up Cooling System Starts from this step, coding background is needed. Experience in writing shell command and editors will be useful.

• Turn on the bias voltage High Voltage to -100V, please gently ramp up the voltage in -1V, -2V, -3V, -4V, ..., -10V, -20V, -30V, ..., -100V. Please set the limit for leakage current to μA level. The thin module at KEK , from KEKQ11 to KEKQ18 have leakage current around 0.2μA initially when no tests are conducted at all.

• Power Cycle and connection check Before further step, it’s better to have power cycle on this module and check the connectivity. Modify the configuration file as below

          vim ../production/scan-operator/configs/so_yarr.json
          # or other editors 
          # belows are part of the content in so_yarr.json
          "scanlist":[
              [std_digitalscan]
          ] 
          # leave as above save and quit
          cd ../production/scan-operator/
          ./ScanOperator -m KEKQxx 

  If the connection is fine, then the digital scan should have perfect occupancy map. However, if any troubles happened, please refer to trouble shouting section.

• Boost Canary Turn On the Peltier Power Supply and change the directory to the ../production/canary and then run the following command

          python3 PCPost.py -p /dev/ttyUSB0 
          # try ttyUSB1/ttyUSB2 if not working

  keep the program running and start work on the other command window. Then open a new command window and connect to the Raspberry Pi.

• Launch Raspberry Pi and start PID Login to pi@titrasp03NOSPAMPLEASE.kek.jp passwd: ItkPixQC

          ssh pi@titrasp03.kek.jp
          cd titech_cool/pid 
          ./bin/pidServer.py

  After runing the commands above the PID server should be able to run properly. The Command shell should be remained open.

4. IV curve and VI curve

• HV After turning on both HV and LV, wait for a while until the temperature is setup to required temperature. In the pre-production QC2 period 30C, 20C and -15C are the temperature people need to take iv curve

          python3 libDCS/iviscan.py -e configs/lr_powersupply.json 
          -j configs/lr_iviscan.json -w iv -o KEKQ[**]_30_QC[*]_iv
          # please replace the output file name 
          # based on the module number and condition

  The process ramp up the High voltage in range of 0-200V, and the current should rise quickly in the very beginning and remain relatively stable.

• LV The HV process above should be able to change back to the original HV set up after taking the VI curve by changing the current flow input from 4.6A to 0, notice there should be sudden drop of voltage due to semiconduct property

          python3 libDCS/iviscan.py -e configs/lr_powersupply.json 
          -j configs/lr_iviscan.json -w vi -o KEKQ[**]_30_QC[*]_vi
          # please replace the output file name 
          # based on the module number and condition

5. Change to QC test Scan Flow

• If the module is a new module without configurations, one should first open the configuration file of ScanOperator.sh in

          vim ~/RD53AQuad/production/scan-operator/configs/so_modules.json
          # copy/and/paste/KEKQxx/save/and/quit 
          cd ..
          ./ScanOperator -m <KEKQxx> -c

  The command above will automatically generate the configs for KEKQxx. Please just run with -c option in the very beginning, then change the configuration file as how it works in Set up Cooling System part.

          vim ../production/scan-operator/configs/so_yarr.json
          # or other editors 
          # belows are part of the content in so_yarr.json
          "scanlist":[
              [std_digitalscan]
          ] 
          # leave as above save and quit
          cd ../production/scan-operator/
          ./ScanOperator -m KEKQxx -W
          # -W means upload to local database

  The scan should be done in such process, and I will leave the rest to the next section because the work flow is somehow subject to change depends on module type.

6. Tests done and Clean Up After finishing the test in different conditions and one hopes to pause or terminate the work, here is the instruction for how to close up or shift the module.

• HV and LV First, turn off the HV and LV setting. Turn off LV should be simple with one click on LV supply. However, turn off HV should be the similar procedure as ramp up, change the voltage from -100, -90, -80,..., -20,-10,-9,-8,...0.

• Temperature and PID Set cooling box temperature to 25C

          # the shell condition is subjects to change
          ./bin/pidClient.py
          setT 25
          # or use the pidWidget and the slide bar
          ./bin/pidWidget.py

  Either method works to change the temperature to normal state. After module is in normal condition 25C then take out the module carefully and shift to other testing unit.

• Uninstall the Module Wait untill the temperature back to room temperature and then uninstall the module carefully using backward procedure as described in previous installation part. If no further tests will be conducted then one could also shut down the PID

          ./bin/pidClient.py 
          kill_server

In general, the quick QC workflow is written here and subjects to be changed anytime the module testing group inside ATLAS collaboration would like to change. However, many things unique to the ATLAS Japan’s group, like Xray and cooling box will remain the same.

RD53A QC work flow

One should go through the previous quick start section and then get into this part for RD53A testing. If it is other type of module, then please check the section specifically.

RD53A is conducted around 30, 20, -35. -15C separately." />

1. Electrical Test at 30 C

• Set the PID setting temperature at 30 C

• Wait for "STABLE" flag on the Grafana

• Sensor IV scan as described

          # folder: production/scan-operator
          python3 libDCS/iviscan.py -e configs/lr_powersupply.json 
          -j configs/lr_iviscan.json -w iv -o KEKQ[**]_30_QC[*]_iv
          # please replace the output file name 
          # based on the module number and condition

• Normal "Tuning Routine" set to scanlist configs and run ScanOperator

          vim configs/so_yarr.json # and modify
          ./ScanOperator.sh -m KEKQxx -W # -W for upload to localDB

• Xray Source Scan

  • Setup the X-Ray Shielding above the cooling box

  • Rotate the key on the shielding and make sure cooling box is properly sealed. This could be checked through the Pad

  • Set up the output power of Xray to 50kV80uA and change scan configs to std_noisescan.json or (TBD)

  • Once the data taking is finished from Scan operator. Immediately turn of the Xray.

• SLDO VI Scan

          python3 libDCS/iviscan.py -e configs/lr_powersupply.json 
          -j configs/lr_iviscan.json -w vi -o KEKQ[**]_30_QC[*]_vi
          # please replace the output file name 
          # based on the module number and condition

• Upload data using QCHelper (TBD) underscore Need documentation help from Yukijo Fujita San.

2. Electrical Test at 20 C

• Set the PID setting temperature at 20 C

• Wait for "STABLE" flag on the Grafana

• Sensor IV scan as described

          # folder: production/scan-operator
          python3 libDCS/iviscan.py -e configs/lr_powersupply.json 
          -j configs/lr_iviscan.json -w iv -o KEKQ[**]_30_QC[*]_iv
          # please replace the output file name 
          # based on the module number and condition

• "Tuning Routine" set to scanlist configs and run ScanOperator

          vim configs/so_yarr.json # and modify
          ./ScanOperator.sh -m KEKQxx -W # -W for upload to localDB

• Xray Source Scan

  • Setup the X-Ray Shielding above the cooling box

  • Rotate the key on the shielding and make sure cooling box is properly sealed. This could be checked through the Pad

  • Set up the output power of Xray to 50kV80uA and change scan configs to std_noisescan.json or (TBD)

  • Once the data taking is finished from Scan operator. Immediately turn of the Xray.

3. SLDO startup test at -35C

• Turn off the LV power for the module by the labRemote

          python3 libDCS/qaqc.py -e configs/lr_powersupply.json 
          -c LV -d configs/idb_dcs.json -i power-on

• Set the PID setting temperature to -35C

• Wait for the "STABLE" flag for PID on the Grafana

• Change the Cooling Control to the fixed Voltage mode from the PID mode.

• (*)Turn on the LV, and perform the digital scan.

• Check if the module is configured and the EnMask plot is all done

• IF NOT repeat at -20C, -5C 10C until success, as illustrated in previous figure.

• Set the Cooling control back to the PID mode

4. Electrical Test at -15C

• Set the PID setting temperature to -15C

• Wait for "STABLE" flag on the Grafana

• Sensor IV scan as described

          # folder: production/scan-operator
          python3 libDCS/iviscan.py -e configs/lr_powersupply.json 
          -j configs/lr_iviscan.json -w iv -o KEKQ[**]_30_QC[*]_iv
          # please replace the output file name 
          # based on the module number and condition

  output json file to the localDB by using the QC Helper. In previous QC1 and QC2, such data is not uploaded until the very end.

• "Tuning Routine" set to scanlist configs and run ScanOperator

          vim configs/so_yarr.json # and modify
          ./ScanOperator.sh -m KEKQxx -W # -W for upload to localDB

• Xray Source Scan

  • Setup the X-Ray Shielding above the cooling box

  • Rotate the key on the shielding and make sure cooling box is properly sealed. This could be checked through the Pad

  • Set up the output power of Xray to 50kV80uA and change scan configs to std_noisescan.json or (TBD)

  • Once the data taking is finished from Scan operator. Immediately turn of the Xray.

After finishing the QC once, it does NOT mean everything is done for such module. Visual Inspections and plenty of other things also need to be checked. Moreover, another round of the QC might be conducted after Parylene Coating and Thermal Cycles as shown in the beginning of the figure in this subsection, which all depends on the decision of Module Testing community.

External Devices

I was confused at first where to put this section in the entire documentation at first. After careful consideration, I put directly after the introduction section because Manipulating Cooling Box is where I made most mistakes and setup of physical condition is the key to all the testing later. Without proper control of physical condition, overheating, broken module, condensation and no data output could happen anytime.

The Chiller

• Product: As-One LTB-125 α

• Heat Capacity: 180W

• Minimum Set Temperature : -30

• * The actual temperature doesnt reach to -30 unless the chiller is thermally isolated.

• Full manual is available here.

• Coolant needs to be filled up to 2 cm below the internal line of the sink.

• Front Panel

• RUN/STOP for Cooling ON/OFF

• "Pump" for switching Pump Circulation

• Use "SET", "SHIFT", "UP/DOWN" to change the temperature

The Cooling Box

Initialization Process

• Toggle the main switch of the chiller

• Turn ON the PUMP of the chiller

• Make sure the dry air 1.5L/min

• SET the chiller temperature +25C

• Turn ON the Peltier Power Supply Takasago ZX-400L

• Launch Raspberry Pi

          titrasp03.kek.jp
          ssh pi@titrasp03.kek.jp 
          cd titech_cool/pid

• Launch the PID Server

          ./bin/pidServer.py

  Wait for 1 min

• Access to Grafana http://atlastit01.kek.jp:3000 Login using username: itkqc passwd: ItkPixQC Check if temperature logs are properly recorded

• Launch the control weight on Rpi:

          ./bin/pidWidget.py &

• Set the chiller Set Temperature to -30C manual operation while trying to lowering the temperature, change the gas flow to 2L/min and 3L/min will be more efficient, sometime necessary for module with larger heat emission.

Canary Board Remote Monitor

Also described in previous introduction section. If one would like to run the canary board in the back end while using the same shell. One could try screen command. Mainly, Canary board is reading the temperature data from RTC temperature resistor and

    # cd ../production/canary
    screen -S canary 
    python3 PCPost.py -p /dev/ttyUSB0 
    # try ttyUSB1 if command not working
    ^j d # ctrl+j+d to detach
    screen -r canary # to reattach

Grafana

The Grafana is the interface helping scan operator directly observe the physical condition of the module. Dew Point is quite important parameter when one need to open the cooling box. The dew point is the temperature the air needs to be cooled to in order to achieve a relative humidity of 100%. Simply explain, the higher the dew point rises, the greater the amount of moisture in the air. The dew point is highly related with the cooling box interlock logic. Please always pay attention dew point should be higher than carrier temperature to open the module.

LV Remote Monitor

1. Low Voltage Control one could use the code to monitor the PS

    python3 libDCS/qaqc.py -e configs/lr_powersupply.json 
    -c LV -d configs/idb_dcs.json -i measure

This starts monitoring the "LV" channel. For the HV, change the LV to HV and see next subsection. Leave the above script running inside a screen.

    screen -S lv_monitor
    ^ j d # to detach 

It might also possible in the future to run the module with detailed current and voltage recorded TBD

HV Remote Monitor

1. High Voltage Control one could use the code inside qaqc.py

    # cd ../production/scan-operator
    python3 libDCS/qaqc.py -e configs/lr_powersupply.json 
    -c HV -d configs/idb_dcs.json set-voltage -1
    # The Command above setup the HV to -1 
    python3 libDCS/qaqc.py -h

Notice Please change the HV Config from GPIB mode to RD-232 mode before operating. When one reboot the HV supply, some other people used the HV, such situation might lead to the change of condition of DAQ. In such scenario, always check setting first. 2. High Voltage Limit Set Up Some people might encounter the limitation is set too low and the HV could not be increased remotely. Setting leakage current limit should solve such issue.

Xray Shielding and interlock

ATLASPC9 Yarr and Scan

As shown in previous section Yarr System and FPGA Firmware with Osaka boards are important for the DAQ at B4. The signals and data from the RD53A are transmitted through these adapters and collected in the ATLASPC9. This section is designed to help the beginner who would not like to use yarr as a blackbox. Many parts of the yarr should be explained with more details to help with specific hack and changes.

Configuration Files

Before going to further step on how codes work or how to understand the entire yarr system (where some detailed parts may not worth your time for input), understanding current configuration is quite essential for one to modify and make their own configuration when specific testing hope to be conducted.

    # Config inside scan operator
    /home/atlasj/work/RD53AQuad/production/scan-operator/configs/so_yarr.json
    ../production/scan-operator/configs/so_modules.json
    # Scan Configs inside yarr
    # here just list several typical scan type 
    # Some configs are subject to change and has been studied in 
    # later sections
    ../production/yarr/configs/scans/rd53a/std_digitalscan.json
    ../production/yarr/configs/scans/rd53a/std_analogscan.json
    ../production/yarr/configs/scans/rd53a/std_thresholdscan.json
    ../production/yarr/configs/scans/rd53a/std_crosstalkscan.json      
    ../production/yarr/configs/scans/rd53a/std_noisescan.json
    ../production/yarr/configs/scans/rd53a/std_xrayscan.json        

For example, here the std_xrayscan.json is made from std_noisescan.json and all type of scans with external source are called sourcescan within the Module Testing Community. Change the data taking frequency in the configs Files may help with the data taking rate and also help to distinguish the component from PCB area.

Digital and Analog Scan

Digital Scan is mostly used when checking the functionality of the circuit itself. Analog scan is using random trigger to have 100 injections on each pixels. The acceptable Scan should look like following figure.

Masking and Mask Loop

The usual digital scan and analog scan provide the mask for ITk’s later study, like tracking and etc. The noisy pixel or broken pixel with no data output will be block from the final calculation process. Inside yarr system, for testing purpose, such part of the code is located in:

    /home/atlasj/work/RD53AQuad/production/yarr/src/libRd53a
    # Specific file name
    /home/atlasj/work/RD53AQuad/production/yarr/src/libRd53a/Rd53aMaskLoop.cpp

And the masking loop is inside the Rd53aMaskLoop.cpp file is also shown below. Usually when people are taking scans, the standard mask is obtained but not applied. If the scan is taken with certain option, then the mask will be applied to analog scan, threshold scan and etc.

    void Rd53aMaskLoop::execPart1() {
        // Some code here before loop  

        //Loop over all pixels - n_Col=400, n_Row=192 - from Rd53aPixelCfg.h
        for(unsigned col=0; col<Rd53a::n_Col; col++) {
            for(unsigned row=0; row<Rd53a::n_Row; row++) {
                if (applyMask(col,row)){
                    if (m_maskType == StandardMask){
                      //---------------------------------------------
                      //standard map, inj and read out the same pixel
                      //---------------------------------------------
                      rd53a->setEn(col, row, 1);
                      rd53a->setInjEn(col, row, 1);
                      //---------------------------------------------
                      // Hack could be done here if one would like to 
                      // block certain area 
                      //---------------------------------------------
                      //iwata (KEKQ10ver)
                      //if (col<5&&row>51&&row<60){
                      //  rd53a->setEn(col, row, 0);
                      //}
                      //KEKQ15ver
                      // if (col<4&&row<104&&row>35){
                      //   rd53a->setEn(col, row, 0);
                      // }                      
                    }
                    if (m_maskType == CrossTalkMask  ){
                      // Some code here 
                    }
                    else if (m_maskType == CrossTalkMaskv2 ){
                      // Some code here
                    }
    }

Notice, there is no consensus on such method. While one pixel is broken and the broken pixel area is extremely noisy, using such method to block certain pixel to continue QC may help with complete the process, but fundamentally such module might be bad and emitted.

Threshold Tuning

This subsection threshold Tuning part is based heavily on the RD53A module. RD53A has three Front End (FE). It basically represent the Front End Circuit Structures are different. The tuning process for Differential FE (Diff FE), Linear FE (Lin FE) and Synchronous FE(Syn FE) are all different. For Lin FE, the ASICS could stor 4 bits in each pixel and the value ranges from 0 to 15. For Diff FE, the ASICS could store 5 bits in each pixel, which means the tunable range is -15 to 15 in total 31 values. The tuning sequence used in the RD53A QC flow written in the current configuration file is

    # file location:
    # /home/atlasj/work/RD53AQuad/production/scan-operator/configs/so_yarr.json
    "Tuning Routine":
    [
        ["diff_tune_globalthreshold", 1500],
        ["diff_tune_pixelthreshold", 1500],
        ["diff_retune_globalthreshold", 1500],
        ["diff_retune_pixelthreshold", 1500],
        ["lin_tune_globalthreshold", 2000],
        ["lin_tune_pixelthreshold", 2000],
        ["lin_retune_globalthreshold", 1500],
        ["lin_retune_pixelthreshold", 1500],
        ["syn_tune_globalthreshold", 1500],
        ["syn_tune_globalthreshold", 1500], 
        ...

The value actually being changed is the TDAC value inside the pixel circuit. For differential front end. One could also refer to the Diff FE user guide. . The differential user guide listed the detailed parameters that might influence the final tuning and etc. If one would like to check how the tuning process is specifically defined, then here is the specific code block for tuning in feedbackloop:

    # /home/atlasj/work/RD53AQuad/production/yarr/src/libRd53a
    void Rd53aPixelFeedback::init() {
        if (m_resetTdac) {
                    ...
                    for (unsigned col=1; col<=Rd53a::n_Col; col++) {
                        for (unsigned row=1; row<=Rd53a::n_Row; row++) {
                            //Initial TDAC in mid of the range
                            if (128<col && col<=264 && tuneLin) {
                                rd53a->setTDAC(col-1, row-1, 8);
                                linCnt++;
                            } else if (264<col && tuneDiff) {
                                rd53a->setTDAC(col-1, row-1, 0);
                                diffCnt++;
                            }

Somehow Synchronous FE is ignored in this section. For the upper level where produced the feedback histogram matrix, the code blocks look like this:

    # /home/atlasj/work/RD53AQuad/production/yarr/src/libYarr
    void OccPixelThresholdTune::processHistogram(HistogramBase *h) {
        ... 
        for (unsigned i=0; i<fbHisto->size(); i++) {
            double occ = occMaps[ident]->getBin(i);
            if ((occ/(double)injections) > m_occHighCut) {
                fbHisto->setBin(i, -1);
            } else if ((occ/(double)injections) < m_occLowCut) {
                fbHisto->setBin(i, +1);
            } else {
                fbHisto->setBin(i, 0);
            }
            mean += occMaps[ident]->getBin(i);
            occDist->fill(occMaps[ident]->getBin(i));
        }
        ...

m_occLowCut is the value set up for tuning and decide which direction the TDAC value would be changed.

Threshold Scan

How to obtain S-Curve for differential FE and more. This should be helpful for the specific pixels’ study. The basic idea for taking threshold scan is to inject 100 charge at different threshold. The output occupancy will output S-curve and fitted by errnode function.

    # part of the diff_thresholdscan.json file 
    {
        "config": {
          "max": 200,
          "min": 0,
          "step": 1,
          "parameter":"InjVcalDiff"
        },
        "loopAction": "Rd53aParameterLoop"
      },
      {
        "config": {
          "max": 50,
          "min": 33,
          "step": 1,
          "nSteps": 2,
          "delayArray": [0]
        },
        "loopAction": "Rd53aCoreColLoop"
      },

"InjVcalDiff" is basically the parameter to control the injection charge. Basically 200 roughly means 2000 electrons injection charge. Here, max and min control the threshold scan range. If tuning at 2000e, then the scan range 0 to 200 is definitely not helping with fitting the S-Curve.

Source Scan

This section is written for xray source scan user. The source scan may not be conducted in all ATLAS group’s testing sites, as preparation of xray might be hard for some group. However, since there are plenty of study being conducted, a section of how to use the xray and what the data is presented here.

    # part of the diff_noisescan.json file 
    "loops": [
        {
            "config": {
                "count": 0,
                "delay": 48,
                "extTrigger": false,
                "frequency": 200000,
                "noInject": true,
                "time": 60,
                "sendEcr": false
            },
            "loopAction": "Rd53aTriggerLoop"
        },
        {
            "loopAction": "StdDataGatherer"
        }
      ],
      "name": "NoiseScan",

The above loops is written inside the configuration, other source scan or xray scan should have similar one. "frequency" is set to 200,000 Hz and "time" for data taking is 60s. In such trigger, some study is presented and shown in this link https://indico.cern.ch/event/1100116/ iwata qaqc.

Disconnect Bump Scan

Here the simple initial comparison is provided. Such comparison is presented in module testing group. However the expected bump disconnection scans should look like figure below.

Software Tools and Database

Scan Operator

PlotFromDir

Local Database

Debugging Appendix

This section is intended to be introduced for general problems one could possibly encounter during QC. Some of them are human errors which could be solved and some others show that the pixel detector is just broken.

Cabling and Connection issue

1. Adapter Broken Case [Non Human Error] While connecting the cable, some accident happened before that the lock in the Osaka board end might be broken when lifted up. Please move that part carefully and report the broken issue once happene again.

2. Lost HV or LV [Likely Human Error] This may happened during the installation process. After closing the Dulrin case and the HV suddenly lost, in such scenario the scan operator output sample would look like this:

    [17:57:43:349][  info  ][    SpecRx     ]: Active Rx channels: 0x1
    [17:57:43:349][  info  ][    SpecRx     ]: Active Rx lanes: 0xf
    [17:57:43:349][  info  ][    SpecRx     ]: Rx Status 0x0
    [17:57:43:349][  info  ][    SpecRx     ]: Number of lanes: 4
    [17:57:43:349][ error  ][    SpecRx     ]: Channel 0 Lane 0 not synchronized!
    [17:57:43:349][ error  ][    SpecRx     ]: Channel 0 Lane 1 not synchronized!
    [17:57:43:349][ error  ][    SpecRx     ]: Channel 0 Lane 2 not synchronized!
    [17:57:43:349][ error  ][    SpecRx     ]: Channel 0 Lane 3 not synchronized!
    [17:57:43:359][ error  ][     Rd53a     ]: Did not receive any data for chip1
    [17:57:43:359][critical][  scanConsole  ]: Cant establish communication, aborting!
    [ warn ][sl] After 1 attempts, couldnt succesfully finish std_digitalscan.
    [ info ][so] Wrapping and exiting after getting an error

3. Cable between Osaka board and module [Human Error] When one connect the data cable between adapter and the silicon detector, during the first boosting process, digital scan or analog scan might give interesting sequential output.

This could also caused by definition of certain parameter inside the configuration file. For example the file in configs show that:

    # Configuration file ../production/scan-operator/configs/so_modules.json
    "GlobalConfig":
    {
        "SldoAnalogTrim": 22,
        "SldoDigitalTrim": 22,
        "OutputActiveLanes": 7,
        "CmlEn": 7,
        "CmlEnTap": 1,
        "CmlInvTap": 1,
        "CmlTapBias0": 600,
        "CmlTapBias1": 100,
        "CdrSelSerClk": 1,
                    "AuroraCbSend": 1,
        "DiffLcc": 400,
        "DiffLccEn": 1,
        "CdrPdDel": 4
    },

One could clearly see the parameter "CmlTapBias0" and "CmlTapBias1" are the one controling output waveform.

RD53A the Tap value "CmlTapBias0" around 600 and "CmlTapBias1" and 100, these two should be modified in the in the testing stage. " />

4. SHT85s Temperature Monitor broken [Non Human Error] After one year’s operation, the temeprature monitor SHT85s is broken. Suzuki Takahito San from Waseda is investigating the reason behind it. It could be the reason that people lift the lid to heavily or the radiation from the Xray breaks the actual module.

5. High Voltage Limitation [Human Error] This was once happened when people trying to ramp up the temperature but the limitation to the leakage current is set to 1μA. Someone not familiar with the HV manipulation could refer to the notation here.

Cooling Box Issues or Errors

1. The coolant flow problem [Non human error] Some time the flow of the cooling box will stops or interupted, or one might be able to observe the vacancy in the pipeline. This has been modified for several time however it is still not properly changed.

2. Cooling box does not work properly [human error] This also happened to me first when I am not so familiar with cooling unit. However, the nitrogen vaccum chamber and chiller are always the first things to be checked.

3. PID widget crash, GUI not being used properly [Human Error] This might happen when one was trying to open too many windows. If there is already one PID Widget opened and one is trying to change the temperature setup from the other PID widget, then such action is mostly like to be failed. Only one PID widget could communicate with PID temperature control. Close all the widgets and open a new one should simply solve such issue.

4. Interlock Could not change to Open Status This is mostly likely due to the dew point setup is way lower than room temperature. One should wait for module recovers to normal state in order to open the interlock and change the module.

5. Grafana Not refreshing data monitored The situation happened when the canary board is not properly uploading the data. It might be two reason: 1. the canary board is not upload data for proper USB port. This is also written in the detailed explanation in the quick start section. 2. Switching the module may interrupt the data uploading process. Usually people used screen for running the program in the back end.

    # cd to production/canary
    screen -S canary
    python3 PCPost.py -p /dev/ttyUSB0 
    ^ j d # detach 
    # try ttyUSB1/ttyUSB2 if not working

Since this method is not so stable, changing to running in the command line Status might be the best way for monitoring module data.

    # cd to production/canary
    python3 PCPost.py -p /dev/ttyUSB0 
    ^ j d # detach 
    # try ttyUSB1/ttyUSB2 if not working

which is also written in the current version of the User guide.

Assembly Appendix

Although this part is not the main purpose of the QC documentation, understanding the overall structure of RD53A module or ItkPix v1.x module will be helpful to debug the potential problems in scan.

Data, code and figures are provided in electronic form.