This page contains information on how to configure the system and to take data in the ComLab.

Columns commissioning

The full test protocol is available at https://www.overleaf.com/read/mrqtdxndrrck. Refer to the Troubleshooting section of this page to solve issues.

To check HV and DCS information, from the run control fsm, click Select and plot, and select (up to eight) variables to plot.

If the run control panel is not running, send

/home/lbrich/WinCC/shortcuts/fsm.sh

from a terminal on workspace 6 of pcsyslab02.

Columns installation

Follow the validation section at https://www.overleaf.com/read/mrqtdxndrrck.

See the instructions at https://edms.cern.ch/ui/file/2301748/1/EDMS-2301748-v1-Chiller-Instruction-for-Commissioning-CF191216-1105.pdf to turn on/off the Nove chiller.

Columns test

From the run control fsm click on the ComLab button. Select the column ID and press the Run all steps button. Check from time to time the temperaturs, low and high voltages and current are OK and updating, using the Select and plot button.

The processing of the runs is automated. Only the DAC scans need a further processing. To produce the corresponding output run on lbrichpcie40-00 (first argument is the list of run numbers, the second number the list of thresholds)

/home/lbrich/bin/dacOutput.sh [RICH_ID] [column number] [commaSeparatedListOfRunNumbers] [commaSeparatedListOfThresholds]

Once the test is finished, run from pcsyslab02 (the first argument is the RICH ID, the second the column ID):

./compressConditions.sh 2 10

for RICH2 COL_10 in this example.

At this points data can be copied to eos under /eos/lhcb/wg/RICH/ComLab/RICH2

Setup the system from scratch

  • Switch on the file server storerich01.
  • Switch on the chiller for the MARATON power supply.
  • Switch on the MARATON and CAEN power supplies.
  • Switch on the pcie40 server (lbrichpcie40-00). In case of a power cut, these server automatically startup. However, in order for things to be properly configured, send a poweroff command and switch it on again.
  • Once storerich01 is pinging, switch on pcsyslab02, lbrichecs01 and pclhcb33.
  • The /home/WinCC and /data folders must be mounted on pcsyslab02, lbrichecs01 and lbrichpcie40-00. If not, mount it on both machines. Check on /etc/fstab the right paths (first column is the remote folder, second column the local path).
  • From pcsyslab02, ssh to lbrichpcie40-00.
  • Send the sudo pcie40_systemd -r to load the firmwares.
  • Open the GBT server log with sudo pcie40_systemd -l.
  • On pcsyslab02 open a terminal and type startPA.
  • Run MultiMiniDAQ on workspace 6.
  • On pcsyslab02 do multiple ssh to lbrichecs01 to run the projects as shown in the following:
  • Run R2DCS1 on workspace 7. Open the D&N, FSM tab, start/restart FSM. Take the FSM. The lock next to the top FSM node must be blue (=> shared).
  • Run R2HV1 on workspace 8. Open the D&N, FSM tab, start/restart FSM. Take the FSM. The lock next to the top FSM node must be blue (=> shared).
  • The CAEN and Wiener OPC servers should run as services (you can check with sudo systemctl status OpcUaCAEN.service and sudo systemctl status OpcUaWiener.service).
  • Run RIECS1 on workspace 9.
  • On workspace 10, ssh to pclbrich02 and send the startDB command.
  • Run the elmb project on pclhcb33. Run the CAN OPC server with ~/OpcUa/ELMB/runCANserver.sh OPCUACANOpenServer.xml.
  • Check that on pcsyslab02, lbrichecs01 and lbrichpcie40-00 DIM_DNS_NODE=lbrichecs01.dyndns.cern.ch and DIM_HOST_NODE is the pcsyslab02.dydns.cern.ch, lbrichecs01.dyndns.cern.ch, lbrichpcie40-00.dyndns.cern.ch. Try sending a did & command from pcsyslab02. View->AllServer. If you see DNS is down, you need to send a sudo systemctl restart dsnd. DNS must be down on lbrichpcie40-00.
  • Open the D&N of MultiMiniDAQ, FSM tab, Open the MiniDAQ FSM and take it. Press restart ctrl managers.
  • Browse to TFC->SOL40->sol40_004 to make disappear the SOL40 links not declared in the SOL40 firmware.
  • On the main operations panel, press the FW Versions... button. Press Configure under config PLLs in the following order: lab004, lab010, lab018. Press Configure GBTs with the same order. Press Configure SFP0 followed by Configure SFP1 in the same order.
  • Reset the TFC partition.
  • On the main operations panel, under other devices, select top_018, press the RXReady button and check Rx TFC 0 and Tx TFC 1 are green. If not, tick RESYNCHRONIZE, Apply commands, clear commands. Repeat the procedure for top_010.
  • Open and start the RICH_SAFETY FSM from the ECS project.
  • Send a START_RUN command. If the FSM fails to go to the RUNNING state, reload the recipes on richtell40_010 and richtell40_018. To reload the recipes RESET and CONFIGURE until you don't see any "NOOOOOOO" in the logviewer (the one next to BAAAAAAD is fine).
  • Remote desktop to pclbtb05 (use the lhcbrich account). Browse to Documents/LaserPulser/programmerv2.20180116 and launch the exe Jar file. Press the Scan button (it should find GBTx 1). Press the Import button and select the golden file located in Documents/LaserPulser/. Read (state goes to PauseForConfig). Write. Read (state goes to Idle).

Troubleshooting

How to restart the GBT server

ps ax | grep Gbt

find the process ID corresponding to lab004 (sol40) and kill it. The process will restart automatically after 30 seconds. Restart ctrl managers from the main operations panel after the GBT server connection circle is green.

Column temperature TESTO sensor and/or humidity sensor not updating

Restart the OPC UA Client -num 9 of the elmb project on pclhcb33.

Failed to switch on either the LV or the HV

From lbrichecs01, send

sudo systemctl restart OpcUaWiener.service

or

sudo systemctl restart OpcUaCAEN.service

Failed to lock ALL the links on a tell40

On the main operations panel, press the FW Versions... button. Press Configure GBTs on the corresponding tell40.

Temperatures reading via GBT-SCA not updating

Check also the Control Manager -num 6 of the R2DCS1 project (on workspace 7) is running. If not, press the green traffic light.

Check the PDMDBs monitoring is running (from the ComLab panel, press the start monitoring button).

WinCC projects

Name Project folder on server Running on server UI on pcsyslab02 workspace Shortcut CommentSorted ascending
R2DCS1 storerich01 lbrichecs01 7 dcs DCS FSM
R2HV1 storerich01 lbrichecs01 8 hv HV FSM
IBCDB pclbrich02 pclbrich02 10 startDB RICH database
MultiMiniDAQ storerich01 pcsyslab02 6 fsm Run control FSM
RIECS1 storerich01 lbrichecs01 9 ecs safety FSM

The shortcuts to open the corresponding gedi are available under the local folder /home/lbrich/WinCC/shortcuts.

Power the frontend

Click on RICH_DCS. Take the partition by clicking on the lock. Issue a Switch_ON command. From the Power Supplies Overview in the top operations panel, you should see the LV channel switching on with voltage of 8.0 V and current around 8 A.

HV recipes

When sending an action to the HV partition, you can write the following recipes in the text field: INSTALLATION, CALIBRATION and PHYSICS. Three actions are available within each of them: Go_STANDBY1, Go_STANDBY2, Go_READY. Note that to allow threshold scans to be performed from higher voltages, the current limit within the corresponding recipes are set to the limit at 1 kV, and then corrected to proper values when the READY state is reached.

Recipe name vSet [V] iSet [uA] (H-type/R-type) rampUpSpeed [V/s] rampDownSpeed [V/s] Comment
INSTALLATION/Go_STANDBY1 700 1200/3750 20 40 To be used the first time a column is installed
INSTALLATION/Go_STANDBY2 850 1450/4600 5 40 To be used the first time a column is installed
INSTALLATION/Go_READY 1000 1750/5700 2 40 To be used the first time a column is installed
CALIBRATION/Go_STANDBY1 800 1620/5650 20 40 Threshold scans/SIN
CALIBRATION/Go_STANDBY2 850 1620/5650 20 40 Threshold scans/SIN
CALIBRATION/Go_READY 1050 1750/5750 2 40 HV training
PHYSICS/Go_STANDBY1 900 1620/5350 20 40 Threshold scans/SIN
PHYSICS/Go_STANDBY2 950 1620/5350 20 40 Threshold scans/SIN
PHYSICS/Go_READY 1000 1620/5350 20 40 Basic runs/Threshold scans/SIN

Power the MaPMTs in PHYSICS mode

Click on RICH_HV. Take the partition by clicking on the lock. Issue a Go_STANDBY1 command (Select the PHYSICS mode when asked for). Wait few minutes. Issue a Go_READY command. Open the power supplies overview to check everything is green (colour calculated depending on the expected current drawn by a baseboard in the dark, taking into account the tolerance of the voltage dividers).

Configure the frontend

Browse to DAQ->R2A_DAQ_FE and send a Configure command. The current of the LV for RICH2 columns goes up to 12.85 A when everything is configured properly.

Check the frontend is properly configured

DAQ->R2A_DAQ_FE->R2A_DAQ_FE_COL00. Enable the calibration pattern. Browse down to the corresponding TELL40s (tell40_10_0 for the upper half of the column, tell40_18_0 for the lower half of the column), move to the monitoring tab. Enable the loop monitoring and check if the calibration pattern (9F everywhere) is there.

How to find the latency

Ensure that the 840120 address of the TELL40s in the LBUS tab of the GBT client is set to 00001000. Browse to DAQ->R2A_DAQ_FE. Select action: DAC scan. Threshold 7. Start Step: 255. Enable the Step Run with 1 step and click START_RUN. This allows to switch on the DAC at the maximum level on all the frontend channels. On the TFC quick control panel, set the limit to 10000 (it allows to acquire around 3 orbits, 1 orbit being 3564). Enable the random trigger source with rate 40078 (the exact rate of the LHC) and Calib A with periodicity 1. Disable the TAE. Press Configure Links and enable at most 4 links per TELL40. Take a run. Data are written under the /data folder of lbrichpcie40-00. Open the raw files with daq40_frgreader and search for a F by using \F. Sign down the BXID where you have the signal (be careful to not confuse it with the EvtID). Take the difference deltaT between this number and 3087 (the bxid values writtein next to the Calib A text field in the TFC quick control panel). Convert the value of (3564-deltaT) in hex and set it for both TELL40s in the 840120 address of the TELL40s in the LBUS tab of the GBT client (the format i 00001XYZ where XYZ is the hex number).

How to acquire data with the laser

Turn the laser on (turn the driver key). Exclude the DATAFLOW partition, check that the Enable in the Step Run box is not ticked and click Apply. Click on TFC quick control in the TFC box, a panel will open where you have to make sure that the box Limited is not ticked and the box Calib A is ticked. Click Apply and close the TFC panel. Click CONFIGURE from the top, followed by a a START_RUN and a GO action. You will see the number of triggers increasing in the Triggers from TFC box. Check the laser driver box where you should be able to now see the red light corresponding to the emission ON. Always make sure that in the top of the RICH partition the state of RICH_HV is READY: if the Subdetector Type was not set correctly this could cause a trip of the HV and consequently the RICH_HV state to go into ERROR.
If the laser is not emitting (i.e. the display on the Kintex evaluation board used to control the laser is empty), use the following procedure:
- log in the Windows7 pclbtb05;
- browse to Documents/LaserPulser/programmerv2.20180116 and launch the exe Jar file;
- press the Scan button (it should find GBTx 1);
- press the Import button and select the golden file located in Documents/LaserPulser/;
- read (state goes to PauseForConfig), write, read (state goes to Idle);
- check that the display is showing oX-00h (X=0,1,2,3);
- open the GBT client->select GBT->!I2C (lbminidaq2-06,GBTid=4,SCA=0,BUS=0,Address=33,Type=GBTx,frequency=100k,SLC=Open Drain,Size=4,write 00000003)
-check that o3 register is 03 on the display, i.e. o3-03h and repeat the procedure at the beginning of this subsection.

BXID offset and Data Links alignment

To change the BXID offset: open the GBT client (Communication: Local, LBUS tab). Select the server corresponding to the TELL40 partition, Address: 840120. Type: 32 bits. Size: 1. Press Read. Copy paste the the Data Out in the Data In field. Change it accordingly (the + shift the readout window to left, the - shift the readout window to the right. Data stay where they are).
Look at hslotpixel histogram of a run with TAE enabled (you need to process it without OR, ./runOffline.sh is fine). Check that the majority of events are in the same bin on the y axis. If not, you need to identify the correct link (each bunch on the x axis corresponds to a data link) that is not aligned with the others.
On the GBT client, on the LBUS tab (size 1). For each address, each nibble corresponds to a link (increasing from right to left).
Address 840100: first six digits to the right are TFC links (from 0 to 5), the following 2 digits are data links 6,7.
Address 840104: data links from 8 to 15 (from right to the left).
Address 840108: data links from 16 to 23.

To move the bunches higher in the hslotpixel, you need to reduce the digit in the corresponding nibble.

How to take a simple run

Set the CLARO parameters from the R2A_DAQ_FE_COL00 panel (hysteresis=1, test pulse disable=1, channel enable=1). Click on TFC quick control. Enabled Limited and set the number of events. Enable Calib A with periodicity 1 (1 pulse per orbit) or Fast Calib A with periodicity 4000 divided by the number of pulses per orbit you want (e.g. 400 if you want a pulse rate of 100 kHz). Enable or disable the TAE. Historically we used a TAE half window of 11, corresponding to 23 consecutive time slots around the BXID where the signal should sit when the right latency has been found. If you want to deliver N pulses to the column, you need to set the number of events to N x ((TAE half window) x 2 +1). Check the Step Run are not enabled at the bottom of the main operations panel.

How to take a scan

DAQ->R2A_DAQ_FE. Select threshold or DAC scan and set the corresponding parameters. Click apply. On the top operations panel, enable the Step Run. Set the appropriate parameters.

How to process data

Move to the $offline folder in lbrichpcie40-00.
Process a single run: use ./run.sh , where is the run number you have in the top operations panel, is (TAE half window x 2) +1 and is whatever character you want if you want to show the offline display.
Process a scan: use ./runScan.sh , where is the run number you have in the top operations panel, is (TAE half window x 2) +1, is the start value you set under DAQ->R2A_DAQ_FE, depends on the number of steps, and is whatever character you want if you want to show the offline display.

Change SOL40 firmware

Comment/uncomment line in /etc/daq40.cfg. pcie40_systemd -r, pcie40_systemd -l.

lab004: Config PLLs, Restart ctrl managers, on FW versions: config SFP 0, config SFP1, config GBTs

lab010 and lab018: config SFP 0. If Rx and Tx TFC 0 are not green, select RESYNCHRONIZE, Apply commands and Clear commands.

Obsolete guidelines

Load the pcie40 cards firmwares with pcie40_program -n Firmware/<nameOfTheFirmware.sof>. is lab004 (SOL40), lab010 (TELL40) and lab018 (TELL40). is written on /etc/daq40.cfg. Load the firmware following the order lab004, lab010, lab018. (If the loading of the firmwares fails: sudo poweroff lbrichpcie40-00. You can switch on again typing lbrichpcie40-00_ipmi.dyndns.cern.ch on the web. Help: p40acceptance.)

-- GiovanniCavallero - 2019-08-13

Edit | Attach | Watch | Print version | History: r19 < r18 < r17 < r16 < r15 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r19 - 2020-07-29 - GiovanniCavallero
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LHCb All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback