FPixPhase1UpgradeTestStandAtFNAL

Software Installation eLog

Installation to the first computer.

Install VME control bus driver, the ROOT, pixel online software etc...

• http://dbweb3.fnal.gov:8080/ECL/cms_fpix/E/show?e=460
• http://dbweb3.fnal.gov:8080/ECL/cms_fpix/E/show?e=462
• http://dbweb3.fnal.gov:8080/ECL/cms_fpix/E/show?e=473
• http://dbweb3.fnal.gov:8080/ECL/cms_fpix/E/show?e=474
• http://dbweb3.fnal.gov:8080/ECL/cms_fpix/E/show?e=479
• http://dbweb3.fnal.gov:8080/ECL/cms_fpix/E/show?e=482
• http://dbweb3.fnal.gov:8080/ECL/cms_fpix/E/show?e=501 [TTCci control]
• http://dbweb3.fnal.gov:8080/ECL/cms_fpix/E/show?e=504 [FED software]
• http://dbweb3.fnal.gov:8080/ECL/cms_fpix/E/show?e=512 [FED software]
• http://dbweb3.fnal.gov:8080/ECL/cms_fpix/E/show?e=522 [FED firmware installation]

Installing POS from scratch follow

• https://twiki.cern.ch/twiki/bin/viewauth/CMS/PixelOnlineSoftwareInstallation?rev=125
• http://dbweb3.fnal.gov:8080/ECL/cms_fpix/E/show?e=659
• http://dbweb3.fnal.gov:8080/ECL/cms_fpix/E/show?e=660

If mFEC firmware installation is needed :

• http://dbweb3.fnal.gov:8080/ECL/cms_fpix/E/show?e=498
• http://dbweb3.fnal.gov:8080/ECL/cms_fpix/E/show?e=500
• Software was left here : ~/ECAL_DAQ

If firmware update for the VMC crate controller (PCI and VME controller) is needed :

• http://dbweb3.fnal.gov:8080/ECL/cms_fpix/E/show?e=507

Installation to cms-sidet003

• cms-sidet003 is SLF which needs extra works, as mentioned in the eLogs below.
• Installation of XDAQ and caen driver (works with ROOT privilege).
  • http://dbweb4.fnal.gov:8080/ECL/cms_fpix/E/show?e=671
• Installation of POS : user account.
  • http://dbweb4.fnal.gov:8080/ECL/cms_fpix/E/show?e=673
  • setting and running POS, and control with firefox :
  • http://dbweb4.fnal.gov:8080/ECL/cms_fpix/E/show?e=676

Computing Environment

• xdaq version 11 (see the install log above, or see /etc/yum.repos.d/xdaq.repo )

Prepare POS in your work directory (FNAL-ized version)

cd
mkdir TriDAS
cd TriDAS

git clone https://github.com/hsatoshi/POSOverlay.git pixel
cd pixel
git checkout fnal_basedon_release2p2

# Open following files, and find "JordanPOS". Modify the line for your environment.
# POSRelease/setenv.sh
# BPixelTools/setup.bsh
# PixelAnalysisTools/test/configuration/PixelAliveAnalysis.xml

cd ~/TriDAS
ln -s pixel/POSRelease/setenv.sh
# edit the top lines in setenv.sh to point to your xdaq and root 
# install paths, and the absolute path to TriDAS in BUILD_HOME
# and at the bottom, uncomment the little block for sourcing root env
.  setenv.sh   # yes do this before actually making anything

cd ~/TriDAS
scp lxplus.cern.ch:~tucker/work/public/pixelpilot/backups/DiagSystem.save.tgz .
tar zxf DiagSystem.save.tgz
export BUILD_SAVE=$BUILD_HOME
unset BUILD_HOME
cd DiagSystem
./configure --enable-compile_for_pixels --with-gwtapplets-path=pixel
make all
export BUILD_HOME=$BUILD_SAVE

cd ~/TriDAS
scp lxplus.cern.ch:~tucker/work/public/pixelpilot/backups/FecSoftwareV3_0-slc6.tgz . 
tar zxf FecSoftwareV3_0-slc6.tgz
cd FecSoftwareV3_0
cp config/FecHeader.linux .
# edit in FecHeader.linux to comment out XDAQ_OS and XDAQ_PLATFORM, and set PCIFEC_COMPILE=no
make Generic
make APIConsoleDebugger
make install

cd ~/TriDAS/pixel
make install  # yes do this first
make Set=pixel

cd XDAQConfiguration
cd PixelRun
mkdir Logs

How to power up the test stand from scratch

(0) Current hardware configuration

The talk "Status of the system test stand at Fermilab", page 3-5. https://indico.cern.ch/event/368149/

(1)Power

Turn ON the PC.

Turn on VEM crate.

Turn on Agilent DC power supply on the table :

• push "Power"
• push "Recal", and again "Recal" -- it says "done".
  • The initial set up is 2.5V for 1ch(Vdig) and 1.9 for 2ch (Vana)
• push "Output on/off" to turn on/off.
  • display shows the voltage and current for 1ch(or 2ch).
  • "Output 1" and "Output 2" buttons show the status of 1ch/2ch.

(2)Initializing TTCci

First, run the xdaq :

cd ~/TriDAS/
source setup_fnalTestBench.sh
cd ~/ttcci_standalone/
sh run_ttcci.csh

Then, open the Firefox and goto a bookmark "XDAQ online" or http://dhcp-131-225-179-132.dhcp.fnal.gov:1974/urn:xdaq-application:service=hyperdaq . Proceed to "ttc TTCCiControl". Click "configure" and then "enable".

If you need to change trigger rate, go to "Main Config",change "Freq" box and then click the "Apply" button.

(3) Initializing CCU and Portcard

cd ~/TriDAS/
source setup_fnalTestBench.sh
cd ~/TriDAS/pixel/BPixelTools/ccu
sh reset_fec.sh      ###  You may need to type "Enter" three times.
sh init_electronics.sh

The last script "init_electronics.sh" setup following items :

• POH
• TPLL reset
• Delay 25 setting
• Gatekeeper
• and show Delay25 setting.

bin/ccu -vmecaenpci -port 2002 <<EOF
i2c -6P 0x10 0x30 0x63
i2c -6P 0x10 0x31 0x63
i2c -6P 0x10 0x32 0x43
i2c -6P 0x10 0x33 0x43
i2c -6P 0x10 0x34 0x43
i2c -6P 0x10 0x20 0x08
i2c -6P 0x10 0x20 0x00
i2c -6P 0x10 0x35 0xf0                                                         
i2c -6P 0x10 0x10 0x22
i2c -6P 0x10 0x11 0x22
i2c -6P 0x10 0x12 0x22
i2c -6P 0x10 0x13 0x22
i2c -6P 0x10 0x14 0x22
i2c -6P 0x10 0x15 0x22
i2c -6P 0x10 0x16 0x22
i2c -6P 0x10 0x18 0x22
delay25 read
exit
EOF

(4) setup Module (TBM and ROCs)

To do this, Step(3) (above) should be done in advance because we need the delay25 to be opened to send command to the modules.

cd ~/TriDAS/
source setup_fnalTestBench.sh
cd ~/TriDAS/pixel/BPixelTools/pxfec
bin/pxfec -port 2000 -$INTERFACE -init  data/satoshi_FL_922_TryJan20

The setup values are written in the file data/satoshi_FL_922_TryJan20.

Now, all the electronics are ready.

Furthermore, if you set some trigger rate at step-(2). the L1 triggers are sent to the modules and data are sent back from the modules. You can validate the L1 trigger is firing by lemo output(bottom one) of the TTCci. You can see signal from POH by probing a pair of the "D_A1" holes on the Piggy board.

(You can skip part below when you are not interested in changing TBM parameters.) In order to access the register on the TBM, use following numbers with function "setA < value >" or setB.

<register> = 
setA 0 XXX : for 0xE0
setA 1 XXX : for 0xE2
setA 3 XXX : for 0xE4
setA 3 XXX : for 0xE6
setA 4 XXX : for 0xE8
setA 5 XXX : for 0xEa
setA 6 XXX : for 0xEc
setA 7 XXX : for 0xEe
setB 7 XXX : for 0xFe

This correspondence comes from the software implementation : given value is shifted by 1 ("register<< 1") and used for # in 0xF#/0xE#.

Below is an example of configuration file.

sys
echo fec
fec  11
echo mfec
mfec 11 8 -6P

cn -6PL12
cn hello
module 15

tbm
reset tbm
reset roc
fullspeed
mode cal
tbmadelay 219 
tbmbdelay 219


roc 0:15
Vdig         6
Vana        80
Vsh         30
Vcomp       12
VwllPr     150
VwllSh     150
VhldDel    250
Vtrim        0 
VthrComp    51
VIBias_Bus  30
PHOffset   170
Vcomp_ADC   50
PHScale    130
VIColOr     10
Vcal       200
CalDel     175
CtrlReg      0
WBC         93
ReadBack   0


mask
arm      6 6
arm      3 3
arm      9 9
arm      11 20


tbm
setA  0 0x80
setB  0 0x80
setA  1 0xc0
setB  1 0xc0

setA  3 0xf0
setB  3 0xf0
setA  4 0x10
setB  4 0x10
setA  5 0x52
setB  5 0x52
setA  6 0
setB  6 0
setA  7 0xC0

(5) FED : Initialize and readout hits from modules

cd ~/TriDAS/
source setup_fnalTestBench.sh
cd ~/pxlpilotfed
./pxlpilotfed

The last commands wait for your input. Choose (1)-> (2) -> (3) -> (4) (type '1' and so on).

Now (a) shows hits

Ongoing work : Online Pixel Software Installation

Setup script

First, the setup script was modified according to the FNAL computer environment : In the file TriDAS/setenv.sh, following items were modified

export ROOTSYS=/home/fpix/PixelOnlineSoftware/TriDAS/../ROOT/root_v5.34.23/
export BUILD_HOME=/home/fpix/PixelOnlineSoftware/TriDAS

Modification if needed.

For FecSoftware (used by TKFECSupervisor), change

  /nfshome0/pixelpilot/build/TriDAS/FecSoftwareV3_0/generic/src/common/FecVmeRingDevice.cc:1907:
   busAdapter = new
HAL::CAENLinuxBusAdapter(HAL::CAENLinuxBusAdapter::V2718,0,0,HAL::CAENLinuxBusAdapter::A3818)
;

to just

  busAdapter = new HAL::CAENLinuxBusAdapter(HAL::CAENLinuxBusAdapter::V2718);

Same change for FE[CD]Supervisors at these lines:

  /nfshome0/pixelpilot/build/TriDAS/pixel/PixelFECSupervisor/src/common/PixelFECSupervisor.cc:2537:
   busAdapter_ = new
HAL::CAENLinuxBusAdapter(HAL::CAENLinuxBusAdapter::V2718,0,0,HAL::CAENLinuxBusAdapter::A3818)
;

  /nfshome0/pixelpilot/build/TriDAS/pixel/PixelFEDSupervisor/src/common/PixelFEDSupervisor.cc:2179:
 busAdapter_ = new
HAL::CAENLinuxBusAdapter(HAL::CAENLinuxBusAdapter::V2718,0,0,HAL::CAENLinuxBusAdapter::A3818)
;

(I am not sure if this is needed.) Edit /home/fpix/PixelOnlineSoftware/TriDAS/TTCSoftware/config/XDAQConfig.xml (1) Somehow 'localhost' can not be used for XXXXX. Put the result of 'hostname' command here. (2) Modify based on which card I use -- PCI or PCIe card? (In the case of PCIe, use CAENPCIe) CAENPCI (3) Correct the file path (full path) if needed. [file=/home/fpix/ttcci_standalone/TTCciConfiguration.txt] and /home/fpix/PixelOnlineSoftware/TriDAS/TTCSoftware/xdaq/libLTCControl.so

compie fecSoftwareV3_0

make Generic
#-- to do this, comment-out '-Werror' from generic/Makefile
#-- added #include <vector> to the file :
# /home/fpix/PixelOnlineSoftware/TriDAS/FecSoftwareV3_0//generic/include/XMLCommonFec.h

make APIConsoleDebugger
make install

compile diagSystem

This time is not the first time to compile DiagSytstem and I did not needed followings, but in general, needed.

# yum install e2fsprogs-devel
# yum install libuuid-devel

#-- Do compile --
./configure
# 'make' gives instruction.
make clean
make all

"pixel" software

In the directory, do followings.

make install # this makes symbolic links.
make Set=pixel # this performs compile.

# if I had
# /usr/bin/ld: cannot find -lAPIFecVme
# FecSoftwareV3_0 is not installed, or check if LD_LIBRARY_PATH has the FecSoftwareV3_0 in it.

# if I had
# /usr/bin/ld: cannot find -lTkDiagBagWizard
# DiagSystem is not installed.

Run XDAQ :

Modify pixel/XDAQConfiguration/ConfigurationNoRU.xml :

                      <BusAdapter xsi:type="xsd:string">CAENPCI</BusAdapter>
                      <Configuration xsi:type="xsd:string">[file=/home/fpix/PixelOnlineSoftware/pxlpilotfed/ttcci_standalone/TTCciConfiguration.txt]</Configuration>

Run :

cd PixelOnlineSoftware/TriDAS/pixel/PixelRun
sh run_noRU.sh

and accessing to http://dhcp-131-225-179-132.dhcp.fnal.gov:1973/urn:xdaq-application:service=hyperdaq with firefox gives DAQ control.

New setup

These are all versions on the fnaltest account.

debug version (currently not running correctly):

cd TriDAS/pixel/PixelRun
. run_noRU.sh

TIF version with adjusted configs (for 4ch PC): still investigations ongoing

cd odyseiTriDAS/pixel/PixelRun
. FNAL_run_noRU.sh

Modified TIF versions to accept 7ch PC: still work ongoing

cd PC7chTriDAS/pixel/PixelRun
. FNAL_run_noRU.sh

To start data taking, need to modify configuration parameter ( /home/fpix/PixelOnlineSoftware/TriDAS/pixel/config ). See "quick setup" below for info.

Run decode :

configuration file : TriDAS/pixel/PixelAnalysisTools/test/configuration/PixelAliveAnalysis.xml . At least, change following :

<!DOCTYPE PixelAnalysis SYSTEM "/home/fpix/PixelOnlineSoftware/TriDAS/pixel/PixelAnalysisTools/dtd/PixelAnalysis.dtd">

then, run

./PixelAnalysisTools/test/bin/linux/x86_64_slc6/PixelAnalysis.exe  PixelAlive ${run number}

The output of the program tells output file path.

[PixelHistoManager::saveHistos()]   Saving file, please wait...
[PixelHistoManager::saveHistos()]   Saved results in /home/fpix/PixelOnlineSoftware/TriDAS/pixel/PixelRun/Runs/Run_0/Run_18/PixelAlive_Fed_40_Run_18.root

Open it with ROOT.

Quick setup

Files to be looked into, for operating modules.

• "aliases.txt"
  • check the key number of measurement program (for example, delay25 5x5 -> 44 ).
• configurations.txt
  • check the key numbers of each hardware configuration for the key you found in the aliases.txt above.
  • the number tells you which directory you need to look for each configuration.
  • for example, if you find "dad 1 " , it means the DAC configuration file is in dad/1/
• "detconfig"
  • the Rocs taking part in the run.
• "nametlanslation"
  • mapping from "name" to FEC/mFEC/,,,FED channel etc.
  • hub-ID of module is one key point.
  • FED channel:
    • 
    • PIGGY UP :

Fiber channel	FED channels
1	x
2	x
3	x
4	x
5	x
6	x
7	1,2
8	3,4
9	5,6
10	7,8
11	9,10
12	11,12

• • • PIGGY DOWN :

Fiber channel	FED channels
1	13,14
2	15,16
3	17,18
4	19,20
5	21,22
6	23,24
7	25,26
8	27,28
9	29,30
10	31,32
11	33,34
12	35,36

• If you connect new module to existing port card (=the port card configuration file already exists.)
  • prepare following configurations
  • when you add entries to existing files, do not forget the last "newline".
    • add configuration file to "tbm"
    • modify configration file in "portcardmap"
    • do nothing for "portcard" (because we already have this configuration file).
    • add entry to "lowvoltagemap"
    • add new file to "mask"
    • add new file to "dad"
    • add new file to "trim"
    • add entry to "maxvsf".

• If your module has a new name also check:
  • the corresponding "calib" files

Temporal procedure

Install pixel software from git-hub, FNAL branch.

   https://github.com/hsatoshi/POSOverlay