This twiki page contains instructions on how to run specific tasks or tests in the CERN Felix TestBed.

Relevant locations

Webpages /services

Felix dashboard/ : Status of all FELIX application run by Supervisord

Felix multivisor : Control applications run by Supervisor on different hosts

Felix User manual : always refer to the manual

Folders

Software releases

releases	/det/tdaq/felix
installed releases	/sw/atlas/tdaq/felix
pointer to last release	/det/tdaq/felix-opearation/scripts/setup_FELIX.sh

Operation directories in /det/tdaq/felix-operation

Supervisor (SV) config files	/det/tdaq/felix-operation/supervisord
Host config file for SV file generation	/det/tdaq/felix-operation/hostconfig
E-link configuration files	/det/tdaq/felix-operation/config
Bus directory	/det/tdaq/felix-operation/bus
Scripts directory	/det/tdaq/felix-operation/scripts
Testbed utilities	/det/tdaq/felix-operation/testbed-utilities

Hardware resources

Vivado hardware server

The Vivado hardware server (version 2019.1) runs on pc-tbed-felix-03

Vivado lab can be started in TBED using

source /afs/cern.ch/user/f/flx/public/xilinx/Vivado_Lab/latest/settings64.sh
/afs/cern.ch/work/f/flx/public/xilinx/Vivado_Lab/latest/bin/vivado_lab

All programmers visible in the Felix Testbed webpage are reachable from this hardware server.

GBT-SCAs (L1DDC & VLDB)

In testbed there are 5 GBT-SCA chips. Four are hosted on NSW L1DDC boards that do not need configuration. One is hosted on a VLDB board.

These chips are connected to e-links 0,1,2,3,4 of device 2 of pc-tbed-felix-07.

Power supply for GBT-SCAs

The L1DDC and VLDB boards are powered by a power supply that can be controlled remotely from pc-tbed-felix-03 as follows

*set up a FELIX release to have Python3*
cd /det/tdaq/felix-operation/testbed-utilities/PowerSupply
(python_env) [cgottard@pc-tbed-felix-03 PowerSupply]$ python RemoteControl.py --help
usage: RemoteControl.py [-h] [--state] [--turn-on] [--turn-off] [--read-voltage]
                        [--read-current] [--set-voltage VOLTS]

optional arguments:
  -h, --help           show this help message and exit
  --state              Display state
  --turn-on            Turn on the power supply
  --turn-off           Turn off the power supply
  --read-voltage       Read voltage
  --read-current       Read current
  --set-voltage VOLTS  Set voltage

VLDB board configuration

To use the GBT-SCA on the VLDB it is necessary to configure the GBTX (all traffic from/to GBT-SCA goes through a GBTX) as follows

cd /det/tdaq/felix-operation/testbed-utilities/PowerSupply
java -jar programmerv2.20180724.jar

If the dongle is not found by the programmer probably a user with permissions needs to rerun udevadm control --reload-rules && udevadm trigger

In the programmer:

1. click on "Read GBTX"
2. click on "Import i..." on top left
3. select all types of file in the pop-up window so that you can choose "GBTX-all2bit-plusEC-mod.txt"
4. click "Write GBTX". If Read is press again the state should be Idle.

After this procedure the link should be aligned, 3 red LEDs should turn on on VLDB and GBT-SCA should be reachable on the EC link.

Reference: gbtxprogrammer git page

TTC system

Testbed is equipped with a TTCVI and an ALTI. TTCVI is connected to pc-tbed-felix-07 and pc-tbed-felix-04. ALTI is connected to pc-tbed-felix-00.

Both TTCVI and ALTI are controlled by sbc-tbed-717.

Control TTCVI + external RaspberryPi trigger

Open the console from sbc-tbed-717

cd /det/tdaq/felix-operation/testbed-utilities/TTCVI
source setup_release.sh
cd ttcvi
.ttcvi

The console allows to issue ECRs, BCRs, a few L1As, random L1As at given rates.

To let the RaspberryPi trigger TTCVI choose "Cosmic" on the main menu.

Control ALTI

ALTI con be controlled by setting up the

Test recipes

Integration with DCS

Run tests with ScaSoftware

ScaSoftware is the part of OPCUA that communicates with GBT-SCAs. Standalone applications included in ScaSoftware are called Demonstrators.

Building ScaSoftware

ScaSoftware is build using a proper FELIX release, not using the x86_64 folder of felix-distribution or software.

1. Get ScaSoftware cloning the repo from https://gitlab.cern.ch/atlas-dcs-common-software/ScaSoftware
2. Check the FELIX release version in setup_paths.sh and change it if needed. Then source the script (PS: if the script believes we are in P1 hack it, we are not in P1).
3. Run build_standalone.sh (at need disable the legacy netio setting in the script -DHAVE_NETIO=0).
4. Find demonstrators in build/Demonstrators
5. To setup ScaSoftware in a new session just source setup_paths.sh

Running a demonstrator (here "repetitive torture")

The following environment variables need to be set to configure FelixClient

export FELIX_CLIENT_bus_group_name="FELIX"
export FELIX_CLIENT_bus_dir="<path to bus in use>"
export FELIX_CLIENT_log_level="error"
export FELIX_CLIENT_local_ip_or_interface="priv0"

The following variables can also be set, for expert use only:

export FELIX_CLIENT_netio_pages="128"
export FELIX_CLIENT_netio_pagesize="1024"
export FELIX_CLIENT_timeout="1000"
export FELIX_CLIENT_verbose_bus="False"

To run a demonstrator both felix-tohost and felix-toflx need to be running. A demonstrator is run passing elinks identifiers (fid).

Below an example run on pc-tbed-felix-07 with SCAs connected to device 2 (device 0 of card 1).

felix-tohost -d 2 --free-cmem --iface priv0 --bus-dir ~/bus --dcs-unbuffered
felix-toflx -d 2 --free-cmem --bus-dir ~/bus --iface priv0 --unbuffered --cdma

./repetitive_torture --addresses netio-next://fid/0x10000020003f8000/0x10000020003f0000 netio-next://fid/0x10000020007f8000/0x10000020007f0000 netio-next://fid/0x1000002000bf8000/0x1000002000bf0000 netio-next://fid/0x1000002000ff8000/0x1000002000ff0000 netio-next://fid/0x10000020013f8000/0x10000020013f0000

Run tests with OPCUA

Build OPCUA with custom FELIX build

To obtain a build of OPCUA that is compiled against a FELIX build

1. prepare a FELIX build using the build-tar.sh script in felix-distribution
2. place the build in a public folder
3. edit .gitlab-ci.yml in the felix-dev branch of OPCUA
4. let the CI run and get the artifact

Run OPCUA

Integration with SW ROD

GBT-mode / FELIX partition

Full-chain tests with external emulators and triggers

Supervisord config files

GBT-mode

FULL-mode

TTC: TTCVI+RaspberryPi, ALTI

-- CarloAlbertoGottardo - 2022-06-30