Preliminary setup after power cycle

Firmware is persistent in the FLX-712, so you don't need to reload it after a power cut. The RunControl takes care of the initialization and link configuration of the FELIX cards when the WIBs are configured.

• #TBA -> add affinity setup scripts automatically run either on systemd or supervisord one-shot.

Initialisation

In case one needs to initialize and enable links manually, it's done by communicating with the FLX RPC servers on every FELIX host.

ssh np04daq@np04-srv-024 
cd /nfs/sw/felix/FLXRPC/
python flxReset.py #TBA

One can do it manually on the server itself:

ssh np04daq@np04-srv-0xx # corresponding FELIX server
cd /nfs/sw/felix/new-software/sw-master-uptodate/
source setup.sh
cd x86_64-centos7-gcc62-opt
./init.sh 0 # init first card in server
./linkconfig.sh 0 5 5 # for the first card in server, enable 5+5 links on the 2 logical units

Troubleshooting

* In general, the Run Control should set the system in a working state. In the case of an error, as reported by flxHealth, first try to reset and reconfigure the corresponding nodes in the Run Control (or simply reset and reconfigure everything from the top) *

Use flxHealth to check status first

cmem errors when launching applications

The driver did not manage to allocate the block of contiguous memory. Check if the driver is in an ERROR state on flxHealth. If so, try to restart the driver on the server:

ssh np04daq@np04-srv-024 
cd /nfs/sw/felix/FLXRPC/
python toggleDriver.py xx # corresponding FELIX server as reported by flxHealth

Debugging

Check OnHost BR logs for sizeErrors... #TBA

Developer Operations

Flashing the card

# setup environment
cd /nfs/sw/felix/new-software/sw-master-uptodate/
source setup.sh
cd x86_64-centos7-gcc62-opt/ftools
# flash all partitions:
./fflash -f 0 /nfs/sw/felix/firmware/MCS-FILE-TO-FLASH.mcs prog
./fflash -f 1 /nfs/sw/felix/firmware/MCS-FILE-TO-FLASH.mcs prog
./fflash -f 2 /nfs/sw/felix/firmware/MCS-FILE-TO-FLASH.mcs prog
./fflash -f 3 /nfs/sw/felix/firmware/MCS-FILE-TO-FLASH.mcs prog

QAT

The OnHost BR is running without compression. (Add FW_counters and QAT driver supervisord process checks.)

-- GiovannaLehmannMiotto - 2018-02-13 -- RolandSipos - 2018-07-13