Some knowledges in the front

FELIX port allocation summary:

To Host (recv): -r set the basis. Each device will have an offset, which is unconfigurable (default 12340)

From Host: this will always be counting from the basis value, for each opened device. (default 12350)

ELink: each device has an offset of 2048.

Example under default port:

scenario 1) I open all the device 0 and 1. 12340/12350 is device 0, 12341/12351 is device 1

scenario 2) I open device 1 only. 12341/12350 is device 1

FELIX simple manual

Some notes before you run

These must be running on felix machine. (um-felix1.cern.ch, um-felix2.cern.ch)

Necessary steps for each terminal

1. Login to one of our felix machine: um-felix1.cern.ch, um-felix2.cern.ch.
2. setup environment(if you want to use this version felixcore 4.0.4, OpcServer 1.3.2), otherwise, the default installed rpm will be the latest.

   source /home/setup/setup.sh

Step that you might need to do from time to time

1. Install felix firmware(if you haven't done it):

   vivado (you might need to use another account to open it)

   Auto connect

   then, select hardware manager. Choose the xcku115_0 FPGA. use the FLX_712 binary file under /afs/cern.ch/user/x/xuwa/public

   restart the felix machine.

Steps after you restart the computer

1. Get the felix driver started:

   sudo /etc/init.d/drivers_flx start

2. GBT phase alignment:
   

   flx-init (flx-init -T 2 if using flx sw older than version 04-00-05)

Steps after you power on FEs

1. Correctly set up "elinkconfig". Need TH_FO, FH_FO to be None for all, and Clock to be coming from TTC. Rate parameters need to be "Generate/Upload" to be effective.

   elinkconfig

   # then "open" either of those .elc file as a starting point. The files below only have link0, and then you can replicate to any other links. Link numbering on the patch panel is determined by how the patch panel is connected in itself.

   # /afs/cern.ch/work/r/rowang/public/FELIX/ElinkConfig/config_felixcore.elc

   # /afs/cern.ch/user/r/rowang/public/FELIX/ElinkConfig/config_felixcore.elc

2. Correct gbtx setting. Check out this script as an example for uploading gbtx config! Please copy it and modify accordingly. Manual inside
   

   /afs/cern.ch/work/r/rowang/public/FELIX/GBTXConfig/config_gbtx.py

3. Check that the ToHostFanOut and FromHostFanOut (TH_FO/FH_FO) are none(using real data). This can be done with the following command for both flx-card:
   

   femu -n -d 0
   femu -n -d 1

   
   This is included in the config_gbtx script under Rongkun's directory.

To run!

1. open felixcore (hard core): (need the -elinks option for swROD to subscribe onto)
   

   felixcore --data-interface=eno1 --elinks=0-500

2. opcserver. you can use the FELIX GUI to generate one, remember it will have a hostname automatically added behind your original name.
   OpcServer can only be properly stopped by `kill -9` or `Ctrl+C` or `Ctrl+\`
   

   /home/ScaOpcUa_1.3.2_N/bin/OpcUaScaServer <Your Opc Server File>.xml

Error?

Set up FELIX

1. Q: During `flx-init -T 2` or `flx-init`, it's prompting me LO register 0x02, and it never ends, what is happening?
   A: It's very likely that the TTCvx module is not pushed in fully in the crate! Or the LEMO cable/connector is bad..

Start up FELIX

1. Q: felixcore command is not found?
   A: You need to setup the environment! See Necessary steps, source ...
2. Q: The felixcore gives me a lot of error and doesn't get stabilized.
   A: Check if you have a correctly configured gbtx and elink config. Sometimes need to check MiniSAS connection. See also the point below.
3. Q: Sometimes felixcore can't stabilize, sometimes the OpcServer gives me endless error of "broken frame".
   A: Congrats ::)) This is by far the most tricky error. In most cases it's unstable MiniSAS connector issue. Could also be overheating. We don't know, honestly. Try everything from reloading firmware, restart, set up, and pray...
4. Q: The OpcServer got a "Deferred blablabla"
   A: It usually comes up the first time you have a good environment for felixcore! Congratulation, finally!
5. Q: The OpcServer /felixcore closes immediately.
   A: Check if there's other processes occupying the resources. Usually check "ps aux | grep "Opc\|felixcore"". Kill with -9 for the OpcUaScaServer if it's there.
6. Q: The OpcServer gives me endless error of "link not found".
   A: Check if you have a correctly configured gbtx and elink config. Sometimes need to check MiniSAS connection.

Configuration

1. Q: After configuring, it kept promping a lot of the following:
   

   Can't write I2c: bad-status: BadOutOfService (0x808d0000)

   Can't read GPIO: OPC-UA read: variable status is not goodBadResourceUnavailable (0x80040000)

   A: Check if OPC server is acting weirdly? If so, please restart felixcore and opc server. If not consider Q2
2. Q: After configuring I get a limited number of

   Can't write I2c: bad-status: BadCommunicationError (0x80050000)

   A: Check if the message appears after trying to configure certain chip. If it's under rocCoreDigital, first you can try to configure gbtx phase again. And then if the problem is still there, you might need to tune parameters like roc internal pll phase.

3. Q: After configuring, it kept promping a lot of the following(in the configure front end):

   0x80ae0000 explanation=' BadConnectionClosed (0x80ae0000)']

   A: Check if you have a correct NodeID existing in the OpcServer file. This error comes from OpcClient cannot find corresponding node! I suggest you to use FELIXGui to generate the OpcServer file!

4. Q: When I use partition to config, it hang there and eventually time out.
   A: Check the log file for record. If there's "timeout", check OpcSerserver status.
5. Q: When using FELIXGUI to do configuration, it hang there and time out eventually, what happened?
   A: We currently use ssh non-interactive shell to login to sbc machine to do all the configuration. This is because after configuration we also need to send sbc signals. In case of hanging, usually the connection to the server is waiting for interactive prompt. Go to here for more.

Data Taking

1. Q: I see no data coming out..
   A: Well, it could be everything! The felixcore might be corrupted, config not right... Check OpcServer to see if the link is still there. It's better to start over.