Verification of v5.4.0 firmware and software release
Details
Date |
19/05/2021 |
Location |
University of Bristol lab |
People |
Sudan |
Test objective
Verify the operation of the software and firmware following the fixing of several bugfixes, and the addition of a new output port in endpoint firmware block.
The bugs fixed are documented below:
Test setup
Hardware
An AIDA 2020 TLU was configured to acts as master using the overlord fw design. An Enclustra AX3+PM3 with a timing FMC (pc053a) acted as an endpoint. The output from the TLU was fed into an 1:8 passive optical splitter via single mode optical fibre. The timing FMC was connected to one of the splitter outputs using a single mode optical fibre.
TLU UID: 0x5410ecbba408
FMC UID: 0x49162b675e3
Firmware
The bitfile used for the TLU was:
https://pdts-fw.web.cern.ch/pdts-fw/mr/28/latest/overlord_tlu_mr28_sha-809ffc54_runner-slu9p8x4-project-19909-concurrent-17_210506_2040.tgz
The endpoint bitfile was
https://pdts-fw.web.cern.ch/pdts-fw/mr/28/latest/endpoint_pc053d_fmc_mr28_sha-809ffc54_runner-6bb6b053-project-19909-concurrent-1_210506_2205.tgz
Software
The software version used for this test was: test/v5.4.0_1105
Device configuration
The TLU was configured using the following commands:
pdtbutler io PROD_MASTER reset --force-pll-cfg DUNE_TLU-DUNTLU09-Registers_62.5_mhz.txt
pdtbutler mst PROD_MASTER synctime
pdtbutler mst PROD_MASTER part 0 configure
The endpoint was configured using the commands below:
pdtbutler io EPT_0 reset --force-pll-cfg Si5394-053endptr_62-5MHz_4kHz-Registers.txt
pdtbutler ept EPT_0 0 enable
pdtbutler ept EPT_0 0 status
Test results
Following the configuration of the TLU and timing partition 0, the partition status command,
pdtbutler mst PROD_MASTER part 0 status
yielded the following output:
Created device PROD_MASTER
-----------Hardware info-----------
+----------------+----------------+
| Board type | tlu |
| Board revision | kTLURev1 |
| Board UID | 0x5410ecbba408 |
| Carrier type | enclustra-a35 |
| Design type | overlord |
+----------------+----------------+
Master FW rev: 0x50400, partitions: 4, channels: 5
-- Master state---
Timestamp: 0x120064b134692f4 -> Wed, 19 May 2021 15:24:16 +0000
--------------Cmd gen counters--------------
+------+-----------------+-----------------+
| | Accept counters | Reject counters |
+------+-----------------+-----------------+
| Chan | cnts | hex | cnts | hex |
+------+--------+--------+--------+--------+
| 0x0 | 0 | 0 | 0 | 0 |
| 0x1 | 0 | 0 | 0 | 0 |
| 0x2 | 0 | 0 | 0 | 0 |
| 0x3 | 0 | 0 | 0 | 0 |
| 0x4 | 0 | 0 | 0 | 0 |
+------+--------+--------+--------+--------+
=> Partition 0
---------Controls--------
+---------------+-------+
| Register | Value |
+---------------+-------+
| buf_en | 0 |
| frag_mask | 0 |
| part_en | 0x1 |
| rate_ctrl_en | 0x1 |
| run_req | 0 |
| spill_gate_en | 0x1 |
| trig_ctr_rst | 0 |
| trig_en | 0 |
| trig_mask | 0xf1 |
+---------------+-------+
--------State-------
+----------+-------+
| Register | Value |
+----------+-------+
| buf_err | 0 |
| buf_warn | 0 |
| in_run | 0 |
| in_spill | 0 |
| part_up | 0x1 |
| run_int | 0 |
+----------+-------+
Event Counter: 0
Buffer status: OK
Buffer occupancy: 0
+-------------+-----------------+-----------------+
| | Accept counters | Reject counters |
+-------------+-----------------+-----------------+
| Cmd | cnts | hex | cnts | hex |
+-------------+--------+--------+--------+--------+
| TimeSync | 69 | 0x45 | 0 | 0 |
| Echo | 0 | 0 | 0 | 0 |
| SpillStart | 0 | 0 | 0 | 0 |
| SpillStop | 0 | 0 | 0 | 0 |
| RunStart | 0 | 0 | 0 | 0 |
| RunStop | 0 | 0 | 0 | 0 |
| WibCalib | 0 | 0 | 0 | 0 |
| SSPCalib | 0 | 0 | 0 | 0 |
| FakeTrig0 | 0 | 0 | 0 | 0 |
| FakeTrig1 | 0 | 0 | 0 | 0 |
| FakeTrig2 | 0 | 0 | 0 | 0 |
| FakeTrig3 | 0 | 0 | 0 | 0 |
| BeamTrig | 0 | 0 | 0 | 0 |
| NoBeamTrig | 0 | 0 | 0 | 0 |
| ExtFakeTrig | 0 | 0 | 0 | 0 |
+-------------+--------+--------+--------+--------+
The endpoint status command
pdtbutler ept EPT_0 0 status
gives the output below.
Created endpoint device EPT_0
-----------Hardware info----------
+----------------+---------------+
| Board type | fmc |
| Board revision | kFMCRev3 |
| Board UID | 0x49162b675e3 |
| Carrier type | enclustra-a35 |
| Design type | endpoint |
+----------------+---------------+
+---------+
| 0 |
| 0x50100 |
+---------+
+------------------+--------------------------------------------+
| Endpoint | 0 |
+------------------+--------------------------------------------+
| State | Waiting for phase adjustment command (0x6) |
| Partition | 0 |
| Address | 0 |
| Timestamp | Wed, 19 May 2021 15:25:29 +0000 |
| Timestamp (hex) | 0x120064bef16618d |
| EventCounter | 0 |
| Buffer status | OK |
| Buffer occupancy | 0 |
+------------------+--------------------------------------------+
--- Endpoint state ---
+------------+----------+
| Endpoint | 0 |
+------------+----------+
| buf_err | 0x0 |
| buf_warn | 0x0 |
| ep_rdy | 0x0 |
| ep_rsto | 0x1 |
| ep_stat | 0x6 |
| in_run | 0x0 |
| in_spill | 0x0 |
| sfp_tx_dis | 0x1 |
+------------+----------+
--- Command counters ---
+--------------+----------+
| Endpoint | 0 |
+--------------+----------+
| TimeSync | 85 |
| Echo | |
| SpillStart | |
| SpillStop | |
| RunStart | |
| RunStop | |
| WibCalib | |
| SSPCalib | |
| FakeTrig0 | |
| FakeTrig1 | |
| FakeTrig2 | |
| FakeTrig3 | |
| BeamTrig | |
| NoBeamTrig | |
| ExtFakeTrig | |
| None | |
+--------------+----------+
Delay adjustment
Sending the delay adjustment command,
pdtbutler mst PROD_MASTER align apply-delay 0x1 0x3 0x7 --force
applies a coarse delay of 0x3, and a fine delay of 0x7 to endpoint address 0x1. The endpoint status after delay adjustment is:
(dbt-pyvenv) -bash-4.2$ pdtbutler ept EPT_0 0 status
Created endpoint device EPT_0
-----------Hardware info----------
+----------------+---------------+
| Board type | fmc |
| Board revision | kFMCRev3 |
| Board UID | 0x49162b675e3 |
| Carrier type | enclustra-a35 |
| Design type | endpoint |
+----------------+---------------+
+---------+
| 0 |
| 0x50100 |
+---------+
+------------------+---------------------------------+
| Endpoint | 0 |
+------------------+---------------------------------+
| State | Ready (0x8) |
| Partition | 0 |
| Address | 1 |
| Timestamp | Wed, 19 May 2021 16:55:42 +0000 |
| Timestamp (hex) | 0x120068af2aad218 |
| EventCounter | 0 |
| Buffer status | OK |
| Buffer occupancy | 0 |
+------------------+---------------------------------+
--- Endpoint state ---
+------------+----------+
| Endpoint | 0 |
+------------+----------+
| buf_err | 0x0 |
| buf_warn | 0x0 |
| cdelay | 0x3 |
| ep_rdy | 0x1 |
| ep_rsto | 0x0 |
| ep_stat | 0x8 |
| fdelay | 0x7 |
| in_run | 0x0 |
| in_spill | 0x0 |
| sfp_tx_dis | 0x0 |
+------------+----------+
--- Command counters ---
+--------------+----------+
| Endpoint | 0 |
+--------------+----------+
| TimeSync | 843 |
| Echo | 5 |
| SpillStart | |
| SpillStop | |
| RunStart | |
| RunStop | |
| WibCalib | |
| SSPCalib | |
| FakeTrig0 | |
| FakeTrig1 | |
| FakeTrig2 | |
| FakeTrig3 | |
| BeamTrig | |
| NoBeamTrig | |
| ExtFakeTrig | |
| None | |
+--------------+----------+
indicating the successful application of the coarse and fine delays. The following combinations of delay values were also verified.
Buffer reset error:
The functioning of the event buffer was verified by configuring a fake triggers generator, and enabling triggering for partition 0. The commands issued were:
pdtbutler mst PROD_MASTER faketrig-conf 0 1
pdtbutler mst PROD_MASTER part 0 configure --no-spill-gate
pdtbutler mst PROD_MASTER part 0 start
pdtbutler mst PROD_MASTER part 0 trig
The presence of events in the buffer was verified by the status command (output below).
Created device PROD_MASTER
-----------Hardware info-----------
+----------------+----------------+
| Board type | tlu |
| Board revision | kTLURev1 |
| Board UID | 0x5410ecbba408 |
| Carrier type | enclustra-a35 |
| Design type | overlord |
+----------------+----------------+
Master FW rev: 0x50400, partitions: 4, channels: 5
-- Master state---
Timestamp: 0x120065480c0a84c -> Wed, 19 May 2021 15:37:45 +0000
--------------Cmd gen counters--------------
+------+-----------------+-----------------+
| | Accept counters | Reject counters |
+------+-----------------+-----------------+
| Chan | cnts | hex | cnts | hex |
+------+--------+--------+--------+--------+
| 0x0 | 24 | 0x18 | 0 | 0 |
| 0x1 | 0 | 0 | 0 | 0 |
| 0x2 | 0 | 0 | 0 | 0 |
| 0x3 | 0 | 0 | 0 | 0 |
| 0x4 | 0 | 0 | 0 | 0 |
+------+--------+--------+--------+--------+
=> Partition 0
---------Controls--------
+---------------+-------+
| Register | Value |
+---------------+-------+
| buf_en | 0x1 |
| frag_mask | 0 |
| part_en | 0x1 |
| rate_ctrl_en | 0x1 |
| run_req | 0x1 |
| spill_gate_en | 0 |
| trig_ctr_rst | 0 |
| trig_en | 0x1 |
| trig_mask | 0xf1 |
+---------------+-------+
--------State-------
+----------+-------+
| Register | Value |
+----------+-------+
| buf_err | 0 |
| buf_warn | 0 |
| in_run | 0x1 |
| in_spill | 0 |
| part_up | 0x1 |
| run_int | 0x1 |
+----------+-------+
Event Counter: 8
Buffer status: OK
Buffer occupancy: 48
+-------------+-----------------+-----------------+
| | Accept counters | Reject counters |
+-------------+-----------------+-----------------+
| Cmd | cnts | hex | cnts | hex |
+-------------+--------+--------+--------+--------+
| TimeSync | 14 | 0xe | 0 | 0 |
| Echo | 0 | 0 | 0 | 0 |
| SpillStart | 0 | 0 | 0 | 0 |
| SpillStop | 0 | 0 | 0 | 0 |
| RunStart | 1 | 0x1 | 0 | 0 |
| RunStop | 0 | 0 | 0 | 0 |
| WibCalib | 0 | 0 | 0 | 0 |
| SSPCalib | 0 | 0 | 0 | 0 |
| FakeTrig0 | 8 | 0x8 | 4 | 0x4 |
| FakeTrig1 | 0 | 0 | 0 | 0 |
| FakeTrig2 | 0 | 0 | 0 | 0 |
| FakeTrig3 | 0 | 0 | 0 | 0 |
| BeamTrig | 0 | 0 | 0 | 0 |
| NoBeamTrig | 0 | 0 | 0 | 0 |
| ExtFakeTrig | 0 | 0 | 0 | 0 |
+-------------+--------+--------+--------+--------+
Checking application of delays
This is currently being debugged.
pdtbutler mst PROD_MASTER align apply-delay 0x1 0x3 0x7 --force
pdtbutler mst PROD_MASTER align toggle-tx 0 --on
(dbt-pyvenv) -bash-4.2$ pdtbutler mst PROD_MASTER align measure-delay 0
Created device PROD_MASTER
-----------Hardware info-----------
+----------------+----------------+
| Board type | tlu |
| Board revision | kTLURev1 |
| Board UID | 0x5410ecbba408 |
| Carrier type | enclustra-a35 |
| Design type | overlord |
+----------------+----------------+
Master FW rev: 0x50400, partitions: 4, channels: 5
2021-May-19 17:09:21,766 LOG [dunedaq::timing::GlobalNode::enable_upstream_endpoint(...) at /users/sp13109/v5.4.0_1105/testtiming/sourcecode/timing/src/GlobalNode.cpp:82] Upstream endpoint reset, waiting for lock
2021-May-19 17:09:21,877 LOG [dunedaq::timing::GlobalNode::enable_upstream_endpoint(...) at /users/sp13109/v5.4.0_1105/testtiming/sourcecode/timing/src/GlobalNode.cpp:104] Endpoint locked: state= 0xe
ERROR ('RuntimeError' exception): ' Timeout whilst waiting for echo. Timeout (ms): 500'
--
SudarshanParamesvaran - 2021-05-19