NB: the selection of the SERDES (VTRx [lane 0], SFP [lane 1] or PCB loopback [lane 2]) is done at the time of compilation of the FPGA fw, so it is fixed for a given programming file (stp).
Version (hex) |
Files |
New features and bugfixes |
Test results |
UART Script |
00.02 |
stp top.v |
Corrected ePortDown. Modified GPIOs. Option to transmit an 84-bit constant. |
Found that crystal Q1 has wrong frequency |
|
00.03 |
stp top.v |
Only Serdes Lane 0 enabled (VTRx). Deserialized data should be viewable with "Active Probes". See comments on top.v |
|
|
00.04 |
stp top.v |
Only Serdes Lane 0 enabled (VTRx), QPLL clocks go to tespoints but are not used by the SERDES (so the SERDES may work even if the QPLL is not set properly ). See comments on top.v |
|
|
00.05 |
stp top.v |
As v 0.4 but SERDES Lane changed to Lane 2, which is looped back on the PCB. This allows testing all the serdes logic without using a VTRx. |
|
|
00.0a |
L1/SFP stp top vhdl |
I2C target in the mezzanine igloo2, etc. Selection of REFCLK0 or REFCLK1 for SERDES done through a switch on the board. The change is effective after a power cycle. See top vhdl file. |
Tested in U of Va and Cern bld904. No problem observed; the link latency is not constant (consistent with the present design) |
|
00.0b |
L1/SFP stp top vhdl |
Redesign of the RX Clock phase aligner logic, etc. see top vhdl file. |
Test at Bld 904: occasionally the optical link does not work after a powerup and needs a power-cycle |
|
02.02 |
L0/VTRX stp L1/SFP stp top vhdl |
The link is auto-reset until BitSlip number = 5 (which should imply constant latency). |
Tests at U of VA: the link gets established without errors and BitSlip number is 5 |
|
02.04 |
L0/VTRX stp L1/SFP stp: n/a top vhdl |
- TEST_MODE now comes from the UART interface and not the switch on the mezzanine board (requires v3.4 UART PYTHON Script like uart_BERtest.py)
- UART now continually outputs a CONNECT message every 0.5 second until it receives the first valid command. This is useful when using a terminal program to try to find the baud rate- keep looking until see data (NON-ASCII).
- Fixed a bug in gbt_rx_decoder_gbtframe_chnsrch.vhd
- Modified igloo2_gtx.vhd to use clock S_EPCS_RX_CLK_NoBuff for the LANE0 RX FIFO instead of RX_WORDCLK. It makes it like the other RX FIFOs and does not seem to impact timing.
- Added some clock synchronization of more signals
- Quite possibly found and fixed an insidious bug that was causing the instability issues between builds of a design. It had to do with using asynchronous resets for logic and creating a race condition such that some builds had the proper winner of the race and some did not.
- Removed unused logic in igloo2_serdes_apb_master.vhd related to "s_UPDATE_SETTINGS". In debugging the instability issues between builds, I saw what appeared to be the state machine going into an invalid state. So I removed this unused state and added an OTHERS clause to the state machine as well to try to improve the design.
- slowSig1 & 2 now have functions. First, to be safe, they are glitch filtered for 8 counts of the internal 50 MHz oscillator. slowSig1 is now connected to the RefClk selection and the mezzanine switch no longer controls RefClk selection. After changing RefClk selection, the GBT logic must be reset in order to use the new setting. slowSig2 is now used to reset the GBT logic.
- Corrected the order of FPGA_MAJOR_VERSION and FPGA_MINOR_VERSION in the I2C register. They did not match what the ngccm server expected.
- Applied triple-modular redundancy for radiation testing and use on the detector.
- RefClk selection reported by I2C is actually the value of refclk when the GBT reset was last released. Therefore, this represents the actual used value of RefClk selection.
- Added flip/flops for checking for SEUs within TMR macros. A running count of errors is kept and reported by the I2C Slave.
- Fixed some missing clock domain sync's \
|
Some basic tests during dry-run tests at CHARM irradiation facility. This f/w version is being used for radiation testing of HF system in June 2015. DO NOT USE - Communications can be unstable |
|
02.05 |
L0/VTRX stp L1/SFP stp: n/a top vhdl |
- created a divide by 4 clock from the RefClk. This is a ~30 MHz clock that is a known clock and does not rely on PLL locking. Decided to make use of this clock for the UART so that the UART works even when the PLLs are not locking. This is needed for debugging of the timing issues. The fabric clock is not stable enough to use for generation of the baud clock for the UART. Could potentially replace the use of the fabric clock with this div4 clock. The reason for div4 instead of div3 is that div4 is easier to create and the clock speed is not too important.
- Created a new COREUART instance, but this time specifically for the Igloo2. COREUART_0 was originally created for SmartFusion2 as that was the only option. Not sure if it makes a difference.
- Added ability to read and write APB registers in SERDES from the UART interface
- Added ability to read APB registers in Rx PLL from the UART interface
- UART is now forced into Reset based on the setting of the TEST_SIGNAL switch on the mezzanine. This allows it to be disabled for normal operations in order to prevent the chance of the UART decoding noise into an actionable command.
- The locking of the RX Pll to a specific BITSLIP is now optional based on the setting of the other TEST_SIGNAL switch on the mezzanine
- LED1 now flashes if any of the following are not in the "normal" mode: test mode, UART enable and BITSLIP locking. If test mode (comm. loopback) is enabled, or the UART is enabled or BIT SLIP locking is disabled, LED1 flashes to indicate it.
- changed the LED2 status logic so that if it is flashing, you know that communications is error free. Any detected issues with communication monitored during a flash period will prevent a flash period. Before, tiny errors could cause the LED to flicker but it appeared that it was flashing to indicate good comm. This should prevent this issue so that if it is flashing regularly, comm. is good.
- Updated UART logic to require a two character prefix before each command and allows the use of parameters which was used for reading and writing APB registers via UART. This also checks for UART errors before using the received UART byte and ignores received data in case of error.
- UART no longer outputs characters at the start - this was used for debugging the UART but appears to cause the USB Serial driver to crash sometimes
- Changed state machine in igloo2_gbt_apb_driver.vhd to match the FSM documented in the Microsemi App Note AC335, fig. 3, which uses PREADY to dynamically add wait states. Was hoping that this change would improve the timing instabilities of the firmware but it did not make a difference. Still, the FSM is better now and less likely to present any issues.
- Changed the initial SERDES register settings a little and changed the logic used for loading them at reset.
- switched to using Igloo2_FABRIC_CLK for outputting the fabric clock. This adds a CLKINT clock buffer which was missing before using the old method.
|
Currently being used for production testing of HF ngCCM. Appears more stable that v2.4 but it still suffers from the build-to-build instability issue |
|
02.06 |
L0/VTRX stp L1/SFP stp: n/a top vhdl |
- Reporting the actual REFCLK selection in UART instead of simply the current state of mezz_slow_sig1.
- Built with automatic TMR generation enabled.
|
Basically v02.05 but with TMR. Tullio: link does not seem to lock with many prod cards, with GLIBv3 in Bld904 |
|
02.07 |
L0/VTRX stp L1/SFP stp: n/a top vhdl |
- Fixed reporting of actual REFCLK selection. Now, the reported value is the actual value written to the SERDES APB registers during SERDES init.
- actual REFCLK selection is now used to switch refclk_net, which impacts REFCLK_DIV4. This prevents a change in mezz_slow_sig1 from directly changing REFCLK_DIV4.
- Since mezz_slow_sig1 & mezz_slow_sig2 now have pull-downs on the production ngCCMs, need to invert their signals within the Glue Logic FPGAs and remove the inversion done in the mezzanine firmware. This way, at power on, without any remote jumpers activated, the counting room jumper signals are '1' and the associated mezz_slow_sig's are '0' before reaching the mezzanine. This prevents a race condition between reset and the pull-downs that sometimes caused REFCLK0 to be selected at power on and sometimes REFCLK1.
- Also, built with automatic TMR generation enabled.
|
Use this version if need to be able to switch between REFCLK0 and REFCLK1 using the counting room jumper signals. |
|
02.08 |
L0/VTRX stp L1/SFP stp: n/a top vhdl |
- Forced reference clock selection to REFCLK0 and disabled the ability of mezz_slow_sig1 from changing it. Also disconnected the ability of mezz_slow_sig2 from resetting GBT communications.
- Also, built with automatic TMR generation enabled.
|
This is v02.07 with reference clock selection forced to REFCLK0, the radiation tolerant QPLL. |
uart_BERtest.py |
02.11 (2 0xb) |
L0/VTRX stp L1/SFP stp: n/a top vhdl |
Updated to match the latest version of the HE/HB ngCCM source code at the time this image was created. Lots and lots of changes to how GBT is handled but the basic, top-level code has changed very little. |
This has been tested in the HF burn-in station. |
uart_BERtest.py |
03.00 |
TOP hdl Register DOC Build Instructions
|
- See GIT tag: v3.0
- Builds meet all timing constraints.
- TMR is enabled for Half FEC Link Rate only
- The build for Full FEC Link Rate could not meet timing if TMR was enabled. So the Full build should only be used on a test stand for debug purposes.
- Built with Libero v11.9 SP6
- Only the version that supports LANE0/VTRX was built and tested. LANE1/SFP was not built.
- If using GLIBv2 as a test stand FEC and HF at half speed,
- If using GLIBv2 as a test stand FEC and HF at full speed,
- The Full bit rate firmware does not support the PRBS error counters. So expect to see PRBS errors increment on both HF ngCCM and FEC. However, the pre-FEC, or RAW, error counters can and should be used to determine communications bit errors.
- This means that when testing GBT on the Full build, could only use RX pre-FEC counters on ngCCM because the GLIBv2 has no pre-FEC error counters. However, many of the other tests would not work without communications working both ways.
- Ported over many of the code improvements from the HE/HB code base including the ability to support half bit-rate (2.4 Gbps) GBT.
- One feature from HE/HB is that the VTR TX link (ngCCM to ngFEC) now returns system status and VTR RX link Health status bits in place of the PRBS test pattern. The PRBS Test Pattern can be enabled by setting the TEST COMM bit to a '0'. It is a '1' by default on reset and if the VTR RX link is not valid (returning RX Health is the default). See the new bits as shown in the GBT Bits Spreadsheets (HF_ngCCM_Full_Speed_GBT_Bits_v9.xlsx & HF_ngCCM_Half_Speed_GBT_Bits_v2.xlsx) The RAW Reed-Soloman Decoder error counter in the ngFEC can be adequately used to monitor the ngCCM to ngFEC link quality in place of the PRBS.
- With the adoption of a half speed GBT bit assignment that is compatible with HE/HB, support for HF slots 2, 7 and 9 had to be dropped. Clocks and shared fast command signals still go to these slots but there is no support for I2C nor JTAG. The Full speed version does still fully support these slots as it has done in the past.
- Another feature from HE/HB is the addition of the Comm_Has_Been_Good counter to the RX serial links for VTR (ngFEC). The counter must be over a certain threshold to allow the incoming GBT frames into the ngCCM for action. Safe defaults are in place until the counter exceeds the threshold. Bad RX Frames have more weight than good frames. So the worse the link quality, the longer it takes to recover. A loss of the RX clock completely resets the counter and requires a 0.2s (FULL bit rate) or 0.4s (HALF bit rate) to recover.
- Removed Lock Bitslip from DIP Switch #1 - test feature no longer needed and using this switch for selecting TX INVERT. This can be used to enable support for multi-mode VTRx.
- CNTRM test was not conducted due to lack of available equipment during COVID lockdown. The needed equipment for the test was stuck in my office when I was not allowed entry.
- CNTRM is no longer used, so not conducting this test is acceptable. In fact, the optocouplers were removed from the on-detector boards, if I remember correctly. So it cannot physically be used on production units anyway.
- *Known Issue*: On my development test stand with GLIBv2, I sometimes power up and find that occasional accesses of mezzanine registers result in I2C errors (usually lost bus arbitration [LBA]). Currently suspect it is unique to my test stand but will investigate if ngFEC also experiences this.
- Testing on the actual test stand with a ngFEC programmed for HE/HB Half bit rate did not report any I2C issues so the issue must have been particular to the GLIBv2 as suspected.
- *Known Issue*: Some builds fail to lock GBT communication on the single HF ngCCM unit in the development test stand. Re-running Place and Route has fixed this in the recent past. Seems very similar to issue seen during original development and based on past investigations, it seems most likely to be an issue with the Place and Router tool.
|
Full:
Test |
L0/VTRX |
L1/SFP |
CNTRM |
n/a |
n/a |
IO |
Pass |
n/a |
I2C |
Pass |
n/a |
TEMP |
Pass |
n/a |
JTAG |
Pass |
n/a |
GBT |
Pass |
n/a |
Clock |
Pass |
n/a |
Half:
Test |
L0/VTRX |
L1/SFP |
CNTRM |
n/a |
n/a |
IO |
Pass |
n/a |
I2C |
Pass |
n/a |
TEMP |
Pass |
n/a |
JTAG |
Pass |
n/a |
GBT |
Pass |
n/a |
Clock |
Pass |
n/a |
|
uart_BERtest.py |
03.01 |
TOP hdl Register DOC Build Instructions
|
- See GIT tag: v3.1
- Builds meet all timing constraints.
- TMR is enabled for both Half and Full FEC Link Rates
- Got lucky with the FULL build and it finally met timing without needing to disable TMR insertion.
- Built with Libero v11.9 SP6
- Only the version that supports LANE0/VTRX was built and tested. LANE1/SFP was not built.
- If using GLIBv2 as a test stand FEC and HF at half speed,
- If using GLIBv2 as a test stand FEC and HF at full speed,
- The Half bit rate firmware does not support I2C to slots 2, 7 & 9. This is due to adoption of the HE/HB bit assignments for half bit rate for HF, which has more I2C channels. However, it was decided that since installed HF crate have no cards in slots 2, 7 & 9 that this will work fine.
- The Full bit rate firmware does not support the PRBS error counters. So expect to see PRBS errors increment on both HF ngCCM and FEC. However, the pre-FEC, or RAW, error counters can and should be used to determine communications bit errors.
- This means that when testing GBT on the Full build, could only use RX pre-FEC counters on ngCCM because the GLIBv2 has no pre-FEC error counters. However, many of the other tests would not work without communications working both ways so the fact that they passed means that communications was good.
- Swapped the polarity of DIP Switch #1 which sets the TX Inversion for the VTRx. This is because HF ngCCMs on the CMS detector have this switch already moved to the ON position. So now, setting DIP Switch #1 to ON disables TX Inversion so that a Single Mode VTRx can be used. Changing DIP Switch #1 to off enables TX Inversion so that a Multi-mode VTRx can be used.
- *Known Issue*: On my development test stand with GLIBv2, I sometimes power up and find that occasional accesses of mezzanine registers result in I2C errors (usually lost bus arbitration [LBA]). Currently suspect it is unique to my test stand but will investigate if ngFEC also experiences this.
- Testing on the actual test stand with a ngFEC programmed for HE/HB Half bit rate did not report any I2C issues so the issue must have been particular to the GLIBv2 as suspected.
- *Known Issue*: Some builds fail to lock GBT communication on the single HF ngCCM unit in the development test stand. Re-running Place and Route has fixed this in the recent past. Seems very similar to issue seen during original development and based on past investigations, it seems most likely to be an issue with the Place and Router tool.
|
Full:
Test |
L0/VTRX |
L1/SFP |
CNTRM |
n/a |
n/a |
IO |
Pass |
n/a |
I2C |
Pass |
n/a |
TEMP |
Pass |
n/a |
JTAG |
Pass |
n/a |
GBT |
Pass |
n/a |
Clock |
Pass |
n/a |
Half:
Test |
L0/VTRX |
L1/SFP |
CNTRM |
n/a |
n/a |
IO |
Pass |
n/a |
I2C |
Pass |
n/a |
TEMP |
Pass |
n/a |
JTAG |
Pass |
n/a |
GBT |
Pass |
n/a |
Clock |
Pass |
n/a |
|
uart_BERtest.py |
03.02 |
TOP hdl Register DOC Build Instructions
|
- See GIT tag: v3.2
- Builds meet all timing constraints.
- FPGA Silicon Signatures (SILSIG):
-
FULL : 21102901 -
HALF : 21100701 NOTE: HALF SILSIG does not properly encode the half bit rate since that digit is a 0 instead of a 2 to indicate half bit rate. The fix would require rebuild and retest so since this build works well, do not want to risk a build with issues by rebuilding. TMR is enabled for both Half and Full FEC Link Rates - A few HDL code and constraint file tweaks were required to get the FULL TMR build to meet timing after the HALF TMR build was tested and deployed. These changes are not expected to make a negative impact on the HALF bit rate build but they should be considered if having future issues building for HALF bit rate.
- FULL TMR build had to strip out the UART logic in order to meet timing, so serial debug access no longer works with the FULL TMR firmware.
- FULL bit rate build had to strip out the PRBS generation and verification logic in order to make room for TMR. As previously stated, PRBS is no longer supported in the FULL bit rate builds starting with v03.00.
Built with Libero v11.9 SP6 Only the version that supports LANE0/VTRX was built and tested. LANE1/SFP was not built. If using GLIBv2 as a test stand FEC with HF at half speed, If using GLIBv2 as a test stand FEC with HF at full speed, Same caveats apply from v03.01 regarding that the Half bit rate firmware does not support I2C to slots 2, 7 & 9 and the Full bit rate firmware does not support the PRBS error counters.ADDED Transceiver I2C MUX. In previous firmware versions, the mezzanine IC (bus 16) I2C bus was routed to the VTRx, then the SFP and then the internal registers. This version puts the external connection to the VTRx and to the SFP behind a multiplexer (MUX) such that by default, the IC I2C bus never goes external to the FPGA unless the transceiver mux is changed. The mux can be set to None, VTRx and SFP with None being the default. This protects the VTRx and SFP from errant writes since the transceiver mux should almost always be set to None *Known Issue*: Some builds fail to lock GBT communication on the single HF ngCCM unit in the development test stand. Re-running Place and Route has fixed this in the recent past. Seems very similar to issue seen during original development and based on past investigations, it seems most likely to be an issue with the Place and Router tool. |
Full:
L0/VTRX tests were with MM VTRx
Test |
L0/VTRX |
L1/SFP |
CNTRM |
n/a |
n/a |
IO |
Pass |
n/a |
I2C |
Pass |
n/a |
TEMP |
Pass |
n/a |
JTAG |
Pass |
n/a |
GBT |
Pass |
n/a |
Clock |
Pass |
n/a |
Half:
L0/VTRX tests were with MM VTRx
Test |
L0/VTRX |
L1/SFP |
CNTRM |
n/a |
n/a |
IO |
Pass |
n/a |
I2C |
Pass |
n/a |
TEMP |
Pass |
n/a |
JTAG |
Pass |
n/a |
GBT |
Pass |
n/a |
Clock |
Pass |
n/a |
|
uart_BERtest.py |
: Test_Procedure.
Test of all cards according to the Test_Procedure above have been done at Univ of MD, unfortunately all results have been written only on physical sheets of paper.
for future testing.
The Serial Number is on the label put during manufacturing, which has a number like WO 301 643 - 008.
We use only the last two digits.
.
The LEDs must be at right angles in order to be seen through the front panel. Three mezzanines were modified at UVa by removing all four LEDs on a mezzanine and then adding new LEDs whose pins were manually bent into right angles. LED spacer mounts with a height of 0.140" were used to make the height off of the board uniform (
). The production mezzanines should be built with these right angle LEDs as well as the test points on the bottom should be installed on the top side of the board. See the below images for reference.
The HF ngCCM mezzanine has a set of two DIP Switches accessible from the back edge of the mezzanine, which likely requires removing the ngCCM from the crate to access. The above photo indicates the location of these DIP Switches along the back edge of the mezzanine. These switches have features that can be enabled by moving the switch toward the front panel. This is marked on the DIP Switch as the ON direction. Also, the numbering of the DIP switches is referenced in the photo.
For any of these switches, just changing the switch is not enough. The control boards only look at the state of the switches at reset. So the best practice is to power down the ngCCM, change the desired DIP switch and power the module back up. Since the ngCCM likely needs to be removed from the rack to change the DIP switches, this should be a natural best practice. These signals are only looked at during reset in order to prevent an erroneous DIP switch input signal being seen by the FPGA during operation and have it change modes when it should not be.
The upper Yellow, front panel LED marked "Mode Status" in the photo indicates the status of the DIP Switches. It either stays solidly on, or it blinks a repeating pattern of 1, 2 or 3 pulses. This corresponds to the binary pattern of (not SW#1) and SW#2 with (not SW#1) acting as the least significant bit. If the LED is on solidly, this corresponds to the binary pattern for 0. SW#1 is inverted before creating the LED flash pattern for historical reasons. Note that the LED does not show the current state of the DIP Switches but rather the state that was captured when the HF ngCCM mezzanine was last in reset.
The mezzanine has a debug serial port which is used to output firmware state information and GBT fiber communications status. It can be accessed with a v2.7/v3.6 python script,
. It is known to work on SLC5, SLC6, SLC7, Windows 7, Windows 10, OS X 10.10+ or macOS 11 using python v2.7 or v3.6. Other versions of python may work and other operating systems may work. The
package must be installed before able to use uart_BERtest.py.
. The FTDI chipset is recommended but not required. One useful cable is this one from Sparkfun:
. It is also possible to make use of a bluetooth module which can be powered from the mezzanine debug port. This bluetooth module from Sparkfun is known to work:
. The bluetooth may require some one-time configuration.
The following image shows how to connect the wires for the TTL UART. The image is of the mezzanine combined with a ngCCM motherboard and the front panel attached. Reference the yellow LEDs on the mezzanine to orient the image for yourself. Note that right column of pins on the mezzanine debug connector are shown unused with the serial connections on the left column, as oriented in the image.
connect the power wire from any of the USB-TTL Serial cables as that is 5V coming from the USB port.
.
IMPORTANT: As of Firmware v2.5, the UART is disabled by default. To enable it, the Test Signal switch 2 (small DIP-style switch on mezzanine) must be in the ON position. LED1 will then blink a pattern of 2 to indicate that the UART is enabled. Be sure to disable the UART when complete with tests by putting switch 2 back to the OFF position.