Repairs and updates of the VMEbus SBCs of ATLAS

Introduction

There are two generations of SBCs in use by ATLAS:
  • VP717 / VP917
    • These SBCs are still fully supported and have no know issues. In case of a malfunction they can of course still be repaired.
  • VPE-24
    • This type of SBCs is based on the Intel Atom CPU. CPUs that have been produced before January 2019 have a known issue that may lead to a failure of the CPU at any time. More than 10 SBCs have already failed in this way. In order to prevent problems during run 3 it is strongly recommended to have the CPUs replaced preventively.

Note: SBCs of type VP-110 and VP-315 are no longer supported and cannot be repaired.

Is my SBC concerned?

Note: Only SBCs with an Atom processor of the "D0" step have to be updated.
There is a Linux command to read a PCI configuration register which will show whether a D0 step or D1 step processor is installed. The command is:
sudo setpci -s 00:1f.0 f8.l
D0: 0x01110F1A
D1: 0x01130F1A

Note: Users may not have the sudo privilege required to run the command. Contact Markus Joos.

Repair / update logistics FAQ

Q: Do I have to have my SBC updated
A: No, the update is not mandatory. But be aware that a D0 CPU may fail at any moment

Q: How much will the update cost and who pays?
A: The preventive replacement of the CPU costs GBP 500. You have to provide a budget code.

Q: How long will it take to have my SBC updated
A: CCT has a turn around time of ~4 weeks (shipping included). As their capacity is limited (10 cards per week) cards will be sent in batches

Q: Can I have a spare SBCs as a temporary replacement?
A: It depends. ATLAS has a small stock of spares. Spares will be provided if available

List of SBCs waiting to be sent

Owner Budget code for repair Serial number IP-name D0 CPU confirmed Preferred shipping date Remark
Rhys Owen TBD TBD TBD no later some SBCs
Tomoyuki Saito TBD TBD TBD no as soon as possible ~10 SBCs
Tomoyuki Saito TBD TBD TBD no once the first batch is back ~10 SBCs

Please note: CCT will be closed from 24th Dec to 4th Jan 2021.

List of SBCs that went for a repair / preventive replacement of the CPU

EDH Shipping date RMA Serial number Owner Budget code for repair IP-name D0 CPU confirmed Remark
8534894 TBD 109678 M27080/041 Rhys Owen T577300 n/a via serial number CPU preventive replacement
8534894 TBD 109678 M27080/042 Rhys Owen T577300 n/a via serial number CPU preventive replacement
8534894 TBD 109678 M27080/043 Rhys Owen T577300 n/a via serial number CPU preventive replacement
8534894 TBD 109678 M27080/045 Rhys Owen T577300 n/a via serial number CPU preventive replacement
8534894 TBD 109678 M27080/046 Rhys Owen T577300 n/a via serial number CPU preventive replacement
8534894 TBD 109678 M27080/048 Rhys Owen T577300 n/a via serial number CPU preventive replacement
8534894 TBD 109678 M27080/049 Rhys Owen T577300 n/a via serial number CPU preventive replacement
8534894 TBD 109678 M27080/071 Rhys Owen T577300 n/a via serial number CPU preventive replacement
8534894 TBD 109678 M27080/002 Antoine Marzin TBD n/a via serial number CPU preventive replacement
8534894 TBD 109678 M27080/012 Antoine Marzin TBD n/a via serial number CPU preventive replacement
8534894 TBD 109678 M27080/056 Kris Korcyl T552060 n/a via serial number CPU preventive replacement
8534894 TBD 109678 M26420/040 Gerhard Brandt T577020 n/a via serial number CPU preventive replacement
8534894 TBD 109678 M26420/041 Gerhard Brandt T577020 n/a via serial number CPU preventive replacement
8490051 17.11.2020 109664 M27080/079 Wainer Vandelli (via Gerhard Brandt) T575080   yes CPU preventive replacement
8490051 17.11.2020 109664 M27080/004 Gerhard Brandt T577020   yes CPU preventive replacement
8490051 17.11.2020 109664 M27080/006 Gerhard Brandt T577020   yes CPU preventive replacement
8490051 17.11.2020 109664 M27080/008 Gerhard Brandt T577020   yes CPU preventive replacement
8490051 17.11.2020 109664 M27080/009 Gerhard Brandt T577020   yes CPU preventive replacement
8490051 17.11.2020 109664 M27080/011 Gerhard Brandt T577020   yes CPU preventive replacement
8490051 17.11.2020 109664 M27080/010 Gerhard Brandt T577020   yes CPU problem
8490051 17.11.2020 109664 M27080/044 Rhys Owen (L1Calo) T577300   yes CPU problem
8490051 17.11.2020 109664 M27080/037 Rhys Owen (L1Calo) T577300   yes CPU problem
8468690 16.10.2020 109641 M27080/007 Gerhard Brandt
8456167 13.10.2020 109640 M27080/082 Masato Aoki
8449868 05.10.2020 109629 M27080/005 Gerhard Brandt
8350165 17.07.2020 109555 M26420/023 Antoine Marzin
8350165 17.07.2020 109555 M27080/101 Antoine Marzin
8343271 15.07.2020 109550 M26420/051 C. Luci
8318667 26.06.2020 109536 M27080/054 K. Korcyl
8143171 05.02.2020 109392 M27080/089 K. Nagai
8143171 05.02.2020 109392 M27080/016 K. Nagai
8143171 05.02.2020 109392 M26420/047 W. Vandelli
8143171 05.02.2020 109392 M27080/014 W. Vandelli
8075529 10.12.2019 109330 M27080/001 Gerhard Brandt
8016972 08.11.2019 109286 M27080/053 B. Barnett
8016972 08.11.2019 109292 M30201/008 C. Sbarra
8016972 08.11.2019 109292 M26420/038 Gerhard Brandt
8016972 08.11.2019 109292 M26420/039 Gerhard Brandt
7868089 12.07.2019 109185 M27080/074 B. Barnett
7781352 15.05.2019 109120 M30201/008 C. Sbarra

My SBC is back from repair. What next?

  • Install the battery for the BIOS backup. You will find it in an orange bag in the box of the SBC.
  • Connect the SBC to a screen and check these BIOS parameters:
    • Main -> Boot Features -> PXE boot -> Set to "enabled"
    • Main -> Boot Features -> Front Panel Eth A -> Set to "enabled"
    • Main -> Boot Features -> Front Panel Eth B -> Set to "enabled"
    • VME -> Outbound Reserved Size -> Set to 2 GB
    • Advanced -> Misc Configuration -> PCI MMIO Size -> 3GB.

Other issues (not CPU related)

Serial number Owner Remark
M26420/051 C. Luci Crashed several times in USA15 (could not be reproduced in lab14)

-- MarkusJoos - 2020-11-12

Edit | Attach | Watch | Print version | History: r7 < r6 < r5 < r4 < r3 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r7 - 2020-11-27 - MarkusJoos
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Sandbox All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback