DAQ PC

The original specification DAQ PCs were specified and assembled by Barry Green (RHUL)

Case

The case is a Compucase 411 . This case can be placed on a shelf in a rack or mounted with rails (not supplied)

Power Supply Unit

The power supply is a CiT 650UT . This is a low cost power supply. The main considerations when replacing is the ability to provide enough current to spin-up the disks and the presence of an axillary 12v connector for the motherboard. The P.S.U.s are internally illuminated with coloured LEDs, these serve no useful purpose!

Motherboard

The motherboard is an MSI K9A2 Platinum
If you have problems with booting the system (at the BIOS Level) try updating the BIOS firmware: BIOS Firmware

The motherboard has 4 PCIe slots suitable for installation of an ODR. To use the 4 th slot the graphics card will have to be replaced with a PCI graphics card. Any old, low spec, PCI graphics card will be suitable as any graphical applications would normally be run on a remote display. The order in which PCIe slots are populated is important and the motherboard manual should be consulted.

An additional PCIe NIC can be added in the PCIe x1 slot if required. NICs based on the Broadcom TG3 are high performance and well supported by SLC5.

CPU

The CPU is an AMD quad core Phenom
  • Model number 9650
  • Clock Frequency 2.3 GHz
  • Total level 2 cache 2MB
  • Level 3 cache 2MB
  • Package socket AM2+
  • Maximum power95W
  • CMOS technology 65nm SOI

Memory

3GB of memory is fitted to the motherboards. A problem was found when using more that 3GB of memory (Probably if the 32bit memory space is exceeded). The problem could be related to an additional layer of memory management or an incorrect variable size being used. The work around was to limit the memory to 3GB.

Operating System

The system is installed with SLC5 (CERN rebuild of RHEL5 linux). Some reconfiguration will be required to allow the system to automatically update from CERN. This might be the file which needs to be forced to install http://linuxsoft.cern.ch/cern/updates/slc5X/i386/RPMS/yum-conf-5X-6.slc5.cern.noarch.rpm . Check with yum listrepos that an RHUL yum config isn't installed. If it is remove it then install the CERN repo conf.

System disk

The system disk is parallel ATA drive. This was used to keep the maximum number of SATA ports available for the RAID array

RAID array

The RAID array is 4 disks in RAID 5 configured as a RAID 5 array. The RIAD is configured by the /etc/raidtab file. There are other ways to configure RAID within the OS which might be more up-to-date. Only 4 drives are used as the operating system only supports (by default) 4 of the SATA channels. Samsung model HD162GJ 160GB drives are supplied.

each disk was partitioned with both parted and fdisk because the relevant parts of each were known.

parted was used to add a partition table

parted /dev/sda

mktable gpt

mkpart primary 0 -1

fdisk was then used to assign the type of the partition to 0xFD (linux raid)

The /etc/mdadm.conf file contains:

DEVICE /dev/sd[abcd]1

ARRAY /dev/md0 devices=/dev/sda1,/dev/sdb1,/dev/sdc1,/dev/sdd1

The array was created with the command:

mdadm -C /dev/md0 --level=raid5 --raid-devices=4 /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1

the filesystem was created: mkfs.xfs /dev/md0

/etc/fstab was modified to mount the partition as /array

Measure write performance using time dd if=/dev/zero of=/array/testfile10g bs=32768 count=327680 give 270 MB/s

Hardware RAID controller

Some of the DAQ PCs have been supplied with hardware Areca RAID controllers. There can be incompatibility between some hard drives and hardware RAID controllers due to the way they handle errors. This controller needs firmware version 1.49 to be reliable. Seagate Baracuda model number ST3000DM001 have been found to work reliably with this controller. Other disks in this series with probably work reliably too.
dd was used to create a 100GB file to measure the data rate of the above configuration. 320MB/s was achieved.

Another test configuration was tried. This uses the RAID controller but with the disk set as JBOD. Linux RAID5 was then used. This achieved 277MB/s with the seagate 5 x ST3000DM001 disks.
10 x ST3000DM001 disks gave 345MB/s (RAID5 SATA150 ncq)). 350MB/s (RAID5 SATA300 ncq)
10 x samsung 160GB disks 322MB/s

Disk Caddy

The disk caddy is an ICYDOCK MB455SPF.
These have been found to be the source of many apparent disk failures. The unit is a convenient way to mount a large number of disks so a fix has been found.
Remove the 8 screws holding the backpanel to the caddy. Remove the 2 screws holding the backplan in place. Remove the 6 screws holding the Right hand side panel in place. The tab at the rear should be removed. This will beak off if bent a couple of times. Reassemble the case without backplane and back panel. The cables and power can now be directly attached to the disks. The PC case may need to be kept closed to maintain cooling.

System information

System Name MB Serial No. MAC Address Location RHUL serial No.
Caldaq0      
Caldaq1 0111260530 00:15:F2:36:AF:14 RHUL  
Caldaq2   00:10:18:30:A3:B6  
Caldaq3 0111262591 00:1D:92:65:07:9F RHUL D05248
Caldaq4 0111262360 00:1D:92:65:06:DA RHUL D05249
Caldaq5 0111262056 00:1D:92:65:05:68 RHUL D05251

Trouble shooting

There are a number of known problems with these systems.

Booting/ The BIOS

The BIOS sometimes doesn't display an image. After a number of power cycles a message appears reporting that overclocking failed and offers del or F11 as key press options. We have found that pressing any other key is the correct response. It then offers you F1 to go to the BIOS setup. Press F1. You can then either reset the defaults or just save and exit. The next restart will then be OK. Flashing the BIOS is probably the only permanent solution. This can possibly be fixed by updating the BIOS version If you have problems with booting the system (at the BIOS Level) try updating the BIOS firmware: BIOS Firmware

Hard Disk RAID array

The systems are supplied with hard drives connected to the motherboard. These are configures as a RAID array using the operating system. These are two other RAID setups mentioned at boot time. Areca and fasttrack. Fasttrack is often known as "fake raid" and not recommended. The Areca RAID is a high performance RAID controller in one of the PCIe slots. This can be configured from its' BIOS or from its' own web interface accessed from its' own ethernet port. If you choose to use this you should change the hard disks to RAID edition disks or ones which are specifically mentioned on the areca.com.tw support pages. If you need to know why research TLER.

-- BarryGreen - 09-Nov-2009

Edit | Attach | Watch | Print version | History: r12 < r11 < r10 < r9 < r8 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r12 - 2012-04-20 - BarryGreen
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CALICE All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback