Material needed

• Data Switch : HP 3500
• Data Switch : HP 2650
• Farm server : Dell SC1425
• Farm nodes : Dell SC1950
• Power cables :
  • one special power cable for the Data switch
  • 2 normal power cables for the control switch and the server
  • normal power cables for the nodes
• Network cables :
  • Control cables (yellow) :
    • 3 x 0,5 m
    • 2 x 2 m for the first 2 nodes
    • 2 x 1,5 m for node 3 & 4
  • Data cables (green) :
    • 12 x 1 m for the storage uplink and one per subnet (11 for 30 nodes)
    • 2 x 2 m for the first 2 nodes
    • 2 x 1,5 m for node 3 & 4

Install the control & Data switches

Take note of the mac addresses of the switches and add them to the dns and the dhcp server.
Their name will be sw-d1xxx-c1 for the control switch and sw-d1xxx-d1 for the data switch , xxx beein the rack name (eg E04 -> sw-d1e04-c1 & sw-d1e04-d1).
Their IP addresses will be composed like this : the first three digit will define the rack subnet (eq 10.130.150 for rack D1E06) as seen on that page and the last digit beeing 251 for the control switch and 252 for the data switch.

Install the nodes

Cable the control switch

Switch Port	Go to	Comment
1..	nodes 1st interface
47	Patch Panel 1	Farm Server
Gb1	SW-D1xxx-D1 port 48	Data switch control link
Gb2	Patch Panel 9	Up link

Cable the data switch

Switch Port	Go to	Comment
1..	nodes 2nd interface
35-46	Patch Panel 11-22 or 3-8	Central DAQ switch
47	Patch Panel 2	Storage Uplink

Connect the uplink in D2 to the main control switch

The patch panel is in rack D2C06.
On the Control Force 10 switch, the connections are mainly made on the line card 0.

Power on the switches

You should now be able to connect to them with telnet.

Update the Data switch firmware

Connect to the data switch with telnet.
Issue the following commands :

copy tftp flash 10.128.16.26 K_12_14.swi
boot system flash

Configure the Data switch

Run the script /admin/scripts/networkscripts/gen_data_switch_vlan.sh d1xxx > d1xxx.txt
That will generate the text file with all commands to configure the switch for the rack you pass as parameter.
Then run the script /admin/scripts/networkscripts/exectelnet.pl sw-d1xxx-d1 admin password d1xxx.txt
That will execute the config file on the given switch.
Check that the DEFAULT_VLAN only list ports 47 & 48 as untagged and no other ports (to avoid network loops).

Install the Nodes

Cable up the first interface to the control switch and the 2nd to the data switch. Node on will go to port 1 and so on.
Take note of their Serial Number and Mac address. They'll be needed later for quattor configuration.

Install the Server

The servers are installed in racks D2B01 (for rack row D1E) to B05 (for rack row D1A).
Register the new server in the dns (with the ipmi interface). Add the ipmi interface to the dhcp server on dns-sx-01.
Copy the template of an existing server and modify the name, mac and ip addresses and the list of nodes.Add the template to cdb.
Run sudo aii-shellfe --configure server_hostname and sudo aii-shellfe --install server_hostname
I hope to have soon a script to generate the server and node templates and configure the dns and dhcp services.
You can now connect and start the server. The installation should work.

Connections & configuration on SW-DAQ-01

The script /admin/scripts/networkscripts/generate_ips.pl with the rackname (eg.: D1E10) and the subnet number as parameters will give the following informations :

• IP number of the data interface of the computers in that subnet.
• For the link between the data switch in the rack and SW-DAQ-01 :
  • IP number of that link on the data switch in the rack
  • IP number of the port on SW-DAQ-01

The IP numbers for the nodes will be used later in the nodes templates. The data switch is already configured.

Connect a patch cable for that subnet (When all line cards will be installed, a page will document on which Line card/port each D1 subnet will be connected to. In the meantime, we'll have to connect it to the `next free connection`).
Telnet to SW-DAQ-01 do (eg. for link gi 0/7 and IP number :192.168.204.70

sw-daq-01#conf
sw-daq-01(conf)#int gi 0/7
sw-daq-01(conf-if-gi-0/7)#no switchport
sw-daq-01(conf-if-gi-0/7)#ip address 192.168.204.70/26
sw-daq-01(conf-if-gi-0/7)#mtu 9252
sw-daq-01(conf-if-gi-0/7)#ip mtu 9234
sw-daq-01(conf-if-gi-0/7)#no shutdown
sw-daq-01(conf-if-gi-0/7)#exit
sw-daq-01(conf)#exit
sw-daq-01#copy running-config startup-config
sw-daq-01#exit

conf
int gi 0/7
no switchport
ip address 192.168.204.70/26
mtu 9252
ip mtu 9234
no shutdown
exit
exit
copy running-config startup-config
exit

Storage Uplink

In rack D2B*, connect the socket coming from the socket 2 of the D1 rack to the Storage switch.
Eg for rack D1E10 :
Connect D2B1C18 (coming from D1E10A02) to port 10 (corresponding to the rack number in the row) on switch SW-D2B01-S1.