Name of the exercise

Event building : Networking and software basics

Responsible for the exercise

Jean-Christophe Garnier, Dan Octavian Savu, and everyone else interested

Description of the exercise

A large experiment Data Acquisition relies on a process called Event Building. It mainly consists in aggregating each part of a physics event into a single block of data, the proper event. In the 4 major experiments at CERN, this part relies on high-end routers.

The idea of the exercise is to ask students to set up a small event building infrastructure, and then to code the required software (without a trigger).
It requires :
1 - Producing a network traffic
2 - Network configuration
3 - Client implementation

This lab is very related to my experience in LHCb, any other idea are welcome. There are many things to show in networking, many protocols, many tools, and though everything is interesting (at least for me), one shall not forget that they have 2 hours max to perform the lab.
The application protocol between the traffic generator and the clients is to be defined, I chose a push protocol for the normal lab, the pull protocol would be studied by the more advanced students (maybe all). The transport protocol will be by default raw IP or UDP, TCP will be studied by the more advanced also.

1 - Producing a network traffic
Software implementations which sends part of events on an ethernet media. The first part of the exercise would rely on UDP or raw IP, the best students will have the time to have a look afterward on a TCP connection.

It needs to send 3 packets from different sources, to emulate a splitted event. Either IP spoofing (have fun for TCP) or 3 NICs. 3 is enough, if you can process 3, you can process n.
- Normal lab: Push protocol The destination are chosen a round-robin way, in order to illustrate the shape of the input of a computing farm.

Packets needs to be tagged to show that they are from the same event. No need of synchronization a priori if we send the 3 packets from the same thread.
I think to provide everything for this point, it is not the most interesting thing to be done in an event building exercise.

• Reserve lab1: TCP protocol
  

TBD

• Reserve lab2: Pull protocol
  

TBD

2a - Network configuration

The labs starts in a kind of teamwork, with the instruction to plug everything.
Visio draw to show the configuration TBD

Then the students log in to the switch via the management interface, set up a basic configuration (VLANs) so that the traffic generator and the clients can communicate, and a port mirroring for debugging.


Reserve lab: For the bests, set up a routing configuration instead of VLANs (need a router then). Add some QoS and traffic shaping ?

2b - Network monitoring

A very short review/presentation about network monitoring, rrd files and snmp library will be done in ~10 minutes before this section.
After everything is up and running (from a networking point of view) a set of scripts that can monitor traffic statistics via SNMP will be provided to the students. Only the first 5 ports on the switch will be monitored by default for input and output traffic (in bytes/s).

The students will be asked to run those scripts according to the README file inside the archive and to watch the network activity on the switch.
In the end they will have to modify those scripts to monitor just the active ports and display data for a larger time interval, like 1 hour (default is 10 minutes).

• DaqAnkara_NetLab_swmonitoring.rar: Set of scripts for switch traffic monitoring (more details in the README file inside the archive).

Reserve lab: For the bests, set up a routing configuration instead of VLANs (need a router then). Add some QoS and traffic shaping ?

3 - Client implementation

Language is free of choice. Solution will be provided at least in C and python.
Now each student is on his own. The aim is to implement a client which gets one packet from each source, builds the event and stores it.

• Normal lab: Reception of the push protocol.
• Reserve lab1: Reception of TCP.
• Reserve lab2: Reception of the pull protocol.

Incomplete sources will be provided.

What will the students learn

• Network configuratio, administration, and monitoring.
  
• Network encapsulation paradigm.
  
• Programming network applications in C/python/whatever (with a special emphasis for python for RAD in my opinion).
  
• File formatting (flat format).
  

Duration

Not sure yet about the time, by experience with summer students (most physicists, a few IT), 2 hours were not enough for only points 2 and 3 (with no reserve lab), point 1 being provided with the lab.
In order to make the duration fitting 2 hours, I will provide incomplete sources for the client implementation, so the students will only have to focus on the point that we wish to highlight : interpretation of a network protocol, building the event, formatting the file. They won't have to play with includes or OO architectures.

List of material

• Student work stations and event builder clients: 4 desktop computers with free network interfaces.
• Traffic generator: 1 server/desktop computer with 3 1GbE NICs.
• 1 router/switch, with CLI, web interface.
• 1 computer with 1GbE NIC for dataflow monitoring.
• Network cables.
• 1 USB/RS232 interface for switch configuration in case of troubles.

SLC5 everywhere, with python, gcc, man pages, a rpm that I provide to add some capabilities, internet access

Relevant information

Required lectures:
Network
System Design

Manuals:
TBD

Instruction Guides and Sheet:
TBD

Solution

TBD