LHCb Software Training: Basics

Prerequisites

lxplus

To get an account on lxplus, please contact the LHCb secretariat. If you already have an lxplus account, but were previously working in another experiment, you should make sure that your account is registered as an LHCb account, in afs group z5 (see Exercise 1).

These instructions have last been checked against the DaVinci v19r3 (and Gaudi v19r4) environment. Please use this version of DaVinci or a more recent version.

Logging in to lxplus

lxplus

ssh

X11

emacs

Choice of shell

tcsh

csh

sh

bash

ksh

zsh

• A.Taddei, Shell Choice: a shell comparison CERN report CN/DCI/162 (1994)

A user may redefine the default shell on lxplus by going to https://cra.cern.ch and changing the appropriate field. In general, unless you have a strong preference, stick to tcsh as this is better tested.

In what follows, <script>.[c]sh is used as shorthand for:

• csh,tcsh: <script>.csh
• sh,bash,ksh,zsh: <script>.sh

LHCb software environment

• /afs/cern.ch/group/z5/group_env.[c]sh
• /afs/cern.ch/group/z5/group_login.[c]sh

• /afs.cern.ch/lhcb/scripts/lhcbsetup.[c]sh

• /afs.cern.ch/lhcb/scripts/CMT.[c]sh

CMT

When not working on lxplus, but on a Linux machine where AFS is available, the standard LHCb environment may be set using:

LHCb flavour of emacs editor

emacs user guide

vim

nedit

Joe's own editor

pico

nano

other

IMPORTANT : If you are working over a slow network connection, it is sometimes useful to work in a 'terminal mode', where a new X window is not opened. This can be done by running

1. Open a file with control-X control-F
2. Save a file with control-X control-S
3. Exit with control-X control-C

Exercise 1

• Login to lxplus.
• Check that X11 connections are enabled, for example by displaying a clock:

  xclock &

• Check the group associated with your account by giving the command:

  echo $GROUP

  For an account in the LHCb group the result should be z5.
• Check which shell you are using, for example using:

  echo $0

  or

  echo $SHELL

  and optionally change your default shell by going to https://cra.cern.ch (you will need to login using your CERN NICE username and password)
• Check that the standard LHCb environment has been set. If it has, then you will have seen a message at login time about your CMT settings, and the result of the command:

  env

  should contain a number of variables with prefix LHCB (LHCBHOME, LHCBRELEASES and so on), and with prefix CMT (CMTROOT, CMTCONFIG and so on). If this isn't the case then you need to setup the environment yourself.
• Setup the LHCb flavour of emacs and (optionally) the EDT keyboard emulation by adding the following lines to the ~/.emacs file:

  (load (expand-file-name "$EMACSDIR/lhcb")) (load (expand-file-name "$EMACSDIR/edt"))

  or

  cp $EMACSDIR/.emacs ~

  LHCb specific emacs commands are documented in the LHCb emacs user guide.

Organisation of LHCb software

Gaudi

Software in LHCb is developed in the context of CMT packages and CMT projects.

• A package is a set of files, organised according to some directory structure, which provides some well-defined, circumscribed functionality. It is the basic unit of software development, meaning that all the files in a given package are tagged with the same version number and are released together. Changing just one file in the package implies releasing a new version of the whole package. Most LHCb packages contain code that can be compiled into one or more libraries.
• Packages with related functionality are collected in groups (or are said to be placed under a hat). The purpose of package groups is simply to group together, in the directory tree, all packages that are related in some way (for example all event model packages under the /Event hat, or all tracking packages under the /Tr hat).
• A project is a set of packages that are grouped together according to some functionality, for example all Gaudi packages (Gaudi project), most public LHCb header files and base class libraries (LHCb project), all reconstruction packages (Rec project), all packages specific to the simulation application (Gauss project). All packages within a given project are released as a single unit: changing just one package in a project implies a releasing a new version of the whole project. Choosing a given version of a project automatically selects a single version of each of the packages in the project.

Projects and packages have an associated version number, of the form vXrY, with X and Y integers, and this is incremented when changes are made in package contents. The hierarchy of the software organisation is then: PROJECT/PROJECT_<version>/<package group>/<package>/<package version>. For example

• src: source code;
• <package> (Sub-directory with same name as package): header files;
• options: job options associated with the package;
• cmt: information for CMT processing;
• doc: package documentation.

CVS

kerberos

Check what tokens you have and when they will expire with the command

New versions of packages, and new packages, are placed in the release area after testing, or may be available in the development area as works in progress. The standard LHCb environment includes the variables:

• LHCBRELEASES: release area;
• LHCBDEV: development area.

How to use Gaudi under CMT

CVS repository

release area

LHCb applications

Gaudi

• Gauss: event generation, using Pythia and EvtGen, and detector simulation based on Geant4;
• Boole: simulation of detector response (digitisation)
• Brunel: event reconstruction
• DaVinci: event selection and data analysis
• Panoramix: graphical display of detector and event information

Additional projects contain software that is shared by several applications. The main ones are:

• LHCb: base classes and public include files
• Lbcom: components common to all applications
• Hlt: components for online (HLT) reconstruction
• Rec: components for online (HLT) and offline reconstruction
• Phys: components for online (HLT) and offline analysis
• Analysis: components for offline analysis

Exercise 2

• Move to the LHCb release area: either

  cd ${LHCBRELEASES}

  or use the web interface to the release area: http://cern.ch/LHCb-release-area/
• Note the different project directories, and enter the project directory for any one of the projects.
• Note the different project versions available, and move to the directory for the latest.
• Note the different package groups, and have a look in two or more of these at what packages are available.
• Choose any two packages, and for each in turn: move to the directory for the latest version, look at which sub-directories are defined, and have a look at the files that are in each.
• Look also at the development area:

  cd ${LHCBDEV}

Configuration Management Tool

CMT

For a package with name myPackage, CMT itself defines an environment variable MYPACKAGEROOT that contains the package location.

The full set of commands that can be used in a requirements file is described in the CMT manual. Some of the most useful commands are shown below.

# Lines beginning with the symbol # are treated as comments.
# Blank lines are ignored.
#==================================================================
# The following commands describe a package.

# Package name.
package myPackage

# Package version, should correspond to CVS tag.
version v1r0

# Structure, i.e. directories to process. 
# A package must always have:
#   a cmt directory (for the requirements file)
#   a doc directory (for the release.notes file)
# It may have:
#   a src directory for source files (.h, .cpp)
#   a options directory for job options (.opts)
branches cmt doc src options

#==================================================================
# The following command specifies packages which the current package
# needs for compilation and linking. Put as many lines as necessary.
# The keyword is followed by package name and version and,
# where applicable, the package group in which the package is placed.
# The version can usually be v* (i.e. any version) because the exact version
# is usually determined by setting the environment in which a package is built
use DaVinciTools v* Phys

#==================================================================
# The following commands tell CMT what to build and how

# Build application with the specified file
application myApp ../src/myApp.cpp

# Build a library with the specified files
library myLib ../src/*.cpp

# Define link options and environment variables for loading component library
apply_pattern component_library library=myLib

# Define link options and environment variables for loading linker library
apply_pattern linker_library library=myLib

#==================================================================
# The following commands define symbols (variables and aliases).
# They consist of a keyword followed by a symbol name, and possibly by a value.

# Previously defined symbols, including environment variables, can be used in
# value assignments by placing brackets - () or {} - around them and a $ in front.

# Treat symbol as a Make macro definition.
# This can be used to define a symbol as a local variable
# (i.e. one that won't be visible from the shell).
macro MYCMTDIR $(HOME)/cmtuser

# Treat symbol as an environment variable.
# The symbol will be visible from the shell after executing SetupProject
set LCGDIR /afs/cern.ch/sw/lcg

# Treat symbol as a path variable.
# The symbol will be visible from the shell after executing SetupProject
# When appending or prepending an item to a path variable not initialised in the 
# requirements file, it's a good idea to first remove the item, to ensure it occurs only once.

# Set path.
path PYTHONPATH $(HOME)/python

# Append item to path.
path_append PYTHONPATH $(HOME)/myPythonPackage

# Remove item from path.
path_remove LD_LIBRARY_PATH $(HOME)/lib

# Prepend item to path.
path_prepend LD_LIBRARY_PATH $(HOME)/lib

# Create alias.
# The alias will be visible from the shell after executing SetupProject
alias myApp $(MYPACKAGEROOT)/$(CMTCONFIG)/myApp.exe
#==================================================================

Most CMT commands, but not all, operate on requirements files, and must be given from a package's cmt directory. If a requirements file includes lines indicating that other packages are used, then these packages will be searched for in locations specified by the project environment, and the requirements files of each used package is processed in turn.

The CMT manual should be consulted for the full set of CMT commands, but a useful subset is given below.

• Perform package configuration - that is, create setup files and Make files:

  cmt config

  Note that when a copy of a package is obtained using getpack the package is configured automatically.
• After configuration, Make commands can be used from the cmt directory:
  • Build the package binaries:

    gmake

    The output of the build operation is placed in a package sub-directory. The name of this sub-directory is given by the value of the environment variable CMTCONFIG.
  • Delete the package binaries:

    gmake binclean

  • Delete all generated files, including copies made to the project InstallArea

    gmake clean

• Show all packages used by the current package:

  cmt show uses

• Execute <shell command(s)> in cmt directory of current package and all used packages within the same project:

  cmt broadcast <shell command(s)>

  Multiple commands need to be separated by semicolons and enclosed in inverted commas. This can be useful, for example, to force rebuilding of all used packages in the user's area:

  cmt broadcast "cmt config ; gmake binclean ; gmake"

• Execute an executable within the environment of the current project

  cmt run <myExec.exe>

To release a new package in the LHCb framework it must be imported into CVS and registered so that getpack can find it. Follow the instructions here.

When executing an application, the following two environment variables are very important:

• PATH: search path for executables;
• LD_LIBRARY_PATH: search path for shared libraries.

Exercise 3

• Create a CMT project directory in which to work.

  setenvDaVinci v19r3

  This command creates a directory ~cmtuser/DaVinci_v19r3, containing a file cmt/project.cmt. Any directory tree containing this file is called a CMT project.
• Look at the value of CMTPROJECTPATH

  echo $CMPROJECTPATH

  The setenvDaVinci command also set up the search path for CMT projects
• Look at the cmt/project.cmt file that was created

  pwd cat cmt/project.cmt

  The setenvDaVinci v19r3 command placed you inside your new CMT project, and created a file that tells CMT that any CMT command issued from inside this directory tree should use the environment defined by the DAVINCI_v19r3 project, which is found somewhere on the CMTPROJECTPATH (look for it!).

  If you want to work with a different application, or with a different version of DaVinci, simply re-issue the setenv<Project> command, e.g. setenvBrunel v31r7. Information about CMT projects, including tips and tricks for working with them, can be found here

• Create a new package in your DaVinci_v19r3 project:

  cd ~cmtuser/DaVinci_v19r3 mkdir -p MyGroup/MyPackage/v1r0/cmt cd MyGroup/MyPackage/v1r0/cmt

• Create a cmt requirements file

  emacs requirements &

  Look at the generated file and add the line

  use GaudiAlg v*

  Save the file. The use statement that you added tells CMT that the files in this package will require to compile and link against files in the GaudiAlg package.
• Try some basic CMT commands:

  cmt show uses ls ls .. cmt config ls ls ..

• Try creating some different types of files with emacs. Each time, make a small modification to the file and save it

  emacs ../doc/release.notes & emacs ../src/MyAlg.h & emacs ../src/MyAlg.cpp & emacs ../src/MyPackage_dll.cpp &

• Use CMT to compile the files you just created, and make a library out of them

  gmake ls .. ls ../slc4_ia32_gcc34 ls ../$CMTCONFIG

• Similarly, make the debug version of the library (i.e. containing debug symbols for use with a debugger)

  setenv CMTCONFIG $CMTDEB gmake ls .. ls ../slc4_ia32_gcc34_dbg ls ../$CMTCONFIG

Exercise 4

Running LHCb Applications

Exercise 5

Exercise 6

Exercise 7

Exercise 8

-- MarcoCattaneo - 18 Jul 2007