Central ILCSOFT builds at CERN

Full GitHub installation via CVMFS

Download iLCInstall

git clone git@github.com:iLCSoft/iLCInstall.git

Initialize necessary tools

cd iLCInstall
source builds/init_x86_64.sh

Set proper path where you want to install by editing builds/release-versions-HEAD.py and set varibles

ilcsoft_install_prefix = '__path_where_to_install__'

Run the installation

./ilcsoft-install builds/release-ilcsoft-fullgit.cfg -i

You might be missing some packages on your local machine, if this is the case execute this

sudo yum install expat-devel giflib giflib-devel git libSM-devel libX11-devel libXext libXext-devel libXft libXft-devel libXmu libXmu-devel libXpm-devel libjpeg-turbo-devel libpng-devel libtiff libtiff-devel mesa-libGLU mesa-libGLU-devel subversion zlib-devel freeglut freeglut-devel which wget bzip2 glibc-devel glibc-utils glibc glibc-headers glibc-static glibc-common libstdc++-devel libstdc++ rsync make openssh-server openssh-clients tar nano openmotif-devel sshpass

Introduction (AFS style deprecated)

In principle, full ILCSOFT releases are regularly built at DESY and deployed via cvmfs. However, due to the rapid software development and daily changes to core software, it is necessary to frequently build HEAD installations locally and CERN and accessible to everyone over afs, in addition to the tagged ILCSOFT releases. Especially for changes to core software like dd4hep and lcio, one needs to recompile the whole of ILCSOFT when developing their own package. For example, when developing DDMalrinPandora, which depends on DD4hep, one cannot simply link DDMArlinPandora against a private check-out of dd4hep combined with a central ILCSOFT installation, since dd4hep is also a dependency of other core packages, like for example MarlinDD4hep (which initializes the geometry centrally for all Marlin Processors). From experience, when the code will usually compile but running e..g Marlin with DDMarlinPandora will result in a segmentation fault.

The builds are generally deployed under:

/afs/cern/eng/clic/work/ilcsoft

To facilitate the procedure, a service account was created ( clidpsw ) with access to an 8-core VM on the CERN cloud ( ddoshepvm00 ). The builds are performed using the ilcsoft-install package which is checked-out at:

/afs/cern.ch/eng/clic/work/ilcinstall-trunk

with the configuration script:

/afs/cern.ch/eng/clic/work/ilcinstall-trunk/releases/HEAD/minimal-ilcsoft.cfg

which is a modified version of the HEAD installation script, release-ilcsoft.cfg provided by the package. As the name suggests, the installation is "minimal" in the sense that it omits many legacy packages that are no longer used in the new reconstruction software or not used for the needs of the CLIC detector and physics study. The differences can readily be seen by using diff. These include:

• Not installed packages: Mokka, PathFinder, MarlinTPC, BBQ, Druid, SLIC and SlicPandora (with other supporting packages) , CondDBMySQL, old Testbeam packages, etc
• Modifications to packages: Disable linking to missing packages, avoid needs to build MySQL etc
• Allow make to run with -j (use 8 cores)
• Use local installation of ROOT and Python: this was necessitated by a hack in some of $ROOTSYS/graf3d/eve/src/TEve*GL.cxx files, where the flag fDLCache = false; had to manually be enabled. Until this fix makes its way into an official ROOT build, we have to use this local installation, otherwise the DD4hep teveDisplay crashes (even over ssh) on intel graphics cards running on Ubuntu and other OSes. Note: We have also modified release-versions-HEAD.py to accommodate these changes! This private build is located at /afs/cern.ch/eng/clic/work/ilcsoft/root/5.34.34.

Procedure to make a new HEAD installation

The default configuration is to build the installation under:

/afs/cern.ch/eng/clic/work/ilcsoft/HEAD-YYYY-MM-DD

where YYYY-MM-DD is today's date. if the directory exists, the ilcsoft-install script will try to resume the installation in that directory, adding missing packages and/or rebuilding specific packages if so requested (see below). For reasons of space shortage under the /afs/cern.ch/eng/clic/work, we often build the installation under the clicdpsw account's public workspace (100 GB):

/afs/cern.ch/work/c/clicdpsw/public/ilcsoft

An easy way to do this is to make a directory in the public area, naming it HEAD-YYYY-MM-DD and then place a symbolic link in the regular installation area. In that way, the installation will "resume" from an empty directory.

To start the installation, one simply has to login to the build machine:

ssh clicdpsw@ddoshepvm00

Assuming you want to install at the regular common workspace (if there's enough space), one simply changes to the ilcsoft-install checkout directory:

cd /afs/cern.ch/eng/clic/work/ilcinstall-trunk/

from which one can issue the command:

./ilcsoft-install releases/HEAD/minimal-ilcsoft.cfg

An installation summary with all the package paths and other information will be printed out. If all looks ok, you can issue the actual installation command (notice -i at the end):

./ilcsoft-install releases/HEAD/minimal-ilcsoft.cfg -i

The installation will begin by checking out all the packages first and then start the compilation. The whole procedure should take about 30 minutes, at the end of which the following message should be shown:

****************************** Finished installation ******************************

&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&
&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&

The installation can readily be used by anyone with a zf account. Login to your usual workstation and issue:

/afs/cern.ch/eng/clic/work/ilcsoft/HEAD-YYYY-MM-DD/init_ilcsoft.sh

Rebuilding specific packages in an existing installation with or without checking out an updated version

The easiest way to force a package to be rebuilt is to delete the directory from the installation. This way, when you launch the installation procedure again, assuming that the date is the same, the script will check out a new version from svn. If you want to target a particular installation (for example if the date has changed and for some reason you prefer to modify an older installation, say to save time), you will need to modify minimal-ilcsoft.cfg changing the ilcsoft_install_dir variable to the full string path of the targeted installation.

If for any reason you need to locally modify a package (not recommended) and recompile, you can always cd to the package build directory and issue make -j install to rebuild the package with any changes. Often, however, one needs to completely remove the build directory contents in which case the cmake configuration is lost. A way to recover this is to use the ilcinstall script after appropriately modifying the configuration file, in this case minimal-ilcsoft.cfg. For example, to rebuild the module PandoraPFANew, add the following line after the module installation line:

#the next line should be there already
ilcsoft.install( PandoraPFANew( PandoraPFANew_version ))
#add the following line or set to 'True'
ilcsoft.module("PandoraPFANew").rebuild = True

This will force the rebuild of the package when you start the installation via the command:

./ilcsoft-install releases/HEAD/minimal-ilcsoft.cfg -i

-- NikiforosNikiforou - 2016-02-03