Under Construction

In the most simple case, you need only to refer to the Quick Start Guide, this twiki explains in details how to manually compile and run the application.

A Public Geant4 Standalone Benchmark

This twiki describes the efforts of the Geant4 collaboration to provide an easy-to-use, self-contained, multi-platform application that can be used to compare the performances of different hardware systems with a typical simulation workload based on a LHC MonteCarlo simulation.

Quick Start Guide: All you need to know

Open a terminal, and type:

$ git clone https://gitlab.cern.ch/adotti/Geant4HepExpMTBenchmark.git
$ cd Geant4HepExpMTBenchmark
#Replace 20 with the number of parallel jobs you want to use
$ ./bootstrap.sh -x -m -j 20 default
$ ls 
#a new directory appears
$ cd <output-directory>
$ ./run.sh 2 10

A job simulating with 2 threads 10 particles in a LHC detector.

The rest of this twiki explains some details on the installation process and you need it only in some non-standard cases.

Introduction

The primary goal of this project is to allow a very easy way of building and running a semi-realistic Geant4 application based on one of the LHC detectors geometry. Courtesy of the ATLAS and CMS experiments a simplified view of their detector geometries is used to perform Geant4 simulations that can be used to perform simple performance monitoring of a Geant4 applications. The benchmark is self contained: all software needed to compile and run the application is distributed with the benchmark itself.

The benchmark is based on a Geant4 application that simulates the interaction of high-energy particles in the CMS detector. All physics processes of a Geant4 application are enabled and tracking in a uniform 4T magnetic field is included. No I/O of hits is implemented (yet). The application, called HepExpMT, has been presented at IEEE/NSS-MIC 2015 Conference by Steven Farrell and Andrea Dotti (proceedings TBP, arXiv).

To simplify the installation, compilation and execution of the test a self-contained tarball is provided that contains all needed software. The included software dependencies are:

• cmake, to configure the projects (as downloaded from https://cmake.org/)
• A patched version of Xerces-C (as downloaded from http://xerces.apache.org/, the patch extends the maximum number of nodes allowed in the XML file)
• Geant4 itself (as downloaded from http://www.geant4.org/)

Reference Manual: Installation

The bootstrap.sh script can be used to install and compile the application. During the installation process code and data will be downloaded (the source code is 45MB, the data-files are much larger). It is possible to disable automatic download of code, in such a case the needed source code and data-libraries should be already present on the host.

Prerequisites

The compilation of Geant4 requires a C++11 compatible compilers. Please refer to Geant4 Installation Guide for more information. Geant4 uses cmake in the configuration step (minimum version is 3.3). A recent cmake version is included in the source code tarball. Finally Xerces-C xml parser is needed to read the experimental geometry. Also this dependency is included in the source code tarball.

The script uses curl command to download the code. You can always manually download from the web the code tarball and avoid this step (see here).

Automatic installation

It is recommended to use the bootstrap.sh installation script to perform an automatic installation. The final size of the installation area is about 4GB.

To perform an automatic installation:

$ git clone https://gitlab.cern.ch/adotti/Geant4HepExpMTBenchmark.git
$ cd Geant4HepExpMTBenchmark

Type ./bootstrap.sh -h for help on script option. To speed up the compilation it is suggested to use the option -j N with N the number of parallel compilation jobs to use. If you want to compile the included cmake add the option -m and to compile the included Xerces-C add -x.

The script uses the system compiler, unless CXX and CC environment variables are set.

So to compile everything, do not worry about dependencies and use 10 parallel compilation jobs:

$ ./bootstrap.sh -m -x -j 10 default

Some advanced use-cases

In some cases you do not want to let the script download everything automatically or you want a finer control on the system. Here are some common use cases.

• Manual download of the large self-contained tarball, compile included Xerces-C and cmake: download the benchmark tarball and place it in the working directory. Instruct the script to use the local tarball with: ./bootstrap.sh -x -m -i <tarballname> [... other options ...]. Installation script will still download data-bases from Geant4 website automatically.
• Manual download Geant4 and application, compile using system-installed Xerces-C and cmake: download Geant4 tarball from Geant4 website; get via git the application, create a tarball out of it: tar czf G4App-src-Current.tar.gz HepExpMT and use system installed cmake and Xerces-C: ./bootstrap.sh -i <g4tarball> -i G4App-src-Current.tar.gz default. Installation script will still download data-bases from Geant4 website automatically.
• As before but in addition do not download automatically Geant4 data-bases: Get databases from Geant4 website, unpack them somewhere add to previous option: ./bootstrap.sh -i <g4tarball> -i G4App-src-Current.tar.gz -DGEANT4_INSTALL_DATADIR=/where/DBs/are default
• Specify Xerces-C system installation path: ./bootstrap.sh -DXERCESC_ROOT_DIR=some/dir [... other options...]
• Do not download application, just compile in a preexisting directory (for example, to recompile the existing code): ./bootstrap.sh [...] -o <existing-dir>

Manual installation (do not use bootstrap script)

This is not a complete manual, only the most common options are shown. The bootstrap script is designed to be used by novice users or by people not interested in the details of Geant4 configuration and installation. Assuming you have cmake (>3.3) installed and Xerces-C installed these are the step to manually compile the application:

1. Download Geant4
2. Download the Geant4 data-bases (see Geant4 download area) and install them somewhere
3. Download the application
4. Compile Geant4, instructions are in the Geant4 Installation Guide. Remember to activate multi-threading and GDML support. This should work in most cases: cmake -DGEANT4_INSTALL_DATA=/some/dir/where/G4DBs/are -DGEANT4_BUILD_MULTITHREADED=ON -DGEANT4_USE_GDML=ON -DCMAKE_INSTALL_PREFIX=/dir/where/installation/goes -DBUILD_SHARED_LIBS=OFF -DBUILD_STATIC_LIBS=ON /dir/where/g4/sources/are && make -j <N> && make install = . Add -DXERCESC_ROOT_DIR=some/dir if Xerces-C is not installed in a location where cmake can find it automatically.
5. Compile the application: cmake -DCMAKE_INSTALL_PREFIX=/dir/where/installation/goes -DGeant4_DIR=/dir/where/installation/goes/lib[64]/Geant4* /dir/where/HepExpMT/sources/are && make -j <N> && make install

Compilation for non x86 architectures

Xeon Phi (KNC)

To compile application for Xeon Phi via intel compiler a toolchain cmake file must be provided, see XeonPhiSupport Geant4 twiki page for an example of its content. Provided this file is available, the benchmark is compiled with: cmake -c icc -p <fullpat-to-cmake-toolchainfile> -DXERCESC_INCLUDE_DIR=$PWD/<odir>/include -DXERCESC_LIBRARY=$PWD/<odir>/lib/libxerces-c.a [-x -m -j...] default Due to a limitation in the script, the XERCESC_LIBRARY options must be passed pointing to the correct directory where the MIC compiled Xerces-C library and code is located. Note that if using the -x directory option the directory will be created by the script itself. It is important to use full paths and not relative paths for these variables. Even if cross-compiling for MIC, by default the auto-created output directory is called x86_64-Linux-icc-<VERSION>-Release-static.

Xeon Phi (KNL)

It is suggested to compile the application natively on a KNL node. In such a case there is no need to have any special configuration.

ARM

PowerPC

MPI support

To manually compile with MPI support you need to perform the following operations:

1. compile Geant4 as usual (see Manual Installation) using your set of cmake options.
2. compile Geant4 MPI interface library (available under <g4src>/examples/extended/parallel/MPI/source) with cmake -DGeant4_DIR=<g4-installation-dir>/lib[64]/Geant4* -DCMAKE_INSTALL_PREFIX=/dir/where/installation/goes -DBUILD_SHARED_LIBS=OFF -DBUILD_STATIC_LIBS=ON <g4-installation-dir>/share/Geant4*/examples/extended/parallel/MPI/source && make -j <N> && make install
3. compile the application enabling MPI: cmake -DCMAKE_INSTALL_PREFIX=/dir/where/installation/goes -DGeant4_DIR=/dir/where/installation/goes/lib[64]/Geant4* -DG4mpi_DIR=/dir/where/installation/goes/lib[64]/G4mpi* -DWITH_MPI=ON /dir/where/HepExpMT/sources/are && make -j <N> && make install

Use mpirun or your system mpi wrapper to execute the application, with the macro file mpi.mac, for example: mpirun -n 2 HepExpMT mpi.mac cms.gdml 0 0 0, run from the HepExpMT subdirectory in the installation area, modify the mpi.mac script to set the total number of events to be used at the ui command: /mpi/beamOn. Remember to source the <g4-installation-dir>/bin/geant4.[c]sh script before running the application.

Results

-- AndreaDotti - 2016-04-28