Under Construction
In the most simple case, you need only to refer to the
Quick Start Guide, this twiki explains in details how to manually compile and run the application.
A Public Geant4 Standalone Benchmark
This twiki describes the efforts of the Geant4 collaboration to provide an easy-to-use, self-contained, multi-platform application that can be used to compare the performances of different hardware systems with a typical simulation workload based on a LHC MonteCarlo simulation.
Quick Start Guide: All you need to know
Open a terminal, and type:
$ git clone https://gitlab.cern.ch/adotti/Geant4HepExpMTBenchmark.git
$ cd Geant4HepExpMTBenchmark
#Replace 20 with the number of parallel jobs you want to use
$ ./bootstrap.sh -x -m -j 20 default
$ ls
#a new directory appears
$ cd <output-directory>
$ ./run.sh 2 10
A job simulating with 2 threads 10 particles in a LHC detector.
The rest of this twiki explains some details on the installation process and you need it only in some non-standard cases.
Introduction
The primary goal of this project is to allow a very easy way of building and running a semi-realistic Geant4 application based on one of the LHC detectors geometry. Courtesy of the ATLAS and CMS experiments a simplified view of their detector geometries is used to perform Geant4 simulations that can be used to perform simple performance monitoring of a Geant4 applications. The benchmark is self contained: all software needed to compile and run the application is distributed with the benchmark itself.
The benchmark is based on a Geant4 application that simulates the interaction of high-energy particles in the
CMS
detector. All physics processes of a Geant4 application are enabled and tracking in a uniform 4T magnetic field is included. No I/O of
hits is implemented (yet).
The application, called HepExpMT, has been presented at
IEEE/NSS-MIC 2015 Conference
by Steven Farrell and Andrea Dotti (proceedings TBP,
arXiv
).
To simplify the installation, compilation and execution of the test a self-contained tarball is provided that contains all needed software. The included software dependencies are:
Reference Manual: Installation
The
bootstrap.sh script
can be used to install and compile the application. During the installation process code and data will be downloaded (the source code is 45MB, the data-files are much larger). It is possible to disable automatic download of code, in such a case the needed source code and data-libraries should be already present on the host.
Prerequisites
The compilation of Geant4 requires a C++11 compatible compilers. Please refer to
Geant4 Installation Guide
for more information. Geant4 uses
cmake
in the configuration step (minimum version is 3.3). A recent cmake version is included in the source code tarball. Finally
Xerces-C xml parser
is needed to read the experimental geometry. Also this dependency is included in the source code tarball.
The script uses
curl
command to download the code. You can always manually download from the web the code tarball and avoid this step (see
here).
Automatic installation
It is recommended to use the
bootstrap.sh
installation script to perform an automatic installation.
The final size of the installation area is about 4GB.
To perform an automatic installation:
$ git clone https://gitlab.cern.ch/adotti/Geant4HepExpMTBenchmark.git
$ cd Geant4HepExpMTBenchmark
Type
./bootstrap.sh -h
for help on script option. To speed up the compilation it is suggested to use the option
-j N
with
N
the number of parallel compilation jobs to use.
If you want to compile the included cmake add the option
-m
and to compile the included Xerces-C add
-x
.
The script uses the system compiler, unless CXX and CC environment variables are set.
So to compile everything,
do not worry about dependencies and use 10 parallel compilation jobs:
$ ./bootstrap.sh -m -x -j 10 default
Some advanced use-cases
In some cases you do not want to let the script download everything automatically or you want a finer control on the system. Here are some common use cases.
- Manual download of the large self-contained tarball, compile included Xerces-C and cmake: download the benchmark tarball
and place it in the working directory. Instruct the script to use the local tarball with: ./bootstrap.sh -x -m -i <tarballname> [... other options ...]
. Installation script will still download data-bases from Geant4 website automatically.
- Manual download Geant4 and application, compile using system-installed Xerces-C and cmake: download Geant4 tarball
from Geant4 website; get via git the application
, create a tarball out of it: tar czf G4App-src-Current.tar.gz HepExpMT
and use system installed cmake and Xerces-C: ./bootstrap.sh -i <g4tarball> -i G4App-src-Current.tar.gz default
. Installation script will still download data-bases from Geant4 website automatically.
- As before but in addition do not download automatically Geant4 data-bases: Get databases from Geant4 website, unpack them somewhere add to previous option:
./bootstrap.sh -i <g4tarball> -i G4App-src-Current.tar.gz -DGEANT4_INSTALL_DATADIR=/where/DBs/are default
- Specify Xerces-C system installation path:
./bootstrap.sh -DXERCESC_ROOT_DIR=some/dir [... other options...]
- Do not download application, just compile in a preexisting directory (for example, to recompile the existing code):
./bootstrap.sh [...] -o <existing-dir>
Manual installation (do not use bootstrap script)
This is not a complete manual, only the most common options are shown. The bootstrap script is designed to be used by novice users or by people not interested in the details of Geant4 configuration and installation.
Assuming you have cmake (>3.3) installed and Xerces-C installed these are the step to manually compile the application:
- Download Geant4
- Download the Geant4 data-bases (see Geant4 download area) and install them somewhere
- Download the application
- Compile Geant4, instructions are in the Geant4 Installation Guide
. Remember to activate multi-threading and GDML support. This should work in most cases: cmake -DGEANT4_INSTALL_DATA=/some/dir/where/G4DBs/are -DGEANT4_BUILD_MULTITHREADED=ON -DGEANT4_USE_GDML=ON -DCMAKE_INSTALL_PREFIX=/dir/where/installation/goes -DBUILD_SHARED_LIBS=OFF -DBUILD_STATIC_LIBS=ON /dir/where/g4/sources/are && make -j <N> && make install =
. Add -DXERCESC_ROOT_DIR=some/dir
if Xerces-C is not installed in a location where cmake can find it automatically.
- Compile the application:
cmake -DCMAKE_INSTALL_PREFIX=/dir/where/installation/goes -DGeant4_DIR=/dir/where/installation/goes/lib[64]/Geant4* /dir/where/HepExpMT/sources/are && make -j <N> && make install
Compilation for non x86 architectures
Xeon Phi (KNC)
To compile application for Xeon Phi via intel compiler a toolchain cmake file must be provided, see
XeonPhiSupport Geant4 twiki page for an example of its content. Provided this file is available, the benchmark is compiled with:
cmake -c icc -p <fullpat-to-cmake-toolchainfile> -DXERCESC_INCLUDE_DIR=$PWD/<odir>/include -DXERCESC_LIBRARY=$PWD/<odir>/lib/libxerces-c.a [-x -m -j...] default
Due to a limitation in the script, the
XERCESC_LIBRARY
options must be passed pointing to the correct directory where the MIC compiled Xerces-C library and code is located. Note that if using the
-x
directory option the directory will be created by the script itself. It is important to use full paths and not relative paths for these variables. Even if cross-compiling for MIC, by default the auto-created output directory is called
x86_64-Linux-icc-<VERSION>-Release-static
.
Xeon Phi (KNL)
It is suggested to compile the application natively on a KNL node. In such a case there is no need to have any special configuration.
ARM
MPI support
To manually compile with MPI support you need to perform the following operations:
- compile Geant4 as usual (see Manual Installation) using your set of cmake options.
- compile Geant4 MPI interface library (available under
<g4src>/examples/extended/parallel/MPI/source
) with cmake -DGeant4_DIR=<g4-installation-dir>/lib[64]/Geant4* -DCMAKE_INSTALL_PREFIX=/dir/where/installation/goes -DBUILD_SHARED_LIBS=OFF -DBUILD_STATIC_LIBS=ON <g4-installation-dir>/share/Geant4*/examples/extended/parallel/MPI/source && make -j <N> && make install
- compile the application enabling MPI:
cmake -DCMAKE_INSTALL_PREFIX=/dir/where/installation/goes -DGeant4_DIR=/dir/where/installation/goes/lib[64]/Geant4* -DG4mpi_DIR=/dir/where/installation/goes/lib[64]/G4mpi* -DWITH_MPI=ON /dir/where/HepExpMT/sources/are && make -j <N> && make install
Use
mpirun
or your system mpi wrapper to execute the application, with the macro file
mpi.mac
, for example:
mpirun -n 2 HepExpMT mpi.mac cms.gdml 0 0 0
, run from the
HepExpMT
subdirectory in the installation area, modify the
mpi.mac
script to set the total number of events to be used at the ui command:
/mpi/beamOn
. Remember to source the
<g4-installation-dir>/bin/geant4.[c]sh
script before running the application.
Results
--
AndreaDotti - 2016-04-28