Introduction

CMake is an Open Source, cross-platform configuration and build tool used in several projects around the world.

Among its advantages we can count the support that comes from a large user base and the CMake configuration language that doubles as a portable and powerful scripting language.

The main drawbacks of CMake are the syntax of its language (not very nice looking) and the lack of support for runtime environment definition, but, thanks to the power of the language and to some home-made Python tools, we can overcome the drawbacks.

Lately we have been developing a CMake-based infrastructure to allow building of Gaudi-based projects is a way similar to what we are used to with CMT. Here I'll describe how to write the CMake configuration for projects and packages. See also the CMakeFAQ

Understanding and Writing CMake Code

A minimal introduction on the CMake syntax and conventions is mandatory for people who never used it. Info The syntax highlighting in the code blocks is not correct because SyntaxHighlightingPlugin does not understand the CMake language.

In CMake every statement is a function call, written as an identifier followed by optional spaces and the arguments to the function as a space-separated list enclosed in parentheses. The arguments to a function can span over several lines, and double quotes (") can be used to pass arguments containing spaces and new-lines. Examples of functions:

message("hello world!")

if(1 GREATER 0)
  message("this is true")
else()
  message("this is false")
endif()

One can add comments to the code using # at the beginning of the comment text (spaces preceding # are ignored), like, e.g., in Unix shells and Python:

# I'm a comment
if(TRUE)
     # this is always printed
     message("it's true")
else()
     # this is never printed
     message("it's false")
endif()

The CMake language supports variables, which are set with the function set and dereferenced with ${...}, e.g.:

set(MyMessage "hello world")
message(${MyMessage})

Dereferencing of variables can be nested and works also in between double quotes:

set(name MyName)
set(${name}_msg "This is ${name}")
message(${${name}_msg})

There is something special in the way CMake functions are invoked. CMake functions and macros (similar to functions) accept variable number of arguments and the only separators between arguments are spaces (tabs and new-lines too), so it might not be obvious how to pass optional arguments (like it's done in Python with named arguments). The solution found by CMake developers is to use keyword-separated lists of arguments. For example, we can imagine a function that requires a mandatory argument name and two optional lists of files, C sources and C++ sources. In CMake you could find it invoked like this:

make_a_library(MyLibrary)

make_a_library(MyLibrary C_SOURCES file1.c file2.c file3.c)

make_a_library(MyLibrary CXX_SOURCES file1.cpp file2.cpp file3.cpp)

make_a_library(MyLibrary
               C_SOURCES file1.c file2.c file3.c
               CXX_SOURCES file1.cpp file2.cpp file3.cpp)

Warning, important Function names are case-insensitive, but variable names are case sensitive as well as string comparison.

More details can be found at http://www.cmake.org/Wiki/CMake/Language_Syntax.

CMake Configuration of a Gaudi-based Project

The configuration of a project is divided in two main concerns:
  • project configuration
  • subdirectory (package) configuration

Project Configuration

At the top level directory of a project there must be two files:
  • toolchain.cmake
  • CMakeLists.txt

The first one (toolchain.cmake) is generic (same content for all projects) and is usually generated by getpack, except for a few cases. The code is available at https://svnweb.cern.ch/trac/lhcb/browser/LbScripts/trunk/LbUtils/data/toolchain.cmake.

The second one (CMakeLists.txt) is the main configuration file. It starts with some boilerplate code needed for bootstrapping:

    1CMAKE_MINIMUM_REQUIRED(VERSION 2.8.5)
    2
    3#---------------------------------------------------------------
    4# Load macros and functions for Gaudi-based projects
    5find_package(GaudiProject)
    6#---------------------------------------------------------------
    7
line 1
mandatory declaration to ensure that the version of CMake being used is recent enough
line 5
load the Gaudi CMake modules and functions used later in this file and in the configuration of the packages
Empty lines and lines starting with # are ignored.

This preamble is followed by the function call that declares the project name and version, the dependencies on other projects and the required data packages with an optional version pattern. For example:

    1# Declare project name and version
    2gaudi_project(MyProject v36r4
    3              USE BaseProject v24r2
    4                  AnotherBase v5r15
    5              DATA Some/DataPkg
    6                   SpecialDatPkg VERSION v7r*
    7                   MoreData)
line 2
we call the function gaudi_project passing as first two arguments the project name and the version (in the format vXrY[pZ] or the special value HEAD), Note that the following lines are all arguments passed to gaudi_project, until the ) character.
line 3
we start the list of the dependencies on other projects with the keyword USE; the projects have to be specified as a list of pairs name and version (e.g. USE Lbcom v13r6p1 Rec v14r6)
line 5
the keyword DATA defines the start of the optional list of required data packages; each data package is specified by its name, optionally followed by the keyword VERSION and a glob pattern used to select the allowed versions (if the VERSION is not specified, we assume any version)

The current project (and all the subdirectrories it contains) will have access to all the libraries built in the projects it depends on and will have access to the data packages required (via the environment variables defined there).

When using the getpack tool to check out the project from the software repository, you will also find a file called Makefile that simplifies the interaction with CMake.

Subdirectory Configuration

A subdirectory (also called package) is a directory inside the project top directory that contains a configuration file (CMakeLists.txt) containing build instructions and the source files used in the build, header files or scripts to be installed, etc.

All the configuration files of the subdirectories should start with the declaration of the version of the package with something like (taken from Kernel/LHCbKernel):

################################################################################
# Package: LHCbKernel
################################################################################
gaudi_subdir(LHCbKernel v14r3)
Where we call the function gaudi_subdir passing to it the name of the subdirectory/package and its version.

After the declaration of the version we can optionally declare dependencies on other subdirectories with a call to gaudi_depends_on_subdirs, passing to it the list of packages that should be build before this one:

gaudi_depends_on_subdirs(GaudiAlg
                         GaudiKernel
                         GaudiObjDesc
                         Kernel/LHCbMath)
Warning, important gaudi_depends_on_subdirs is only needed to hint to CMake about the order in which it should consider the subdirectories, but this is (in principle) not needed and will be dropped as soon as the correct dependencies discovery will be in place (see LBCORE-226).

What usually follows is a list of external software packages we need to build the targets in the current subdirectory (e.g. ROOT, Boost, etc.). While the functions encountered so far are Gaudi-specific (as the gaudi_ prefix suggests), the code to use external packages is standard CMake code:

find_package(ROOT)
find_package(Boost COMPONENTS system)
The calls to the function find_package are described in the CMake documentation, but, briefly, the lines above tell CMake to find ROOT headers and libraries (line 1) and the headers and the system library of Boost (line 2). If the calls succeed (at configuration time), CMake sets some variables to the locations of required headers and libraries.
Warning, important A call to find_package does not mean that the external package is actually used during the build: it must be explicitly mentioned when declaring the build target.

The rest of the configuration file depends on what the subdirectory contains and/or needs to build (a library, a module, Python modules, headers,...). In the following sections are described the most common use cases.

Building a Library (AKA _linker library)

A library is a container of code that can be used by other libraries, applications or modules (see following sections).

The declaration that a subdirectory provides for a library looks more or less like this:

gaudi_add_library(MyCommonCode
                  src/*.cpp
                  PUBLIC_HEADERS MyHeaders
                  INCLUDE_DIRS Boost ROOT
                  LINK_LIBRARIES Boost ROOT RecEvent TrackEvent GaudiAlgLib)

The first argument of the gaudi_add_library function is the name of the library and must be unique in the current project and in all the projects it depends on. The arguments that follow the name, is a list of source files (glob patterns are allowed) that must be compiled in the module.

A library must expose header files to the source files of other build targets, which is done specifying the list of directories containing the headers after the keyword PUBLIC_HEADERS. These directories will be also installed after the build (in InstallArea//include), so that they are visible to other projects. More... Close In the very unlikely case we need to produce a library that does not need to export headers we can replace the PUBLIC_HEADERS section with just the keyword NO_PUBLIC_HEADERS.

Optionally, one can specify a list of include directories (after the INCLUDE_DIRS keyword) and of libraries (after LINK_LIBRARIES) needed to compile and link the module. INCLUDE_DIRS accepts paths to directories and names of external packages (found with calls to find_package) which are automatically converted to the list of include directories. LINK_LIBRARIES, similar to INCLUDE_DIRS, accepts paths to libraries and names of external packages, plus names of libraries built in other subdirectories of the current (or required) projects, in which case the other subdirectories must be specified in the gaudi_depends_on_subdirs call. It should be noted that when linking against a library produced in a subdirectory of a Gaudi project, the include directories required by that library are added automatically to the include path (without having to specify them in the INCLUDE_DIRS section). For example, in the code:

gaudi_add_library(Lib1
                  src/lib1/*.cpp
                  PUBLIC_HEADERS Lib1Hdrs
                  INCLUDE_DIRS Boost ROOT
                  LINK_LIBRARIES Boost ROOT)
gaudi_add_library(Lib2
                  src/lib2/*.cpp
                  PUBLIC_HEADERS Lib2Hdrs
                  LINK_LIBRARIES Lib1)
the library Lib2 will also have access to ROOT and Boost headers and libraries (even if the libraries are declared in different subdirectories).

Building a Module (AKA _component library)

A module is a container of components of the Gaudi Plugin System, and probably is the most common build target in Gaudi projects (technically speaking it is a shared library that is loaded at run-time).

The declaration that a subdirectory provides for a module is quite similar to that of a library:

gaudi_add_module(MyAnalysisAlgs
                 src/*.cpp
                 INCLUDE_DIRS Boost ROOT
                 LINK_LIBRARIES Boost ROOT RecEvent TrackEvent GaudiAlgLib)
where the function is now gaudi_add_module, to which we cannot pass the keyword PUBLIC_HEADERS.

It should be taken into account that there are other differences between a library and a module. In particular:

  • a module cannot be used in the LINK_LIBRARIES section of a target declaration
  • the plugin system of Gaudi is not able to find and load components from libraries

Installing Headers (without a library)

Sometimes a subdirectory provides public headers but not a library (e.g. for interfaces, purely inlined classes, etc.). In this case we cannot install the headers directory via the PUBLIC_HEADERS argument of gaudi_add_library, but we can use the function gaudi_install_headers:
gaudi_install_headers(MyHeaders)
where MyHeaders is the directory containing the headers, like in the case of PUBLIC_HEADERS.

Using GaudiObjDesc (LHCb-specific)

Subdirectories using GaudiObjDesc to generate classes from XML files need to first load the GaudiObjDesc CMake functions:
# the subdirectory GaudiObjDesc shuold be loaded before this one
gaudi_depends_on_subdirs(GaudiObjDesc)
# load the GaudiObjDesc CMake module
include(GaudiObjDesc)
then, assuming that the XML files are in the directory xml, the headers should be created in the directory Event and that we want the dictionaries for the generated classes, we can use a command like:
god_build_dictionary(xml/*.xml
                     HEADERS_DESTINATION Event
                     LINK_LIBRARIES GaudiKernel LHCbKernel)

There are other possible use cases in which we use GaudiObjDesc:

  • generate public headers, but no dictionary
god_build_headers(xml/*.xml DESTINATION Event)

  • generate private headers
god_build_headers(xml/*.xml PRIVATE)

  • generate public headers and a dictionary extending existing dictionary files (example from Kernel/LHCbKernel)
god_build_dictionary(xml/*.xml
                     HEADERS_DESTINATION Kernel
                     EXTEND dict/LHCbKernelDict.h
                     INCLUDE_DIRS ROOT Boost
                     LINK_LIBRARIES ROOT Boost GaudiKernel GaudiObjDescLib LHCbMathLib LHCbKernel)

Info For compatibility with the old build system the following two uses are equivalent:

  • new style
god_build_dictionary(xml/*.xml
                     HEADERS_DESTINATION Event
                     LINK_LIBRARIES GaudiKernel LHCbKernel)
  • old style
god_build_headers(xml/*.xml DESTINATION Event)
god_build_dictionary(xml/*.xml
                     HEADERS_DESTINATION Event
                     LINK_LIBRARIES GaudiKernel LHCbKernel)

Installing Python Modules

When a subdirectory contains a python directory with Python modules that should be installed to be accessible to other subdirectories and projects, the CMakeLists.txt file should include the line:
gaudi_install_python_modules()

Installing Scripts

Similar to the case of Python modules, scripts in the directory scripts can be installed with the dedicated command:
gaudi_install_scripts()

Declaring Tests (QMTest)

When a subdirectory contains the directory tests/qmtest with QMTest tests (see GaudiTestingInfrastructure), CMake should be informed with the line:
gaudi_add_test(QMTest QMTEST)

Info Other types of tests are available, but at the moment QMTest is the most powerful option.

Building ROOT Dictionaries

Either to use them for persistency or interactivity purposes, the creation of ROOT dictionaries can be requested with code like (from GaudiKernel):
gaudi_add_dictionary(GaudiKernel dict/dictionary.h  dict/dictionary.xml
                     LINK_LIBRARIES GaudiKernel) 

Building Executables

It is relatively uncommon in a Gaudi project to produce executables, but it can be done with the command gaudi_add_executable which uses the same arguments of gaudi_add_module, as in
gaudi_add_executable(mySpecialCommand
                     src/*.cpp
                     INCLUDE_DIRS Boost
                     LINK_LIBRARIES Boost)
which will build the executable mySpecialCommand.exe.

Manipulating the Environment

Although it should be avoided as much as possible, it might be needed to set or modify environment variables. It can be done with the function gaudi_env:
gaudi_env(SET var1 value2
          PREPEND my_path some_path
          UNSET unneeded_var)
The sub-commands of gaudi_env are:
  • SET
  • UNSET
  • PREPEND
  • APPEND
  • REMOVE

If some changes are needed only at build or test time (i.e. they do not need to be exposed to other projects), one can use gaudu_build_env instead of gaudi_env.

Warning, important "System" variables like PATH and PYTHONPATH should never be modified explicitly. If you need to give access to scripts or Python modules, install them with the dedicated functions.

Building with CMake

Work in progress, under construction Work in Progress Work in progress, under construction (see LBCORE-12 and LBCORE-174)

Building a Package

The tools needed to easily work and build with CMake are not final yet, but there is a preliminary version in $LHCBDEV that can be tested. This shell (bash) script shows the required steps needed to build and test the package Phys/DaVinciTools in the context of the project Phys build in the nightly build slot lhcb-gaudi-head.

# Prepare the local project directory
lb-dev --name MyDevel --nightly lhcb-gaudi-head Phys HEAD
cd MyDevel

#
# An alternative way is:
#
# lb-dev --nightly lhcb-gaudi-head --dest-dir $User_release_area Phys HEAD
# cd $User_release_area/PhysDev_HEAD

# Get the packages to be built
getpack Phys/DaVinciTools trunk

# Build
make -j 6

# Test
make test

# Run Gaudi
./run gaudirun.py

If you need to checkout (getpack) another package after a build, you must call make configure to ensure that CMake finds it.

make purge is also very useful to ensure that everything in the local directory is built from scratch.

If you need to use the prepared build with lb-run, you must call make install to create all the required files in InstallArea.

When using the production version of LbScripts one needs a few manual steps. In this case the same exercise as above looks like this:
# Common environment (usually not needed, but useful in a script)
. LbLogin.sh -c x86_64-slc6-gcc48-opt

# We need the lhcb-cmake nightly build slot
export NB=$LHCBNIGHTLIES/lhcb-cmake/Fri
. $NB/setupSearchPath.sh

# Enable build with CMake
export USE_CMAKE=1
export CMAKE_PREFIX_PATH=$NB/GAUDI/GAUDI_HEAD/cmake:$CMAKE_PREFIX_PATH

# Prepare the local project directory
cd $User_release_area
mkdir MyDevel
cd MyDevel

# Get the packages to be built
getpack --no-config --no-eclipse Phys/DaVinciTools trunk

# CMake configuration of the local project
cp $NB/LHCB/LHCB_HEAD/{toolchain.cmake,Makefile} .
cat > CMakeLists.txt <<EOF
CMAKE_MINIMUM_REQUIRED(VERSION 2.8.5)
find_package(GaudiProject)

gaudi_project(MyDevel HEAD
              USE Phys HEAD)
EOF

# Build
make -j 6

# Test
make test ; make QMTestSummary

# Run Gaudi
build.$CMTCONFIG/run gaudirun.py

Building a Project

To build a whole project is a bit simpler than with a single package (still, improvements can and will be made).
# Common environment (usually not needed, but useful in a script)
. LbLogin.sh -c x86_64-slc6-gcc48-opt

# Enable build with CMake
export USE_CMAKE=1

# Checkout the project
cd $User_release_area
getpack -PH DaVinci
cd DAVINCI/DAVINCI_HEAD

# Edit the dependencies to use the correct versions
emacs CMakeLists.txt

# Build
make -j 6

# Test
make test

# Run Gaudi
build.$CMTCONFIG/run gaudirun.py

-- MarcoClemencic - 13 Dec 2013

Edit | Attach | Watch | Print version | History: r27 < r26 < r25 < r24 < r23 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r27 - 2015-10-21 - MarcoCattaneo
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LHCb All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback