CMake 3.15

Introduction

CMake

Among its advantages we can count the support that comes from a large user base and the CMake configuration language that doubles as a portable and powerful scripting language.

The main drawbacks of CMake are the syntax of its language (not very nice looking) and the lack of support for runtime environment definition, but, thanks to the power of the language, we can overcome the drawbacks.

Here I'll describe how to write the CMake configuration for projects and packages. See also the CMake Documentation.

Understanding and Writing CMake Code

In CMake every statement is a function call, written as an identifier followed by optional spaces and the arguments to the function as a space-separated list enclosed in parentheses. The arguments to a function can span over several lines, and double quotes (") can be used to pass arguments containing spaces and new-lines. Examples of functions:

message("hello world!")

if(1 GREATER 0)
  message("this is true")
else()
  message("this is false")
endif()

One can add comments to the code using # at the beginning of the comment text (spaces preceding # are ignored), like, e.g., in Unix shells and Python:

# I'm a comment
if(TRUE)
     # this is always printed
     message("it's true")
else()
     # this is never printed
     message("it's false")
endif()

The CMake language supports variables, which are set with the function set and dereferenced with ${...}, e.g.:

set(MyMessage "hello world")
message(${MyMessage})

Dereferencing of variables can be nested and works also in between double quotes:

set(name MyName)
set(${name}_msg "This is ${name}")
message(${${name}_msg})

There is something special in the way CMake functions are invoked. CMake functions and macros (similar to functions) accept variable number of arguments and the only separators between arguments are spaces (tabs and new-lines too), so it might not be obvious how to pass optional arguments (like it's done in Python with named arguments). The solution found by CMake developers is to use keyword-separated lists of arguments. For example, we can imagine a function that requires a mandatory argument name and two optional lists of files, C sources and C++ sources. In CMake you could find it invoked like this:

make_a_library(MyLibrary)

make_a_library(MyLibrary C_SOURCES file1.c file2.c file3.c)

make_a_library(MyLibrary CXX_SOURCES file1.cpp file2.cpp file3.cpp)

make_a_library(MyLibrary
               C_SOURCES file1.c file2.c file3.c
               CXX_SOURCES file1.cpp file2.cpp file3.cpp)

Function names are case-insensitive, but variable names are case sensitive as well as string comparison.

CMake Configuration of a Gaudi-based Project

• Set up the build environment (if not already set up)
• Configure the project with CMake (generate Makefile or build.ninja)
• Compile the source files and link the binaries
• Test the previously build binaries (optional but recommended)

Set up the build environment

• BINARY_TAG: the variable that describe the platform
• CMAKE_PREFIX_PATH: is a path-like variable that must contain the list the list of path to:
  • the compiler e.g. g++, clang++ (the compiler may be a wrapper)
  • the build system e.g. make, ninja
  • all third-party dependencies e.g. Boost, ROOT
• PATH (optional): may contain the paths to the compiler and the build system

There are several ways to set these variables:

• use export several time (or run a shell script that will do so)
• source a view (a shell script that sets the aforementioned variables to a directory of symlinks)
• specify a toolchain to the configuration

With export

export BINARY_TAG="x86_64-centos7-gcc8-opt"
export CMAKE_PREFIX_PATH="/path/to/g++:/path/to/boost:/path/to/ROOT:..."

It is also possible to list the call to export in a shell script to be able to source it later on.

With a view

source /cvmfs/sft.cern.ch/lcg/views/LCG_96/x86_64-centos7-gcc8-opt/setup.sh
# the views may lack some stuff
export CMAKE_PREFIX_PATH="$CMAKE_PREFIX_PATH:/cvmfs/sft.cern.ch/lcg/releases/LCG_95/vectorclass/1.30/x86_64-centos7-gcc8-opt:/cvmfs/projects.cern.ch/intelsw/psxe/linux/x86_64/2019/vtune_amplifier"

Sourcing a view is not the same as sourcing a shell script that uses export. A view is a directory of symbolic links and a setup.sh script.

With a toolchain

# Either use
-D CMAKE_TOOLCHAIN_FILE=/path/to/a/toolchain # at configure time when calling cmake
# or
ln -s /path/to/a/toolchain toolchain.cmake # right away

Configuration of the project

# Check CMake version
cmake --version
# if version < 3.15
export PATH="/cvmfs/lhcb.cern.ch/lib/contrib/CMake/3.15.2/Linux-x86_64/bin:$PATH"

The configuration is the step when CMake is called and produces the files for the build system (e.g. make, ninja).

Two directories must be specified:

• the source tree: contains the sources
• the build tree: will contain the outputs of the build

cmake -S . -B build.$BINARY_TAG
# options can be passed at configure time
cmake -S . -B build.$BINARY_TAG -G Ninja # set the build system
cmake -S . -B build.$BINARY_TAG -D GAUDI_USE_AIDA=OFF # enable/disable a third party dependency
cmake -S . -B build.$BINARY_TAG -D CMAKE_BUILD_TYPE=Developer # select a build type (a set of compile and link options)
cmake -S . -B build.$BINARY_TAG -D CMAKE_TOOLCHAIN_FILE=toolchain.cmake # specify a toolchain
# several options may be specified with: -D ... -D ... -D ...
# ccmake and cmake-gui can be used

cmake -LH build.$BINARY_TAG # to see all the options and their help messages

Compilation of the project

cd build.$BINARY_TAG ; make # or ninja or an IDE
# or directly with CMake
cmake --build build.$BINARY_TAG

Test the binaries

cd build.$BINARY_TAG ; ctest -j `nproc` --output-on-failure --schedule-random

The wrapper

make # configure + compile
make test # run the tests

The wrapper uses a toolchain if a file called toolchain.cmake exists in the current directory. It might be useful to have a shell function to easily switch from one toolchain to another.

function switch_platform
{
    export BINARY_TAG=$1
    rm -f toolchain.cmake
    ln -s /cvmfs/.../toolchains/$1.cmake toolchain.cmake # !!! Use the right path on CVMFS
}

Use the software

cd build.$BINARY_TAG
# ./run <program> <args...>
./run listcomponents -h
./run gaudirun.py

Modify the configuration

The file at the top level directory describes the build of the whole project:

• may contain a licence block
• contains documentation (how to configure it, available options)
• describes the project (name, version)
• fetches the dependencies
• sets options for the build (C++ standard)
• list all the sub-projects (packages in sub-directories)
• handles the installation

• describes the binaries that will be compiled (libraries, modules, executable, ROOT dictionaries)
• register tests for these binaries
• handles the installation of their python packages and scripts

Configuration of sub-projects

    1# {licence block if needed}
    2# {SubdirName} subdirectory
    3
    4# Build the library
    5gaudi_add_library(SubdirNameLib
    6                  SOURCES src/Lib/Counter.cpp
    7                          src/Lib/Event.cpp
    8                  LINK PUBLIC GaudiKernel)
    9
   10# Build the plugin
   11gaudi_add_module(SubdirName
   12                 SOURCES src/AbortEvent/AbortEventAlg.cpp
   13                         src/AlgSequencer/HelloWorld.cpp
   14                 LINK GaudiKernel
   15                      GaudiExamplesLib
   16                      ROOT::Tree
   17                      Rangev3::rangev3)
   18if(GAUDI_USE_AIDA) # optional dependency
   19    target_sources(SubdirName PRIVATE src/EvtColsEx/EvtColAlg.cpp
   20                                         src/Histograms/Aida2Root.cpp)
   21    target_link_libraries(SubdirName PRIVATE AIDA::aida)
   22endif()
   23
   24# Build the executable
   25gaudi_add_executable(Allocator
   26                        SOURCES src/Allocator/Allocator.cpp
   27                                src/Allocator/MyClass1.cpp
   28                        LINK SubdirNameLib
   29                            GaudiKernel)
   30
   31# Generate GaudiExamples_user.confdb
   32gaudi_generate_confuserdb()
   33
   34# Tests
   35gaudi_add_tests(QMTest)
   36gaudi_add_tests(nosetests)
   37
   38# Compiled python module
   39gaudi_add_python_module(PyExample
   40                        SOURCES src/PythonModule/Functions.cpp
   41                                src/PythonModule/PyExample.cpp
   42                        LINK Python::Python
   43                             Boost::python)
   44
   45# ROOT dictionaries
   46gaudi_add_dictionary(SubdirNameDict
   47                     HEADERFILES src/IO/dict.h
   48                     SELECTION src/IO/dict.xml
   49                     LINK SubdirNameLib)
   50
   51# Install python modules
   52gaudi_install(PYTHON)
   53# Install other scripts
   54gaudi_install(SCRIPTS)

Why is the explicit list of sources mandatory? Even tough CMake is able to use glob patterns with file(GLOB...), those glob patterns are expanded at configure time and their results hardcoded in makefile or build.ninja or whichever file used by IDEs. This means that if a new file that matches the pattern is added, there is no way for the build system (make, ninja...) to notice it. The first solution is to reconfigure the project each time a new file is added to update the hardcoded list of sources. (This can be done either by rerunning the configuration command or by touching a CMakeLists.txt.) The other solution would be to forward the glob pattern to the build system. CMake offers a way to do so: file(GLOB ... CONFIGURE_DEPENDS) but for the time being, only Makefiles generators and Ninja are supported, meaning that people using IDEs would still have to reconfigure the project themselves.

Adding a new sub-project to the project

    1add_subdirectory(SubdirName)

The directory of the sub-project must also contain a CMakeLists.txt that looks like the one above.

Adding a new third-party dependency

Then, add the look up of the dependency in the file cmake/{ProjectName}Dependencies.cmake (replace with the name of the dependency and with the minimal required version).

# For mandatory dependencies
find_package(<DependencyName> <minVersion> ${__quiet})
set_package_properties(<DependencyName> PROPERTIES TYPE REQUIRED)

# For optional dependencies
if(GAUDI_USE_<DEPENDENCY_NAME>)
  find_package(<DependencyName> <minVersion> ${__quiet})
  if(CMAKE_FIND_PACKAGE_NAME) # if the lookup is perform from ProjectConfig.cmake
    # then, all optional dependencies become required
    set_package_properties(<DependencyName>  PROPERTIES TYPE REQUIRED)
  else()
    set_package_properties(<DependencyName>  PROPERTIES TYPE RECOMMENDED)
  endif()
endif()
# and add the option GAUDI_USE_<DEPENDENCY_NAME> in the top level =CMakeLists.tx=t

Finally, it is usable.

gaudi_add_<type>(... LINK ... Dependency::Target ...)

Removing an old third-party dependency

Then remove the find module file Find{DependencyName}.cmake in cmake/modules if it exists.

Gaudi CMake functions to help the configuration

here

List of defined functions:

• gaudi_add_library()
• gaudi_add_header_only_library()
• gaudi_add_module()
• gaudi_add_python_module()
• gaudi_add_executable()
• gaudi_add_tests()
• gaudi_add_dictionary()
• gaudi_install()
• gaudi_generate_confuserdb()
• gaudi_check_python_module()
• gaudi_generate_version_header_file()

Building a stack of project at once

Example of a stack: Gaudi, LHCb, Lbcom, Rec, Brunel

mkdir workspace ; cd workspace
git clone project_url # of all the projects of the desired stack
cat <<EOF > CMakeLists.txt
cmake_minimum_required(VERSION 3.15)

project(LHCbFullStack
        LANGUAGES CXX
        DESCRIPTION "LHCb full stack")

enable_testing()

add_subdirectory(Gaudi)
add_subdirectory(LHCb)
# add_subdirectory() ... all the other projects of the stack
EOF

Using GaudiObjDesc (LHCb-specific)

Pro tip: do not document to much GaudiObjDesc so that people begrudge to use GaudiObjDesc.

-- ThomasCluzel - 2019-08-21