CMSSW SCRAM CMS.BuildFile

Complete:

Contents

• Introduction
• SCRAM
• CMS.BuildFile
  • BuildFile generator script
• New BuildFile XML Format
• Purpose of different directories in CMSSW
  • src
  • interface
  • bin
  • test
  • plugins
  • data
  • python
  • scripts
• BuildFile tags
  • Use
  • Export
  • Lib
  • Bin
  • Library
  • Flags ( CXX, SealPlugin, LCG Dict, Controlling genreflex, Rule Checker, Code Gen etc...... )

Introduction

This document will provide you some information about the BuildFile in the CMSSW environment. e.g. what is the purpose of these BuildFiles under CMSSW, different tags (and flags) used in these BuildFiles, how to generate different products (libraries, binaries etc.). But first a short description of CMSSW build system SCRAM and BuildFile

SCRAM

SCRAM is a configuration management tool, a distribution system, a build system and resource manager, with local resources and applications managed in a transparent way. In addition it provides a common development environment. You can find more document about SCRAM here.

BuildFile

It is a configuration document for SCRAM (the build system). For each software unit (library, executable etc.), a BuildFile defines the external/internal dependencies and the interface. The interface supplies the product provided by this unit and its dependencies. For detail document please read the SCRAM documentation at SCRAM.

BuildFile Generator Script

In CMSSW, there is PERL script createBuildFile.pl which one can use to generate clean BuildFile(s). For more detail please read this

New CMS.BuildFile XML Format

SCRAM version V2_* and above support xml format of CMS.BuildFile too. New XML format is same as old format with the following restrictions

• Closing tags: All tags should have closing tags e.g.

  <use name="tool"/> 

  or

  <use name="tool"></use>

  are valid tags.
• Tag's attributes values: Values of a tag's attributes should be enclosed in double quotes (") e.g.

  <lib name="MyLib"/>

  is correct while

  <lib name=MyLib/>

  is wrong. Also if your library has the name, for example,

  libSPlot.so

  the convention is to of "take off the lib, and drop the file extension .so. Thus it should look like

  <lib name=SPlot>

  and NOT

  <lib name=libSPlot.so>

• Tag and its attributes names: All tags and attributes should start with alphabet character and should be at least three characters long. Valid character set for tag names are a-z,0-9 and '_' e.g. in

  <use name="tool"/>

  use is valid tag while in

  <Use name="tool"/>

  Use is not a valid tag name. Tag attributes names could also have A-Z characters e.g.

  <flags EdmPlugin="1"/>

  EdmPlugin is valid tag attribute name.

NOTE: To avoid external dependency, SCRAM has its own xml parser which does not support all the features of XML format e.g. xml style comments, special characters etc.

CMSSW Source code directory structure

All CMSSW software is located under "src" directory. Under "src" directory we have different sub-systems (e.g. Alignment, FWCore, DataFormats etc.) and each sub-system consists of one or more packages (e.g. FWCore/FWLite, FWCore/Framework). Under each package one can have following directories. Here is the short description about the default purpose of these directories.

• src: Used to generate a single shared library only (unless explicitly used for other purpose). All the C++ source code should go in this directory. Shared library generated out of this directory will be named as SubsystemPackage. It is recommended that you create a Subsystem/Package/CMS.BuildFile to provide dependency for this library. The generated shared library is copied to your project's architecture specific lib directory.

• interface: Used to provide C++ header files which declare the interface for the shared library generated out of src directory. No need to provide a interface/CMS.BuildFile

• data: Used to provide the configuration files (e.g .cgi, .cff, .cfg) and data files. All these files will be copied to your project's share/Subsystem/Package/data directory. It is not recommended to add large binary data files in CMSSW CVS. If there are some binary data files needed for your tests then you can add a download.url file under this directory which should contain the URLs for those data files. On running "scram build" command, all the URLs will be fetched using the wget and put in the project's share/Subsystem/Package/data area. A detail document about this download.url file is available at CMSSWBinaryData. There is no need to provide a data/CMS.BuildFile.

• test: Used to generate test libraries and test executables. One can generate more than one product (libs/bins) from this using the proper BuildFile tags in test/BuildFile file. If there is a sub-directory data under this then that sub-directory will be treated the same way as Subsystem/Package/data i.e. all the configurations files under it will be copied to share/Subsystem/Package/data. All the shared libraries are copied to your project's architecture specific lib directory and executables are copied to architecture specific bin directory.

• bin: Used to generate executables intended for general use and libraries for these executables. Technically, the same as the test directory, but there is no support for a bin/data directory. All you can do is to generate different products (libs/bins) by using the proper BuildFile tags in bin/BuildFile.

• plugins: Same as bin directory but the only difference is that all the shared libraries generated out of it are by default plugins (SealPlugins).
  NOTE: This directory is only available for CMSSW nightly builds, CMSSW_1_3_0_preX and above.

• scripts: Used to provide perl/python/shell/... scripts. All these scripts are copied to your project's architecture specific bin directory.

• python: Used to provide python scripts and generating python bindings. It will copy all the .py files into your project's python/Subsystem/Package directory. It will also create __init__.py for each directory under your project's python directory if not already exist. If there is a subdirectory under this python directory then that will also be copied to project's python/Subsystem/Package directory.
  NOTE: This directory is only available for CMSSW nightly builds, CMSSW_1_3_0_preX and above.

CMSSW CMS.BuildFile

For writing CMS.BuildFile in the CMSSW environment most of the time you will use the following CMS.BuildFile tags (tags and their attributes are not case sensitive).

• use: This tag is used to add a dependency on external/internal software unit. e.g.

  <use name=boost>

  means add dependency on external software unit boost and

  <use name=Subsystem/Package>

  means add dependency on an internal software unit Subsystem/Package.

• export: This tag is used to export the product generated(if any) by this software unit (mostly the shared library name) and its dependencies for the other software units which depend on it. e.g. having something like

  <use name=root>
  <use name=boost>
  <export>
    <use name=root>
    <use name=boost>
    <lib name=XY>
  </export>

  means that this software unit is going to generate a share library XY and its interface depends on two external software units "root" and "boost". Please note that all you need to export the product name and the interface dependencies. e.g. if boost is the only software unit used in your *.h files then your CMS.BuildFile should look like

  <use name=root>
  <use name=boost>
  <export>
    <use name=boost>
    <lib name=XY>
  </export>

  In this example, adding

  <use name=root>

  in the export is not harmful but it is a good practice to only export those statements which are used in your interface/*.h files. Normally you should have export section only in the Subsystem/Package/BuildFile.

• lib: This tag is used to add a direct dependency on a library. It is recommended to only use this tag within the export section of a BuildFile. e.g. adding something

  <lib name=abc>

  outside the export section mean now your product (lib/bin) will be linked against this library abc too. When added in the export section then make sure that the lib name should match the library name generated from this software unit. Also note, as also mentioned above, if your library has the name, for example,

  libSPlot.so

  the convention is to of "take off the lib, and drop the file extension .so. Thus it should look like

  <lib name=SPlot>

  and NOT

  <lib name=libSPlot.so>

• library: This tag is used to generate a shared library. e.g. having something like

  <library name=XYZ file=*.cpp></library>

  in test/BuildFile, bin/BuildFile or plugins/BuildFile means generate a shared library called XYZ. One can specify the dependency using use tag within the

  <library ...></library>

  tag. e.g. having something like this in bin/BuildFile

  <use name=boost>
  <library name=XYCommon file=common/*.cpp>
    <use name=qt>
  </library>
  <library name=X file=X.cpp>
    <lib name=XYCommon>
    <use name=qt>
  </library>
  <library name=Y file=Y.cpp>
    <lib name=XYCommon>
    <use name=qt>
    <use name=root>
  </library>

  means that generate 3 products (please note that boost external software unit is used for all 3 products)
  • Shared library XYCommon which depends on external software units boost and qt
  • Shared library X which depends on library XYCommon and external software units boost and qt
  • Shared library Y which depends on library XYCommon and external software units boost, qt and root

• bin: Same as library tag but in this case the generated product will be an executable. So adding something like

  <bin name=XYZ file=*.cpp></bin>

  in test/BuildFile, bin/BuildFile or plugins/BuildFile means generate a executable XYZ. You can add dependency using the use tag in the same way as it was done for library tag.

• flags: This tag is used to provide additional information to the build system (i.e. scram) about the product (lib/bin). There are different tags which serve different purposes. A flag added in b/w

  <library></library>
  OR
  <bin></bin>

  tag pair is only valid for that library or binary. Detail description of the available flags are here. The syntex of this tag is

  <Flags FlagName=FlagValue>

  where
  • FlagName is the name of the flag and it is not case sensitive. Space character (" ") is not allowed in the FlagName
  • FlagValue is the value for this flag. If there are space characters in it then please enclose this in "". e.g.

    <Flags FlagName="Flag Value">

    IMPORTANT Normally you should not add this tag flags in your export section of BuildFile.

CPPDEFINES, CXXFLAGS, CFLAGS, FFLAGS, LDFLAGS

To provide additional flags to be used by C/C++/Fortran compiler/linker. Please note that you do not need to add "-D" for CPPDEFINES flag e.g.

<Flags CppDefines="DEBUG=1">

means that now compiler will add -DDEBUG=1 flag. You can add these flags for all library, binary, python bindings shared libraries. Please note that if you want to add a CPPDEFINE of the form MYDEFINES="value" then please add something like this in your CMS.BuildFile

<Flags CPPDEFINES='MYDEFINES=\"value\"'>

You can also add environment variables and gmake variables by adding something like

<Flags CPPDEFINES='PathAtBuildTime=\"$(PATH)\"'>

NOTE: Space character (" ") is not allowed in the CPPDEFINES value e.g. something like this

<Flags CPPDEFINES='MyDefines=\"value1 value2\"'>

is not allowed.

SEAL_PLUGIN_NAME

To tell the build system to register this library as a SealPlugin. Please remember that all the shared libraries generated out of plugins directory are by default SealPlugin, so no need to add this flag for those. Adding something like

<Flags SEAL_PLUGIN_NAME=mylibname>

for a library will declare that library as a SealPlugin. Please make sure that library name here in this tag matches the actual library name.

SEALPLUGIN

It is same as SEAL_PLUGIN_NAME tag but value of this tag could be '0' or '1'. By default all the libraries generated are not SealPlugin. So to declare shared library as SealPlugin, you can add SealPlugin=1 flag for that library. As all the libraries generated out of plugins directory are by default SealPlugin, so there is no need to set this flag to '1' for those libraries. But if for some reason you want to generate a non SealPlugin library from plugins directory then set SealPlugin=0 for that library.
NOTE: This flag is only available for all release of IGUANA, CMSSW nightly builds, CMSSW_1_3_0_preX and above.

SKIP_FILES

This flag allows you to skip some files. If used for a shared library generated out of src, bin or test directory, then it means build the shared library without compiling and linking these files. If used in scripts/BuildFile then this mean do not copy these scripts in to the project's specific bin directory.
NOTE: This flag is only available for all release of IGUANA, CMSSW nightly builds, CMSSW_1_3_0_preX and above.

INSTALL_SCRIPTS

This tag allows you to copy scripts in to project's specific bin area before building a shared library (from src, bin or test directory) or a binary (from src or bin directory). e.g. having something like

<Flags INSTALL_SCRIPTS=script1.pl>

for a library or binary will first copy this script to project's specific bin area and then build the product (lib or bin).
NOTE: This flag is only available for all release of IGUANA, CMSSW nightly builds, CMSSW_1_3_0_preX and above.

NO_LIB_CHECKING

If this flag is set to 'yes' or '1' then all the library testing steps (symbol checking, library loading check) will be skipped.
NOTE: This flag is only available for CMSSW nightly builds, CMSSW_1_3_0_preX and above.

LCG_DICT_HEADER, LCG_DICT_XML

These flags allow you to provide the name of header (*.h) file and selection XML file (*.xml) for the generation of LCG dictionarties. The default values for these flags are

LCG_DICT_HEADER=classes.h
LCG_DICT_XML=classes_def.xml

In CMSSW environment it is recommended to use the default values, so you should create classes.h and classes_def.xml files if you want to generate LCG dictionaries. You can provide multiple *.h and *.xml files using these flags (make sure that number and order of *.h files matches the number and order of *.xml files).
NOTE: This flag is only available for CMSSW nightly builds, CMSSW_1_3_0_preX and above.

IMPORTANT: Please do not add
<Flags SEAL_PLUGIN_NAME=mylibname>
OR
<Flags SEALPLUGIN=1>
flag for a shared library for which you are also going to generate LCG dictionaries

GENREFLEX_ARGS

This flag allows you to pass extra command-line arguments for genreflex for LCG dictionaries generation. Default value is --deep. If you do not want the default value then you can either have

<Flags GENREFLEX_ARGS="--new-arg1 --new-arg2">

in order to use --new-arg1 --new-arg2 as command-line arguments for genreflex or you can have

<Flags GENREFLEX_ARGS="--">

which will not pass any extra command-line arguments to genreflex.

GENREFLEX_FAILES_ON_WARNS

By default --fail_on_warnings flag is used for building LCG dictionaries but if for some reason you want to disable this then please add the following flag in your CMS.BuildFile

<Flags GENREFLEX_FAILES_ON_WARNS=no>
OR
<Flags GENREFLEX_FAILES_ON_WARNS=0>

NOTE: This flag is only available for CMSSW nightly builds, CMSSW_1_3_0_preX and above.

ROOTMAP

One can use this flag in CMS.BuildFile to tell the build system to also generate libNameCapabilities.rootmap file while generating the LCG dictionaries. This libNameCapabilities.rootmap will be copied to your project's architecture specific lib directory. Flags declaration in CMS.BuildFile should look like

<Flags ROOTMAP=yes>
OR
<Flags ROOTMAP=1>

NOTE: This flag is only available for CMSSW nightly builds, CMSSW_1_3_0_preX and above.

TEST_RUNNER_ARGS

One can add this flag to pass the command-line arguments for a test executable. The flag is only vaild in test/CMS.BuildFile. You can add it multiple times e.g. having something like

<Flags TEST_RUNNER_ARG="-p myconfig.cfg"/>
<bin file="myTest1.cpp" name="myTest1">
  <Flags TEST_RUNNER_ARG="arg_for_test1"/>
</bin>
<bin file="myTest2.cpp" name="myTest2">
  <Flags TEST_RUNNER_ARG="arg_for_test2"/>
</bin>

mean that now when you will run scramv1 b runtests then it will run the following two test

• myTest1 -p myconfig.cfg arg_for_test1
• myTest2 -p myconfig.cfg arg_for_test2

NO_TESTRUN

One can add this flag to skip running of the test via scram. e.g. if you have some thing like this

<bin file="myTest1.cpp" name="myTest1">
  <Flags NO_TESTRUN="1"/>
</bin>
<bin file="myTest2.cpp" name="myTest2">
</bin>

then only myTest2 will be run via "scram build runtest" and myTest1 will be skipped.

TEST_RUNNER_CMD

Use this flag to tell scram how to run your test executable. e.g. if you have

<bin file="myTest1.cpp" name="myTest1">
  <Flags TEST_RUNNER_CMD="cd $(CMSSW_BASE)/src/MySubSystem/MyPackage/test; myTest1 arg1 arg2"/>
</bin>

then scram will run

cd $(CMSSW_BASE)/src/MySubSystem/MyPackage/test; myTest1 arg1 arg2

The command will be run using a bash shell.

ADD_SUBDIR

If this flag is set to 'yes' or '1' then for a shared library, generated out of src directory only, build system will also search for C/C++/Fortran source files in the sub directories of src directory.
NOTE: This flag is only available for CMSSW nightly builds, CMSSW_1_3_0_preX and above.

CODEGENPATH

This flags is used to provide the codegen.py script installtion path for generating the code. Build system will assume that codegen.py script is available under $CODEGENPATH/bin directory.

RULES_IGNORED

It is a rule checker specific flag. It allows to skip checking of rules for different C/C++ files. This flag is only valid for shared library generated from src directory. Please see the rule checker documentation for more detail.

RULE_VIOL_SKIP_FILES

It is a rule checker specific flag. It allows to skip any C/C++ files for rule checking. This flag is only valid for shared library generated from src directory. Please see the rule checker documentation for more detail.

Review Status

Editor/Reviewer and Date	Comments
Main.muzaffar - 19 Dec 2006	Page author
ConstantinLoizides - 20 Feb 2007	Page content last modified
JennyWilliams - 20 Feb 2007	edited for inclusion in swguide

Responsible: Main.muzaffar
Last reviewed by: MostRecentReviewer