SCRAM for developers of CMSSW

Complete: 5

Contents

Introduction

This document is for developers of SCRAM-based CMS projects (CMSSW, IGUANA, IGNOMINY ). This will describe how one can use different scram commands for creating developer area and how to compile/build code.

SCRAM

SCRAM is a configuration management tool, a distribution system, a build system and resource manager, with local resources and applications managed in a transparent way. In addition it provides a common development environment. In this document we will only look those SCRAM commands which are useful for a developer in SCRAM-based CMS project area.

SCRAM commands

Running scram command, without any argument, will print the scram help about all the available commands. You do not need to learn all these commands, being a developer of a SCRAM-based project, you only need to know some basic scram commands. e.g. commands to
  • search SCRAM about available SCRAM-based project for a given architecture
  • create a developer area based on a given release where a developer can compile/build/test his/her code
  • compile/build code
  • get help on different scram commands

Searching for available releases

The first thing a developer needs to know is the architecture he/she is working on and if there any release available for that architecture.
arch
"scram arch" is the command which tells you what architecture you are working on. You can force a architecture to be a different by setting the SCRAM_ARCH environment variable. e.g. on a SLC3, 32 bit machine with GCC compiler version 3.2.3 the "scram arch" command will print something like slc3_ia32_gcc323. Now if one wants to work on a debug version (if available) then he/she can set the SCRAM_ARCH environment variable to slc3_ia32_gcc323_dbg. e.g. doing something like this csh/tcsh shell
In csh/tcsh:
> setenv SCRAM_ARCH slc3_ia32_gcc323_dbg
OR In bash/sh:
> SCRAM_ARCH=slc3_ia32_gcc323_dbg; export SCRAM_ARCH
will force the scram architecture to slc3_ia32_gcc323_dbg unless one explicitly forces the scram arch using -arch scram-arch command line argument.
list
"scram list ProjectName" is the command which shows which releases are available for a given scram architecture. e.g. doing something like
> scram list CMSSW
will print the list of all available CMSSW releases based on current scram arch. OR
> scram --arch slc4_ia32_gcc345 list CMSSW
will show the list of all CMSSW projects released for slc4_ia32_gcc345. e.g.
> scram list CMSSW
will print something like this
Listing installed projects....

--------------------------------------------------------------------------------
| Project Name  | Project Version          | Project Location                  |
--------------------------------------------------------------------------------
  CMSSW           CMSSW_x_y_y                
                                         --> /afs/cern.ch/cms/sw/slc3_ia32_gcc323/cms/cmssw/CMSSW_x_y_z
  CMSSW           CMSSW_a_b_c_pre1           
                                         --> /afs/cern.ch/cms/sw/slc3_ia32_gcc323/cms/cmssw/CMSSW_a_b_c_pre1
help
The one very useful scram command is "scram scram-command -help" which provides online scram help on different scram commands. e.g. "scram list -help", "scram arch -help" etc.

Creating a developer area based on release area

Once you know the architecture you want to work on (e.g. slc3_ia32_gcc323) and the base release for that architecture (e.g. CMSSW_x_y_z_pre1) then the next step is to create a developer's area based on it.
project
"scram project ProjectName ProjectVersion" is the command to do this. e.g. doing
> scram project CMSSW CMSSW_x_y_z_pre1
will create a CMSSW_x_y_z_pre1 developer's area in your current working directory. The next step is to check out the code (on which you want to work) from github in to the src directory of newly created dev area.

> cd CMSSW_x_y_z_pre1/src
> cmsenv
> git cms-addpkg Subsystem/Package

Please remember that if you want to create a new Subsystem/Package in CMSSW repository then please use the CMS tag collector to generate a request. Details about the sub-directories under Subsystem/Package are available here.

Compiling/Building code

Now you have your developer area ready to be build. For building your source code you need to run "scram build" OR in short "scram b" command.
build
This command "scram b" will build every thing available under your current working directory. e.g. running it from your dev's src directory (e.g. CMSSW_x_y_z_pre1/src) will build every Subsystem/Package available and running it from a specific Subsystem/Package directory (e.g. CMSSW_x_y_z_pre1/src/subsystemA/packageB) will only build this package and all the other packages it depends on (if needed). Please remember that once a Subsystem/Package has been built then changing
  • src/*.cc files should only re-compile those src/*.cc files and re-link the shared library
  • interface/*.h files should only re-compile all those src/*.cc files which depend on those headers and re-link the shared library.
  • BuildFile file should re-compile all the src/*.cc files and re-link the library and also re-build all other Subsystem/Packages which depends on this package. e.g. if subsystemA/packageC and subsystemA/packageD depends on subsystemA/packageB then changing the subsystemA/packageB/BuildFile will not only rebuild subsystemA/packageB but also subsystemA/packageC and subsystemA/packageD.
Command-line flags for scram b
There are some useful command-line arguments which you can pass to the "scram b" command. e.g. following are the command-line arguments which scram passes to the gmake (the actual building tool)
  • -k: Normally "scram b" command stops its processing at first compile/build error. If you want to keep on running then add this command-line argument e.g. "scram b -k".
  • -j n: You can use -j n (where n is an integer) to run n simultaneous processes for compilation/building. e.g. "scram b -j 4" will run 4 simultaneous processes to compile/build.
    NOTE: This command-line option will work properly only for CMSSW nightly builds, CMSSW_1_3_0_preX and above.

Following are the command-line arguments which are used directly by scram for building your code.
  • -f: Adding this command-line argument tells scram to skip the checking of the configuration files and BuildFiles changes and go directly to the building. It is only useful when you have already build a package and now you have only changed any *.h or *.cc file.
    NOTE: If you have either checked out a new package from git or deleted a existing package or modified the BuildFile of already built package then please do not use this command-line argument. See tips/tricks for more detail.
  • -r: Adding this command-line argument tells scram to re-build the project cache (i.e. by reading all the BuildFiles and updating the internal cache) before running any gmake commands.
    NOTE: For CMSSW nightly builds, CMSSW_1_3_0_preX and above, you do not need to use this command-line argument unless you have changed any .tmpl file in your project's config directory.
  • -v: By default in CMSSW, scram b prints information about what the build system is doing not how it is doing. Adding this command-line option also prints the commands used for building. Please note that if you already have run scram b (i.e. without -v option) then running scram b -v next time will have no effect (unless the internal cache was updated for some reason). The alternative to this is SCRAM_BUILDVERBOSE environment variable. Setting this environment variable, to a non-empty value, will always run scram build in verbose mode.
Command-line targets for scram b
There are some special command-line argument (actually the gmake targets) available which are good to know. e.g.
  • clean: "scram b clean" will remove already build products (libraries/binraries/scripts) and the tmp area used to build those e.g. the *.o files etc. Running this form a subsystem/package will remove all the products generated from this package and running it from a subsystem directory will remove all the products generated from all the packages of this sub-system.
  • echo_xyz: "scram b echo_xyz" (where xyz is an internal gmake variable e.g. CXX, CPPDEFINES etc.) will print the value of xyz gmake variable.
  • ToolUpdated_toolname: "scram b ToolUpdated_toolname" (where toolname is an external tool setup for your project e.g. qt, boost, root, rootcore etc.) will update the internal build cache for that tool so that when next time you do a "scram b" then all the products, which depend on this tool, could re-compile/re-build properly using new path of that tool. You need to run this after running any "scram setup ...." command.
    NOTE: This target is only available in CMSSW nightly builds, CMSSW_1_3_0_preX and above.
  • ExternalLinks: "scram b ExternalLinks" will update all the symlinks for all the tools setup locally in your developement area. You only need to run this if you had done "scram setup".
  • subsystem: "scram b subsystem" (where subsystem is the name of subsystem you want to build e.g. DataFormats, FWCore etc.) will build every thing for that subsystem.
  • subsystem_clean: "scram b subsystem_clean" (where subsystem is the name of subsystem you want to clean e.g. DataFormats, FWCore etc.) will remove all the product generated from the subsystem and also clean the tmp area for that subsystem.
  • prodname: "scram b prodname" (where prodname is the name of the product(lib/bin) you want to build e.g. for a library generated from subsystemA/PackageB/src directory the prodname will be subsystemAPackageB
    NOTE: This target is only available in CMSSW nightly builds, CMSSW_1_3_0_preX and above.
  • prodname_clean: "scram b prodname_clean" (where prodname is the name of the product(lib/bin)) will remove the product prodname and also clean all the tmp files for that product.
    NOTE: This target is only available in CMSSW nightly builds, CMSSW_1_3_0_preX and above.
  • subsystem/package: "scram b subsystem/package" (where subsystem/package is the package you want to build e.g. FWCore/Framework etc.) will build every thing for that subsystem/package.
Running test executables generated from test directory
If you have generated some test executables/binaries from your test directory then by running "scram b runtests" will run all the tests and pass the required command-line arguments (defined in the test/BuildFile) to each test. Please see test runner flag for more details.
TIPS and TRICKS
  • Check out only the packages which you want to edit/modify. e.g. if you want to edit/modify a subsystemA/packageB then only checkout subsystemA/packageB and not the whole subsystemA. Less packages in your developer area means less time to compile/build and more time for testing.
  • -f: Make use of -f option, it will save you some compilation time. Please do not use this flag if you have
    • freshly checked out a new package
    • removed an already check out package
    • modified or added a CMS.BuildFile
    • added a new package's sub-directory e.g. data, test, bin, scripts, python, plugins
    • added Qt Q_OBJECT macro in any *.h file, so that the Qt moc should run for this
    • added new files for
      • Root dictionaries: src/*LinkDef.h
      • LCG dictionaries: src/classes.h and src/classes_def.xml
      • Code generation: src/*desc.xml
      • Flex/Bison: src/*lex.l and src/*parse.y
It is safe to use -f option if you have modified/added *.h, *.cc files.
  • -j: Make use of -j N option (where N is ~= twice the number of processors available). When you have multiple packages in your development area then its hard to read the output of scram b -j n. If you set BUILD_LOG environment variable to yes then doing scram b will print the build log for each package separately. e.g. one can do
    • for tcsh/csh: env BUILD_LOG=yes scram build
    • for bash/sh: BUILD_LOG=yes scram build
  • You do not need to use -r flag ever. The only time you need to use this option is when you have changed any .tmpl file in your project's config directory.
  • Try not creating any log file under your project's src tree (any where under project's src directory and its sub-directories). e.g. running your test executable and re-directing the output to some log file in your project's src directory tree is not good. Because creating/removing a file under your project's src directory will change the timestamp of that directory and this means that next "scram b" command is going to update your project cache (unless there is -f flags used), so next compilation will take a bit more time.

Getting list of external tools (and their versions) setup for your dev area

Doing scram tool list will print the list of all external tools (and their versions) available in your SCRAM-based project's area.

Getting details of an external tool

Doing scram tool info toolname will print information about that external tool. e.g. its include/lib/bin paths, shared/static libraries of this tool, other tools on which it depends on etc.

Building debug shared libraires for a non-debug SCRAM ARCH

If you want to generate debug shared libraries/executables for a non-debug SCRAM arch then you can add the following line

<Flags CXXFLAGS="-g">

in either
  • top level CMS.BuildFile (i.e. config/CMS.BuildFile) to generate all libraries/executables from this dev area with debugging information OR
  • in specific CMS.BuildFile (i.e. src/Subsystem/Package/CMS.BuildFile, src/Subsystem/Package/test/CMS.BuildFile, src/Subsystem/Package/bin/CMS.BuildFile etc.) to generate all libraries/executables from this specific package with debugging information.
OR with SCRAM version V2_* you can do
  • scram b clean; scram b USER_CXXFLAGS="-g"
By default CMS packages are compiled with the -O2 flag, which causes your code to be optimised and not be exactly the same as the one you have written. For instance, temporary variables you declare in your code and you might want to check can be optimised out for the sake of execution speed. To disable the compiler optimisations the file $CMSSW_BASE/config/toolbox/$SCRAM_ARCH/tools/selected/gcc-cxxcompiler.xml must be edited to remove -O2 from the line specifying it. To make sure no optimisation is used it is better if -O2 is replaced by -O0. The -g option can also be added to compile with debug symbols by default ( specifying USER_CXXFLAGS="-g" at compilation time is not necessary if this is done). Every time the gcc-cxxcompiler.xml file has been modified the commands scram setup gcc-cxxcompiler and scram b clean must be run to update SCRAM with the new options and to clean the build environment.

Skipping shared library checking tests during the compilation of your code

It is not recommended to skip these tests as these make sure that your library is fully-bound(no missing symbols) and loadable. But if for some reason you need to skip these tests then you can do the following.
  • Skipping shared library load check: Set SCRAM_NOLOADCHECK environment variable to a non-empty value.
    NOTE: If the SEAL external tool is not setup for your project then this test will also be skipped as it needs SealModuleLoadCheck tool to run this test.
  • Skipping missing symbols check: Set SCRAM_NOSYMCHECK environment variable to a non-empty value.

Renaming/Moving a SCRAM-based project's dev area

If for some reason you want to move/rename your current working SCRAM-based project's development area then here is what you need to do
  • Copy or rename the existing project's area e.g.
    cp -rp /home/dev/ProjectOldDir  /home/dev/ProjectNewDir
    OR
    mv /home/dev/ProjectOldDir  /home/dev/ProjectNewDir
  • Go to the new project's dev area and do
    scram b ProjectRename
    NOTE: This is only available for CMSSW nightly builds, CMSSW_1_3_0_preX and above.

Compiling against a different version of external tool

Some time you need to test a different version of an external tool (e.g. qt, root, boost, python etc.) as compare to the one used by the release. To do this,
  • With SCRAM V2_* series: Just edit the
    <devarea>/config/toolbox/<arch>/tools/selected/<toolname>.xml
    file and change the tool paths or add/remove libs and then simply do
    scram setup <toolname>
  • With SCRAM V1_* series: You can run scram setup -i toolname and it will ask you the new path. If you want to add/remove new libs variables for a tool then follow this
    • scram setup toolname
    • cp .SCRAM/InstalledTools/toolname toolname.new
    • scram tool remove tooname
    • scram setup tooname toolversion file:toolname.new

Please note that there are some external tools with different names but point to the same external area. e.g.

  • rootcore, rootrflx etc. all point to same root installation
  • boost, boost_python etc. all points to same boost installation.
So now if you want to change the path for rootcore then you should also change the paths for root*. Below is a small trick to find out the tools which point to the same location (run this from your project's top level directory)
  • SCRAM V2_* series:
     grep ' name=".*_BASE"' config/toolbox/`scram arch`/tools/selected/* | sed 's|.*name="||;s|" default="|=|;s|" value="|=|;s|"/>||' | \
           grep "_BASE=" | sed 's|_BASE=| |' | awk '{print $2"="$1}' 

  • SCRAM V1_* series:
       grep "_BASE' => '" .SCRAM/`scram arch`/ToolCache.db | sed "s|,||g;s|'||g;s|_BASE||" | awk '{print $3"="$1}' | sort | uniq
The output you get will look like
   ....
   .../external/boost/1.33.1=BOOST
   .../external/boost/1.33.1=BOOST_PYTHON
   .../external/bz2lib/1.0.2=BZ2LIB
   .../external/castor/2.1.1-4=CASTOR
   ...
   .../lcg/root/5.13.04e=ROOTCORE
   .../lcg/root/5.13.04e=ROOTRFLX
Note that in the above output
  • boost and boost_python tools point to same external location. So if you want to change the path for boost tool then you should change the paths for boost_python too.
  • rootcore and rootrflx point to same external location. So if you want to change the path for rootcore tool then you should change the paths for rootrflx too.
You can add "| grep -i root" at the end of above command in order to just see the root tools.
NOTE:You need run "scram b ToolUpdated_toolname " so that next time when you do a "scram b" then all the products in your dev area, which depend on this tool, could properly re-compile/re-build using new paths of that tool. This is only available in CMSSW nightly builds, CMSSW_1_3_0_preX and above.

Missing/Adding library for an external tool

All the libraries, of an external tool, are not added in the external tool description. If you want to use a library from an external tool which is not available via any other external tool (setup for your project) then you need to contact the configuration manager and ask him/her to add this missing lib in either an existing tool or create a new tool for this lib. This can take some time so for your testing you can add the following lines in your src/subsystem/package[/[bin|test]]/CMS.BuildFile. Please note that this is only for your testing and once the new lib is available (either via an existing tool or via a new external tool) you need to fix your CMS.BuildFile accordingly.
<Use name=toolname>
<Use lib=newlib>

Adding new external tools

Please do not do any development against any tool which is not available in your project's configuration. It is highly recommended that you first contact CMS projects' configuration manager for adding new external tools.

Changing an external tool description

Please contact CMS projects' configuration manager for this.

scram build Errors/Warnings

Here is detail list of errors/warnings, and their possible solutions, one could get while doing scram build
  • Missing packages/tools
       WARNING: Unable to find package/tool called subsystemX/packageX
          in current project area (declared at src/subsystemY/packageY/...)
    Although this is just a warning and does not stop the build process but it is highly recommended that you fix this or ask the responsible developer to fix this. To fix this just edit all the CMS.BuildFile(s) under src/subsystemY/packageY directory and remove any occurrence of
    <Use name=subsystemX/packageX>
  • Mismatch library and plugin name
       ****ERROR: Plugin name should be same as the library name. Please fix the ".../CMS.BuildFile" file and replace "YourPluginName" with "YourLibName"
                  Please fix the above error otherwise library "YourLibName" will not be registered as plugin
    All you need to do it to edit the CMS.BuildFile mentioned in error and replace the line
    <Flags SEAL_PLUGIN_NAME=YourPluginName>
    with
    <Flags SEAL_PLUGIN_NAME=YourLibName>
  • SealPlugin and SEAL_PLUGIN_NAME both flags defined in CMS.BuildFile
       ****ERROR: Both flags "SEALPLUGIN" and "SEAL_PLUGIN_NAME" are set for "YourLibName" library in ".../CMS.BuildFile" file.
                  You only need to provide one flag (preferred flag is SEALPLUGIN). Please fix this first otherwise plugin will not be registered
    All you need to do is to edit the CMS.BuildFile mentioned in this error and remove either line
    <Flags SealPlugin=1>
    OR
    <Flags SEAL_PLUGIN_NAME=YourLibName>
  • Wrong value for SealPlugin flag
       ****ERROR: Only allowed values for SEALPLUGIN flag are "0" OR "1". Please fix this for "YourLibName" library in ".../BuildFile" file.
    All you need to do is to edit the BuildFile mentioned in this error and fix the line
    <Flags SealPlugin=X>
    where replace X with either "1" or "0". See detail about this on BuildFile SealPlugin flag
  • SealPlugin and LCG dictionaries
       ****ERROR: One should not set SEAL_PLUGIN_NAME or SEALPLUGIN flag for a library which is also going to generate LCG dictionaries
                  Please take appropriate action to fix this by either removing the
                  SEAL_PLUGIN_NAME or SEALPLUGIN flag from the "../CMS.BuildFile" file for library "YourLibName"
                  OR LCG DICT header/xml files for this seal plugin library
    This happens when you have SEAL_PLUGIN_NAME or SEALPLUGIN flag set in your CMS.BuildFile and also you have LCG DICT header (by default classes.h) and xml (by default classes_def.xml) files. Build system will refuse to build these kind of libraries. So you need to fix this either by removing the SEAL_PLUGIN_NAME/SEALPLUGIN flag from the CMS.BuildFile mentioned in the error or moving the LCG DICT header/xml files in a separate package.
  • Missing LCG Dictionary header or XML file
    ****WARNING: Not going to generate LCG DICT from "YouPackageDir" because NO. of .h ("LCGDictHeaderFiles") does not match NO. of .xml ("LCGDictXMLFiles") files.
    You need to provide both LCG dictionaries header and XML files in order to generate LCG dictionaries. Missing of any of these files will not create LCG dictionaries from your package. More detail on generating LCG dictionaries in CMSSW is available here.
  • ="scram b"= did not re-build any product after changing an external tool
    You have changed an external tool (e.g. by doing "scram setup -i toolname") but "scram b" did not re-build any product which depend on this tool. In this case, may be you have forgotten to do "scram b ToolUpdated_toolname ". In future this will be done automatically.
  • Product type not supported
    WARNING: Product type "ProdType" not supported yet from "YourPackagePath".
    There are some limitations on which products (libraries, binaries) one can build from a sub-directory of his/her package. e.g. from
    • src: One can only generate a single shared library
    • bin, test, plugins: One can generate shared libraries (using library tag), binaries (using bin tag).
If you have any tag in your BuildFile which is not supported from a package sub-directory or not recognized by build system then you will get the above error. You need to fix your BuildFile. Please contact your release manager for this.
  • Error opening directory for reading
    ERROR: Can not open "ReqDir" directory. ".../CMS.BuildFile" is refering for files in this directory.
    This happens when you have something like this in your CMS.BuildFile
    <bin name=XYZ file=mysubdir/myTest.cpp></bin>
    OR
    <library name=XYZ file=mysubdir/*.cc></library>
    In this case if mysubdir does not exist then you will get the above error. Please make sure that you have correct path for files in your CMS.BuildFile.
  • Error reading/writing any file
    ERROR: Can not open xyzfile file for reading/writing
    Make sure you have read/write access for this xyzfile.
  • Moving/Renaming of a project's dev area
       **** ERROR: You have moved/renamed this project area "NewProjectDir" from "OldProjectDir"
                   Please first run "scram b ProjectRename" command
    You will see this error when you copy or move your dev area into a new directory and do a "scram b" in the new area. As project's internal cache still has references of the original area so you need to run
    scram b ProjectRename
    in order to fix those references.
    NOTE: This can take some time (some secs to couple of mins, depending on the number of packages in your dev area).
  • Overriding commands for target
       warning: overriding commands for target 'ProductName_all'
       warning: ignoring old commands for target 'ProductName_all'
    These are gmake errors and reason is that there are two or more products with same name (ProductName). In order to find from where these two products are coming please run the following command from your project's top level directory.
       grep "ProductName_BuildFile *:=" tmp/`scram arch`/Makefile
    It will show you the list of BuildFiles from where these products are generated. Please either rename the product you are trying to build or contact other developers to fix this issue as soon as possible.
  • No rule to make target
    gmake: *** No rule to make target `src/Subsystem/Package/.../*.h', needed by `.../src/SubSystem/Package/.../*o'.  Stop.
    You will see these kind of errors if you have removed any source files (.h or .cc) which were present for the previous successful build. To fix this go to your "src/SubSystem/Package/..." directory and do "scram b clean" there. After this "scram b" should work properly.

Review Status

Editor/reviewer and date Comments
Main.muzaffar - 19 Dec 2006 Page author
Main.muzaffar - 20 Feb 2007 page content last edited
JennyWilliams - 21 Feb 2007 page edited to fit in with swguide
SimoneBologna - 28 Nov 2017 Added instructions on how to remove optimisation flags for debugging

Responsible: Main.muzaffar
Last reviewed by: MostRecentReviewer

Edit | Attach | Watch | Print version | History: r38 < r37 < r36 < r35 < r34 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r38 - 2018-11-08 - MaryHillHadley



 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback