TWiki> CMSPublic Web>SWGuide>SWGuideDevelopersGuide (revision 51)EditAttachPDF

CMSSW Developer's Guide

Complete: 0

This page is obsolete, and the information it contains does not reflect any more the organisation and development process of the CMS software

Introduction

This guide is intendend to orient developers in the world of CMSSW. The suggested solutions will not be the only ones. Please feel free to contribute to this page, in case you think there is a better way to do things.

Collaboration

In CMSSW there are three kind of contributors to the integration process

  • developers
  • administrators
  • release managers

Developers can commit code in a given package. They are listed as such in MySubsystem/MyPackage/.admin/developers file.
Package Administrators can commit and publish official tags to the TagCollector. They are listed as such in the MySubsystem/MyPackage/.admin/developers file.
Release Managers can commit and tag anywhere. They control the integration process and build the whole system. They are responsible for installation at Cern.

CMSSW source tree

The Source Tree is organized in two levels: Subsystems and Package . A package is identified by these two coordinates, as in DataFormats/DetId

CVS

CVS (several guides accessible through Google) is our version management system. Additional machinery on top of CVS is used for access control through the developers file. Creation of new SubSystems and Packages is controlled by .admin/NewModule and .admin/NewLeaf respectively. This is reponsibility of release managers.

In addition to general CVS guides available on the web, CERN IT provides documentation on accessing CVS projects hosted at CERN (such as CMSSW).

.admin/developers for CMSSW CVS access control

CMSSW CVS access control for commits and tags are done via .admin/developers file. Detail document on this is available here

SCRAM

Scram is our build and configuration management system. Useful wiki pages that describe the use of SCRAM with CMSSW are available :

The CMS TagCollector

The CMS TagCollector is the release management system. Its functions are:

  • Submission of package tags to the system for nightly builds and releases
  • Query release configuration
  • Request for new packages
  • Manage package administrative privileges

and for Release Managers:

  • create new releases
  • manage releaes
  • manage new package requests
  • manage tag queues

Read the TagCollector's Documentation and Tutorial for more information.

Integration Builds (former "Nightly" builds)

A new system to provide Integration Builds for the CMSSW software is in production. More information is available at this wiki page.

Recommended developement cycle

A possible way of working is the one in which the developer checks out a nightly, works for some days on it, then switches to a more recent nightly. In particular, near a scheduled release of prerelease, it's a good idea to have your packages working in the nightly. The release schedule is available at https://twiki.cern.ch/twiki/bin/view/CMS/ReleaseSchedule

Checking out or updating code

Most probably, as a developer, you want to develop against a recent nighly build. Here are the steps to checkout a nightly build:

  • Find available nightlies scram list CMSSW
  • Setup the SCRAM area scram project CMSSW CMSSW_yyyy-mm-dd
  • Acquire the environment: in the project area: eval `scramv1 runtime -csh` (or -sh if you use the bash shell)
  • Checkout your packages: addpkg MySubsystem/MyPackage  Vxx-xx-xx . For example addkg CMS.PhysicsTools/PatAlgos V07-11-31. You can also have the list of packages in a MyPackages.txt file and do addpkg -f MyPackages.txt.

Writing code

Code should be written according to the CMS coding conventions document and SWGuideFrameWork

Compiling the code

The command to compile is scramv1 b -v -k. The -v options asks for verbosity, the -k tells scram not to stop at the first error.

Checking the status of your code

At some point, you'll probably want to check the status of the code you have worked on against the one you or some other developer had submitted for integration. The script ProjectCVSStatus.pl does this. Try it with the --help option for more details.

Documenting the code

Your classes should contain the appropriate doxygen markup. Every package should contain a package.doc file in the doc directory, containing the main documentation page. See the reference manual itself for more details. It explains how to use the documentation templates.

Checking the generated documentation

Type scramv1 b referencemanual and point your browser to your $CMSSW_BASE/html.

Add tests to your package

Authors of CMSSW packages are encouraged to have unit tests for their packages. A unit test is a software verification and validation method in which one can test if individual units of source code are fit for use. A unit is the smallest testable part of an application/code. The unit tests used by CMSSW Framework developers are provided in FWCore/Framework/test directory. In CMSSW, one runs the unit tests for a package by creating a working area, checking out that package, building it with scram, and then giving the command scram b runtests. A typical output from runing this command in the directory FWCore/Framework/test is shown HERE.

Unit tests are also run as part of the Integration Builds (IB). One can look at the unit tests performed on all the packages in a IB by looking at the CMSSW Integration build webpage. On this webpage click on any "details" in the "builds" column. This will bring another webpage where one can look (in the column UnitTest logfile) at the log file for the unit tests run for different packages as part of IB. Not all packages have a unit tests.

Another thing to pay attention is that IB infrastructure expects a 0 exit code for successful tests, and any other number indicates a failure. cmsRun applications already obey this rule. Developers writing their own scripts as tests should be aware of this. In the unit test out put HERE search for word "status" and see if a unit test ran fine i.e. has status = 0.

The unit tests should be defined as a bin target in the CMS.BuildFile file that is in the test directory of a package or a lower directory. All the .cc .cpp files that your unit test needs must also reside in the test directory. Look at the following as examples:

  • FWCore/Framework/test/CMS.BuildFile.xml
  • FWCore/Integration/test/CMS.BuildFile.xml
  • CMS.PhysicsTools/FWLite/test/CMS.BuildFile

Let us take couple of snippets from a BuildFile to see what they do/mean:

The first snippet:

  • <bin   name="TestFWCoreFrameworkView" file="View_t.cpp">
     <use   name="DataFormats/Common"/>
     <use   name="cppunit"/>
    </bin>
    

The 'bin' part defines a new test that should be run where 'name' is going to be the name for the executable and 'file' contains a comma separated list of all files which should be compiled together to create the binary. The 'use' within the 'bin' tells scram what packages (DataFormats/Common) or external tools (cppunit) are needed to compile and link the executable.

The second snippet:

  • <bin   name="TestFWCoreFrameworkInputTagFailure" file="TestDriver.cpp">
     <flags   TEST_RUNNER_ARGS=" /bin/bash FWCore/Framework/test test_InputTag_cache_failure.sh"/>
     <use   name="FWCore/Utilities"/>
    </bin>
    

To know what this snippet means have look HERE.

Here are some links that may serve as examples of different kind of unit tests:

  • test that runs a simple executable with a "main" function. The output of this test is HERE.
  • test that uses the cppunit package which is an external product specifically designed to be used for unit tests. The output of this test is HERE.
  • one that uses a shell script to run cmsRun. The output of this test is HERE.

The unit tests are expected to be fast. Less than a second is good, a few seconds is OK, but if it takes minutes to run (or longer), then it should probably be redesigned to run faster or not be a unit test.

Check-in (or commit) your code

While you are working on your code, tests and documentation (but at latest when you are done) you have to commit (or check-in) your code/documents back into the CVS repository. To do this, you do:

  • cd MySubsystem/MyPackage
  • if you have new files to add, you will need to mark them with cvs add filename
  • cvs ci # this will prompt you for a comment which will tell you and other people what the changes are
  • OR cvs ci -m"a comment describing your changes" here you give the descriptive comment in-line

you can also give a list of files to the cvs ci command if you want (usually) to give a separate comment to each (or a group/subset) of your files, for example:

  • cd MySubsystem/MyPackage
  • cvs ci -m"changed to new plugin management" CMS.BuildFile # only the CMS.BuildFile is committed.
  • cvs ci -m"adapted to design iteration for naming of header files" src/*.cpp # all cpp files in src/ are committed.

Removing files from cvs

If you need to remove files from the cvs directory (i.e. obsolete files), do:

  • cvs update -A
  • cvs delete
  • cvs commit -m 'removing stuff'

You must not cvs delete files starting from an existing tag without first doing a cvs update -A (or else you will change that tag).

Tag your package

When you are satisfied with your developement, you might apply an unofficial tag , which is a tag not in the form Vxx-yy-zz, and ask your package administrator to turn it into an anofficial tag and publish it.
If you are a package administrator, then you probably want to publish the tag yourself:

  • cd MySubsystem/MyPackage
  • cvs status -v CMS.BuildFile to see the list of tags
  • cvs tag Vxx-yy-zz . Compared with the previous tag you should increase, the zz number for patches, the yy number for a feature addition, the xx number for a (backward incompatible) change in interface.

Check your tag

It is very good practice to perform a fresh check out of your package in a clean area, build it and test it to make sure you did not forget to tag or commit any file. This can be done with cvs co -r Vxx-yy-zz ~MySubsystem/MyPackage , preferably in a new area.

Submit your tag to the integration system

So much for the tagging. You then need to sumbit the new tag to the TagCollector:

  • Sign in to the CMS TagCollector.
  • Go to the Package Publication and Queue page (button is in the header/top of the page).
  • Fill the text box with MySubsystem/MyPackage Vxx-yy-zz (you can also use the controls on the right to query CVS for the tags and to add them to the text box).
  • Select the type of the change.
  • Select the architectures you tested the changes on.
  • Select the releases to queue for.
  • Click on Publish & Queue.
  • A result page should appear.

Read the TagCollector's Documentation and Tutorial for more information.

Checking the nightly

You can follow the status of your packages at the nightly page. The system will send you an email in case of build failure.

Changing CMSSW version during development cycle

If your development takes a long time, you may need to switch from an old to a new CMSSW version, before you have completed it. This can be done by:

  • scramv1 project CMSSW CMSSW_new
  • cd CMSSW_new/src
  • cp -pr ../../CMSSW_old/src/* .
  • cvs update -r CMSSW_new (Warning, if any new subdirectories have been added, these will only be given to you if you use cvs update -d).
  • cvscheck (and then correct by hand any conflicts it shows).

Requesting a New Package

To request a new package, use the Create New Package page of the CMS TagCollector. The request will be forwarded to the list hn-cms-newPackageRequests@cernNOSPAMPLEASE.ch. If there is no objection from Subsystem Administrators, one of the Release Managers will create the package in CVS (via Subsystem/.admin/NewLeaf) and approve your package request in TagCollector.

Start filling the new package

You might find convenient to use a script to create templates of framework producers, analyzers, etc: templates

The Release Procedure

See this page, ReleaseSchedule, for information about release policy, and particular releases.

FAQ

Here is the link to the FAQ page

Review Status

Reviewer/Editor and Date (copy from screen) Comments
Main.argiro - 10 Apr 2006 page created
BenediktHegner - 20 Dec 2006 page content last edited
JennyWilliams - 07 Feb 2007 twiki tidying
AndreasPfeiffer - 31 May 2007 added ref to coding rules in SWGuide
AndreasPfeiffer - 25 July 2007 updated Integration Builds (former "nightly")
IanTomalin - 28 Aug 2008 Added "changing CMSSW version during development cycle" section
ShahzadMuzaffar - 14 Nov 2008 Added ".admin/developers for CMSSW CVS access control" section
SudhirMalik - 13 April 2010 Added description of the unit tests for a release
MiguelOjeda - 05 May 2011 Updated several sections regarding TagCollector

Responsible: Main.argiro
Last reviewed by: Reviewer

Topic attachments
I Attachment History Action Size Date Who Comment
Texttxt TestFWCoreFrameworkInputTagFailure.txt r1 manage 3.5 K 2010-04-15 - 19:09 SudhirMalik  
Texttxt TestFWCoreFrameworkRegistryTemplate.txt r1 manage 0.2 K 2010-04-15 - 19:07 SudhirMalik  
Texttxt TestFWCoreFrameworkeventprincipal.txt r1 manage 0.3 K 2010-04-15 - 19:08 SudhirMalik  
Texttxt UnitTestOutput.txt r1 manage 193.3 K 2010-04-16 - 07:23 SudhirMalik  
Edit | Attach | Watch | Print version | History: r53 < r52 < r51 < r50 < r49 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r51 - 2016-06-25 - AndreaBocci



 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback