TWiki> CMSPublic Web>CRAB>WorkBookCRAB2Tutorial (revision 1)EditAttachPDF

Running CMSSW code on the Grid using CRAB


Recipe for the tutorial

For this tutorial we will use:


we will use already prepared CMSSW analysis code to analyze a XXX sample, which replicates a real analysis scenario.

  • Location:


  • CRAB_2_2_2

using the central installation available at CERN

The example is written to use the csh shell family

If you want to use sh replace csh with sh.

Setup local Environment and prepare user analysis code

In order to submit jobs to the Grid, you must have an access to a LCG User Interface (LCG UI). It will allow you to access WLCG-affiliated resoures in a fully transparent way. LXPLUS users can get an LCG UI via AFS by:

source /afs/

Install CMSSW project in a directory of your choice:

mkdir Tutorial
cd Tutorial
scramv1 p CMSSW CMSSW_x_y_z
cd CMSSW_x_y_z/src
eval `scramv1 runtime -csh`

get and compile the example of the user analysis code

tar zxvf Demo.tgz
scramv1 b

CRAB setup

Most users (particularly those on LXPLUS) do not need to install CRAB. They only need to set it up.

CRAB is intended to be installed in a private area for use by a single person, or in a common area for use by all system users. A public installation is available on CERN's LXPLUS. At CERN on LXPLUS, users may access CRAB at (shown for arbitrary version X_Y_Z):

To know the latest release check CRAB web page or proper HyperNews forum.

Setup on lxplus:

In order to setup and use CRAB from any directory, source the the script crab.(c)sh located in /afs/, which always points to the latest version of CRAB. After the source of the script it's possible to use CRAB from any directory (tipically use it on your CMSSW working directory).

source /afs/

NOTE: The source of|csh script have to be done after the installation and at the start of every new session, but you need to run configureBoss only the very first time.

Data selection

To select data you want to access, use the DBS web page where available datasets are listed DBS Data Discovery (see links on CRAB home page). For this tutorial we'll use :

Keyword sarch for:
  • *RelVal131Higgs-ZZ-4Mu*

CRAB configuration

Modify the CRAB configuration file crab.cfg according to your needs: a fully documented template is available at $CRABDIR/python/crab.cfg . For guidance, see the list and description of configuration parameters. For this tutorial, the only relevant sections of the file are [CRAB], [CMSSW] and [USER] .

The configuration file should be located at the same location as the CMSSW parameter-set to be used by CRAB. Please change directory to :

cd  Demo/MyTrackAnalyzer/test/ 
and save the crab configuration file:
with the following content:





Run Crab

Once your crab.cfg is ready and the whole underlying environment is set up, you can start run CRAB. CRAB support a command line help which can be useful for the first time. You can get it via:
crab -h
in particular there is a HOW TO RUN CRAB FOR THE IMPATIENT USER section where the base command are reported.

Job Creation

The job creation checks the availability of the selected dataset and prepares all the jobs for submission according to the selected job splitting specifyed on the crab.cfg

The creation process creates a CRAB project directory (default: crab_0__

CRAB allows the user to chose the project name, so that it can be used later to distinguish multiple CRAB projects in the same directory.

crab -create

which should produce a similar screen output like:

fill me

Job Submission

With the submission command it's possible to specify a combination of jobs and job-ranges separated by comma (e.g.: =1,2,3-4), the deafult is all. To submit all jobs of the last created project with the default name, it's enough to execute the following command:

crab -submit 
to submit a specific project:
crab -submit -c  <dir name>

which should produce a similar screen output like:

fill me

Job Status Check

Check the status of the jobs in the latest CRAB project with the following command:
crab -status 
for check a specific project:
crab -status -c  <dir name>

which should produce a similar screen output like:

fill me

Job Output Retrieval

For the jobs which are in status done it's possible to retrieve di output. The following command retrieves the output of all jobs of the last created CRAB project:
crab -getoutput 
to get the output of a specific project:
crab -getoutput -c  <dir name>

the job results will be copied in the res subdir of your crab project, and it's specified by a message like

fill me

Final plot

All 10 jobs produce a histogram output file which can be combined using ROOT in the res directory:

hadd histograms.root histograms_*.root

The final histograms.root opened in ROOT contains the final plot:




In parallel to the execution of the first 10 jobs, the user can try the new GRID submission modes of CRAB by changing

scheduler = edg


scheduler = glite

to use the gLite RB submission mode or to

scheduler = glitecoll

to use the gLite RB bulk submission mode repeating the creation, submission, status check and getoutput steps described above.

CRAB with writing out CMSSW ROOT files

New CMSSW parameter-set

To write out a CMSSW ROOT file in this example, please create a new CMSSW parameter-set named


with following content:

fill me

Prepare Castor area for storage element interaction

For CRAB to be able to write into your Castor user directory:


we have to create a destination directory and change the file permissions:

rfmkdir /castor/<u>/<username>/tutorial 
rfchmod +775 /castor/<u>/<username>/tutorial

replacing <u> with the first letter of your username and <username> with your username.

Prepare new crab.cfg

Now the cmssw parameter-set produces an output file (output.root) which the user can include into output file and can ask to CRAB to copy it in the Sorage Slement (castor). Please modify the crab.cfg as in the following example:



Edit | Attach | Watch | Print version | History: r120 | r4 < r3 < r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r1 - 2008-06-10 - MattiaCinquilli
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback