Complete: 5
Detailed Review status

under review

This document describes how to use CRAB at CERN for direct submission to the batch system LSF or CERN Analysis Facility (CAF). The responsable of CAF is Peter Kreuzer. Useful information of CAF can be found in the CAF twiki page.


  • Dataset you want to access has to be available at the CAF, so it must be registered in the CAF DBS
  • If you run on CAF, you have to be authorized to do so. In this page: you can find the sub-groups and the correspond leader. If you know your sub-group, you can contact the leader for the authorization.
  • CRAB StandAlone (direct submission)
    • Jobs has to be submitted from an afs directory, from a node with LSF access for exemple on lxplus
    • Since in this case you are effectively using CRAB as a convenience tool to do LSF submission from your shell, you need to setup the environment as usual:
      • make sure you setup the environment in the following order *
        1.source /afs/
        2. cmsenv
        3. source /afs/
        • in the above replace sh with csh if you are using tcsh
    • Please note that you must be sure to have enough quota on your afs area. Large output should be put on castor (look at CAF stageout below)
    • Even if you decided to send the output to castor, the stdout/err and the Framework Job Report will be returned back to your afs area in any case.
    • Removes the requirement to use an AFS directory and a host with LSF access, so can also submit from your desktop/laptop


The workflow is exactly the same of that you would follow to access data on the Grid (see: CRAB Tutorial). So you setup your CMSSW area, you develop your code, test it on a (small) part of a dataset and then you configure CRAB to create and submit identical jobs to CAF to analyze the full Dataset. In the crab.cfg configuration file, you have just to put under the [CRAB] section:
scheduler = caf

The available CAF queues are:


Running on the CAF, using caf as scheduler instead of lsf, the longest queue will be selected automatically (cmscaf1nw). If you need to select a different queue you can fill the parameter queue under the [CAF] section with either cmscaf1nh or cmscaf1nd (i.e. queue = cmscaf1nh). If you know that your jobs are short, it should be more efficient to use shorter queues.

CAF stageout

If you are running jobs at CAF then the required stageout configuration is:

  • Stage out into CAF user area (T2_CH_CERN is the offical site name for CAF):

copy_data = 1

the path where data will be stored is /store/caf/user/<username>/<user_remote_dir>

There is no support for staging out to the CAF-T1 from the GRID. The above instructions only apply for jobs running on the CAF itself.

Further details on CRAB and Stage out configurations available at this page.

CAF publication

You need the following in crab configuration file:

  • (NOTE: the storage element where the data are copied have to be T2_CH_CERN):
copy_data = 1
publish_data_name = data-name-to-publish  (e.g. publish_data_name = JohnSmithTestDataVersion666 )
dbs_url_for_publication =

The path where data will be stored is /store/caf/user/<username>/<primarydataset>/<publish_data_name>/<PSETHASH>

Review status

Reviewer/Editor and Date (copy from screen) Comments
MarcoCalloni - 18 Dec 2008 linked to workbook, needs to be renamed swguide -> workbook
StefanoLacaprara - 13 Mar 2008 created the page

Responsible: MarcoCalloni
Last reviewed by:

Edit | Attach | Watch | Print version | History: r43 < r42 < r41 < r40 < r39 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r42 - 2011-09-02 - StephenJGowdy
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2023 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback