CRAB Prerequisites

Grid certificate

Get a Grid certificate and the registration to CMS VO

CRAB submits jobs to the Grid (LCG), so you need to run it from an User Interface, with a valid certificate, issued by your appropriate Certification Authority, and have a valid proxy. You need also to be registered on VORMS server. To get a certificate from CERN CA and register to CMS VO, you can find detailed instruction in the SWGuideLcgAccess page. If you get a certificate from another Certification Authority, the procedure to register to CMS VO with your certificate should be the same.

Setup your certificate for LCG

See instructions in this Offline Workbook page

Test your grid certificate

  1. Is your personal certificate able to generate Grid proxies? To find out, after having setup your environment run this command:
    grid-proxy-init -debug -verify 
    In case of failure, the possible causes are:
    • the certificate/key pair is not installed under $HOME/.globus/
    • the certificate has expired
    • the certificate and the private key do not match
    In the first case, you either do not have a certificate at all or have to install it on the UI; in the second case, you should get a new certificate; in the third case you probably have incorrectly installed your certificate.
  2. Are you a member of the CMS VO? To see if this is the case, you can execute this command:
    voms-proxy-init -voms cms 
    If you get an error, chances are that you did not register to the CMS VO, or your registration expired. In this case, please follow the instructions in the SWGuideLcgAccess page

SiteDB registration

Read the page https://twiki.cern.ch/twiki/bin/view/CMS/SiteDBForCRAB to check if you have to take some action to register in SiteDB (and eventually follow the instructions there).

CMSSW

Get CMSSW

Please refer to section Access CMSSW Code of the CMS Offline WorkBook.

Test the code locally

Before launching a million event analysis job on Grid, be sure to test your code locally in a clean area.

  1. Build a new CMSSW area (for example, CMSSW_7_4_15_...; pick as appropriate to your job):
    scramv1 project -n CMSSW_7_4_15_gridjobs CMSSW CMSSW_7_4_15 
    cd CMSSW_7_4_15_gridjobs/src
    eval `scramv1 runtime -csh`  *or*  eval `scramv1 runtime -sh` 
  2. Check-out from the cvs repository only the code or configuration files you need to modify, and build your local libraries including your analysis code.
  3. Make sure that the code you check-out is compatible with the CMSSW version you are using.
  4. Make sure that the CMSSW version you are using is compatible with the data you intend to read.
  5. Prepare a test job accessing the data you will access in your Grid job. There are several ways to read the proper data:
    • The easiest way is to use the xrootd service to read data directly from a remote site. How to do this is explained in Using Xrootd Service for remote Data Accessing.
    • You can also use the xrootd service to copy a data file from a suitable dataset to your local machine (to work w/o network e.g.), as explained in File download with command-line tools.
    • If no suitable files exist, you can generate some events using the configuration file which is available from the DAS service. ( OLD: choose Configs after the dataset name from the DBS view).
  6. Test your CMSSW configuration file locally in order to avoid problems with the ParameterSet parsing.
  7. Run the job interactively (e.g. at CERN on lxplus):
    cmsRun your-pset-config-file.py 

Validate a CMSSW config file

In CRAB2, a user can validate its CMSSW configuration file by launching crab -validateCfg after creating the task with crab -create. In this way the configuration file will be controlled and validated by a corresponding python API. Note that it is not enough to check that the configuration file runs interactively, because in interactive mode CMSSW is too tolerant with python errors in that configuration file. At times a user may worry that the problem is in CRAB or CRAB validation rather than in the configuration file; in this case, one can use the following test, which does not involve CRAB:

edmConfigHash your-pset-config-file.py
Note that this is needed, but not necessarely sufficient, to have a valid CMSSW configuration file. Other problem could be related to some hidden charatecters (^M) in the configuration file, overall if it was downloaded from the web. To discover them you can use the command
cat -v your-pset-config-file.py
and remove them with the command
perl -pi -e 'tr/\cM//d;' your-pset-config-file.py
Then you can revalidate the configuration file again.

Figuring out what data to analyze

This section is documented in chapter Locating Data Samples of the CMS Offline WorkBook.

CRAB client installation and setup the environment

Use CRAB at CERN

The following prescription works on lxplus. On other UI's at remote sites, site admins should make sure the locally installed LCG UI and CRAB versions are the same (or compatible) with what used there. To prepare correctly your local environment, please execute the following recipe respecting the order:

  1. Make sure you are starting in a clean shell (e.g. right after ssh lxplus).
  2. Have an access to a LCG User Interface (LCG UI):
    source /afs/cern.ch/cms/LCG/LCG-2/UI/cms_ui_env.(c)sh 
  3. Setup the CMS environment:
    cd CMSSW_X_Y_Z/src/
    cmsenv 
    where CMSSW_X_Y_Z is the CMSSW project installed in a your local directory.
  4. Setup the CRAB environment:
    source /afs/cern.ch/cms/ccs/wm/scripts/Crab/crab.(c)sh 

Use CRAB outside CERN

FNAL LPC

The setup at FNAL is similar to CERN one, but with slightly different commands. Here is for SL5
source /uscmst1/prod/grid/gLite_SL5.(c)sh
cmsenv
source /uscmst1/prod/grid/CRAB/crab.(c)sh

Notice that these settings change roughly every few months. The most up to date instructions are always at SWGuideCrab#CRAB_Usage_at_LPC_CAF_T3_US_FNAL which may also explain how to use other environments.

Other sites

Preferred way: use CVMFS
  1. Setup the Grid UI according to your site directions
  2. Setup the CMS environment:
    cd CMSSW_X_Y_Z/src/
    cmsenv 
    where CMSSW_X_Y_Z is the CMSSW project installed in a your local directory.
  3. Setup the CRAB environment:
    source /cvmfs/cms.cern.ch/crab/crab.(c)sh 
Alternative way, use tarball
  • Download the latest CRAB_X_Y_Z tgz archive :
    wget http://cmsdoc.cern.ch/cms/ccs/wm/www/Crab/Docs/CRAB_current.tgz 
    • this will download a file with a version number, like CRAB_X_Y_Z.tgz
  • Untar the tgz file in a directory "YOUR_DIR" of your UserInterface:
    tar -xvzf CRAB_X_Y_Z.tgz 
  • Run the configure file you can find in "YOUR_DIR"/CRAB_X_Y_Z/ from a clean shell:
    cd YOUR_DIR/CRAB_X_Y_Z/
    ./configure 
    It will configure the installation and create the crab.(c)sh file. This can be done by you or by your sistem-administrator for the site-wide installation. This has to be done just once.
  • Close your installation shell.
  • Open a new shell, load the environment and follow the source file order:
    • Source the appropriate file to have the Grid UI software available.
    • Setup the CMS environment. From your CMSSW directory, do:
      eval `scramv1 runtime -(c)sh` 
    • Setup the CRAB environment:
      source YOUR_DIR/CRAB_X_Y_Z/crab.(c)sh 

Responsible: StefanoBelforte

-- MarcoCalloni - 28-Jan-2010

Edit | Attach | Watch | Print version | History: r29 | r26 < r25 < r24 < r23 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r24 - 2014-09-20 - StefanoBelforte
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback