CRAB Prerequisites page

Grid certificate

Get a Grid certificate and the registration to CMS VO

CRAB submits jobs to the Grid (LCG), so you need to run it from an User Interface, with a valid certificate, issued by your appropriate Certification Authority, and have a valid proxy. You need also to be registered on VORMS server. To get a certificate from CERN CA and register to CMS VO, you can find detailed instruction here: https://twiki.cern.ch/twiki/bin/view/CMS/WebHome?topic=SWGuideLcgAccess. If you get a certificate from another Certification Authority, the procedure to register to CMS VO with your certificate should be the same.

Setup your certificate for LCG

Please look at the guide from CERN Certification Authority here: https://ca.cern.ch/ca/Help/?kbid=024010 . For more information you can see the gLite User Guide: https://edms.cern.ch/file/722398/1.4/gLite-3-UserGuide.pdf .

  • if you are renewing an existing certificate make sure you delete the old .pem files first.

Test your grid certificate

  1. Is your personal certificate able to generate Grid proxies? To find out, after having setup your environment run this command:
    grid-proxy-init -debug -verify 
    In case of failure, the possible causes are:
    • the certificate/key pair is not installed under $HOME/.globus/
    • the certificate has expired
    • the certificate and the private key do not match
    In the first case, you either do not have a certificate at all or have to install it on the UI; in the second case, you should get a new certificate; in the third case you probably have incorrectly installed your certificate.
  2. Are you a member of the CMS VO? To see if this is the case, you can execute this command:
    voms-proxy-init -voms cms 
    If you get an error, chances are that you did not register to the CMS VO, or your registration expired. In this case, please follow the instructions in this page: https://twiki.cern.ch/twiki/bin/view/CMS/WebHome?topic=SWGuideLcgAccess .

SiteDB registration

Read the page https://twiki.cern.ch/twiki/bin/view/CMS/SiteDBForCRAB to see if you have to take some action to register in SiteDB (and eventually follow the instructions there).

CMSSW

Get CMSSW

Look at the CMS Offline WorkBook chapter 2.3 CMSSW Application Framework section Access CMSSW Code.

Test the code locally

Before launching a million event analysis job on Grid, be sure to test your code locally in a clean area.

  1. Build a new CMSSW area (for example, CMSSW_7_4_15_...; pick as appropriate to your job):
    scramv1 project -n CMSSW_7_4_15_gridjobs CMSSW CMSSW_7_4_15 
    cd CMSSW_7_4_15_gridjobs/src
    eval `scramv1 runtime -csh`  *or*  eval `scramv1 runtime -sh` 
  2. Check-out from the cvs repository only the code or configuration files you need to modify, and build your local libraries including your analysis code.
  3. Make sure that the code you check-out is compatible with the CMSSW version you are using.
  4. Make sure that the CMSSW version you are using is compatible with the data you intend to read.
  5. Prepare a test job accessing the data you will access in your Grid job. There are several ways to read the proper data:
  6. Test your CMSSW configuration file locally in order to avoid problems with the ParameterSet parsing.
  7. Run the job interactively (e.g. at CERN on lxplus):
    cmsRun your-pset-config-file.py 

Validate a CMSSW config file

In CRAB2, a user can validate its CMSSW configuration file by launching crab -validateCfg after creating the task with crab -create. In this way the configuration file will be controlled and validated by a corresponding python API. Note that it is not enough to check that the configuration file runs interactively, because in interactive mode CMSSW is too tolerant with python errors in that configuration file. At times a user may worry that the problem is in CRAB or CRAB validation rather than in the configuration file; in this case, one can use the following test, which does not involve CRAB:

edmConfigHash your-pset-config-file.py
Note that this is needed, but not necessarely sufficient, to have a valid CMSSW configuration file. Other problem could be related to some hidden charatecters (^M) in the configuration file, overall if it was downloaded from the web. To discover them you can use the command
cat -v your-pset-config-file.py
and remove them with the command
perl -pi -e 'tr/\cM//d;' your-pset-config-file.py
Then you can revalidate the configuration file again.

Figuring out what data to analyze

Look at the CMS Offline WorkBook chapter 5.4 Locating Data Samples.

CRAB client installation and setup the environment

Use CRAB at CERN

The following prescription works on lxplus. On other UI's at remote sites, site admins should make sure the locally installed LCG UI and CRAB versions are the same (or compatible) with what used there. To prepare correctly your local environment, please execute the following recipe respecting the order:

  1. Make sure you are starting in a clean shell (e.g. right after ssh lxplus).
  2. Have an access to a LCG User Interface (LCG UI):
    source /afs/cern.ch/cms/LCG/LCG-2/UI/cms_ui_env.(c)sh 
  3. Source the CMS environment:
    cd CMSSW_X_Y_Z/src/
    cmsenv 
    where CMSSW_X_Y_Z is the CMSSW project installed in a your local directory.
  4. Source the CRAB environment:
    source /afs/cern.ch/cms/ccs/wm/scripts/Crab/crab.(c)sh 

Use CRAB outside CERN

  • Download the latest CRAB_X_Y_Z tgz archive (specified in https://twiki.cern.ch/twiki/bin/view/CMSPublic/SWGuideCrab#CRAB_Releases_Notes):
    wget http://cmsdoc.cern.ch/cms/ccs/wm/www/Crab/Docs/CRAB_X_Y_Z.tgz 
  • Untar the tgz file in a directory "YOUR_DIR" of your UserInterface:
    tar -xvzf CRAB_X_Y_Z.tgz 
  • Run the configure file you can find in "YOUR_DIR"/CRAB_X_Y_Z/ from a clean shell:
    cd YOUR_DIR/CRAB_X_Y_Z/
    ./configure 
    It will configure the installation and create the crab.(c)sh file. This can be done by you or by your sistem-administrator for the site-wide installation. This has to be done just once.
  • Close your installation shell.
  • Open a new shell, load the environment and follow the source file order:
    • Source the appropriate file to have the Grid UI software available.
    • Source the CMS environment. From your CMSSW directory, do:
      eval `scramv1 runtime -(c)sh` 
    • Source the CRAB environment:
      source YOUR_DIR/CRAB_X_Y_Z/crab.(c)sh 

Responsible: StefanoBelforte

-- MarcoCalloni - 28-Jan-2010

Edit | Attach | Watch | Print version | History: r29 | r22 < r21 < r20 < r19 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r20 - 2014-04-23 - AndresTanasijczuk
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback