SWGuideRunningGridPrerequisites < CMSPublic

CMSPublic Web>SWGuide>SWGuideCrab>SWGuideRunningGridPrerequisites (2020-05-13, StefanoBelforte) (raw view)
---+ !CRAB Prerequisites

%TOC{title="Contents:"}%

---++ Grid certificate

---+++Get a Grid certificate and the registration to CMS VO

<!--STARTSWGUIDEPREREQ0-->
CRAB submits jobs to the Grid (LCG), so you need to run it from an User Interface, with a valid certificate, issued by your appropriate Certification Authority, and have a valid proxy. You need also to be registered on VORMS server. To get a certificate from CERN CA and register to CMS VO, you can find detailed instruction in the  SWGuideLcgAccess page. If you get a certificate from another Certification Authority, the procedure to register to CMS VO with your certificate should be the same. 
<!--STOPSWGUIDEPREREQ0-->

---+++Setup your certificate for LCG

<!--STARTSWGUIDEPREREQ1-->
See instructions in [[WorkBookStartingGrid#ObtainingCert][this Offline Workbook page]]

<!--STOPSWGUIDEPREREQ1-->

---+++Test your grid certificate

<!--STARTSWGUIDEPREREQ2-->
<ol>
<li>Is your personal certificate able to generate Grid proxies? To find out, after having setup your environment run this command:<pre>
grid-proxy-init -debug -verify </pre>
In case of failure, the possible causes are:
<ul>
<li> the certificate/key pair is not installed in =$HOME/.globus/usercert.pem $HOME/.globus/userkey.pem= (a.k.a. "pem files")
<li> the certificate has expired
<li> the certificate and the private key do not match
</ul>
In the first case, you either do not have a certificate at all or have to install it on the UI; in the second case, you should get a new certificate; in the third case you probably have incorrectly installed your certificate.
<li>Are you a member of the CMS VO? To see if this is the case, you can execute this command: <pre>
voms-proxy-init -voms cms </pre>
If you get an error, chances are that you did not register to the CMS VO, or your registration expired. In this case, please follow the instructions in the SWGuideLcgAccess page
<li> You can verify the expiration date of your certificate with:<pre>
openssl x509  -subject -dates -noout  -in $HOME/.globus/usercert.pem </pre>
<li> see also: https://twiki.cern.ch/twiki/bin/view/CMSPublic/SWGuideVomsFAQ
</ol>
<!--STOPSWGUIDEPREREQ2-->

---++ !SiteDB registration

There is no such need since end of 2018. Forget about SiteDB

---++ CMSSW

---+++ Get CMSSW

Please refer to section [[WorkBookCMSSWFramework#AccessCmsswCode][Access CMSSW Code]] of the CMS Offline !WorkBook.

---+++ Test the code locally

<!--STARTSWGUIDEPREREQ3-->
Before launching a million event analysis job on Grid, be sure to test your code locally in a clean area.
   1 Build a new CMSSW area (for example, CMSSW_%WBRELEASE%_...; pick as appropriate to your job): <pre>
cmsrel CMSSW_%WBRELEASE%
cd CMSSW_%WBRELEASE%/src
cmsenv</pre>
   1 Check-out from the cvs repository only the code or configuration files you need to modify, and build your local libraries including your analysis code.
   1 Make sure that the code you check-out is compatible with the CMSSW version you are using.
   1 Make sure that the CMSSW version you are using is compatible with the data you intend to read.
   1 Prepare a test job accessing the data you will access in your Grid job. There are several ways to read the proper data:
      * The easiest way is to use the xrootd service to read data directly from a remote site. How to do this is explained in [[WorkBookXrootdService][Using Xrootd Service for remote Data Accessing]].
      * You can also use the xrootd service to copy a data file from a suitable dataset to your local machine (to work w/o network e.g.), as explained in [[WorkBookXrootdService#DownCmd][File download with command-line tools]].
      * If no suitable files exist, you can generate some events using the configuration file which is available from the [[https://cmsweb.cern.ch/das/][DAS]] service.
   1 Test your CMSSW configuration file locally in order to avoid problems with the !ParameterSet parsing.
   1 Run the job interactively (e.g. at CERN on lxplus): <pre>
cmsRun your-pset-config-file.py </pre>
<!--STOPSWGUIDEPREREQ3-->

---+++Validate a CMSSW config file

<!--STARTSWGUIDEPREREQ4-->
In CRAB2, a user can validate its CMSSW configuration file by launching =crab -validateCfg= after creating the task with =crab -create=. In this way the configuration file will be controlled and validated by a corresponding python API. Note that it is *not* enough to check that the configuration file runs interactively, because in interactive mode CMSSW is too tolerant with python errors in that configuration file.
At times a user may worry that the problem is in CRAB or CRAB validation rather than in the configuration file; in this case, one can use the following test, which does not involve CRAB:
<pre>
edmConfigHash your-pset-config-file.py
</pre>
Note that this is needed, but not necessarely sufficient, to have a valid CMSSW configuration file.
Other problem could be related to some hidden charatecters (^M) in the configuration file, overall if it was downloaded from the web. To discover them you can use the command
<pre>
cat -v your-pset-config-file.py
</pre>
and remove them with the command
<pre>
perl -pi -e 'tr/\cM//d;' your-pset-config-file.py
</pre>
Then you can revalidate the configuration file again.
<!--STOPSWGUIDEPREREQ4-->

---++ Figuring out what data to analyze

This section is documented in chapter [[WorkBookLocatingDataSamples][Locating Data Samples]] of the CMS Offline !WorkBook.

---++ CRAB client installation and setup the environment

---+++ Use !CRAB at CERN

<!--STARTSWGUIDEPREREQ5-->
please see SWGuideCrab, in particular : https://twiki.cern.ch/twiki/bin/view/CMSPublic/CRAB3CheatSheet#Environment_setup
<!--STOPSWGUIDEPREREQ5-->

---+++Use !CRAB outside CERN

<!--STARTSWGUIDEPREREQ6-->
---++++ Preferred way: use CVMFS
   1 Setup the Grid UI according to your site directions
   1 Follow same instructions as at CERN (CVMFS is globally available)

<!--STOPSWGUIDEPREREQ6-->

%RESPONSIBLE% Main.StefanoBelforte %BR% 

-- Main.MarcoCalloni - 28-Jan-2010
Topic revision: r29 - 2020-05-13 - StefanoBelforte
CMSPublic
Create a LeftBar
Public webs
- Cern Search
- TWiki Search
- Google Search
CMSPublic All webs
Copyright &© 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback