SWGuideCrabAtCAF < CMSPublic

CMSPublic Web>SWGuide>SWGuideCrab>SWGuideCrabAtCAF (2011-10-13, StephenJGowdy) (raw view)
---+ !CRAB at !CAF/LSF at CERN

%COMPLETE5%
%BR%
[[#ReviewStatus][Detailed Review status]]


%TOC{title="Contents:"}%

| *Newsbox* |
| under review|


<hr><!-- ====================================================================== -->


This document describes how to use !CRAB at CERN for direct submission to the batch system LSF or CERN Analysis Facility (!CAF). The responsable of CAF is Peter Kreuzer.
Useful information of !CAF can be found in the [[https://twiki.cern.ch/twiki/bin/view/CMS/CAF][CAF twiki page]].

---+++ Prerequisites

<!--STARTCRABATCAF0-->

   * Dataset you want to access has to be available at the CAF, so it must be registered in the CAF !DBS
   * If you run on !CAF, you have to be authorized to do so. In this page: https://twiki.cern.ch/twiki/bin/view/CMS/CAF#User_Permissions you can find the sub-groups and the correspond leader. If you know your sub-group, you can contact the leader for the authorization.
   * !CRAB !StandAlone (direct submission)
      * Jobs has to be submitted from an afs directory, from a node with !LSF access for exemple on lxplus
      * Since in this case you are effectively using !CRAB as a convenience tool to do LSF submission from your shell, you need to setup the environment as usual:
         * make sure you setup the environment in the following order
            *<pre>
1.source /afs/cern.ch/cms/LCG/LCG-2/UI/cms_ui_env.sh
2. cmsenv
3. source /afs/cern.ch/cms/ccs/wm/scripts/Crab/crab.sh
</pre>
            * in the above replace =sh= with =csh= if you are using tcsh
      * Please note that you must be sure to have enough quota on your afs area. Large output should be put on castor (look at !CAF stageout below)
      * Even if you decided to send the output to castor, the stdout/err and the Framework Job Report will be returned back to your afs area in any case.
      * %RED%Removes the requirement to use an AFS directory and a host with !LSF access, so can also submit from your desktop/laptop%ENDCOLOR%

<!--STOPCRABATCAF0-->

---+++ Running 
<!--STARTCRABATCAF1-->
The workflow is exactly the same of that you would follow to access data on the Grid (see: [[https://twiki.cern.ch/twiki/bin/view/CMS/WorkBookCRABTutorial][CRAB Tutorial]]). So you setup your !CMSSW area, you develop your code, test it on a (small) part of a dataset and then you configure !CRAB to create and submit identical jobs to !CAF to analyze the full Dataset.
In the crab.cfg configuration file, you have just to put under the =[CRAB]= section: 
%BR% 
=scheduler = caf= %BR% 
<!--STOPCRABATCAF1-->

The available !CAF queues are:

cmscaf1nh %BR% 
cmscaf1nd %BR% 
cmscaf1nw %BR% 

Running on the !CAF, using =caf= as scheduler instead of =lsf=, the longest queue will be selected automatically (cmscaf1nw). 
If you need to select a different queue you can fill the parameter =queue= under the =[CAF]= section with either cmscaf1nh or cmscaf1nd (i.e. =queue = cmscaf1nh=).
If you know that your jobs are short, it should be more efficient to use shorter queues.


---+++ !CAF stageout
<!--STARTCRABATCAF2-->
If you are running jobs at !CAF then the required stageout configuration is:

   * Stage out into !CAF user area  (T2_CH_CERN is the offical site name for !CAF):

    =[USER]=  %BR% 
    =copy_data = 1= %BR% 
    =storage_element=T2_CH_CERN= %BR% 
    =user_remote_dir=xxx= %BR% 

the path where data will be stored is /store/caf/user/&lt;username&gt;/&lt;user_remote_dir&gt;

There is no support for staging out to the CAF-T1 from the GRID. The above instructions only apply for jobs running on the CAF itself.

<!--STOPCRABATCAF2-->

Further details on !CRAB and Stage out configurations available at [[SWGuideCrabFaq#4_Stage_out_in_your_own_director][this page]]. 

---+++ !CAF publication

You need the following in crab configuration file:
   *  (NOTE: the storage element where the data are copied have to be T2_CH_CERN):
<pre> 
[USER]
copy_data = 1
storage_element=T2_CH_CERN
publish_data=1
publish_data_name = data-name-to-publish  (e.g. publish_data_name = JohnSmithTestDataVersion666 )
dbs_url_for_publication = https://cmsdbsprod.cern.ch:8443/cms_dbs_caf_analysis_01_writer/servlet/DBSServlet
</pre>

The path where data will be stored is /store/caf/user/&lt;username&gt;/&lt;primarydataset&gt;/&lt;publish_data_name&gt;/&lt;PSETHASH&gt; 



#ReviewStatus
---++ Review status 

<!-- Add your review status in this table structure with 2 columns delineated by three vertical bars. Add comments for editing, reviewing, etc -->

| *Reviewer/Editor and Date (copy from screen)* | *Comments* |
| Main.MarcoCalloni - 18 Dec 2008 | linked to workbook, needs to be renamed swguide -> workbook |
| Main.StefanoLacaprara - 13 Mar 2008 | created the page |

<!-- Here the "responsible" is the contact person for maintenance of the page - often initially the page author -->
<!-- The "review" person is the most recent person to carry out a full review of the page  - on this field please include the date in the format shown -->
%RESPONSIBLE% Main.MarcoCalloni  %BR%
%REVIEW%
Topic revision: r43 - 2011-10-13 - StephenJGowdy
CMSPublic
Create a LeftBar
Public webs
- Cern Search
- TWiki Search
- Google Search
CMSPublic All webs
Copyright &© 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback