Difference: SWGuideCrab (1 vs. 163)

Revision 1632019-09-20 - MargueriteTonjes

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 136 to 136
 

Monitoring

CRAB Monitoring links for users (and ops):
Changed:
<
<
>
>
 
CRAB Monitoring links for operators

Revision 1622019-09-20 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 132 to 132
 
Notes about CRAB3 release management
Documentation about releasing CRAB3 patches
Notes about CRAB3 validation
Some notes about the deployment procedure and validation process of new CRAB3 versions
Deleted:
<
<
#Monitoring

Monitoring

New Monitoring:
 
Changed:
<
<
>
>

Monitoring

CRAB Monitoring links for users (and ops):
CRAB Monitoring links for operators
 
Old Monitoring:
Deleted:
<
<
 
Deleted:
<
<
 

Links to relevant projects

Line: 224 to 223
 
Added:
>
>
 
Deleted:
<
<
 
Changed:
<
<
>
>
 

Revision 1602019-04-01 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 101 to 101
 
CRAB3 frontend installation
Documentation about how to deploy the CRAB server frontend (i.e. CRAB REST Interface and CRAB Cache).
<!-- $  CRAB3 frontend installation/ slc7: Documentation about how to deploy the CRAB server frontend on an slc7 machine (i.e. CRAB REST Interface and CRAB Cache on an slc7 machine). -->
CRAB3 backend installation
Documentation about how to deploy the CRAB server backend, a.k.a. the CRAB TaskWorker.
Added:
>
>
CRAB Publisher installation
Documentation about how to deploy the CRAB Standalone Publisher (i.e. not part of ASO server)
 
CRAB3 Schedd installation and deployment
Documentation for CRAB3 Schedd deployment.
CRAB3 client installation
Documentation for operators of the CRAB3 client.
ASO installation and deployment
Documentation for ASO deployment and operations.

Revision 1592018-11-30 - TodorTrendafilovIvanov

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 131 to 131
 
Notes about CRAB3 release management
Documentation about releasing CRAB3 patches
Notes about CRAB3 validation
Some notes about the deployment procedure and validation process of new CRAB3 versions
Added:
>
>
#Monitoring
 

Monitoring

New Monitoring:

Changed:
<
<
  • [[][Instructions on Puppet maintenance of the scripts to feed monitoring data to ES ]]
>
>
 
Old Monitoring:

Revision 1582018-11-30 - TodorTrendafilovIvanov

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 105 to 105
 
CRAB3 client installation
Documentation for operators of the CRAB3 client.
ASO installation and deployment
Documentation for ASO deployment and operations.
Notes about server deployment
Notes about the current deployment of CRAB3 servers.
Changed:
<
<
CRAB Puppet profile
Documentation of the current Puppet profile of the project
>
>
CRAB3 Puppet profile
Documentation of the current Puppet profile of the project
 

Technical documents

Line: 132 to 132
 
Notes about CRAB3 validation
Some notes about the deployment procedure and validation process of new CRAB3 versions

Monitoring

Added:
>
>
New Monitoring:

Old Monitoring:
 
Deleted:
<
<
 

Revision 1572018-09-21 - TodorTrendafilovIvanov

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 232 to 232
 
Added:
>
>
 

Revision 1562018-09-20 - TodorTrendafilovIvanov

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 105 to 105
 
CRAB3 client installation
Documentation for operators of the CRAB3 client.
ASO installation and deployment
Documentation for ASO deployment and operations.
Notes about server deployment
Notes about the current deployment of CRAB3 servers.
Added:
>
>
CRAB Puppet profile
Documentation of the current Puppet profile of the project
 

Technical documents

Revision 1552018-06-14 - LeonardoCristella

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 151 to 151
 
CRAB vs CAF in Run 2
Notes about CAF use cases after discussion between Stefano and several DPG people.
Local Resource Provisioning
Notes on the different possibilities to submit to resources dedicated to local users.
Changed:
<
<
Submitting jobs to the CERN condor pool using preparelocal
A different approach on how to independently submit jobs to a local batch system.
>
>
Submitting jobs to the CERN HTCondor pool using 'crab preparelocal'
A different approach on how to independently submit CRAB jobs to a local batch system.
 

Getting started

Line: 167 to 167
 

Help Getting support

Changed:
<
<
BEFORE CONTACTING Computing Tools, PLEASE CONSULT THE CRAB TROUBLESHOOTING GUIDE AT THIS link
>
>
BEFORE CONTACTING Computing Tools, PLEASE CONSULT THE CRAB TROUBLESHOOTING GUIDE AT THIS LINK:
CRAB3Troubleshoot
 and the CRAB3 Frequently Asked Questions page (and maybe even the CRAB2 Frequently Asked Questions page).

A common mistake is to test the code locally only on a small number of events which is not representative of the whole dataset. If the log of a failed job contains a CMSSW error (e.g. exit code 134, 139), please run your code interactively on the very same event on which the CRAB job failed.

Revision 1542018-06-07 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 113 to 113
 
CMS job exit codes
List of known exit codes for CMSSW and CRAB3.
Notes about CRAB3 service Certificates management
Documentation about the management of all the service certificates in the CRAB3 machines.
Notes about CRAB3 Oracle Database management
Documentation about the management of the oracle database in the CRAB3 project.
Changed:
<
<
>
>
Inside Documentation
Miscellaneous collection of useful insights into how CRAB pieces work internally
 

Code repositories

CrabCodeDevelopment
Short guide to using github for Crab

Revision 1532018-06-06 - MarcoMascheroni

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 151 to 151
 
CRAB vs CAF in Run 2
Notes about CAF use cases after discussion between Stefano and several DPG people.
Local Resource Provisioning
Notes on the different possibilities to submit to resources dedicated to local users.
Added:
>
>
Submitting jobs to the CERN condor pool using preparelocal
A different approach on how to independently submit jobs to a local batch system.
 

Getting started

Revision 1522018-04-17 - TodorTrendafilovIvanov

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 112 to 112
 
CRAB3 State Machine
Documentation about the Taks status tracking inside CRAB Server Data Base
CMS job exit codes
List of known exit codes for CMSSW and CRAB3.
Notes about CRAB3 service Certificates management
Documentation about the management of all the service certificates in the CRAB3 machines.
Changed:
<
<
Notes about CRAB3 Oracle Database management
Documentation about the management of the oracle database in the CRAB3 project.
>
>
Notes about CRAB3 Oracle Database management
Documentation about the management of the oracle database in the CRAB3 project.
 

Code repositories

Revision 1512018-03-28 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 131 to 131
 

Monitoring

Changed:
<
<
>
>
 

Revision 1502018-03-20 - TitasKulikauskas

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->

Revision 1492018-03-13 - TodorTrendafilovIvanov

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 99 to 99
 

Installation

CRAB3 frontend installation
Documentation about how to deploy the CRAB server frontend (i.e. CRAB REST Interface and CRAB Cache).
Added:
>
>
<!-- $  CRAB3 frontend installation/ slc7: Documentation about how to deploy the CRAB server frontend on an slc7 machine (i.e. CRAB REST Interface and CRAB Cache on an slc7 machine). -->
 
CRAB3 backend installation
Documentation about how to deploy the CRAB server backend, a.k.a. the CRAB TaskWorker.
CRAB3 Schedd installation and deployment
Documentation for CRAB3 Schedd deployment.
CRAB3 client installation
Documentation for operators of the CRAB3 client.

Revision 1482018-02-23 - MarcoMascheroni

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 94 to 94
 

Documentation for CRAB3 operators, developers, admins, experts

Added:
>
>
New operator TODO list
TODO list new operators should go through
 

Installation

CRAB3 frontend installation
Documentation about how to deploy the CRAB server frontend (i.e. CRAB REST Interface and CRAB Cache).

Revision 1472018-02-21 - TodorTrendafilovIvanov

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 136 to 136
 
Added:
>
>
 

Links to relevant projects

Revision 1462018-02-18 - LeonardoCristella

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 165 to 165
 BEFORE CONTACTING Computing Tools, PLEASE CONSULT THE CRAB TROUBLESHOOTING GUIDE AT THIS link
and the CRAB3 Frequently Asked Questions page (and maybe even the CRAB2 Frequently Asked Questions page).
Changed:
<
<
A common mistake is to test the code locally only on a small number of events which is not representative of the whole dataset. If the log of a failed job contains a CMSSW error (e.g. exit code 134), please run your code interactively on the very same event on which the CRAB job failed.
>
>
A common mistake is to test the code locally only on a small number of events which is not representative of the whole dataset. If the log of a failed job contains a CMSSW error (e.g. exit code 134, 139), please run your code interactively on the very same event on which the CRAB job failed.
  All CRAB users must subscribe to the CERN Computing Announcements HyperNews Forum. It is very low traffic and if there is a global problem you will know about it faster in that way.

Revision 1452018-02-15 - LeonardoCristella

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 12 to 12
 pre.note {background-color: white;}
Changed:
<
<
CRAB Logo
>
>
CRAB Logo
 

Software Guide on CRAB

Line: 70 to 69
 

CRAB3 documentation

Changed:
<
<
CRAB3 releases
Summary of CRAB3 releases since October 2014 with list of new features, improvements and bug fixes.
>
>
CRAB3 releases
Summary of CRAB3 releases since October 2014 with list of new features, improvements and bug fixes.
 

Help Documentation for beginner users

Changed:
<
<
Quickstart guide for CRAB3
A "cheat sheet" for getting started with CRAB3.
CRAB tutorial (introductory)
This is the most complete and recommended documentation about CRAB3 for beginners.
Help CRAB Troubleshooting
What to do if something does not work. A must read
CRAB tutorial (advanced)
Exercises for CRAB3 Advanced Tutorial 22 May 2015. Covers:
>
>
Quickstart guide for CRAB3
A "cheat sheet" for getting started with CRAB3.
CRAB tutorial (introductory)
This is the most complete and recommended documentation about CRAB3 for beginners.
Help CRAB Troubleshooting
What to do if something does not work. A must read
CRAB tutorial (advanced)
Exercises for CRAB3 Advanced Tutorial 22 May 2015. Covers:
  dryrun, recovery task, scriptExe, LHE, CRABAPI library
Changed:
<
<
CRAB configuration file
Documentation of the CRAB3 configuration file parameters.
CRAB commands
Documentation of the CRAB3 commands.
CRAB user functions
Documentation of the CRAB3 user functions.
CRAB advanced topics
Provisory page with documentation of different topics about CRAB3 (e.g. generating MC with LHE files, using scriptExe).
Tips and tricks about CRAB
Tips and tricks about CRAB3.
CRAB3 frequently asked questions
You may also want to consult the CRAB2 frequently asked questions page as some questions are common for CRAB2 and CRAB3.
>
>
CRAB configuration file
Documentation of the CRAB3 configuration file parameters.
CRAB commands
Documentation of the CRAB3 commands.
CRAB user functions
Documentation of the CRAB3 user functions.
CRAB advanced topics
Provisory page with documentation of different topics about CRAB3 (e.g. generating MC with LHE files, using scriptExe).
Tips and tricks about CRAB
Tips and tricks about CRAB3.
CRAB3 frequently asked questions
You may also want to consult the CRAB2 frequently asked questions page as some questions are common for CRAB2 and CRAB3.
 

Documentation for more expert users

Changed:
<
<
Data handling in CRAB
An overview of how CRAB3 handles input data, output data and data publication.
CRAB3 task flow
A description of the steps involved in a typical CRAB3 task with description of CRAB3 architecture.
CRAB3 job states
In-depth information about job states in CRAB3.
CRAB3 client API
Details on how to use the CRAB3 Client library API.
>
>
Data handling in CRAB
An overview of how CRAB3 handles input data, output data and data publication.
CRAB3 task flow
A description of the steps involved in a typical CRAB3 task with description of CRAB3 architecture.
CRAB3 job states
In-depth information about job states in CRAB3.
CRAB3 client API
Details on how to use the CRAB3 Client library API.
 

Documentation for CRAB3 operators, developers, admins, experts

Installation

Changed:
<
<
CRAB3 frontend installation
Documentation about how to deploy the CRAB server frontend (i.e. CRAB REST Interface and CRAB Cache).
CRAB3 backend installation
Documentation about how to deploy the CRAB server backend, a.k.a. the CRAB TaskWorker.
CRAB3 Schedd installation and deployment
Documentation for CRAB3 Schedd deployment.
CRAB3 client installation
Documentation for operators of the CRAB3 client.
ASO installation and deployment
Documentation for ASO deployment and operations.
Notes about server deployment
Notes about the current deployment of CRAB3 servers.
>
>
CRAB3 frontend installation
Documentation about how to deploy the CRAB server frontend (i.e. CRAB REST Interface and CRAB Cache).
CRAB3 backend installation
Documentation about how to deploy the CRAB server backend, a.k.a. the CRAB TaskWorker.
CRAB3 Schedd installation and deployment
Documentation for CRAB3 Schedd deployment.
CRAB3 client installation
Documentation for operators of the CRAB3 client.
ASO installation and deployment
Documentation for ASO deployment and operations.
Notes about server deployment
Notes about the current deployment of CRAB3 servers.
 

Technical documents

Changed:
<
<
Operator debugging tips
Handy commands for operators.
CRAB3 State Machine
Documentation about the Taks status tracking inside CRAB Server Data Base
CMS job exit codes
List of known exit codes for CMSSW and CRAB3.
Notes about CRAB3 service Certificates management
Documentation about the management of all the service certificates in the CRAB3 machines.
Notes about CRAB3 Oracle Database management
Documentation about the management of the oracle database in the CRAB3 project.
>
>
Operator debugging tips
Handy commands for operators.
CRAB3 State Machine
Documentation about the Taks status tracking inside CRAB Server Data Base
CMS job exit codes
List of known exit codes for CMSSW and CRAB3.
Notes about CRAB3 service Certificates management
Documentation about the management of all the service certificates in the CRAB3 machines.
Notes about CRAB3 Oracle Database management
Documentation about the management of the oracle database in the CRAB3 project.
 

Code repositories

Changed:
<
<
CrabCodeDevelopment
Short guide to using github for Crab
CRAB Client
CRAB Server
REST, CrabCache, TW, new Publisher
ASO (old)
old ASO + Publisher in separate server
Submission Infrastructure Scripts
includes e.g. all scripts that we run in various ways on CRAB schedd's
Oracle Database management scripts
Puppet
>
>
CrabCodeDevelopment
Short guide to using github for Crab
CRAB Client
CRAB Server
REST, CrabCache, TW, new Publisher
ASO (old)
old ASO + Publisher in separate server
Submission Infrastructure Scripts
includes e.g. all scripts that we run in various ways on CRAB schedd's
Oracle Database management scripts
Puppet
 

Release management

Changed:
<
<
Notes about CRAB3 release management
Documentation about releasing CRAB3 patches
Notes about CRAB3 validation
Some notes about the deployment procedure and validation process of new CRAB3 versions
>
>
Notes about CRAB3 release management
Documentation about releasing CRAB3 patches
Notes about CRAB3 validation
Some notes about the deployment procedure and validation process of new CRAB3 versions
 

Monitoring

Line: 143 to 139
 

Links to relevant projects

Changed:
<
<
FNAL LPC Submission
submission via CRAB3 to FNAL LPC
glideInWMS for CMS documentation
main entry to Submission Infrastructure
>
>
FNAL LPC Submission
submission via CRAB3 to FNAL LPC
glideInWMS for CMS documentation
main entry to Submission Infrastructure
 

Local Submission

Changed:
<
<
CRAB vs CAF in Run 2
Notes about CAF use cases after discussion between Stefano and several DPG people.
Local Resource Provisioning
Notes on the different possibilities to submit to resources dedicated to local users.
>
>
CRAB vs CAF in Run 2
Notes about CAF use cases after discussion between Stefano and several DPG people.
Local Resource Provisioning
Notes on the different possibilities to submit to resources dedicated to local users.
 

Getting started

Line: 171 to 165
 BEFORE CONTACTING Computing Tools, PLEASE CONSULT THE CRAB TROUBLESHOOTING GUIDE AT THIS link
and the CRAB3 Frequently Asked Questions page (and maybe even the CRAB2 Frequently Asked Questions page).
Added:
>
>
A common mistake is to test the code locally only on a small number of events which is not representative of the whole dataset. If the log of a failed job contains a CMSSW error (e.g. exit code 134), please run your code interactively on the very same event on which the CRAB job failed.
 All CRAB users must subscribe to the CERN Computing Announcements HyperNews Forum. It is very low traffic and if there is a global problem you will know about it faster in that way.

To send feedback and/or get support about CRAB3, please write to the Computing Tools CMS HyperNews forum. You can do it either via e-mail to hn-cms-computing-tools@cernNOSPAMPLEASE.ch or using the web form https://hypernews.cern.ch/HyperNews/CMS/get/computing-tools.html (the later is recommended, as messages via e-mail have a significant delay to reach other users subscribed to the CMS HyperNews). All CRAB users may find it useful to subscribe to this forum.

Line: 184 to 180
 
    • the URL printed in green by the crab status command at the line: Task URL to use for HELP
  • if a problem on the client side is suspected, upload the crab.log file (use the crab uploadlog command) corresponding to the task for which you are reporting the problem
<!--/twistyPlugin twikiMakeVisibleInline-->
Changed:
<
<
  • This command:
    crab status -d  crab_20170616_143152
    
    Produces:
>
>
  • This command:
    crab status -d  crab_20170616_143152
    Produces:
 CRAB project directory: /afs/cern.ch/work/b/belforte/CRAB3/TC3/crab_20170616_143152 Task name: 170616_123158:belforte_crab_20170616_143152 Grid scheduler: crab3@vocms0197NOSPAMPLEASE.cern.ch
Line: 195 to 190
 Dashboard monitoring URL: http://dashb-cms-job.cern.ch/dashboard/templates/task-analysis/#user=belforte&refresh=0&table=Jobs&p=1&records=25&activemenu=2&status=&site=&tid=170616_123158%3Abelforte_crab_20170616_143152 Status on the scheduler: COMPLETED
Changed:
<
<
Jobs status: finished 100.0% (2/2) So you would have to paste in your mail at least this URL
https://cmsweb.cern.ch/crabserver/ui/task/170616_123158%3Abelforte_crab_20170616_143152
while of course also copying/pasting the full text above would be good, and even better.
>
>
Jobs status: finished 100.0% (2/2) So you would have to paste in your mail at least this URL
https://cmsweb.cern.ch/crabserver/ui/task/170616_123158%3Abelforte_crab_20170616_143152
while of course also copying/pasting the full text above would be good, and even better.
 
<!--/twistyPlugin-->

Other relevant CMS HyperNews forums

Line: 206 to 200
 

Useful links

  • General
Changed:
<
<
>
>
 
Line: 225 to 219
 
Changed:
<
<
>
>
 
Changed:
<
<
>
>
 
Changed:
<
<
>
>
 

Revision 1442018-01-29 - TodorTrendafilovIvanov

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 110 to 110
 
CRAB3 State Machine
Documentation about the Taks status tracking inside CRAB Server Data Base
CMS job exit codes
List of known exit codes for CMSSW and CRAB3.
Notes about CRAB3 service Certificates management
Documentation about the management of all the service certificates in the CRAB3 machines.
Changed:
<
<
<!--   $ Notes about CRAB3 Oracle Database management: Documentation about the management of the oracle database in the CRAB3 project.  -->
>
>
Notes about CRAB3 Oracle Database management
Documentation about the management of the oracle database in the CRAB3 project.
 

Code repositories

Line: 120 to 120
 
CRAB Server
REST, CrabCache, TW, new Publisher
ASO (old)
old ASO + Publisher in separate server
Submission Infrastructure Scripts
includes e.g. all scripts that we run in various ways on CRAB schedd's
Added:
>
>
Oracle Database management scripts
 
Puppet
Added:
>
>
 

Release management

Notes about CRAB3 release management
Documentation about releasing CRAB3 patches

Revision 1432018-01-23 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 114 to 114
 

Code repositories

Added:
>
>
CrabCodeDevelopment
Short guide to using github for Crab
 
CRAB Client
CRAB Server
REST, CrabCache, TW, new Publisher
ASO (old)
old ASO + Publisher in separate server

Revision 1422018-01-19 - TodorTrendafilovIvanov

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 110 to 110
 
CRAB3 State Machine
Documentation about the Taks status tracking inside CRAB Server Data Base
CMS job exit codes
List of known exit codes for CMSSW and CRAB3.
Notes about CRAB3 service Certificates management
Documentation about the management of all the service certificates in the CRAB3 machines.
Added:
>
>
<!--   $ Notes about CRAB3 Oracle Database management: Documentation about the management of the oracle database in the CRAB3 project.  -->
 

Code repositories

Revision 1412017-11-22 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 95 to 95
 

Documentation for CRAB3 operators, developers, admins, experts

Added:
>
>

Installation

 
CRAB3 frontend installation
Documentation about how to deploy the CRAB server frontend (i.e. CRAB REST Interface and CRAB Cache).
CRAB3 backend installation
Documentation about how to deploy the CRAB server backend, a.k.a. the CRAB TaskWorker.
CRAB3 Schedd installation and deployment
Documentation for CRAB3 Schedd deployment.
CRAB3 client installation
Documentation for operators of the CRAB3 client.
ASO installation and deployment
Documentation for ASO deployment and operations.
Added:
>
>
Notes about server deployment
Notes about the current deployment of CRAB3 servers.

Technical documents

 
Operator debugging tips
Handy commands for operators.
CRAB3 State Machine
Documentation about the Taks status tracking inside CRAB Server Data Base
CMS job exit codes
List of known exit codes for CMSSW and CRAB3.
Deleted:
<
<
Notes about server deployment
Notes about the current deployment of CRAB3 servers.
Notes about CRAB3 release management
Documentation about releasing CRAB3 patches
 
Notes about CRAB3 service Certificates management
Documentation about the management of all the service certificates in the CRAB3 machines.
Added:
>
>

Code repositories

CRAB Client
CRAB Server
REST, CrabCache, TW, new Publisher
ASO (old)
old ASO + Publisher in separate server
Submission Infrastructure Scripts
includes e.g. all scripts that we run in various ways on CRAB schedd's
Puppet

Release management

Notes about CRAB3 release management
Documentation about releasing CRAB3 patches
 
Notes about CRAB3 validation
Some notes about the deployment procedure and validation process of new CRAB3 versions
Added:
>
>

Monitoring

Links to relevant projects

 
FNAL LPC Submission
submission via CRAB3 to FNAL LPC
glideInWMS for CMS documentation
main entry to Submission Infrastructure
Line: 191 to 222
 
Deleted:
<
<
 

Revision 1402017-11-22 - TitasKulikauskas

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 103 to 103
 
Operator debugging tips
Handy commands for operators.
CRAB3 State Machine
Documentation about the Taks status tracking inside CRAB Server Data Base
CMS job exit codes
List of known exit codes for CMSSW and CRAB3.
Changed:
<
<
Server deployment notes
Notes about the current deployment of CRAB3 servers.
>
>
Notes about server deployment
Notes about the current deployment of CRAB3 servers.
 
Notes about CRAB3 release management
Documentation about releasing CRAB3 patches
Notes about CRAB3 service Certificates management
Documentation about the management of all the service certificates in the CRAB3 machines.
Notes about CRAB3 validation
Some notes about the deployment procedure and validation process of new CRAB3 versions

Revision 1392017-11-17 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 178 to 178
 
Added:
>
>
 

Revision 1382017-08-30 - TodorTrendafilovIvanov

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 96 to 96
 

Documentation for CRAB3 operators, developers, admins, experts

CRAB3 frontend installation
Documentation about how to deploy the CRAB server frontend (i.e. CRAB REST Interface and CRAB Cache).
Deleted:
<
<
CRAB3 State Machine
Documentation about the Taks status tracking inside CRAB Server Data Base
 
CRAB3 backend installation
Documentation about how to deploy the CRAB server backend, a.k.a. the CRAB TaskWorker.
Added:
>
>
CRAB3 Schedd installation and deployment
Documentation for CRAB3 Schedd deployment.
 
CRAB3 client installation
Documentation for operators of the CRAB3 client.
Deleted:
<
<
Server deployment notes
Notes about the current deployment of CRAB3 servers.
 
ASO installation and deployment
Documentation for ASO deployment and operations.
Deleted:
<
<
Schedd installation and deployment
Documentation for CRAB3 Schedd deployment.
 
Operator debugging tips
Handy commands for operators.
Added:
>
>
CRAB3 State Machine
Documentation about the Taks status tracking inside CRAB Server Data Base
 
CMS job exit codes
List of known exit codes for CMSSW and CRAB3.
Added:
>
>
Server deployment notes
Notes about the current deployment of CRAB3 servers.
 
Notes about CRAB3 release management
Documentation about releasing CRAB3 patches
Added:
>
>
Notes about CRAB3 service Certificates management
Documentation about the management of all the service certificates in the CRAB3 machines.
 
Notes about CRAB3 validation
Some notes about the deployment procedure and validation process of new CRAB3 versions
FNAL LPC Submission
submission via CRAB3 to FNAL LPC
glideInWMS for CMS documentation
main entry to Submission Infrastructure

Revision 1372017-07-15 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 106 to 106
 
CMS job exit codes
List of known exit codes for CMSSW and CRAB3.
Notes about CRAB3 release management
Documentation about releasing CRAB3 patches
Notes about CRAB3 validation
Some notes about the deployment procedure and validation process of new CRAB3 versions
Added:
>
>
FNAL LPC Submission
submission via CRAB3 to FNAL LPC
 
glideInWMS for CMS documentation
main entry to Submission Infrastructure

Revision 1362017-07-09 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 129 to 129
 

Help Getting support

Changed:
<
<
Before contacting Computing Tools, please consult the CRAB Troubleshooting Guide and the CRAB3 Frequently Asked Questions page (and maybe even the CRAB2 Frequently Asked Questions page).
>
>
BEFORE CONTACTING Computing Tools, PLEASE CONSULT THE CRAB TROUBLESHOOTING GUIDE AT THIS link
and the CRAB3 Frequently Asked Questions page (and maybe even the CRAB2 Frequently Asked Questions page).
  All CRAB users must subscribe to the CERN Computing Announcements HyperNews Forum. It is very low traffic and if there is a global problem you will know about it faster in that way.

To send feedback and/or get support about CRAB3, please write to the Computing Tools CMS HyperNews forum. You can do it either via e-mail to hn-cms-computing-tools@cernNOSPAMPLEASE.ch or using the web form https://hypernews.cern.ch/HyperNews/CMS/get/computing-tools.html (the later is recommended, as messages via e-mail have a significant delay to reach other users subscribed to the CMS HyperNews). All CRAB users may find it useful to subscribe to this forum.

Changed:
<
<
When contacting Computing Tools, you should:
>
>
When contacting Computing Tools, you must:
 
Changed:
<
<
  • write a brief description of the problem and of relevant error messages
>
>
  • write a brief description of the problem and copy/paste text of relevant error messages
 
  • try to be specific ("I have errors, please help me" will not get you an useful reply)
  • give pointers to relevant task information , e.g. by copy/pasting the first few lines form crab status output. In general make sure that you report at least:
    • task name (and a job number, if applicable)

Revision 1352017-07-07 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 129 to 129
 

Help Getting support

Changed:
<
<
Before contacting Computing Tools, please consult the CRAB Troubleshooting Guide and the
>
>
Before contacting Computing Tools, please consult the CRAB Troubleshooting Guide and the
 CRAB3 Frequently Asked Questions page (and maybe even the CRAB2 Frequently Asked Questions page).

All CRAB users must subscribe to the CERN Computing Announcements HyperNews Forum. It is very low traffic and if there is a global problem you will know about it faster in that way.

Revision 1342017-07-05 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 96 to 96
 

Documentation for CRAB3 operators, developers, admins, experts

CRAB3 frontend installation
Documentation about how to deploy the CRAB server frontend (i.e. CRAB REST Interface and CRAB Cache).
Added:
>
>
CRAB3 State Machine
Documentation about the Taks status tracking inside CRAB Server Data Base
 
CRAB3 backend installation
Documentation about how to deploy the CRAB server backend, a.k.a. the CRAB TaskWorker.
CRAB3 client installation
Documentation for operators of the CRAB3 client.
Server deployment notes
Notes about the current deployment of CRAB3 servers.
Line: 165 to 166
 

Useful links

Added:
>
>
 
  • Certificate Related
Added:
>
>
 
Deleted:
<
<
<!--      * gLite User Guide  -->
 
Added:
>
>
 
Deleted:
<
<
 
Deleted:
<
<
    • T2 and analysis groups Tier-2 Associations to Analysis Groups
      <!-- * [[http://goc.grid.sinica.edu.tw/gstat/][GIIS] -->
 
  • Services
Deleted:
<
<
 
Changed:
<
<
>
>
 
  • CMSSW
Changed:
<
<
<!-- * CMSSW versions installed on remote GRID sites -->
>
>
 
Deleted:
<
<
 

Revision 1332017-07-04 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 72 to 72
 
CRAB3 releases
Summary of CRAB3 releases since October 2014 with list of new features, improvements and bug fixes.
Changed:
<
<

Documentation for beginner users

>
>

Help Documentation for beginner users

 
Quickstart guide for CRAB3
A "cheat sheet" for getting started with CRAB3.
CRAB tutorial (introductory)
This is the most complete and recommended documentation about CRAB3 for beginners.
Changed:
<
<
CRAB Troubleshooting
What to do if something does not work. A must read
>
>
Help CRAB Troubleshooting
What to do if something does not work. A must read
 
CRAB tutorial (advanced)
Exercises for CRAB3 Advanced Tutorial 22 May 2015. Covers: dryrun, recovery task, scriptExe, LHE, CRABAPI library
CRAB configuration file
Documentation of the CRAB3 configuration file parameters.
Line: 126 to 126
  Some basic workflows are documented in the CRAB3 Tutorial page. If you have never used CRAB before, it is a good place to learn how to use the tool.
Changed:
<
<

Getting support

>
>

Help Getting support

Before contacting Computing Tools, please consult the CRAB Troubleshooting Guide and the CRAB3 Frequently Asked Questions page (and maybe even the CRAB2 Frequently Asked Questions page).

  All CRAB users must subscribe to the CERN Computing Announcements HyperNews Forum. It is very low traffic and if there is a global problem you will know about it faster in that way.

To send feedback and/or get support about CRAB3, please write to the Computing Tools CMS HyperNews forum. You can do it either via e-mail to hn-cms-computing-tools@cernNOSPAMPLEASE.ch or using the web form https://hypernews.cern.ch/HyperNews/CMS/get/computing-tools.html (the later is recommended, as messages via e-mail have a significant delay to reach other users subscribed to the CMS HyperNews). All CRAB users may find it useful to subscribe to this forum.

Deleted:
<
<
Before contacting Computing Tools, please consult the CRAB3 Frequently Asked Questions page (and maybe even the CRAB2 Frequently Asked Questions page).
 When contacting Computing Tools, you should:

  • write a brief description of the problem and of relevant error messages

Revision 1322017-07-04 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 76 to 76
 
Quickstart guide for CRAB3
A "cheat sheet" for getting started with CRAB3.
CRAB tutorial (introductory)
This is the most complete and recommended documentation about CRAB3 for beginners.
Added:
>
>
CRAB Troubleshooting
What to do if something does not work. A must read
 
CRAB tutorial (advanced)
Exercises for CRAB3 Advanced Tutorial 22 May 2015. Covers: dryrun, recovery task, scriptExe, LHE, CRABAPI library
CRAB configuration file
Documentation of the CRAB3 configuration file parameters.
Line: 85 to 86
 
Tips and tricks about CRAB
Tips and tricks about CRAB3.
CRAB3 frequently asked questions
You may also want to consult the CRAB2 frequently asked questions page as some questions are common for CRAB2 and CRAB3.
Deleted:
<
<
CRAB3 common errors
 

Documentation for more expert users

Data handling in CRAB
An overview of how CRAB3 handles input data, output data and data publication.

Revision 1312017-07-03 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 133 to 133
  To send feedback and/or get support about CRAB3, please write to the Computing Tools CMS HyperNews forum. You can do it either via e-mail to hn-cms-computing-tools@cernNOSPAMPLEASE.ch or using the web form https://hypernews.cern.ch/HyperNews/CMS/get/computing-tools.html (the later is recommended, as messages via e-mail have a significant delay to reach other users subscribed to the CMS HyperNews). All CRAB users may find it useful to subscribe to this forum.
Added:
>
>
Before contacting Computing Tools, please consult the CRAB3 Frequently Asked Questions page (and maybe even the CRAB2 Frequently Asked Questions page).
 When contacting Computing Tools, you should:
Changed:
<
<
  • write a brief description of the problem and of relevant error messages;
>
>
  • write a brief description of the problem and of relevant error messages
 
  • try to be specific ("I have errors, please help me" will not get you an useful reply)
Changed:
<
<
  • always specify a task name (and a job number, if applicable);
  • upload the crab.log file (use the crab uploadlog command) corresponding to the task for which you are reporting the problem;
  • report the URL provided by the crab uploadlog command in the e-mail.

Before contacting Computing Tools, please consult the CRAB3 Frequently Asked Questions page (and maybe even the CRAB2 Frequently Asked Questions page).

>
>
  • give pointers to relevant task information , e.g. by copy/pasting the first few lines form crab status output. In general make sure that you report at least:
    • task name (and a job number, if applicable)
    • the URL printed in green by the crab status command at the line: Task URL to use for HELP
  • if a problem on the client side is suspected, upload the crab.log file (use the crab uploadlog command) corresponding to the task for which you are reporting the problem
<!--/twistyPlugin twikiMakeVisibleInline-->
  • This command:
    crab status -d  crab_20170616_143152
    
    Produces:
    CRAB project directory:       /afs/cern.ch/work/b/belforte/CRAB3/TC3/crab_20170616_143152
    Task name:          170616_123158:belforte_crab_20170616_143152
    Grid scheduler:          crab3@vocms0197.cern.ch
    Status on the CRAB server: SUBMITTED
    Task URL to use for HELP:    https://cmsweb.cern.ch/crabserver/ui/task/170616_123158%3Abelforte_crab_20170616_143152
    Dashboard monitoring URL:    http://dashb-cms-job.cern.ch/dashboard/templates/task-analysis/#user=belforte&refresh=0&table=Jobs&p=1&records=25&activemenu=2&status=&site=&tid=170616_123158%3Abelforte_crab_20170616_143152
    Status on the scheduler:    COMPLETED
    
    Jobs status:                    finished      100.0% (2/2)
    
    So you would have to paste in your mail at least this URL
    https://cmsweb.cern.ch/crabserver/ui/task/170616_123158%3Abelforte_crab_20170616_143152
    while of course also copying/pasting the full text above would be good, and even better.
<!--/twistyPlugin-->
  Other relevant CMS HyperNews forums

Revision 1302017-05-03 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 94 to 94
 
CRAB3 job states
In-depth information about job states in CRAB3.
CRAB3 client API
Details on how to use the CRAB3 Client library API.
Changed:
<
<

Documentation for CRAB3 operators

>
>

Documentation for CRAB3 operators, developers, admins, experts

 
CRAB3 frontend installation
Documentation about how to deploy the CRAB server frontend (i.e. CRAB REST Interface and CRAB Cache).
CRAB3 backend installation
Documentation about how to deploy the CRAB server backend, a.k.a. the CRAB TaskWorker.
Line: 106 to 106
 
CMS job exit codes
List of known exit codes for CMSSW and CRAB3.
Notes about CRAB3 release management
Documentation about releasing CRAB3 patches
Notes about CRAB3 validation
Some notes about the deployment procedure and validation process of new CRAB3 versions
Added:
>
>
glideInWMS for CMS documentation
main entry to Submission Infrastructure
 

Local Submission

Revision 1292017-02-07 - MarcoMascheroni

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 105 to 105
 
Operator debugging tips
Handy commands for operators.
CMS job exit codes
List of known exit codes for CMSSW and CRAB3.
Notes about CRAB3 release management
Documentation about releasing CRAB3 patches
Added:
>
>
Notes about CRAB3 validation
Some notes about the deployment procedure and validation process of new CRAB3 versions
 

Local Submission

Revision 1282016-12-21 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 127 to 127
 

Getting support

Changed:
<
<
To send feedback and/or get support about CRAB3, please write to the Computing Tools CMS HyperNews forum. You can do it either via e-mail to hn-cms-computing-tools@cernNOSPAMPLEASE.ch or using the web form https://hypernews.cern.ch/HyperNews/CMS/get/computing-tools.html (the later is recommended, as messages via e-mail have a significant delay to reach other users subscribed to the CMS HyperNews). All CRAB users should subscribe to this forum.
>
>
All CRAB users must subscribe to the CERN Computing Announcements HyperNews Forum. It is very low traffic and if there is a global problem you will know about it faster in that way.

To send feedback and/or get support about CRAB3, please write to the Computing Tools CMS HyperNews forum. You can do it either via e-mail to hn-cms-computing-tools@cernNOSPAMPLEASE.ch or using the web form https://hypernews.cern.ch/HyperNews/CMS/get/computing-tools.html (the later is recommended, as messages via e-mail have a significant delay to reach other users subscribed to the CMS HyperNews). All CRAB users may find it useful to subscribe to this forum.

  When contacting Computing Tools, you should:

  • write a brief description of the problem and of relevant error messages;
Added:
>
>
  • try to be specific ("I have errors, please help me" will not get you an useful reply)
  • always specify a task name (and a job number, if applicable);
 
  • upload the crab.log file (use the crab uploadlog command) corresponding to the task for which you are reporting the problem;
Changed:
<
<
  • report the URL provided by the crab uploadlog command in the e-mail for the support.
>
>
  • report the URL provided by the crab uploadlog command in the e-mail.
 
Changed:
<
<
But before contacting Computing Tools, please consult the CRAB3 Frequently Asked Questions page (and maybe even the CRAB2 Frequently Asked Questions page).
>
>
Before contacting Computing Tools, please consult the CRAB3 Frequently Asked Questions page (and maybe even the CRAB2 Frequently Asked Questions page).
  Other relevant CMS HyperNews forums

Revision 1272016-08-09 - JadirSilva

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 101 to 101
 
CRAB3 client installation
Documentation for operators of the CRAB3 client.
Server deployment notes
Notes about the current deployment of CRAB3 servers.
ASO installation and deployment
Documentation for ASO deployment and operations.
Added:
>
>
Schedd installation and deployment
Documentation for CRAB3 Schedd deployment.
 
Operator debugging tips
Handy commands for operators.
CMS job exit codes
List of known exit codes for CMSSW and CRAB3.
Notes about CRAB3 release management
Documentation about releasing CRAB3 patches

Revision 1262016-07-26 - AndrewMelo

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 60 to 60
 
<!--STOPCRAB2vsCRAB3_1-->
Changed:
<
<

How running CRAB3 is different than CRAB2

>
>

How is running CRAB3 is different than CRAB2

 
<!--STARTCRAB2vsCRAB3_2-->
  • Commands do not require the - in front of them. That is, a CRAB2 command crab -<command> became crab <command> in CRAB3.

Revision 1252016-06-01 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 75 to 75
 

Documentation for beginner users

Quickstart guide for CRAB3
A "cheat sheet" for getting started with CRAB3.
Changed:
<
<
CRAB tutorial (introductory)
This is the most complete and recommended documentation about CRAB3 at the moment for beginners.
CRAB tutorial (advanced)
Exercises for CRAB3 Advanced Tutorial 22 May 2015
>
>
CRAB tutorial (introductory)
This is the most complete and recommended documentation about CRAB3 for beginners.
CRAB tutorial (advanced)
Exercises for CRAB3 Advanced Tutorial 22 May 2015. Covers: dryrun, recovery task, scriptExe, LHE, CRABAPI library
 
CRAB configuration file
Documentation of the CRAB3 configuration file parameters.
CRAB commands
Documentation of the CRAB3 commands.
CRAB user functions
Documentation of the CRAB3 user functions.
Changed:
<
<
CRAB miscellaneous topics
Provisory page with documentation of different topics about CRAB3 (e.g. generating MC with LHE files, using scriptExe).
>
>
CRAB advanced topics
Provisory page with documentation of different topics about CRAB3 (e.g. generating MC with LHE files, using scriptExe).
 
Tips and tricks about CRAB
Tips and tricks about CRAB3.
CRAB3 frequently asked questions
You may also want to consult the CRAB2 frequently asked questions page as some questions are common for CRAB2 and CRAB3.

Revision 1242016-05-02 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 80 to 80
 
CRAB configuration file
Documentation of the CRAB3 configuration file parameters.
CRAB commands
Documentation of the CRAB3 commands.
CRAB user functions
Documentation of the CRAB3 user functions.
Changed:
<
<
CRAB miscellaneous topics
Provisory page with documentation of different topics about CRAB3 (e.g. generating MC with LHE files, using scriptExe).
>
>
CRAB miscellaneous topics
Provisory page with documentation of different topics about CRAB3 (e.g. generating MC with LHE files, using scriptExe).
 
Tips and tricks about CRAB
Tips and tricks about CRAB3.
CRAB3 frequently asked questions
You may also want to consult the CRAB2 frequently asked questions page as some questions are common for CRAB2 and CRAB3.

Revision 1232016-02-15 - EmilisAntanasRupeika

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 102 to 102
 
ASO installation and deployment
Documentation for ASO deployment and operations.
Operator debugging tips
Handy commands for operators.
CMS job exit codes
List of known exit codes for CMSSW and CRAB3.
Changed:
<
<
CRABClient patch release
Documentation about releasing a new CRABClientpatch.
>
>
Notes about CRAB3 release management
Documentation about releasing CRAB3 patches
 

Local Submission

Revision 1222016-02-12 - EmilisAntanasRupeika

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 102 to 102
 
ASO installation and deployment
Documentation for ASO deployment and operations.
Operator debugging tips
Handy commands for operators.
CMS job exit codes
List of known exit codes for CMSSW and CRAB3.
Added:
>
>
CRABClient patch release
Documentation about releasing a new CRABClientpatch.
 

Local Submission

Revision 1212015-12-10 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 74 to 74
 

Documentation for beginner users

Deleted:
<
<
CRAB3 task flow
A description of the steps involved in a typical CRAB3 task with description of CRAB3 architecture.
 
Quickstart guide for CRAB3
A "cheat sheet" for getting started with CRAB3.
CRAB tutorial (introductory)
This is the most complete and recommended documentation about CRAB3 at the moment for beginners.
CRAB tutorial (advanced)
Exercises for CRAB3 Advanced Tutorial 22 May 2015
Line: 90 to 89
 

Documentation for more expert users

Data handling in CRAB
An overview of how CRAB3 handles input data, output data and data publication.
Added:
>
>
CRAB3 task flow
A description of the steps involved in a typical CRAB3 task with description of CRAB3 architecture.
 
CRAB3 job states
In-depth information about job states in CRAB3.
CRAB3 client API
Details on how to use the CRAB3 Client library API.

Revision 1202015-09-26 - AndresTanasijczuk

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 76 to 76
 
CRAB3 task flow
A description of the steps involved in a typical CRAB3 task with description of CRAB3 architecture.
Quickstart guide for CRAB3
A "cheat sheet" for getting started with CRAB3.
Changed:
<
<
CRAB3 tutorial
This is the most complete and recommended documentation about CRAB3 at the moment for beginners.
>
>
CRAB tutorial (introductory)
This is the most complete and recommended documentation about CRAB3 at the moment for beginners.
 
CRAB tutorial (advanced)
Exercises for CRAB3 Advanced Tutorial 22 May 2015
CRAB configuration file
Documentation of the CRAB3 configuration file parameters.
CRAB commands
Documentation of the CRAB3 commands.

Revision 1192015-09-24 - MarcoMascheroni

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 107 to 107
 

Local Submission

CRAB vs CAF in Run 2
Notes about CAF use cases after discussion between Stefano and several DPG people.
Changed:
<
<
Local Resource Provisioning
Notes on the different possibilities to submit to resources dedicated to local users.
>
>
Local Resource Provisioning
Notes on the different possibilities to submit to resources dedicated to local users.
 

Getting started

Revision 1182015-09-03 - MarcoMascheroni

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 106 to 106
 

Local Submission

Changed:
<
<
CRAB vs CAF in Run 2
Notes about CAF use cases after discussion between Stefano and several DPG people.
>
>
CRAB vs CAF in Run 2
Notes about CAF use cases after discussion between Stefano and several DPG people.
 
Local Resource Provisioning
Notes on the different possibilities to submit to resources dedicated to local users.

Getting started

Revision 1172015-09-02 - AndresTanasijczuk

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 81 to 81
 
CRAB configuration file
Documentation of the CRAB3 configuration file parameters.
CRAB commands
Documentation of the CRAB3 commands.
CRAB user functions
Documentation of the CRAB3 user functions.
Changed:
<
<
CRAB3 miscellaneous topics
Provisory page with documentation of different topics about CRAB3 (e.g. generating MC with LHE files, using scriptExe).
>
>
CRAB miscellaneous topics
Provisory page with documentation of different topics about CRAB3 (e.g. generating MC with LHE files, using scriptExe).
 
Tips and tricks about CRAB
Tips and tricks about CRAB3.
CRAB3 frequently asked questions
You may also want to consult the CRAB2 frequently asked questions page as some questions are common for CRAB2 and CRAB3.

Revision 1162015-08-20 - MarcoMascheroni

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 103 to 103
 
Operator debugging tips
Handy commands for operators.
CMS job exit codes
List of known exit codes for CMSSW and CRAB3.
Added:
>
>

Local Submission

CRAB vs CAF in Run 2
Notes about CAF use cases after discussion between Stefano and several DPG people.
Local Resource Provisioning
Notes on the different possibilities to submit to resources dedicated to local users.
 

Getting started

This section describes how to begin using CRAB to perform your analysis on the Grid.

Revision 1152015-06-15 - AndresTanasijczuk

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 127 to 127
  But before contacting Computing Tools, please consult the CRAB3 Frequently Asked Questions page (and maybe even the CRAB2 Frequently Asked Questions page).
Changed:
<
<

Other relevant CMS HyperNews forums

>
>
Other relevant CMS HyperNews forums
 

Revision 1142015-06-05 - AndresTanasijczuk

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 95 to 95
 

Documentation for CRAB3 operators

Changed:
<
<
CRAB3 interface installation
Documentation for operators of the CRAB3 Interface.
CRAB3 backend installation
Documentation for operators of the CRAB3 backend.
>
>
CRAB3 frontend installation
Documentation about how to deploy the CRAB server frontend (i.e. CRAB REST Interface and CRAB Cache).
CRAB3 backend installation
Documentation about how to deploy the CRAB server backend, a.k.a. the CRAB TaskWorker.
 
CRAB3 client installation
Documentation for operators of the CRAB3 client.
Server deployment notes
Notes about the current deployment of CRAB3 servers.
ASO installation and deployment
Documentation for ASO deployment and operations.

Revision 1132015-05-30 - AndresTanasijczuk

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 95 to 95
 

Documentation for CRAB3 operators

Deleted:
<
<
CRAB3 backend installation
Documentation for operators of the CRAB3 backend.
 
CRAB3 interface installation
Documentation for operators of the CRAB3 Interface.
Added:
>
>
CRAB3 backend installation
Documentation for operators of the CRAB3 backend.
 
CRAB3 client installation
Documentation for operators of the CRAB3 client.
Server deployment notes
Notes about the current deployment of CRAB3 servers.
ASO installation and deployment
Documentation for ASO deployment and operations.

Revision 1122015-05-19 - AndresTanasijczuk

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 77 to 77
 
CRAB3 task flow
A description of the steps involved in a typical CRAB3 task with description of CRAB3 architecture.
Quickstart guide for CRAB3
A "cheat sheet" for getting started with CRAB3.
CRAB3 tutorial
This is the most complete and recommended documentation about CRAB3 at the moment for beginners.
Changed:
<
<
CRAB3 configuration file
Documentation of the CRAB3 configuration file parameters.
CRAB3 commands
Documentation of the CRAB3 commands.
CRAB3 user functions
Documentation of the CRAB3 user functions.
>
>
CRAB tutorial (advanced)
Exercises for CRAB3 Advanced Tutorial 22 May 2015
CRAB configuration file
Documentation of the CRAB3 configuration file parameters.
CRAB commands
Documentation of the CRAB3 commands.
CRAB user functions
Documentation of the CRAB3 user functions.
 
CRAB3 miscellaneous topics
Provisory page with documentation of different topics about CRAB3 (e.g. generating MC with LHE files, using scriptExe).
Tips and tricks about CRAB
Tips and tricks about CRAB3.
CRAB3 frequently asked questions
You may also want to consult the CRAB2 frequently asked questions page as some questions are common for CRAB2 and CRAB3.
Line: 88 to 89
 

Documentation for more expert users

Changed:
<
<
Data handling in CRAB3
An overview of how CRAB3 handles input data, output data and data publication.
>
>
Data handling in CRAB
An overview of how CRAB3 handles input data, output data and data publication.
 
CRAB3 job states
In-depth information about job states in CRAB3.
CRAB3 client API
Details on how to use the CRAB3 Client library API.

Revision 1112015-05-07 - AndresTanasijczuk

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 80 to 80
 
CRAB3 configuration file
Documentation of the CRAB3 configuration file parameters.
CRAB3 commands
Documentation of the CRAB3 commands.
CRAB3 user functions
Documentation of the CRAB3 user functions.
Added:
>
>
CRAB3 miscellaneous topics
Provisory page with documentation of different topics about CRAB3 (e.g. generating MC with LHE files, using scriptExe).
 
Tips and tricks about CRAB
Tips and tricks about CRAB3.
CRAB3 frequently asked questions
You may also want to consult the CRAB2 frequently asked questions page as some questions are common for CRAB2 and CRAB3.

Revision 1102015-05-05 - AndresTanasijczuk

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 16 to 16
 

Software Guide on CRAB

Changed:
<
<
Complete: 1
>
>
Complete: 3
  This page contains documentation about CRAB3. The old CRAB2 page has been moved to SWGuideCrab2.
Changed:
<
<
CRAB is a utility to submit CMSSW jobs to distributed computing resources. Currently we are in a transition phase, encouraging users to move from CRAB version 2 towards CRAB version 3 if possible (CRAB3 is still in a beta testing phase and there are some workflows that CRAB3 still doesn't support). This page is being updated accordingly to contain documentation about CRAB3. The old CRAB2 page has been moved to SWGuideCrab2.
>
>
CRAB is a utility to submit CMSSW jobs to distributed computing resources. Currently we are in a transition phase, encouraging users to move from CRAB version 2 towards CRAB version 3 if possible. This page is being updated accordingly to contain documentation about CRAB3. The old CRAB2 page has been moved to SWGuideCrab2.
 
Line: 68 to 68
 
  • The option used to specify the configuration file in the submission command is --config/-c.
<!--STOPCRAB2vsCRAB3_2-->
Changed:
<
<

CRAB2 Documentation

SWGuideCrab2

CRAB3 Documentation

>
>

CRAB3 documentation

 
CRAB3 releases
Summary of CRAB3 releases since October 2014 with list of new features, improvements and bug fixes.
Changed:
<
<

Beginning User Documentation

>
>

Documentation for beginner users

 
CRAB3 task flow
A description of the steps involved in a typical CRAB3 task with description of CRAB3 architecture.
Quickstart guide for CRAB3
A "cheat sheet" for getting started with CRAB3.
Line: 85 to 81
 
CRAB3 commands
Documentation of the CRAB3 commands.
CRAB3 user functions
Documentation of the CRAB3 user functions.
Tips and tricks about CRAB
Tips and tricks about CRAB3.
Changed:
<
<
CRAB3 frequently asked questions
>
>
CRAB3 frequently asked questions
You may also want to consult the CRAB2 frequently asked questions page as some questions are common for CRAB2 and CRAB3.
  CRAB3 common errors
Changed:
<
<

Expert User Documentation

>
>

Documentation for more expert users

 
Data handling in CRAB3
An overview of how CRAB3 handles input data, output data and data publication.
CRAB3 job states
In-depth information about job states in CRAB3.
CRAB3 client API
Details on how to use the CRAB3 Client library API.
Changed:
<
<

Operator Documentation

>
>

Documentation for CRAB3 operators

 
CRAB3 backend installation
Documentation for operators of the CRAB3 backend.
CRAB3 interface installation
Documentation for operators of the CRAB3 Interface.
Line: 105 to 101
 
Operator debugging tips
Handy commands for operators.
CMS job exit codes
List of known exit codes for CMSSW and CRAB3.
Added:
>
>

Getting started

 
Changed:
<
<




OLD CRAB2 DOCUMENTATION TEMPORARELY COPIED HERE FOR UPDATING



CRAB Client and Server

<!--STARTCRABHOMESERVER0-->
AS OF AUGUST 2013, CRAB2 SERVER HAS BEEN TURNED OFF, RECOMMENDED SCHEDULER IS REMOTEGLIDEIN.
<!--STOPCRABHOMESERVER0-->

Getting Started

This section describes how to begin using CRAB to perform your analysis on the Grid.

>
>
This section describes how to begin using CRAB to perform your analysis on the Grid.
 

Prerequisites

Changed:
<
<
To use CRAB to submit your CMSSW jobs to the Grid you must meet some prerequisites, which you can find in the CRAB Prerequisites page. There you will also find information on how to setup your environment to use CRAB at CERN and how to do a private installation of CRAB.

Basic Workflows

You can find a guide to the basic functionality of CRAB in the CRAB Basic Workflows page. If you have never used CRAB before, it is a good place to learn how to use the tool.

Latest Tutorial

Regular tutorials are given on using CRAB. The course material for these tutorials is stored online for future reference, although it refers to specific CRAB and CMSSW versions that may no longer be available.

Advanced CRAB Documentation

The advanced user should refer to the following specific documentation as appropriate.

>
>
To use CRAB to submit CMSSW jobs to the Grid, you must meet some prerequisites, which you can find in the CRAB Prerequisites page.
 
Changed:
<
<

CRAB Online Manual

>
>

Basic workflows

 
Changed:
<
<
The CRAB Online Manual provides detailed documentation about CRAB commands and configuration parameters. This is the same documentation you get from the crab -help command.
>
>
Some basic workflows are documented in the CRAB3 Tutorial page. If you have never used CRAB before, it is a good place to learn how to use the tool.
 
Changed:
<
<

CRAB Publication to DBS

CRAB Usage at CERN CAF

CRAB Usage at LPC CAF (T3_US_FNALLPC)

>
>

Getting support

 
Changed:
<
<
LPC CAF at FNAL is a local analysis facility only accessible to users with a local account. Like for CERN CAF, data there are not accessible to grid jobs.
>
>
To send feedback and/or get support about CRAB3, please write to the Computing Tools CMS HyperNews forum. You can do it either via e-mail to hn-cms-computing-tools@cernNOSPAMPLEASE.ch or using the web form https://hypernews.cern.ch/HyperNews/CMS/get/computing-tools.html (the later is recommended, as messages via e-mail have a significant delay to reach other users subscribed to the CMS HyperNews). All CRAB users should subscribe to this forum.
 
Changed:
<
<
To use CRAB on the FNAL LPC CAF, you need to do roughly as you do at CERN: 1) Setup CMSSW 2) Setup CRAB
>
>
When contacting Computing Tools, you should:
 
Changed:
<
<
For SL5 (cmslpc-sl5):
cmsrel CMSSW_X_Y_Z  (only first time)
cd CMSSW_X_Y_Z
cmsenv
source /cvmfs/cms.cern.ch/crab/crab.(c)sh
>
>
  • write a brief description of the problem and of relevant error messages;
  • upload the crab.log file (use the crab uploadlog command) corresponding to the task for which you are reporting the problem;
  • report the URL provided by the crab uploadlog command in the e-mail for the support.
 
Changed:
<
<
If you want to use CRAB to submit to the local condor batch system, please see the additional instructions at CRABonLPCCAF. If you created a .no_glite_setup file during the CRAB 26x/27x transition, you can safely delete that file now as it is no longer used.
>
>
But before contacting Computing Tools, please consult the CRAB3 Frequently Asked Questions page (and maybe even the CRAB2 Frequently Asked Questions page).
 
Changed:
<
<

Setup on other places

>
>

Other relevant CMS HyperNews forums

 
Changed:
<
<
Please be aware that CRAB needs gLite_3_2 on sl5 and gLite_3_1 on SL4 (i.e. works with the official gLite UI version for the two operating systems).
>
>
 
Added:
>
>

Useful links

 
Deleted:
<
<

CRAB on PBS

CRAB Developers Documentation

Using CRAB to generate MonteCarlo events

FAQ, Troubleshooting, Getting Support

Frequently asked questions (FAQ)

List of CRAB exit codes

Diagnosing Problems

If a problem is encountered with CRAB you can begin with the following template. This template will help you to identify the nature of the problem and if a solution already exists:

Before contacting the CRAB feedback hypernews, you should consult the Frequently Asked Questions (FAQ) page.

Getting Support

If you have still not found the solution to your problem, you may wish to consult the CRAB feedback hypernews forum.

If you are running locally in LPC, in case of problems please contact: lpc-howto@fnalNOSPAMPLEASE.gov

When contacting crabFeedback, you must

  • write brief description of the problem and of relevant error messages
  • upload all files needed for debug. There is a crab option that provides the upload of your logs:
    • if the problem affects some or all jobs, pick one with the issue (jobid) and do
      • if jobs are in "done" status:
        crab -getoutput <jobid> 
        so the stdout and fjr will be retrieved, then
        crab -uploadLog <jobid> 
        to upload all the necessary files
      • if your jobs are aborted:
        crab -postMortem <jobid>
        so the postMortem file will be generated
        crab -uploadLog <jobid> 
        to upload all the files
  • report the URL provided by the crab -uploadLog command in the email for the support.
  • note: of course if your crab working directory is not the default <username>_0_... you need to add the -c option like for any other crab command

Useful Links

 
Line: 252 to 167
 
Deleted:
<
<

CRAB Releases Notes

  • CRAB client - standalone
    • currently recommended version of the CRAB client, i.e. the one which is default on lxplus, is available at CRAB_current.tgz. Release notes here.
    • Older CRAB versions here
    • Even older CRAB versions here

CRAB references

CRAB presentations, posters, publications and tutorials page
 

Revision 1092015-04-15 - FreyaBlekman

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 137 to 137
  Regular tutorials are given on using CRAB. The course material for these tutorials is stored online for future reference, although it refers to specific CRAB and CMSSW versions that may no longer be available.
Changed:
<
<
>
>
 

Advanced CRAB Documentation

Revision 1082015-01-08 - AndresTanasijczuk

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 84 to 84
 
CRAB3 configuration file
Documentation of the CRAB3 configuration file parameters.
CRAB3 commands
Documentation of the CRAB3 commands.
CRAB3 user functions
Documentation of the CRAB3 user functions.
Added:
>
>
Tips and tricks about CRAB
Tips and tricks about CRAB3.
  CRAB3 frequently asked questions

Revision 1072015-01-08 - AndresTanasijczuk

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 82 to 82
 
Quickstart guide for CRAB3
A "cheat sheet" for getting started with CRAB3.
CRAB3 tutorial
This is the most complete and recommended documentation about CRAB3 at the moment for beginners.
CRAB3 configuration file
Documentation of the CRAB3 configuration file parameters.
Added:
>
>
CRAB3 commands
Documentation of the CRAB3 commands.
CRAB3 user functions
Documentation of the CRAB3 user functions.
  CRAB3 frequently asked questions

Revision 1062014-12-12 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 56 to 56
  An up-to-date roadmap to what works and what does not in CRAB3 is in CRAB3Functionalities.
Added:
>
>
The plan for migration user community from Crab2 to CRAB3 is in CRAB3Migration.
 
<!--STOPCRAB2vsCRAB3_1-->

How running CRAB3 is different than CRAB2

Revision 1052014-12-10 - AndresTanasijczuk

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 62 to 62
 
<!--STARTCRAB2vsCRAB3_2-->
  • Commands do not require the - in front of them. That is, a CRAB2 command crab -<command> became crab <command> in CRAB3.
Changed:
<
<
  • The --continue/-c option was replaced by --task/-t.
>
>
  • The --continue/-c option was replaced by --dir/-d.
 
  • The option used to specify the configuration file in the submission command is --config/-c.
<!--STOPCRAB2vsCRAB3_2-->

Revision 1042014-12-04 - AndresTanasijczuk

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 54 to 54
 
  • Storage in an arbitrary location is not supported. (CRAB3 only supports storage in /store/user/ and /store/group/.)
  • Local submission is not implemented.
Changed:
<
<
An uptodate roadmap to what is works and what does not in CRAB3 is here
>
>
An up-to-date roadmap to what works and what does not in CRAB3 is in CRAB3Functionalities.
 
<!--STOPCRAB2vsCRAB3_1-->

Revision 1032014-12-04 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 52 to 52
 There are also features that are not yet implemented in CRAB3:

  • Storage in an arbitrary location is not supported. (CRAB3 only supports storage in /store/user/ and /store/group/.)
Deleted:
<
<
  • MC generation is only possible with the Pythia generator.
 
  • Local submission is not implemented.
Added:
>
>
An uptodate roadmap to what is works and what does not in CRAB3 is here
 
<!--STOPCRAB2vsCRAB3_1-->

How running CRAB3 is different than CRAB2

Revision 1022014-11-10 - BurtHolzman

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
<!-- /ActionTrackerPlugin -->
Line: 148 to 148
 account. Like for CERN CAF, data there are not accessible to grid jobs.

To use CRAB on the FNAL LPC CAF, you need to do roughly as you do at CERN:

Changed:
<
<
1) Setup of the gLite UI 2) Setup CMSSW 3) Setup CRAB

For SL4 (cmslpc):

source /uscmst1/prod/grid/gLite_SL4.(c)sh
cmsrel CMSSW_X_Y_Z  (only first time)
cd CMSSW_X_Y_Z
cmsenv
source /uscmst1/prod/grid/CRAB/crab.(c)sh
>
>
1) Setup CMSSW 2) Setup CRAB
  For SL5 (cmslpc-sl5):
Deleted:
<
<
source /uscmst1/prod/grid/gLite_SL5.(c)sh
 cmsrel CMSSW_X_Y_Z (only first time) cd CMSSW_X_Y_Z cmsenv
Changed:
<
<
source /uscmst1/prod/grid/CRAB/crab.(c)sh
>
>
source /cvmfs/cms.cern.ch/crab/crab.(c)sh
 

If you want to use CRAB to submit to the local condor batch system, please see the additional instructions at CRABonLPCCAF. If you created a .no_glite_setup file during the CRAB 26x/27x transition, you can safely delete that file now as it is no longer used.

Revision 1012014-10-25 - AndresTanasijczuk

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
Changed:
<
<
CRAB Logo
>
>
<!-- /ActionTrackerPlugin -->
 
Changed:
<
<
Complete: 1
>
>
CRAB Logo

Software Guide on CRAB

Complete: 1
  This page contains documentation about CRAB3. The old CRAB2 page has been moved to SWGuideCrab2.
Line: 70 to 83
  CRAB3 common errors
Deleted:
<
<
CRAB3 known bugs
 

Expert User Documentation

Data handling in CRAB3
An overview of how CRAB3 handles input data, output data and data publication.
CRAB3 job states
In-depth information about job states in CRAB3.
Line: 254 to 265
 

CRAB references

CRAB presentations, posters, publications and tutorials page
Changed:
<
<
>
>

  -- MarcoCalloni - 27-Jan-2010

Revision 1002014-10-23 - AndresTanasijczuk

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
CRAB Logo
Line: 60 to 60
 
CRAB3 releases
Summary of CRAB3 releases since October 2014 with list of new features, improvements and bug fixes.

Beginning User Documentation

Added:
>
>
CRAB3 task flow
A description of the steps involved in a typical CRAB3 task with description of CRAB3 architecture.
 
Quickstart guide for CRAB3
A "cheat sheet" for getting started with CRAB3.
CRAB3 tutorial
This is the most complete and recommended documentation about CRAB3 at the moment for beginners.
CRAB3 configuration file
Documentation of the CRAB3 configuration file parameters.

Changed:
<
<
CRAB3 FAQ
>
>
CRAB3 frequently asked questions

CRAB3 common errors

 
Changed:
<
<
CRAB3 Common Errors
>
>
CRAB3 known bugs
 

Expert User Documentation

Data handling in CRAB3
An overview of how CRAB3 handles input data, output data and data publication.

Revision 992014-10-18 - AndresTanasijczuk

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
CRAB Logo
Line: 64 to 64
 
CRAB3 tutorial
This is the most complete and recommended documentation about CRAB3 at the moment for beginners.
CRAB3 configuration file
Documentation of the CRAB3 configuration file parameters.
Added:
>
>
CRAB3 FAQ

CRAB3 Common Errors

 

Expert User Documentation

Data handling in CRAB3
An overview of how CRAB3 handles input data, output data and data publication.
CRAB3 job states
In-depth information about job states in CRAB3.

Revision 982014-10-15 - AndresTanasijczuk

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
CRAB Logo
Line: 57 to 57
 

CRAB3 Documentation

Added:
>
>
CRAB3 releases
Summary of CRAB3 releases since October 2014 with list of new features, improvements and bug fixes.
 

Beginning User Documentation

Quickstart guide for CRAB3
A "cheat sheet" for getting started with CRAB3.
CRAB3 tutorial
This is the most complete and recommended documentation about CRAB3 at the moment for beginners.

Revision 972014-10-10 - AndresTanasijczuk

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
CRAB Logo
Line: 7 to 7
  This page contains documentation about CRAB3. The old CRAB2 page has been moved to SWGuideCrab2.
Changed:
<
<
CRAB is a utility to submit CMSSW jobs to distributed computing resources. Currently we are in a transition phase, encouraging users to move from CRAB version 2 towards CRAB version 3 if possible (CRAB3 is still in a testing phase and there are some workflows that CRAB3 still doesn't support). This page is being updated accordingly to contain documentation about CRAB3. The old CRAB2 page has been moved to SWGuideCrab2.
>
>
CRAB is a utility to submit CMSSW jobs to distributed computing resources. Currently we are in a transition phase, encouraging users to move from CRAB version 2 towards CRAB version 3 if possible (CRAB3 is still in a beta testing phase and there are some workflows that CRAB3 still doesn't support). This page is being updated accordingly to contain documentation about CRAB3. The old CRAB2 page has been moved to SWGuideCrab2.
 
Line: 23 to 23
  For detailed documentation about CRAB, see section Advanced CRAB Documentation further down in this page.
Added:
>
>

Differences between CRAB2 and CRAB3

<!--STARTCRAB2vsCRAB3_1-->
For those who already know about CRAB2 and have to face the transition to CRAB3, here is a list of architecture improvements (or just differences) in the new version.

  • Asynchronous copy of the output files to the destination storage element by an external service called Asynchronous Stage Out (ASO). This allows to have managed file transfers and releases the worker node immediately when the processing job has finished. Impact: the user will no longer have jobs that run and then fail at the end because they cannot move their data.
  • Automatic publication of the output dataset in DBS. No need to run the crab -publish command anymore.
  • Automatic job resubmission for certain failures.
  • Job splitting takes place in the server:
    • The crab -create command is therefore not available anymore and the workflow starts directly from the submission command.
    • The submission command does not accept anymore the range of jobs option (all jobs that are created by the server are also submitted to run).
  • The CRAB3 configuration file is written in python.

There are also features that are not yet implemented in CRAB3:

  • Storage in an arbitrary location is not supported. (CRAB3 only supports storage in /store/user/ and /store/group/.)
  • MC generation is only possible with the Pythia generator.
  • Local submission is not implemented.
<!--STOPCRAB2vsCRAB3_1-->

How running CRAB3 is different than CRAB2

<!--STARTCRAB2vsCRAB3_2-->
  • Commands do not require the - in front of them. That is, a CRAB2 command crab -<command> became crab <command> in CRAB3.
  • The --continue/-c option was replaced by --task/-t.
  • The option used to specify the configuration file in the submission command is --config/-c.
<!--STOPCRAB2vsCRAB3_2-->
 

CRAB2 Documentation

SWGuideCrab2

Line: 41 to 69
 

Operator Documentation

Changed:
<
<
CMSCrabTaskWorker
Documentation for operators of the CRAB3 backend.
>
>
CRAB3 backend installation
Documentation for operators of the CRAB3 backend.
 
CRAB3 interface installation
Documentation for operators of the CRAB3 Interface.
CRAB3 client installation
Documentation for operators of the CRAB3 client.
Server deployment notes
Notes about the current deployment of CRAB3 servers.

Revision 962014-10-09 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
CRAB Logo
Line: 50 to 50
 
CMS job exit codes
List of known exit codes for CMSSW and CRAB3.
Added:
>
>




OLD CRAB2 DOCUMENTATION TEMPORARELY COPIED HERE FOR UPDATING



 

CRAB Client and Server

<!--STARTCRABHOMESERVER0-->
AS OF AUGUST 2013, CRAB2 SERVER HAS BEEN TURNED OFF, RECOMMENDED SCHEDULER IS REMOTEGLIDEIN.

Revision 952014-10-07 - AndresTanasijczuk

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
CRAB Logo
Line: 36 to 36
 

Expert User Documentation

Data handling in CRAB3
An overview of how CRAB3 handles input data, output data and data publication.
Changed:
<
<
CRAB3 job states
In-depth information about job states in CRAB3.
>
>
CRAB3 job states
In-depth information about job states in CRAB3.
 
CRAB3 client API
Details on how to use the CRAB3 Client library API.

Operator Documentation

Changed:
<
<
CRAB3 backend installation
Documentation for operators of the CRAB3 backend.
CRAB3 interface installation
Documentation for operators of the CRAB3 Interface.
CRAB3 client installation
Documentation for operators of the CRAB3 client.
Server deployment notes
Notes about the current deployment of CRAB3 servers.
ASO installation and deployment
Documentation for ASO deployment and operations.
Operator debugging tips
Handy commands for operators.
>
>
CMSCrabTaskWorker
Documentation for operators of the CRAB3 backend.
CRAB3 interface installation
Documentation for operators of the CRAB3 Interface.
CRAB3 client installation
Documentation for operators of the CRAB3 client.
Server deployment notes
Notes about the current deployment of CRAB3 servers.
ASO installation and deployment
Documentation for ASO deployment and operations.
Operator debugging tips
Handy commands for operators.
 
CMS job exit codes
List of known exit codes for CMSSW and CRAB3.

Revision 942014-10-07 - AndresTanasijczuk

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
CRAB Logo
Changed:
<
<
This page contains documentation about CRAB3. The old CRAB2 page has been moved to SWGuideCrab2
>
>
Complete: 1
 
Changed:
<
<
This page is a spring board to the available documentation of CRAB3.
>
>
This page contains documentation about CRAB3. The old CRAB2 page has been moved to SWGuideCrab2.

CRAB is a utility to submit CMSSW jobs to distributed computing resources. Currently we are in a transition phase, encouraging users to move from CRAB version 2 towards CRAB version 3 if possible (CRAB3 is still in a testing phase and there are some workflows that CRAB3 still doesn't support). This page is being updated accordingly to contain documentation about CRAB3. The old CRAB2 page has been moved to SWGuideCrab2.

 
Changed:
<
<
CRAB is an utility to create and submit CMSSW jobs to distributed computing resources. Currently the utility is in its version number 2 (therefore we call it CRAB2). A newer version (namely CRAB3) is under final steps of development and testing, and it is planned to become available to users during summer 2014. Documentation about CRAB is being updated accordingly.
>
>

Introduction

<!--STARTCRABHOME0-->
CRAB is a utility to submit CMSSW jobs to distributed computing resources. By using CRAB you will be able to:
  • Access CMS data and Monte-Carlo which are distributed to CMS aligned centres worldwide.
  • Exploit the CPU and storage resources at CMS aligned centres.
<!--STOPCRABHOME0-->
Before starting to learn about CRAB, you may want to get an overview of the Grid model and of a typical analysis workflow. For that purpose read the following chapters of the CMS Offline WorkBook: For detailed documentation about CRAB, see section Advanced CRAB Documentation further down in this page.
 

CRAB2 Documentation

Changed:
<
<
CMS Offline WorkBook - Chapter 5.6
This is the recommended starting point for new users who wish to learn how to run their analysis jobs on the GRID. This documentation is currently written for CRAB2, but will soon be updated for CRAB3.
CMS Software Guide - CRAB
This is a more complete documentation than the workbook (indeed, the workbook is a summary of the guide). It is more appropriate for advanced use cases of CRAB; it has detailed documentation of advanced features.
>
>
SWGuideCrab2
 

CRAB3 Documentation

Changed:
<
<
CRAB3 is the next generation of CMS analysis tools, featuring a redesigned workflow and software implementation.
>
>

Beginning User Documentation

Quickstart guide for CRAB3
A "cheat sheet" for getting started with CRAB3.
CRAB3 tutorial
This is the most complete and recommended documentation about CRAB3 at the moment for beginners.
CRAB3 configuration file
Documentation of the CRAB3 configuration file parameters.
 
Changed:
<
<

Beginning User Documentation

Quickstart guide for CRAB3
A "cheat sheet" for getting started with CRAB3.
CRAB3 tutorial
This is the most complete and recommended documentation about CRAB3 at the moment for beginners.
CRAB3 configuration file
Documentation of the CRAB3 configuration file parameters.

Expert User Documentation

Data Handling in CRAB3
An overview of how CRAB3 handles input data, output data, and data publication.
>
>

Expert User Documentation

Data handling in CRAB3
An overview of how CRAB3 handles input data, output data and data publication.
 
CRAB3 job states
In-depth information about job states in CRAB3.
Changed:
<
<
CRABClientLibraryAPI
Detail on how to use the CRABClient library API
>
>
CRAB3 client API
Details on how to use the CRAB3 Client library API.
 
Changed:
<
<

Operator Documentation

>
>

Operator Documentation

 
CRAB3 backend installation
Documentation for operators of the CRAB3 backend.
CRAB3 interface installation
Documentation for operators of the CRAB3 Interface.
CRAB3 client installation
Documentation for operators of the CRAB3 client.
Changed:
<
<
Server deployment notes
Notes about the current deployment of CRAB3 servers.
>
>
Server deployment notes
Notes about the current deployment of CRAB3 servers.
 
ASO installation and deployment
Documentation for ASO deployment and operations.
Changed:
<
<
Operator debugging tips
Handy commands for operators.
CMS job exit codes
List of known exit codes for CMSSW and CRAB3.

Introduction

>
>
Operator debugging tips
Handy commands for operators.
CMS job exit codes
List of known exit codes for CMSSW and CRAB3.
 
Deleted:
<
<
<!--STARTCRABHOME0-->
CRAB is a utility to submit CMSSW jobs to distributed computing resources. By using CRAB you will be able to:
  • Access CMS data and Monte-Carlo which are distributed to CMS aligned centres worldwide.
  • Exploit the CPU and storage resources at CMS aligned centres.
<!--STOPCRABHOME0-->
Before starting to learn about CRAB, you may want to get an overview of the Grid model and of a typical analysis workflow. For that purpose read the following chapters of the CMS Offline WorkBook: For detailed documentation about CRAB, see section Advanced CRAB Documentation further down in this page.
 

CRAB Client and Server

<!--STARTCRABHOMESERVER0-->
Line: 61 to 60
 

Prerequisites

Changed:
<
<
To use CRAB to submit your CMSSW job to the Grid you must meet some prerequisites, which you can find in the CRAB Prerequisites page. There you will also find information on how to setup your environment to use CRAB at CERN and how to do a private installation of CRAB.
>
>
To use CRAB to submit your CMSSW jobs to the Grid you must meet some prerequisites, which you can find in the CRAB Prerequisites page. There you will also find information on how to setup your environment to use CRAB at CERN and how to do a private installation of CRAB.
 

Basic Workflows

Revision 932014-10-07 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
CRAB Logo
Changed:
<
<
This page contains documentation about CRAB3. The corresponding CRAB2 page has been moved to SWGuideCrab2
>
>
This page contains documentation about CRAB3. The old CRAB2 page has been moved to SWGuideCrab2
  This page is a spring board to the available documentation of CRAB3.

Revision 922014-10-07 - AndresTanasijczuk

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"
Changed:
<
<

CMS Remote Analysis Builder - CRAB

>
>
CRAB Logo

This page contains documentation about CRAB3. The corresponding CRAB2 page has been moved to SWGuideCrab2

This page is a spring board to the available documentation of CRAB3.

 
Added:
>
>
CRAB is an utility to create and submit CMSSW jobs to distributed computing resources. Currently the utility is in its version number 2 (therefore we call it CRAB2). A newer version (namely CRAB3) is under final steps of development and testing, and it is planned to become available to users during summer 2014. Documentation about CRAB is being updated accordingly.
 
Changed:
<
<

Introduction

>
>

CRAB2 Documentation

CMS Offline WorkBook - Chapter 5.6
This is the recommended starting point for new users who wish to learn how to run their analysis jobs on the GRID. This documentation is currently written for CRAB2, but will soon be updated for CRAB3.
CMS Software Guide - CRAB
This is a more complete documentation than the workbook (indeed, the workbook is a summary of the guide). It is more appropriate for advanced use cases of CRAB; it has detailed documentation of advanced features.

CRAB3 Documentation

 
Changed:
<
<
CRAB3ConfigurationFile
>
>
CRAB3 is the next generation of CMS analysis tools, featuring a redesigned workflow and software implementation.

Beginning User Documentation

Quickstart guide for CRAB3
A "cheat sheet" for getting started with CRAB3.
CRAB3 tutorial
This is the most complete and recommended documentation about CRAB3 at the moment for beginners.
CRAB3 configuration file
Documentation of the CRAB3 configuration file parameters.

Expert User Documentation

Data Handling in CRAB3
An overview of how CRAB3 handles input data, output data, and data publication.
CRAB3 job states
In-depth information about job states in CRAB3.
CRABClientLibraryAPI
Detail on how to use the CRABClient library API

Operator Documentation

CRAB3 backend installation
Documentation for operators of the CRAB3 backend.
CRAB3 interface installation
Documentation for operators of the CRAB3 Interface.
CRAB3 client installation
Documentation for operators of the CRAB3 client.
Server deployment notes
Notes about the current deployment of CRAB3 servers.
ASO installation and deployment
Documentation for ASO deployment and operations.
Operator debugging tips
Handy commands for operators.
CMS job exit codes
List of known exit codes for CMSSW and CRAB3.

Introduction

 
<!--STARTCRABHOME0-->
CRAB is a utility to submit CMSSW jobs to distributed computing resources. By using CRAB you will be able to:

Revision 912014-10-02 - AndresTanasijczuk

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 7 to 7
 

Introduction

Added:
>
>
CRAB3ConfigurationFile
 
<!--STARTCRABHOME0-->
CRAB is a utility to submit CMSSW jobs to distributed computing resources. By using CRAB you will be able to:
  • Access CMS data and Monte-Carlo which are distributed to CMS aligned centres worldwide.

Revision 902014-09-05 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 19 to 19
 

CRAB Client and Server

<!--STARTCRABHOMESERVER0-->
Changed:
<
<
AS OF AUGUST 2013, CRAB2 SERVER HAS BEEN TURNED OFF, RECOMMENDED SCHEDULER IS REMOTEGLIDEIN, GLITE IS ALSO SUPPORTED.
>
>
AS OF AUGUST 2013, CRAB2 SERVER HAS BEEN TURNED OFF, RECOMMENDED SCHEDULER IS REMOTEGLIDEIN.
 
<!--STOPCRABHOMESERVER0-->

Getting Started

Revision 892014-05-22 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 167 to 167
 

CRAB Releases Notes

  • CRAB client - standalone
    • currently recommended version of the CRAB client, i.e. the one which is default on lxplus, is available at CRAB_current.tgz. Release notes here.
Deleted:
<
<
      • at the moment that version is CRAB_2_10_2:
 
    • Older CRAB versions here
    • Even older CRAB versions here

Revision 882014-05-05 - AndresTanasijczuk

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 91 to 91
 

FAQ, Troubleshooting, Getting Support

Frequently asked questions (FAQ)

Changed:
<
<

List of CRAB exit codes

>
>

List of CRAB exit codes

 

Diagnosing Problems

Line: 156 to 156
 
Changed:
<
<
>
>
 

Revision 872014-04-24 - AndresTanasijczuk

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 19 to 19
 

CRAB Client and Server

<!--STARTCRABHOMESERVER0-->
Changed:
<
<
AS OF AUGUST 2013 CRAB2 SERVER HAS BEEN TURNED OFF, RECOMMENDED SCHEDULER IS REMOTEGLIDEIN, GLITE IS ALSO SUPPORTED

The workflow of CRAB standalone is based on a direct submission from the UI, where the user is working. This standalone model has the advantage of simplicity, but it lacks some features, which can be provided by a more advanced architecture client-server where a server is placed between the user and the Grid to perform a set of actions.

The main goals of the client-server architecture is to automate as much as possible the whole analysis work flow and to improve the scalability of the system. The client-server implementation is transparent to the end users: the interface, the installation, the configuration procedure and the usage remains exactly the same as for standalone.

We currently recommend to use CRAB with the remoteGlidein scheduler, think of it as the most reliable and lightweight form of a Crab Server, see CrabFAQ

>
>
AS OF AUGUST 2013, CRAB2 SERVER HAS BEEN TURNED OFF, RECOMMENDED SCHEDULER IS REMOTEGLIDEIN, GLITE IS ALSO SUPPORTED.
 
<!--STOPCRABHOMESERVER0-->
Deleted:
<
<

Benefits of the CRAB Server

<!--STARTCRABHOMESERVER1-->
The CRAB server provides several advantages over the client submission. See: https://twiki.cern.ch/twiki/bin/view/CMSPublic/SWGuideCrabFaq#Should_I_use_crab_server_or_not
<!--STOPCRABHOMESERVER1-->
 

Getting Started

Line: 64 to 46
 

CRAB Online Manual

Changed:
<
<
The CRAB2 Online Manual provides detailed documentation of CRAB2 commands and configuration parameters. This is the same documentation you get from the crab -help command.
>
>
The CRAB Online Manual provides detailed documentation about CRAB commands and configuration parameters. This is the same documentation you get from the crab -help command.
 

CRAB Publication to DBS

Deleted:
<
<

CRAB Server Usage

 

CRAB Usage at CERN CAF

CRAB Usage at LPC CAF (T3_US_FNALLPC)

Line: 155 to 134
 
<!--      * gLite User Guide  -->
Deleted:
<
<
 

Revision 862014-04-23 - AndresTanasijczuk

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 12 to 12
 
  • Access CMS data and Monte-Carlo which are distributed to CMS aligned centres worldwide.
  • Exploit the CPU and storage resources at CMS aligned centres.
<!--STOPCRABHOME0-->
Changed:
<
<
More information on the Grid model and a typical analysis workflow can be found in the following chapters of the WorkBook:
>
>
Before starting to learn about CRAB, you may want to get an overview of the Grid model and of a typical analysis workflow. For that purpose read the following chapters of the CMS Offline WorkBook:
 
Changed:
<
<
For more details on CRAB, see the CRAB manual (this is the same documentation you get from crab -help).
>
>
For detailed documentation about CRAB, see section Advanced CRAB Documentation further down in this page.
 

CRAB Client and Server

<!--STARTCRABHOMESERVER0-->
Line: 48 to 46
 

Prerequisites

Changed:
<
<
To use CRAB to submit your CMSSW job to the Grid you must meet some prerequisites. You can find them and how to use CRAB at CERN and how to do a private installation of CRAB, in this page.
>
>
To use CRAB to submit your CMSSW job to the Grid you must meet some prerequisites, which you can find in the CRAB Prerequisites page. There you will also find information on how to setup your environment to use CRAB at CERN and how to do a private installation of CRAB.
 

Basic Workflows

Changed:
<
<
In this page: Crab Basic Workflows you can find a guide to the basic functionality of CRAB. If you have never used CRAB before it is a good place to learn how to use the tool.
>
>
You can find a guide to the basic functionality of CRAB in the CRAB Basic Workflows page. If you have never used CRAB before, it is a good place to learn how to use the tool.
 

Latest Tutorial

Changed:
<
<
Regular tutorials are given on using CRAB. The course material for these tutorials is stored online for future reference, although refers to specific CRAB and CMSSW versions that may no longer be available.
>
>
Regular tutorials are given on using CRAB. The course material for these tutorials is stored online for future reference, although it refers to specific CRAB and CMSSW versions that may no longer be available.
 
Changed:
<
<
>
>
 

Advanced CRAB Documentation

The advanced user should refer to the following specific documentation as appropriate.

Changed:
<
<

CRAB Online Manual

>
>

CRAB Online Manual

The CRAB2 Online Manual provides detailed documentation of CRAB2 commands and configuration parameters. This is the same documentation you get from the crab -help command.

 

CRAB Publication to DBS

Revision 852014-04-23 - AndresTanasijczuk

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 48 to 48
 

Prerequisites

Changed:
<
<
To use CRAB to submit you CMSSW job to the Grid you must meet some prerequisites. You can find them and how to use CRAB at CERN and how to do a private installation of CRAB, in this page.
>
>
To use CRAB to submit your CMSSW job to the Grid you must meet some prerequisites. You can find them and how to use CRAB at CERN and how to do a private installation of CRAB, in this page.
 

Basic Workflows

Changed:
<
<
In this page: Crab Basic Workflows you can find a guide to the basic functionality of Crab. If you have never used CRAB before it is a good place to learn how to use the tool.
>
>
In this page: Crab Basic Workflows you can find a guide to the basic functionality of CRAB. If you have never used CRAB before it is a good place to learn how to use the tool.
 

Latest Tutorial

Revision 842014-04-23 - AndresTanasijczuk

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 8 to 8
 

Introduction

<!--STARTCRABHOME0-->
Changed:
<
<
CRAB is a utility to create and submit CMSSW jobs to distributed computing resources. By using CRAB you will be able to

  • Access CMS data and Monte-Carlo which is distributed to CMS aligned centres worldwide
  • Exploit the CPU and storage resources at CMS aligned centres
>
>
CRAB is a utility to submit CMSSW jobs to distributed computing resources. By using CRAB you will be able to:
  • Access CMS data and Monte-Carlo which are distributed to CMS aligned centres worldwide.
  • Exploit the CPU and storage resources at CMS aligned centres.
 
<!--STOPCRABHOME0-->
Changed:
<
<
More infos on Grid model and a typical analysis workflow can be found in the following pages:
>
>
More information on the Grid model and a typical analysis workflow can be found in the following chapters of the WorkBook:
 
Changed:
<
<
>
>
For more details on CRAB, see the CRAB manual (this is the same documentation you get from crab -help).
 
Deleted:
<
<
For more details on CRAB, see the !CRAB manual
 

CRAB Client and Server

<!--STARTCRABHOMESERVER0-->
AS OF AUGUST 2013 CRAB2 SERVER HAS BEEN TURNED OFF, RECOMMENDED SCHEDULER IS REMOTEGLIDEIN, GLITE IS ALSO SUPPORTED

Revision 832014-03-06 - SaizSantosLola

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 129 to 129
 
Added:
>
>
If you are running locally in LPC, in case of problems please contact: lpc-howto@fnalNOSPAMPLEASE.gov
 When contacting crabFeedback, you must
  • write brief description of the problem and of relevant error messages
  • upload all files needed for debug. There is a crab option that provides the upload of your logs:

Revision 822014-01-07 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 187 to 187
 

CRAB Releases Notes

  • CRAB client - standalone
Changed:
<
<
    • currently recommended version of the CRAB client, i.e. the one which is default on lxplus, is available at CRAB_current.tgz
    • at the moment that is
    • Older CRAB versions here
  • CRABSERVER
    • CRABSERVER_1_1_4: release note here
    • Older CRABSERVER versions here
>
>
    • currently recommended version of the CRAB client, i.e. the one which is default on lxplus, is available at CRAB_current.tgz. Release notes here.
      • at the moment that version is CRAB_2_10_2:
    • Older CRAB versions here
    • Even older CRAB versions here
 

CRAB references

CRAB presentations, posters, publications and tutorials page

Revision 812013-12-16 - FedericaFanzago

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 181 to 181
 
Added:
>
>
 

Revision 802013-12-13 - FedericaFanzago

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 180 to 180
 
Added:
>
>
 

Revision 792013-08-30 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 31 to 31
 The client-server implementation is transparent to the end users: the interface, the installation, the configuration procedure and the usage remains exactly the same as for standalone.

We currently recommend to use CRAB with the remoteGlidein scheduler, think of it as the most reliable and lightweight form of a Crab Server, see CrabFAQ

Changed:
<
<
>
>
 
<!--STOPCRABHOMESERVER0-->
Added:
>
>
 

Benefits of the CRAB Server

Added:
>
>
 
<!--STARTCRABHOMESERVER1-->
Added:
>
>
 The CRAB server provides several advantages over the client submission. See: https://twiki.cern.ch/twiki/bin/view/CMSPublic/SWGuideCrabFaq#Should_I_use_crab_server_or_not
Deleted:
<
<
<!--STOPCRABHOMESERVER1-->
 
Added:
>
>
<!--STOPCRABHOMESERVER1-->
 

Getting Started

This section describes how to begin using CRAB to perform your analysis on the Grid.

Revision 782013-08-06 - FedericaFanzago

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 143 to 143
 
Added:
>
>
 

Revision 772013-08-06 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 22 to 22
 For more details on CRAB, see the !CRAB manual

CRAB Client and Server

<!--STARTCRABHOMESERVER0-->
Added:
>
>
AS OF AUGUST 2013 CRAB2 SERVER HAS BEEN TURNED OFF, RECOMMENDED SCHEDULER IS REMOTEGLIDEIN, GLITE IS ALSO SUPPORTED

  The workflow of CRAB standalone is based on a direct submission from the UI, where the user is working. This standalone model has the advantage of simplicity, but it lacks some features, which can be provided by a more advanced architecture client-server where a server is placed between the user and the Grid to perform a set of actions.

The main goals of the client-server architecture is to automate as much as possible the whole analysis work flow and to improve the scalability of the system.

Line: 35 to 38
 The CRAB server provides several advantages over the client submission. See: https://twiki.cern.ch/twiki/bin/view/CMSPublic/SWGuideCrabFaq#Should_I_use_crab_server_or_not
<!--STOPCRABHOMESERVER1-->
Changed:
<
<
>
>
 

Getting Started

This section describes how to begin using CRAB to perform your analysis on the Grid.

Line: 61 to 64
 

CRAB Online Manual

CRAB Publication to DBS

Added:
>
>
 

CRAB Server Usage

Added:
>
>
 

CRAB Usage at CERN CAF

CRAB Usage at LPC CAF (T3_US_FNALLPC)

Revision 762013-03-20 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 137 to 137
 
Added:
>
>
 

Revision 752013-03-17 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 176 to 176
 
  • CRAB client - standalone
    • currently recommended version of the CRAB client, i.e. the one which is default on lxplus, is available at CRAB_current.tgz
    • at the moment that is
Changed:
<
<
>
>
 
    • Older CRAB versions here
  • CRABSERVER
    • CRABSERVER_1_1_4: release note here

Revision 742013-03-06 - FedericaFanzago

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 164 to 164
 
Changed:
<
<
>
>
 

Revision 732013-02-19 - FedericaFanzago

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 165 to 165
 
<!-- * CMSSW versions installed on remote GRID sites -->
Added:
>
>
 
Added:
>
>
 

Revision 722013-02-14 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 26 to 26
  The main goals of the client-server architecture is to automate as much as possible the whole analysis work flow and to improve the scalability of the system. The client-server implementation is transparent to the end users: the interface, the installation, the configuration procedure and the usage remains exactly the same as for standalone.
Added:
>
>
We currently recommend to use CRAB with the remoteGlidein scheduler, think of it as the most reliable and lightweight form of a Crab Server, see CrabFAQ
 
<!--STOPCRABHOMESERVER0-->

Benefits of the CRAB Server

<!--STARTCRABHOMESERVER1-->
Changed:
<
<
The CRAB server provides several advantages over the client submission.

  • More advanced job monitoring with a web based front end.
  • Automatic resubmission of aborted jobs.
  • Notification by e-mail of submitted jobs progress.
  • Improved user support from Analysis Operations
>
>
The CRAB server provides several advantages over the client submission. See: https://twiki.cern.ch/twiki/bin/view/CMSPublic/SWGuideCrabFaq#Should_I_use_crab_server_or_not
 
<!--STOPCRABHOMESERVER1-->
Deleted:
<
<
Below some useful links:
 

Getting Started

This section describes how to begin using CRAB to perform your analysis on the Grid.

Line: 49 to 44
  To use CRAB to submit you CMSSW job to the Grid you must meet some prerequisites. You can find them and how to use CRAB at CERN and how to do a private installation of CRAB, in this page.
Deleted:
<
<

Available CRAB Servers

To use the CRAB Server mode (recommended) you need to select an available Server.

 

Basic Workflows

In this page: Crab Basic Workflows you can find a guide to the basic functionality of Crab. If you have never used CRAB before it is a good place to learn how to use the tool.

Line: 186 to 173
 
  • CRAB client - standalone
    • currently recommended version of the CRAB client, i.e. the one which is default on lxplus, is available at CRAB_current.tgz
    • at the moment that is
Changed:
<
<
>
>
 
    • Older CRAB versions here
  • CRABSERVER
    • CRABSERVER_1_1_4: release note here

Revision 712013-02-08 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 19 to 19
 
Changed:
<
<
For more details on crab, see the crab manuals for specific versions.
>
>
For more details on CRAB, see the !CRAB manual
 

CRAB Client and Server

<!--STARTCRABHOMESERVER0-->
The workflow of CRAB standalone is based on a direct submission from the UI, where the user is working. This standalone model has the advantage of simplicity, but it lacks some features, which can be provided by a more advanced architecture client-server where a server is placed between the user and the Grid to perform a set of actions.

Revision 702013-01-29 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 187 to 187
 
  • CRAB client - standalone
    • currently recommended version of the CRAB client, i.e. the one which is default on lxplus, is available at CRAB_current.tgz
    • at the moment that is
Changed:
<
<
>
>
 
    • Older CRAB versions here
  • CRABSERVER
    • CRABSERVER_1_1_4: release note here

Revision 692013-01-18 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 187 to 187
 
  • CRAB client - standalone
    • currently recommended version of the CRAB client, i.e. the one which is default on lxplus, is available at CRAB_current.tgz
    • at the moment that is
Changed:
<
<
>
>
 
    • Older CRAB versions here
  • CRABSERVER
    • CRABSERVER_1_1_4: release note here

Revision 682013-01-11 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 185 to 185
 

CRAB Releases Notes

  • CRAB client - standalone
Changed:
<
<
>
>
    • currently recommended version of the CRAB client, i.e. the one which is default on lxplus, is available at CRAB_current.tgz
    • at the moment that is
 
    • Older CRAB versions here
  • CRABSERVER
    • CRABSERVER_1_1_4: release note here

Revision 672013-01-09 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 185 to 185
 

CRAB Releases Notes

  • CRAB client - standalone
Changed:
<
<
>
>
 
    • Older CRAB versions here
  • CRABSERVER
    • CRABSERVER_1_1_4: release note here

Revision 662012-12-19 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 185 to 185
 

CRAB Releases Notes

  • CRAB client - standalone
Changed:
<
<
>
>
 
    • Older CRAB versions here
  • CRABSERVER
    • CRABSERVER_1_1_4: release note here

Revision 652012-11-27 - FedericaFanzago

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 165 to 165
 
Added:
>
>
 
<!-- * [[http://goc.grid.sinica.edu.tw/gstat/][GIIS] -->

Revision 642012-10-16 - FedericaFanzago

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 156 to 156
 
<!--      * gLite User Guide  -->
  • Data Discovery
Changed:
<
<
>
>
 

Revision 632012-10-10 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 183 to 183
 

CRAB Releases Notes

  • CRAB client - standalone
Changed:
<
<
>
>
 
    • Older CRAB versions here
  • CRABSERVER
    • CRABSERVER_1_1_4: release note here

Revision 622012-08-20 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 72 to 72
  The advanced user should refer to the following specific documentation as appropriate.
Changed:
<
<

CRAB_2_8_2 Online Manual

>
>

CRAB Online Manual

 

CRAB Publication to DBS

CRAB Server Usage

Revision 612012-08-09 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 72 to 72
  The advanced user should refer to the following specific documentation as appropriate.
Changed:
<
<

CRAB_2_8_1 Online Manual

>
>

CRAB_2_8_2 Online Manual

 

CRAB Publication to DBS

CRAB Server Usage

Line: 183 to 183
 

CRAB Releases Notes

  • CRAB client - standalone
Changed:
<
<
>
>
 
    • Older CRAB versions here
  • CRABSERVER
    • CRABSERVER_1_1_4: release note here

Revision 602012-03-26 - FedericaFanzago

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 72 to 72
  The advanced user should refer to the following specific documentation as appropriate.
Changed:
<
<

CRAB_2_8_0 Online Manual

>
>

CRAB_2_8_1 Online Manual

 

CRAB Publication to DBS

CRAB Server Usage

Line: 183 to 183
 

CRAB Releases Notes

  • CRAB client - standalone
Changed:
<
<
>
>
 
    • Older CRAB versions here
  • CRABSERVER
    • CRABSERVER_1_1_4: release note here

Revision 592012-03-19 - FedericaFanzago

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 168 to 168
 
<!-- * [[http://goc.grid.sinica.edu.tw/gstat/][GIIS] -->
Changed:
<
<
>
>
 

Revision 582012-02-16 - SpigaDaniele

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 72 to 72
  The advanced user should refer to the following specific documentation as appropriate.
Changed:
<
<

CRAB_2_7_9 Online Manual

>
>

CRAB_2_8_0 Online Manual

 

CRAB Publication to DBS

CRAB Server Usage

Line: 183 to 183
 

CRAB Releases Notes

  • CRAB client - standalone
Changed:
<
<
>
>
 
    • Older CRAB versions here
  • CRABSERVER
    • CRABSERVER_1_1_4: release note here

Revision 572011-12-06 - FedericaFanzago

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 183 to 183
 

CRAB Releases Notes

  • CRAB client - standalone
Changed:
<
<
>
>
 
    • Older CRAB versions here
  • CRABSERVER
Changed:
<
<
    • CRABSERVER_1_1_3: release note here
>
>
    • CRABSERVER_1_1_4: release note here
 
    • Older CRABSERVER versions here

CRAB references

Revision 562011-11-07 - FedericaFanzago

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 72 to 72
  The advanced user should refer to the following specific documentation as appropriate.
Changed:
<
<

CRAB_2_7_8 Online Manual

>
>

CRAB_2_7_9 Online Manual

 

CRAB Publication to DBS

CRAB Server Usage

Revision 552011-10-03 - AndreasPfeiffer

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Revision 542011-07-22 - FedericaFanzago

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 153 to 153
 
Added:
>
>
 
<!--      * gLite User Guide  -->

Revision 532011-07-15 - FedericaFanzago

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 182 to 182
 

CRAB Releases Notes

  • CRAB client - standalone
Added:
>
>
 

Revision 522011-05-27 - FedericaFanzago

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 171 to 171
 
Added:
>
>
 

Revision 512011-05-26 - FedericaFanzago

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 170 to 170
 
Added:
>
>
 

Revision 502011-05-16 - SpigaDaniele

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 180 to 180
 

CRAB Releases Notes

  • CRAB client - standalone
Added:
>
>
 
  • CRABSERVER

Revision 492011-05-12 - unknown

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 180 to 180
 

CRAB Releases Notes

  • CRAB client - standalone
Changed:
<
<
>
>
 
    • Older CRAB versions here
  • CRABSERVER
    • CRABSERVER_1_1_3: release note here

Revision 482011-05-10 - FedericaFanzago

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 169 to 169
 
Added:
>
>
 

Revision 472011-04-25 - SpigaDaniele

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 72 to 72
  The advanced user should refer to the following specific documentation as appropriate.
Changed:
<
<

CRAB_2_7_7 Online Manual

>
>

CRAB_2_7_8 Online Manual

 

CRAB Publication to DBS

CRAB Server Usage

Line: 179 to 179
 

CRAB Releases Notes

  • CRAB client - standalone
Changed:
<
<
>
>
 
    • Older CRAB versions here
  • CRABSERVER
    • CRABSERVER_1_1_3: release note here

Revision 462011-04-15 - FedericaFanzago

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 167 to 167
 
<!-- * [[http://goc.grid.sinica.edu.tw/gstat/][GIIS] -->
Added:
>
>
 

Revision 452011-04-11 - FedericaFanzago

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 136 to 136
  When contacting crabFeedback, you must
  • write brief description of the problem and of relevant error messages
Changed:
<
<
  • if the problem affects the task as a whole but not the jobs, do
    • crab -uploadLog <jobid>
      to upload the configuration and crab.log files
>
>
  • upload all files needed for debug. There is a crab option that provides the upload of your logs:
 
  • if the problem affects some or all jobs, pick one with the issue (jobid) and do
Changed:
<
<
    • if your jobs are in "done" status, so the stdout and fjr will be uploaded
      • crab -getoutput <jobid> 
    • if your jobs are aborted, so the postMortem file will be uploaded
      • crab -postMortem <jobid>
    • crab -uploadLog <jobid>
>
>
      • if jobs are in "done" status:
        crab -getoutput <jobid> 
        so the stdout and fjr will be retrieved, then
        crab -uploadLog <jobid> 
        to upload all the necessary files
      • if your jobs are aborted:
        crab -postMortem <jobid>
        so the postMortem file will be generated
        crab -uploadLog <jobid> 
        to upload all the files
 
  • report the URL provided by the crab -uploadLog command in the email for the support.
  • note: of course if your crab working directory is not the default <username>_0_... you need to add the -c option like for any other crab command

Revision 442011-04-11 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 145 to 145
 
      • crab -postMortem <jobid>
    • crab -uploadLog <jobid>
  • report the URL provided by the crab -uploadLog command in the email for the support.
Changed:
<
<
  • note: of course if your crab working directory is not the default _0_... you need to add the -c option like for any other crab command
>
>
  • note: of course if your crab working directory is not the default <username>_0_... you need to add the -c option like for any other crab command
 

Useful Links

Revision 432011-04-06 - MikeAnderson

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 19 to 19
 
Added:
>
>
For more details on crab, see the crab manuals for specific versions.
 

CRAB Client and Server

<!--STARTCRABHOMESERVER0-->
The workflow of CRAB standalone is based on a direct submission from the UI, where the user is working. This standalone model has the advantage of simplicity, but it lacks some features, which can be provided by a more advanced architecture client-server where a server is placed between the user and the Grid to perform a set of actions.

Revision 422011-04-05 - FedericaFanzago

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 135 to 135
 When contacting crabFeedback, you must
  • write brief description of the problem and of relevant error messages
  • if the problem affects the task as a whole but not the jobs, do
Changed:
<
<
    • crab -uploadLog <jobid>
  • if the problem affects some or all jobs, pick one with the issue and do
>
>
    • crab -uploadLog <jobid>
      to upload the configuration and crab.log files
  • if the problem affects some or all jobs, pick one with the issue (jobid) and do
    • if your jobs are in "done" status, so the stdout and fjr will be uploaded
 
    • crab -getoutput <jobid> 
Added:
>
>
    • if your jobs are aborted, so the postMortem file will be uploaded
 
    • crab -postMortem <jobid>
    • crab -uploadLog <jobid>
  • report the URL provided by the crab -uploadLog command in the email for the support.

Revision 412011-03-31 - FedericaFanzago

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 135 to 135
 When contacting crabFeedback, you must
  • write brief description of the problem and of relevant error messages
  • if the problem affects the task as a whole but not the jobs, do
Changed:
<
<
    • crab -uploadLog 
>
>
    • crab -uploadLog <jobid>
 
  • if the problem affects some or all jobs, pick one with the issue and do
    • crab -getoutput <jobid> 
    • crab -postMortem <jobid>

Revision 402011-03-29 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 141 to 141
 
    • crab -postMortem <jobid>
    • crab -uploadLog <jobid>
  • report the URL provided by the crab -uploadLog command in the email for the support.
Added:
>
>
  • note: of course if your crab working directory is not the default _0_... you need to add the -c option like for any other crab command
 

Useful Links

Revision 392011-03-10 - FedericaFanzago

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 162 to 162
 
<!-- * [[http://goc.grid.sinica.edu.tw/gstat/][GIIS] -->
Added:
>
>
 
Changed:
<
<
>
>
 

Revision 382011-03-01 - SpigaDaniele

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 171 to 171
 

CRAB Releases Notes

  • CRAB client - standalone
Changed:
<
<
>
>
 
    • Older CRAB versions here
  • CRABSERVER
    • CRABSERVER_1_1_3: release note here

Revision 372011-02-03 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 70 to 70
  The advanced user should refer to the following specific documentation as appropriate.
Changed:
<
<

CRAB_2_7_7 Online Manual

>
>

CRAB_2_7_7 Online Manual

 

CRAB Publication to DBS

CRAB Server Usage

Line: 129 to 129
 

Getting Support

If you have still not found the solution to your problem, you may wish to consult the CRAB feedback hypernews forum.

Added:
>
>
 
Changed:
<
<

When contacting the hypernews forum, you must provide, if possible, the following information:

  • task name and in case crab server name
  • URL to task monitoring in dashboard as printed by crab -status command
  • Make available your configuration and log files:
    • retrieve the output of "done" jobs using the command
      crab -getoutput <jobid> 
    • in the case of "abort" jobs please run the
      crab -postMortem <jobid>
    • upload your logs and configuration files to the central repository using the command
      crab -uploadLog <jobid>
      and report the URL provided by the command in the email for the support.
>
>
When contacting crabFeedback, you must
  • write brief description of the problem and of relevant error messages
  • if the problem affects the task as a whole but not the jobs, do
    • crab -uploadLog 
  • if the problem affects some or all jobs, pick one with the issue and do
    • crab -getoutput <jobid> 
    • crab -postMortem <jobid>
    • crab -uploadLog <jobid>
  • report the URL provided by the crab -uploadLog command in the email for the support.
 

Useful Links

Revision 362011-02-03 - SpigaDaniele

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 70 to 70
  The advanced user should refer to the following specific documentation as appropriate.
Changed:
<
<

CRAB_2_7_5 Online Manual

CRAB_2_7_6_patch1 Online Manual

>
>

CRAB_2_7_7 Online Manual

 

CRAB Publication to DBS

CRAB Server Usage

Line: 171 to 170
 

CRAB Releases Notes

  • CRAB client - standalone
Changed:
<
<
>
>
 
    • Older CRAB versions here
  • CRABSERVER
    • CRABSERVER_1_1_3: release note here

Revision 352011-01-31 - FedericaFanzago

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 153 to 153
 
Changed:
<
<
>
>
 
Changed:
<
<
    • SiteView - In this page you can view some information about the sites
    • GIIS monitoring to see the status of EGEE CE
>
>
 
Changed:
<
<
>
>
<!-- * CMSSW versions installed on remote GRID sites -->
 

Revision 342011-01-31 - FedericaFanzago

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 138 to 138
 
  • task name and in case crab server name
  • URL to task monitoring in dashboard as printed by crab -status command
  • Make available your configuration and log files:
Changed:
<
<
    • retrieve the output of "done" jobs using the command
      crab -getoutput JobNum
    • in the case of "abort" jobs please run the
      crab -postMortem JobNum
    • upload your logs and configuration files to the central repository using the command
      crab -uploadLog JobNum
      and report the URL provided by the command in the email for the support.
>
>
    • retrieve the output of "done" jobs using the command
      crab -getoutput <jobid> 
    • in the case of "abort" jobs please run the
      crab -postMortem <jobid>
    • upload your logs and configuration files to the central repository using the command
      crab -uploadLog <jobid>
      and report the URL provided by the command in the email for the support.
 

Useful Links

Revision 332011-01-27 - FedericaFanzago

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 158 to 158
 
Added:
>
>
 

Revision 322011-01-17 - FedericaFanzago

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 137 to 137
 When contacting the hypernews forum, you must provide, if possible, the following information:
  • task name and in case crab server name
  • URL to task monitoring in dashboard as printed by crab -status command
Changed:
<
<
  • Make available your crab task directory, for example in your lxplus public area

If you have problems with the last request, for example you have insufficient disk space, you can make available or send individually the following (if they exist):

  • Your crab configuration file
  • The crab log file.
    • This is located in the "log" directory in the CRAB task directory.
  • The stdout and stderr of the job concerned
    • These are located in the "res" directory in the CRAB task directory
>
>
  • Make available your configuration and log files:
    • retrieve the output of "done" jobs using the command
      crab -getoutput JobNum
    • in the case of "abort" jobs please run the
      crab -postMortem JobNum
    • upload your logs and configuration files to the central repository using the command
      crab -uploadLog JobNum
      and report the URL provided by the command in the email for the support.
 

Useful Links

Line: 160 to 155
 
Deleted:
<
<
 

Revision 312011-01-11 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 76 to 76
 

CRAB Publication to DBS

CRAB Server Usage

CRAB Usage at CERN CAF

Changed:
<
<

CRAB Usage at LPC CAF

>
>

CRAB Usage at LPC CAF (T3_US_FNALLPC)

LPC CAF at FNAL is a local analysis facility only accessible to users with a local account. Like for CERN CAF, data there are not accessible to grid jobs.

  To use CRAB on the FNAL LPC CAF, you need to do roughly as you do at CERN: 1) Setup of the gLite UI
Line: 101 to 104
 source /uscmst1/prod/grid/CRAB/crab.(c)sh
Changed:
<
<
If you want to use CRAB to submit to the local condor batch system, please see the additional instructions at CRABonLPCCAF. If you created a .no_glite_setup file during the CRAB 26x/27x transition, you can safely delete that file now as it is no longer used.
>
>
If you want to use CRAB to submit to the local condor batch system, please see the additional instructions at CRABonLPCCAF. If you created a .no_glite_setup file during the CRAB 26x/27x transition, you can safely delete that file now as it is no longer used.
 

Setup on other places

Line: 114 to 117
 

FAQ, Troubleshooting, Getting Support

Frequently asked questions (FAQ)

Changed:
<
<

List of CRAB exit codes

>
>

List of CRAB exit codes

 

Diagnosing Problems

Line: 164 to 167
 
Changed:
<
<
>
>
 
  • Editing the twiki
Changed:
<
<
>
>
 

CRAB Releases Notes

  • CRAB client - standalone
Changed:
<
<
>
>
 
    • Older CRAB versions here
  • CRABSERVER
Changed:
<
<
    • CRABSERVER_1_1_3: release note here
    • Older CRABSERVER versions here
>
>
    • CRABSERVER_1_1_3: release note here
    • Older CRABSERVER versions here
 

CRAB references

CRAB presentations, posters, publications and tutorials page
Line: 184 to 187
  -- MarcoCalloni - 27-Jan-2010
Changed:
<
<
META TOPICMOVED by="calloni" date="1268149830" from="CMS.SWGuideCrabDRAFT" to="CMS.SWGuideCrab"
>
>
META TOPICMOVED by="calloni" date="1268149830" from="SWGuideCrabDRAFT" to="SWGuideCrab"

Revision 302010-12-14 - SpigaDaniele

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 71 to 71
 The advanced user should refer to the following specific documentation as appropriate.

CRAB_2_7_5 Online Manual

Changed:
<
<

CRAB_2_7_6 Online Manual

>
>

CRAB_2_7_6_patch1 Online Manual

 

CRAB Publication to DBS

CRAB Server Usage

Line: 171 to 171
 

CRAB Releases Notes

Changed:
<
<
>
>
 
    • Older CRAB versions here
  • CRABSERVER
    • CRABSERVER_1_1_3: release note here

Revision 292010-11-22 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 171 to 171
 

CRAB Releases Notes

Changed:
<
<
>
>
 
    • Older CRAB versions here
  • CRABSERVER
    • CRABSERVER_1_1_3: release note here

Revision 282010-11-19 - SpigaDaniele

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 71 to 71
 The advanced user should refer to the following specific documentation as appropriate.

CRAB_2_7_5 Online Manual

Added:
>
>

CRAB_2_7_6 Online Manual

 

CRAB Publication to DBS

CRAB Server Usage

Line: 169 to 170
 

CRAB Releases Notes

  • CRAB client - standalone
Deleted:
<
<
 
Added:
>
>
 
    • Older CRAB versions here
  • CRABSERVER
    • CRABSERVER_1_1_3: release note here

Revision 272010-11-09 - SpigaDaniele

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 170 to 170
 

CRAB Releases Notes

  • CRAB client - standalone
Added:
>
>
 
    • Older CRAB versions here
  • CRABSERVER
    • CRABSERVER_1_1_3: release note here

Revision 262010-10-08 - SpigaDaniele

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 69 to 69
 

Advanced CRAB Documentation

The advanced user should refer to the following specific documentation as appropriate.

Deleted:
<
<

CRAB_2_7_2 Online Manual

  • You can also try the new format
  • CRAB_2_7_2 has a serious deficiency in that the timeout for stageout is broken. When stageout fails, your job will generally hang on the worker node forever, or until the batch slot lease times out. This means that the site will account generally many hours of wall clock time against your use of the site. You thus will loose priority running at the site., and generally irritate the site admins.
  • We thus strongly discourage using this version of crab.
 
Changed:
<
<

CRAB_2_7_4 Online Manual

>
>

CRAB_2_7_5 Online Manual

 

CRAB Publication to DBS

CRAB Server Usage

Line: 175 to 169
 

CRAB Releases Notes

  • CRAB client - standalone
Changed:
<
<
>
>
 
    • Older CRAB versions here
  • CRABSERVER
    • CRABSERVER_1_1_3: release note here

Revision 252010-08-30 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Revision 242010-08-25 - SpigaDaniele

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 76 to 76
  of wall clock time against your use of the site. You thus will loose priority running at the site., and generally irritate the site admins.
  • We thus strongly discourage using this version of crab.
Changed:
<
<

CRAB_2_7_3 Online Manual

>
>

CRAB_2_7_4 Online Manual

 

CRAB Publication to DBS

CRAB Server Usage

Line: 175 to 175
 

CRAB Releases Notes

  • CRAB client - standalone
Changed:
<
<
>
>
 
    • Older CRAB versions here
  • CRABSERVER
Changed:
<
<
    • CRABSERVER_1_1_2: release note here
>
>
    • CRABSERVER_1_1_3: release note here
 
    • Older CRABSERVER versions here

CRAB references

Revision 232010-07-18 - FrankWuerthwein

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 71 to 71
 The advanced user should refer to the following specific documentation as appropriate.

CRAB_2_7_2 Online Manual

Added:
>
>
  • CRAB_2_7_2 has a serious deficiency in that the timeout for stageout is broken. When stageout fails, your job will generally hang on the worker node forever, or until the batch slot lease times out. This means that the site will account generally many hours of wall clock time against your use of the site. You thus will loose priority running at the site., and generally irritate the site admins.
  • We thus strongly discourage using this version of crab.
 

CRAB_2_7_3 Online Manual

Deleted:
<
<

Use MultiCRAB - To submit the same task to multiple dataset in one go.

 

CRAB Publication to DBS

CRAB Server Usage

CRAB Usage at CERN CAF

Line: 107 to 110
 

Setup on other places

Changed:
<
<
Please be aware that CRAB 266 and earlier needs gLite_3_1 UI, while CRAB 2_7_1 and later needs gLite_3_2 on sl5 and gLite_3_1 on SL4 (i.e. works with the official gLite UI version for the two operating systems)
>
>
Please be aware that CRAB needs gLite_3_2 on sl5 and gLite_3_1 on SL4 (i.e. works with the official gLite UI version for the two operating systems).
 

CRAB on PBS

Line: 124 to 127
 
Changed:
<
<
Before contacting the CRAB feedback hypernews, you should consult the Frequently Asked Questions (FAQ) page.
>
>
Before contacting the CRAB feedback hypernews, you should consult the Frequently Asked Questions (FAQ) page.
 

Getting Support

Revision 222010-07-06 - EricVaandering

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 103 to 103
 source /uscmst1/prod/grid/CRAB/crab.(c)sh
Changed:
<
<
If you want to use CRAB to submit to the local condor batch system, please see the additional instructions at CRABonLPCCAF.
>
>
If you want to use CRAB to submit to the local condor batch system, please see the additional instructions at CRABonLPCCAF. If you created a .no_glite_setup file during the CRAB 26x/27x transition, you can safely delete that file now as it is no longer used.
 

Setup on other places

Revision 212010-06-22 - SpigaDaniele

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 46 to 46
 

Prerequisites

Changed:
<
<
To use CRAB to submit you CMSSW job to the Grid you must meet some prerequisites. You can find them and how to use CRAB at CERN and how to do a private installation of CRAB, in this page.
>
>
To use CRAB to submit you CMSSW job to the Grid you must meet some prerequisites. You can find them and how to use CRAB at CERN and how to do a private installation of CRAB, in this page.
 

Available CRAB Servers

Line: 69 to 69
 

Advanced CRAB Documentation

The advanced user should refer to the following specific documentation as appropriate.

Added:
>
>

CRAB_2_7_2 Online Manual

CRAB_2_7_3 Online Manual

 
Deleted:
<
<

Online Manual

 

Use MultiCRAB - To submit the same task to multiple dataset in one go.

CRAB Publication to DBS

CRAB Server Usage

Line: 170 to 172
 

CRAB Releases Notes

  • CRAB client - standalone
Changed:
<
<
>
>
 
  • CRABSERVER
Changed:
<
<
    • CRABSERVER_1_1_1: release note here
>
>
    • CRABSERVER_1_1_2: release note here
 
    • Older CRABSERVER versions here

CRAB references

Changed:
<
<
CRAB presentations, posters, publications and tutorials page
>
>
CRAB presentations, posters, publications and tutorials page
 

Revision 202010-06-11 - EricVaandering

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 78 to 78
 

CRAB Usage at CERN CAF

CRAB Usage at LPC CAF

Changed:
<
<
In order to use you need to, one time only:

touch ~/.no_glite_setup then log out and back in again.

After doing this, you need to do roughly as you do at CERN:

>
>
To use CRAB on the FNAL LPC CAF, you need to do roughly as you do at CERN:
 1) Setup of the gLite UI 2) Setup CMSSW 3) Setup CRAB
Line: 99 to 94
  For SL5 (cmslpc-sl5):
Changed:
<
<
source /uscmst1/prod/grid/gLite_SL5_CRAB_27x.(c)sh
>
>
source /uscmst1/prod/grid/gLite_SL5.(c)sh
 cmsrel CMSSW_X_Y_Z (only first time) cd CMSSW_X_Y_Z cmsenv source /uscmst1/prod/grid/CRAB/crab.(c)sh
Changed:
<
<
If you want to use CRAB to submit to the local condor batch system, please see the instructions at CRABonLPCCAF.
>
>
If you want to use CRAB to submit to the local condor batch system, please see the additional instructions at CRABonLPCCAF.
 

Setup on other places

Revision 192010-05-28 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 133 to 133
  If you have still not found the solution to your problem, you may wish to consult the CRAB feedback hypernews forum.
Changed:
<
<
>
>
  When contacting the hypernews forum, you must provide, if possible, the following information:
  • task name and in case crab server name

Revision 182010-05-07 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 71 to 71
 The advanced user should refer to the following specific documentation as appropriate.

Online Manual

Added:
>
>
 

Use MultiCRAB - To submit the same task to multiple dataset in one go.

CRAB Publication to DBS

CRAB Server Usage

Revision 172010-05-06 - SpigaDaniele

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 174 to 174
 

CRAB Releases Notes

  • CRAB client - standalone
Changed:
<
<
>
>
 
    • Older CRAB versions here
  • CRABSERVER
    • CRABSERVER_1_1_1: release note here

Revision 162010-05-03 - FedericaFanzago

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 173 to 173
 

CRAB Releases Notes

Changed:
<
<
  • CRAB_2_7_2 : Complete list available here
  • Other CRAB versions here
>
>
  • CRAB client - standalone
  • CRABSERVER
    • CRABSERVER_1_1_1: release note here
    • Older CRABSERVER versions here
 

CRAB references

CRAB presentations, posters, publications and tutorials page

Revision 152010-05-03 - SpigaDaniele

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 70 to 70
  The advanced user should refer to the following specific documentation as appropriate.
Changed:
<
<

Online Manual

>
>

Online Manual

 

Use MultiCRAB - To submit the same task to multiple dataset in one go.

CRAB Publication to DBS

CRAB Server Usage

Line: 173 to 173
 

CRAB Releases Notes

Changed:
<
<
  • CRAB_2_7_1 : Complete list available here
>
>
  • CRAB_2_7_2 : Complete list available here
 
  • Other CRAB versions here

CRAB references

Revision 142010-04-19 - MarcoCalloni

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 115 to 114
 

CRAB on PBS

CRAB Developers Documentation

Added:
>
>

Using CRAB to generate MonteCarlo events

 

FAQ, Troubleshooting, Getting Support

Frequently asked questions (FAQ)

Revision 132010-04-08 - EricVaandering

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 77 to 77
 

CRAB Server Usage

CRAB Usage at CERN CAF

CRAB Usage at LPC CAF

Added:
>
>
In order to use you need to, one time only:

touch ~/.no_glite_setup then log out and back in again.

After doing this, you need to do roughly as you do at CERN: 1) Setup of the gLite UI 2) Setup CMSSW 3) Setup CRAB

For SL4 (cmslpc):

source /uscmst1/prod/grid/gLite_SL4.(c)sh
cmsrel CMSSW_X_Y_Z  (only first time)
cd CMSSW_X_Y_Z
cmsenv
source /uscmst1/prod/grid/CRAB/crab.(c)sh

For SL5 (cmslpc-sl5):

source /uscmst1/prod/grid/gLite_SL5_CRAB_27x.(c)sh
cmsrel CMSSW_X_Y_Z  (only first time)
cd CMSSW_X_Y_Z
cmsenv
source /uscmst1/prod/grid/CRAB/crab.(c)sh

If you want to use CRAB to submit to the local condor batch system, please see the instructions at CRABonLPCCAF.

Setup on other places

Please be aware that CRAB 266 and earlier needs gLite_3_1 UI, while CRAB 2_7_1 and later needs gLite_3_2 on sl5 and gLite_3_1 on SL4 (i.e. works with the official gLite UI version for the two operating systems)

 

CRAB on PBS

CRAB Developers Documentation

Revision 122010-04-02 - MarcoCalloni

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 70 to 70
  The advanced user should refer to the following specific documentation as appropriate.
Changed:
<
<

Online Manual

>
>

Online Manual

 

Use MultiCRAB - To submit the same task to multiple dataset in one go.

CRAB Publication to DBS

Revision 112010-03-27 - FedericaFanzago

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 137 to 137
 

CRAB Releases Notes

Changed:
<
<
  • CRAB_2_6_6 : Complete list available here
>
>
  • CRAB_2_7_1 : Complete list available here
 
  • Other CRAB versions here

CRAB references

Revision 102010-03-10 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 32 to 32
 
  • More advanced job monitoring with a web based front end.
  • Automatic resubmission of aborted jobs.
Changed:
<
<
  • Notification of the progress of jobs submitted by e-mail.
>
>
  • Notification by e-mail of submitted jobs progress.
 
  • Improved user support from Analysis Operations
<!--STOPCRABHOMESERVER1-->

Revision 92010-03-09 - MarcoCalloni

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 146 to 146
 

-- MarcoCalloni - 27-Jan-2010 \ No newline at end of file

Added:
>
>
META TOPICMOVED by="calloni" date="1268149830" from="CMS.SWGuideCrabDRAFT" to="CMS.SWGuideCrab"

Revision 82010-03-09 - MarcoCalloni

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 46 to 46
 

Prerequisites

Changed:
<
<
To use CRAB to submit you CMSSW job to the Grid you must meet the following prerequisites.
>
>
To use CRAB to submit you CMSSW job to the Grid you must meet some prerequisites. You can find them and how to use CRAB at CERN and how to do a private installation of CRAB, in this page.
 

Available CRAB Servers

Revision 72010-03-08 - unknown

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Changed:
<
<
<!--STARTCRABHOME0-->
>
>
 

Introduction

Added:
>
>
<!--STARTCRABHOME0-->
 CRAB is a utility to create and submit CMSSW jobs to distributed computing resources. By using CRAB you will be able to

  • Access CMS data and Monte-Carlo which is distributed to CMS aligned centres worldwide
Line: 19 to 20
 

CRAB Client and Server

Changed:
<
<
>
>
<!--STARTCRABHOMESERVER0-->
  The workflow of CRAB standalone is based on a direct submission from the UI, where the user is working. This standalone model has the advantage of simplicity, but it lacks some features, which can be provided by a more advanced architecture client-server where a server is placed between the user and the Grid to perform a set of actions.

The main goals of the client-server architecture is to automate as much as possible the whole analysis work flow and to improve the scalability of the system. The client-server implementation is transparent to the end users: the interface, the installation, the configuration procedure and the usage remains exactly the same as for standalone.

Changed:
<
<
>
>
<!--STOPCRABHOMESERVER0-->
 

Benefits of the CRAB Server

Changed:
<
<
>
>
<!--STARTCRABHOMESERVER1-->
 The CRAB server provides several advantages over the client submission.

  • More advanced job monitoring with a web based front end.
  • Automatic resubmission of aborted jobs.
  • Notification of the progress of jobs submitted by e-mail.
  • Improved user support from Analysis Operations
Added:
>
>
<!--STOPCRABHOMESERVER1-->
  Below some useful links:

Revision 62010-03-08 - unknown

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 131 to 131
 
Added:
>
>
 

CRAB Releases Notes

  • CRAB_2_6_6 : Complete list available here

Revision 52010-03-05 - MarcoCalloni

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Changed:
<
<
<!--STARTWORKBOOK-->
>
>
<!--STARTCRABHOME0-->
 

Introduction

CRAB is a utility to create and submit CMSSW jobs to distributed computing resources. By using CRAB you will be able to

  • Access CMS data and Monte-Carlo which is distributed to CMS aligned centres worldwide
  • Exploit the CPU and storage resources at CMS aligned centres
Changed:
<
<
<!--STOPWORKBOOK-->
>
>
<!--STOPCRABHOME0-->
  More infos on Grid model and a typical analysis workflow can be found in the following pages:
Line: 44 to 44
 

Prerequisites

Changed:
<
<
To use CRAB to submit you CMSSW job to the Grid you must meet the following prerequisites. The links below give detailed documentation on how to achieve each prerequisite.
>
>
To use CRAB to submit you CMSSW job to the Grid you must meet the following prerequisites.
 
Deleted:
<
<
  1. Grid certificate:
  2. siteDB registration:
  3. CMSSW:
  4. Figuring out what data to analyze
  5. Setup the environment:

Getting CRAB

<!--STARTWORKBOOK-->
The latest CRAB client can always be accessed at CERN on lxplus by executing the following command:

  • source /afs/cern.ch/cms/ccs/wm/scripts/Crab/crab.(c)sh

If you are working on a machine without access to CERN AFS, you can make a private installation, always make it sure to use current production release

  • Download tar ball as instructed here
  • Extract the tar ball, change to the resulting CRAB_X_Y_Z directory and run ./configure

That will create a crab.(c)sh file in your CRAB_X_Y_Z directory that you can use to setup CRAB environment. Always be careful to setup grid UI, cmssw and crab in the proper order (see above)

You need only do the first two steps once.

<!--STOPWORKBOOK-->
 

Available CRAB Servers

To use the CRAB Server mode (recommended) you need to select an available Server.

Changed:
<
<
>
>
 

Basic Workflows

Line: 98 to 64
 
Changed:
<
<

Advanced CRAB Documentation

The advanced user should refer as appropriate to the following specific documentation as appropriate.

>
>

Advanced CRAB Documentation

 
Changed:
<
<

Online Manual

>
>
The advanced user should refer to the following specific documentation as appropriate.
 
Changed:
<
<
<!-- ++++ Use MultiCRAB  -->
<!-- To submit the same task to multiple dataset in one go.-->
>
>

Online Manual

 
Changed:
<
<

CRAB Publication to DBS

CRAB Server Usage

CRAB Usage at CERN CAF

CRAB Usage at LPC CAF

CRAB on PBS

CRAB Developers Documentation

>
>

Use MultiCRAB - To submit the same task to multiple dataset in one go.

CRAB Publication to DBS

CRAB Server Usage

CRAB Usage at CERN CAF

CRAB Usage at LPC CAF

CRAB on PBS

CRAB Developers Documentation

FAQ, Troubleshooting, Getting Support

Frequently asked questions (FAQ)

List of CRAB exit codes

 
Deleted:
<
<

FAQ, HOWTO, Error Diagnosis

Troubleshooting

 

Diagnosing Problems

Changed:
<
<
If a problem is encountered with CRAB you should begin with the following template. This template will help you to identify the nature of the problem and if a solution already exists:
>
>
If a problem is encountered with CRAB you can begin with the following template. This template will help you to identify the nature of the problem and if a solution already exists:
 
Changed:
<
<
Before contacting the CRAB feedback hypernews, you should consult the Frequently Asked Questions page:

>
>
Before contacting the CRAB feedback hypernews, you should consult the Frequently Asked Questions (FAQ) page.
 

Getting Support

Line: 151 to 110
 
  • The stdout and stderr of the job concerned
    • These are located in the "res" directory in the CRAB task directory
Changed:
<
<

Useful Links

>
>

Useful Links

 

Revision 42010-03-03 - MarcoCalloni

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Line: 35 to 35
 
  • Improved user support from Analysis Operations

Below some useful links:

Changed:
<
<
>
>
 

Getting Started

Line: 71 to 69
 
  • source /afs/cern.ch/cms/ccs/wm/scripts/Crab/crab.(c)sh
Changed:
<
<
If you are working on a machine without access to CERN AFS, you can make a private installation:
>
>
If you are working on a machine without access to CERN AFS, you can make a private installation, always make it sure to use current production release
 
Changed:
<
<
  • Download tar ball here.
>
>
  • Download tar ball as instructed here
 
  • Extract the tar ball, change to the resulting CRAB_X_Y_Z directory and run ./configure
Changed:
<
<
  • Source the resulting crab.(c)sh
>
>
That will create a crab.(c)sh file in your CRAB_X_Y_Z directory that you can use to setup CRAB environment. Always be careful to setup grid UI, cmssw and crab in the proper order (see above)
  You need only do the first two steps once.
<!--STOPWORKBOOK-->
Changed:
<
<
For developers you can check out CRAB source codes from:
setenv CVSROOT :pserver:anonymous@cmscvs.cern.ch:/cvs_server/repositories/CMSSW 
cvs login   # for anonymous user: pwd='98passwd'
cvs co [-r Tag_version] CRAB
>
>

Available CRAB Servers

To use the CRAB Server mode (recommended) you need to select an available Server.

 

Basic Workflows

In this page: Crab Basic Workflows you can find a guide to the basic functionality of Crab. If you have never used CRAB before it is a good place to learn how to use the tool.

Added:
>
>

Latest Tutorial

Regular tutorials are given on using CRAB. The course material for these tutorials is stored online for future reference, although refers to specific CRAB and CMSSW versions that may no longer be available.

 
Added:
>
>

Advanced CRAB Documentation

 
Changed:
<
<

CRAB on-line manual and tutorial

To visit on-line CRAB documentation : To visit the specific CRABSERVER documentation: For specific informations related on How to Run CRAB at CERN CAF: For specific informations related on How to Run CRAB at LPC CAF: For specific informations related on How to Run CRAB on PBS: For specific informations related on CRAB publication:
  • read here: [[][]]
Last CRAB Tutorial (CERN):
>
>
The advanced user should refer as appropriate to the following specific documentation as appropriate.

Online Manual

<!-- ++++ Use MultiCRAB  -->
<!-- To submit the same task to multiple dataset in one go.-->

CRAB Publication to DBS

CRAB Server Usage

CRAB Usage at CERN CAF

CRAB Usage at LPC CAF

CRAB on PBS

CRAB Developers Documentation

 

FAQ, HOWTO, Error Diagnosis

Line: 118 to 120
 
Changed:
<
<

Getting Support

>
>

Troubleshooting

Diagnosing Problems

If a problem is encountered with CRAB you should begin with the following template. This template will help you to identify the nature of the problem and if a solution already exists:

Before contacting the CRAB feedback hypernews, you should consult the Frequently Asked Questions page:

Getting Support

  If you have still not found the solution to your problem, you may wish to consult the CRAB feedback hypernews forum.
Line: 126 to 139
 
    • The mail gateway is hn-cms-crabFeedback(at)cern.ch.

When contacting the hypernews forum, you must provide, if possible, the following information:

Changed:
<
<
>
>
  • task name and in case crab server name
  • URL to task monitoring in dashboard as printed by crab -status command
 
  • Make available your crab task directory, for example in your lxplus public area
Changed:
<
<
If you have problems with the above request, for example you have insufficient disk space, you can make available or send individually the following (if they exist):
>
>
If you have problems with the last request, for example you have insufficient disk space, you can make available or send individually the following (if they exist):
 
  • Your crab configuration file
  • The crab log file.
Line: 137 to 151
 
  • The stdout and stderr of the job concerned
    • These are located in the "res" directory in the CRAB task directory
Changed:
<
<
  • to report a bug or ask for a feature CRAB, use savannah CRAB page
  • Get help for LCG/OSG (i.e: job aborted ...)
    • in case you find some LCG/OSG related problem, such as if your jobs are aborted, please open a ticket on GGUS or send a mail to helpdesk@ggusNOSPAMPLEASE.org providing any useful information.
    • CRAB has the option "-postMortem" that put relevant information about job history on a log file. crab -postMortem JobId. Please include this file to GGUS ticket
  • Get help for data access or DBS
    • in case you find some problem related with the data access or DBS, please open a ticket on savannah computing infrastructure providing any useful information as the datasetpath and the file you are not able to read, the WN where your job run, the CRAB.stderr and stdout.

Links

>
>

Useful Links

 
Changed:
<
<
  • DBS Data Discovery - The data discovery page
  • CMSSW versions installed on remote GRID sites - Looking at each site, you can see the software installed (you can use also this link but is not an official page)
>
>
  • CRAB Related
 
Added:
>
>
  • Certificate Related
<!--      * gLite User Guide  -->
 
Added:
>
>
  • Sites
 
Changed:
<
<

Complete List of CRAB related useful links (Grid pages, CMS pages...)

>
>
 

CRAB Releases Notes

  • CRAB_2_6_6 : Complete list available here

Revision 32010-01-28 - MarcoCalloni

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Added:
>
>
<!--STARTWORKBOOK-->
 

Introduction

CRAB is a utility to create and submit CMSSW jobs to distributed computing resources. By using CRAB you will be able to

  • Access CMS data and Monte-Carlo which is distributed to CMS aligned centres worldwide
  • Exploit the CPU and storage resources at CMS aligned centres
Added:
>
>
<!--STOPWORKBOOK-->
  More infos on Grid model and a typical analysis workflow can be found in the following pages:
Changed:
<
<
  • [[][Chapter Overview -- Getting Started]]
  • [[][Analysis workflow]]
>
>
 

CRAB Client and Server

Line: 32 to 34
 
  • Notification of the progress of jobs submitted by e-mail.
  • Improved user support from Analysis Operations
Changed:
<
<

Links

  • Crab Server page for Users
  • Crab Server WorkBook page
  • Crab Server page for Developers
>
>
Below some useful links:
 

Getting Started

Changed:
<
<
This section describes how to begin using CRAB to perform your analysis on the Grid. In what follows, it is assumed that you have developed and tested a CMSSW analysis job on your local User Interface, e.g. lxplus. <----AGGIUNGI INFOS QUI...AD ESEMPIO LINK TO HOWTO DEVEL CMSSW CODE
>
>
This section describes how to begin using CRAB to perform your analysis on the Grid.
 

Prerequisites

Changed:
<
<
To use CRAB to submit you CMSSW job to the Grid you must meet the following prerequisites. If you are not sure if you have done any of these steps, the links give detailed documentation on how to achieve each prerequisite.
>
>
To use CRAB to submit you CMSSW job to the Grid you must meet the following prerequisites. The links below give detailed documentation on how to achieve each prerequisite.
 
  1. Grid certificate:
Line: 62 to 63
 
Added:
>
>
 

Getting CRAB

Added:
>
>
<!--STARTWORKBOOK-->
 The latest CRAB client can always be accessed at CERN on lxplus by executing the following command:

  • source /afs/cern.ch/cms/ccs/wm/scripts/Crab/crab.(c)sh
Line: 75 to 78
 
  • Source the resulting crab.(c)sh

You need only do the first two steps once.

Added:
>
>
<!--STOPWORKBOOK-->
 
Changed:
<
<

Latest Tutorial

Regular tutorials are given on using CRAB. The course material for these tutorials is stored online for future reference, although refers to specific CRAB and CMSSW versions that may no longer be available.

Advanced CRAB Documentation

The advanced user should refer as appropriate to the following specific documentation as appropriate.

Online Manual

>
>
For developers you can check out CRAB source codes from:
setenv CVSROOT :pserver:anonymous@cmscvs.cern.ch:/cvs_server/repositories/CMSSW 
cvs login   # for anonymous user: pwd='98passwd'
cvs co [-r Tag_version] CRAB
 
Deleted:
<
<
<!-- ++++ Use MultiCRAB  -->
<!-- To submit the same task to multiple dataset in one go.-->
 
Changed:
<
<

CRAB Server Usage

CRAB Usage at CERN CAF

CRAB Usage at LPC CAF

CRAB Developers Documentation

>
>

Basic Workflows

 
Changed:
<
<

Troubleshooting

Diagnosing Problems

>
>
In this page: Crab Basic Workflows you can find a guide to the basic functionality of Crab. If you have never used CRAB before it is a good place to learn how to use the tool.
 
Deleted:
<
<
If a problem is encountered with CRAB you should begin with the following template. This template will help you to identify the nature of the problem and if a solution already exists:
 
Deleted:
<
<
 
Changed:
<
<
Before contacting the CRAB feedback hypernews, you should consult the Frequently Asked Questions page:
>
>

CRAB on-line manual and tutorial

To visit on-line CRAB documentation : To visit the specific CRABSERVER documentation: For specific informations related on How to Run CRAB at CERN CAF: For specific informations related on How to Run CRAB at LPC CAF: For specific informations related on How to Run CRAB on PBS: For specific informations related on CRAB publication:
  • read here: [[][]]
Last CRAB Tutorial (CERN):
 
Added:
>
>

FAQ, HOWTO, Error Diagnosis

 
Added:
>
>
 
Changed:
<
<

Getting Support

>
>

Getting Support

  If you have still not found the solution to your problem, you may wish to consult the CRAB feedback hypernews forum.
Line: 126 to 137
 
  • The stdout and stderr of the job concerned
    • These are located in the "res" directory in the CRAB task directory
Deleted:
<
<

Useful Links

  • Certificate Related
<!--      * gLite User Guide  -->

CRAB Releases Notes

  • CRAB_2_6_5 : Complete list available here
  • Other CRAB versions here

References

CRAB presentations, posters, publications and tutorials page

How to Start with CRAB

To start running jobs with CRAB some prerequisites are needed:
  1. Grid certificate:
  2. siteDB registration:
  3. CMSSW:
  4. Discover data available with DBS
  5. Setup the environment:

Follow the crab tutorial: Tutorial 26th Juny 2009

Information on the crab server can be found in: SWGuideCrabServerForUsers

How to get CRAB

The latest CRAB release is always installed at CERN on lxplus.cern.ch, so you can use it by sourcing:
  • source /afs/cern.ch/cms/ccs/wm/scripts/Crab/crab.csh

For local or private installation you can always find the last CRAB tar ball:

For developers you can check out CRAB source codes from:

setenv CVSROOT :pserver:anonymous@cmscvs.cern.ch:/cvs_server/repositories/CMSSW 
cvs login   # for anonymous user: pwd='98passwd'
cvs co [-r Tag_version] CRAB

CRAB on-line manual and tutorial

To visit on-line CRAB documentation : To visit the specific CRABSERVER documentation: For specific informations related on How to Run CRAB at CERN CAF: For specific informations related on How to Run CRAB at LPC CAF: For specific informations related on How to Run CRAB on PBS: Last CRAB Tutorial (CERN):

How to get support

  • CRAB support
 
  • Get help for LCG/OSG (i.e: job aborted ...)
    • in case you find some LCG/OSG related problem, such as if your jobs are aborted, please open a ticket on GGUS or send a mail to helpdesk@ggusNOSPAMPLEASE.org providing any useful information.
Line: 241 to 144
 
  • Get help for data access or DBS
    • in case you find some problem related with the data access or DBS, please open a ticket on savannah computing infrastructure providing any useful information as the datasetpath and the file you are not able to read, the WN where your job run, the CRAB.stderr and stdout.
Deleted:
<
<

FAQ, HOWTO, Error Diagnosis

 

Links

  • DBS Data Discovery - The data discovery page
  • CMSSW versions installed on remote GRID sites - Looking at each site, you can see the software installed (you can use also this link but is not an official page)
Deleted:
<
<
 

Revision 22010-01-27 - MarcoCalloni

Line: 1 to 1
 
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Introduction

Changed:
<
<
CRAB is a Python program intended to simplify the process of creation and submission of CMS analysis jobs into a grid environment. You'll use it to run your jobs on the grid. More introductory information on the Grid:
>
>
CRAB is a utility to create and submit CMSSW jobs to distributed computing resources. By using CRAB you will be able to

  • Access CMS data and Monte-Carlo which is distributed to CMS aligned centres worldwide
  • Exploit the CPU and storage resources at CMS aligned centres

More infos on Grid model and a typical analysis workflow can be found in the following pages:

  • [[][Chapter Overview -- Getting Started]]
  • [[][Analysis workflow]]

CRAB Client and Server

The workflow of CRAB standalone is based on a direct submission from the UI, where the user is working. This standalone model has the advantage of simplicity, but it lacks some features, which can be provided by a more advanced architecture client-server where a server is placed between the user and the Grid to perform a set of actions.

The main goals of the client-server architecture is to automate as much as possible the whole analysis work flow and to improve the scalability of the system. The client-server implementation is transparent to the end users: the interface, the installation, the configuration procedure and the usage remains exactly the same as for standalone.

Benefits of the CRAB Server

The CRAB server provides several advantages over the client submission.

  • More advanced job monitoring with a web based front end.
  • Automatic resubmission of aborted jobs.
  • Notification of the progress of jobs submitted by e-mail.
  • Improved user support from Analysis Operations

Links

  • Crab Server page for Users
  • Crab Server WorkBook page
  • Crab Server page for Developers

Getting Started

This section describes how to begin using CRAB to perform your analysis on the Grid. In what follows, it is assumed that you have developed and tested a CMSSW analysis job on your local User Interface, e.g. lxplus. <----AGGIUNGI INFOS QUI...AD ESEMPIO LINK TO HOWTO DEVEL CMSSW CODE

Prerequisites

To use CRAB to submit you CMSSW job to the Grid you must meet the following prerequisites. If you are not sure if you have done any of these steps, the links give detailed documentation on how to achieve each prerequisite.

  1. Grid certificate:
  2. siteDB registration:
  3. CMSSW:
  4. Figuring out what data to analyze
  5. Setup the environment:

Getting CRAB

The latest CRAB client can always be accessed at CERN on lxplus by executing the following command:

  • source /afs/cern.ch/cms/ccs/wm/scripts/Crab/crab.(c)sh

If you are working on a machine without access to CERN AFS, you can make a private installation:

  • Download tar ball here.
  • Extract the tar ball, change to the resulting CRAB_X_Y_Z directory and run ./configure
  • Source the resulting crab.(c)sh

You need only do the first two steps once.

Latest Tutorial

Regular tutorials are given on using CRAB. The course material for these tutorials is stored online for future reference, although refers to specific CRAB and CMSSW versions that may no longer be available.

Advanced CRAB Documentation

The advanced user should refer as appropriate to the following specific documentation as appropriate.

Online Manual

<!-- ++++ Use MultiCRAB  -->
<!-- To submit the same task to multiple dataset in one go.-->

CRAB Server Usage

CRAB Usage at CERN CAF

CRAB Usage at LPC CAF

CRAB Developers Documentation

Troubleshooting

Diagnosing Problems

If a problem is encountered with CRAB you should begin with the following template. This template will help you to identify the nature of the problem and if a solution already exists:

Before contacting the CRAB feedback hypernews, you should consult the Frequently Asked Questions page:

Getting Support

If you have still not found the solution to your problem, you may wish to consult the CRAB feedback hypernews forum.

When contacting the hypernews forum, you must provide, if possible, the following information:

  • Make available your crab task directory, for example in your lxplus public area

If you have problems with the above request, for example you have insufficient disk space, you can make available or send individually the following (if they exist):

  • Your crab configuration file
  • The crab log file.
    • This is located in the "log" directory in the CRAB task directory.
  • The stdout and stderr of the job concerned
    • These are located in the "res" directory in the CRAB task directory

Useful Links

  • Certificate Related
<!--      * gLite User Guide  -->

CRAB Releases Notes

  • CRAB_2_6_5 : Complete list available here
  • Other CRAB versions here

References

CRAB presentations, posters, publications and tutorials page

 

How to Start with CRAB

Revision 12010-01-27 - MarcoCalloni

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="SWGuide"

CMS Remote Analysis Builder - CRAB

Introduction

CRAB is a Python program intended to simplify the process of creation and submission of CMS analysis jobs into a grid environment. You'll use it to run your jobs on the grid. More introductory information on the Grid:

How to Start with CRAB

To start running jobs with CRAB some prerequisites are needed:
  1. Grid certificate:
  2. siteDB registration:
  3. CMSSW:
  4. Discover data available with DBS
  5. Setup the environment:

Follow the crab tutorial: Tutorial 26th Juny 2009

Information on the crab server can be found in: SWGuideCrabServerForUsers

How to get CRAB

The latest CRAB release is always installed at CERN on lxplus.cern.ch, so you can use it by sourcing:
  • source /afs/cern.ch/cms/ccs/wm/scripts/Crab/crab.csh

For local or private installation you can always find the last CRAB tar ball:

For developers you can check out CRAB source codes from:

setenv CVSROOT :pserver:anonymous@cmscvs.cern.ch:/cvs_server/repositories/CMSSW 
cvs login   # for anonymous user: pwd='98passwd'
cvs co [-r Tag_version] CRAB

CRAB on-line manual and tutorial

To visit on-line CRAB documentation : To visit the specific CRABSERVER documentation: For specific informations related on How to Run CRAB at CERN CAF: For specific informations related on How to Run CRAB at LPC CAF: For specific informations related on How to Run CRAB on PBS: Last CRAB Tutorial (CERN):

How to get support

  • CRAB support
  • Get help for LCG/OSG (i.e: job aborted ...)
    • in case you find some LCG/OSG related problem, such as if your jobs are aborted, please open a ticket on GGUS or send a mail to helpdesk@ggusNOSPAMPLEASE.org providing any useful information.
    • CRAB has the option "-postMortem" that put relevant information about job history on a log file. crab -postMortem JobId. Please include this file to GGUS ticket
  • Get help for data access or DBS
    • in case you find some problem related with the data access or DBS, please open a ticket on savannah computing infrastructure providing any useful information as the datasetpath and the file you are not able to read, the WN where your job run, the CRAB.stderr and stdout.

FAQ, HOWTO, Error Diagnosis

Links

Complete List of CRAB related useful links (Grid pages, CMS pages...)

CRAB Releases Notes

  • CRAB_2_6_6 : Complete list available here
  • Other CRAB versions here

CRAB references

CRAB presentations, posters, publications and tutorials page

-- MarcoCalloni - 27-Jan-2010

 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback