Submitting jobs to the CERN HTCondor pool

This twiki explains how to use the preparelocal CRAB command to send jobs to the CERN condor pool (

Preliminary setup

Once you have setup the environment, you need to create the CRAB project directory for your task. If you

  • have already submitted the task you can
    • simply cd to the project directory created at submission time
    • or create it with the crab remake command;
  • have not yet submitted the task, do it with the --dryrun option

Once the CRAB project directory is created, execute

crab preparelocal --dir = <PROJECTDIR>

HTCondor submission

Add #!/bin/bash as first line of the file in the <PROJECTDIR>/local directory.

Create a "batch" subdirectory in order to keep HTCondor files separated:

mkdir <PROJECTDIR>/local/batch
cd <PROJECTDIR>/local/batch

and place the following task.jdl example file therein:

Universe  = vanilla
Executable = ../
Arguments = $(I)
Log = log/job.$(Cluster).$(Process).log
Output = out/job.$(Cluster).$(Process).out
Error = err/job.$(Cluster).$(Process).err
transfer_input_files = ../, ../CMSRunAnalysis.tar.gz, ../InputArgs.txt, ../Job.submit, ../, ../, ../input_files.tar.gz, ../run_and_lumis.tar.gz, ../sandbox.tar.gz
should_transfer_files = YES
when_to_transfer_output = ON_EXIT

# Resources request
RequestCpus = 1
RequestMemory = 2000
+JobFlavour = "workday"

# Jobs selection
Queue I from (

This configuration example will submit only the first 4 jobs of the task. In order to customise the resources request (number of CPUs, maximum memory, maximum runtime, etc.) and the jobs submission please refer to the CERN Batch Service documentation:

Create the auxiliary directories and submit the jobs with:

mkdir out err log
condor_submit task.jdl

You can check the jobs status with:

condor_q -nobatch

-- MarcoMascheroni - 2018-06-06

Edit | Attach | Watch | Print version | History: r5 < r4 < r3 < r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r4 - 2018-06-14 - LeonardoCristella
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback