General information:

A circumflex (^) signifies a return for multiline commands in certain places.

Discussion of SchedConfig Parameter Definitions

  • ACCESSCONTROL - Allows access to specific groups of people. Set to grouplist if desired.
  • ALLOWDIRECTACCESS - jobs with transferType=direct will be matched by brokerage to sites with allowdirectaccess=True, allowdirectaccess=True for all sites which have copysetup true for direct access. Default False
  • ALLOWEDGROUPS - Sets allowed VOMS groups for the queue. Example: /atlas/.+/Role=production
  • ALLOWEDNODE - Determines allowed worker node names within the queue. Example: (brndt3head|wrk[0-9]+prv)\.hep\.brandeis\.edu
  • ALLOWFAX - Is FAX allowed or not? (True/False)
  • APPDIR - Directory in which the ATLAS Athena software releases are installed
  • AVAILABLECPU - Amount of the real CPU capacity of the site as seen by ATLAS in HS06 (cf. PLEDGEDCPU). It could be the entire capacity of the site for an ATLAS-only site, the capacity of the ATLAS share, or the capacity of the ATLAS share plus an average capacity increment from opportunistic ATLAS use of non-ATLAS resources at the site.
  • AVAILABLESTORAGE - Amount of real storage compared to pledged PLEDGEDSTORAGE
  • CACHEDSE - Enable PD2P by setting to 1. Only relevant for ANALY sites.
  • CELIST - A JSON representation of the list of CEs that a virtual queue (often seen in ANALY queues) may address. To populate, work by analogy from other queues where celist is set. This is ONLY for AGIS compatibility, to allow changes to schedconfig to auto-populate AGIS.
  • CLOUD - The home cloud, possibly at the start of a comma-separated list of clouds. The site can contribute to any cloud in the list(Multi-cloud) If disabled, the config file will be found in "Disabled"
  • CMD - Batch queue submit command for the queue. Examples: condor_submit -verbose %s, qsub -q panda %s
  • CMTCONFIG - Set to the appropriate CMTCONFIG for a queue. Mostly i686-slc5-gcc43-opt
  • COMMENT_ - Any comment necessary to the queue, to be properly set when the queue status (STATUS) is set. NOT changeable in AGIS. NOT stored in the SVN.
  • COPYPREFIX - Prefix for SRM copies out from the queue. Example: ^srm:// If COPYPREFIXIN is None, this covers all copies in and out of the queue.
  • COPYPREFIXIN - Prefix for SRM copies into the queue. Example: ^srm://
  • COPYSETUP - Commands to be run before (inbound) outbound copies; may be a script, or a set of commands like /somepath/^srm://^dcap://^False^True. If COPYSETUPIN is None, this covers all copies in and out of the queue. For more info see
  • COPYSETUPIN - Commands to be run before inbound copies; essentially the same as COPYSETUP but can be used to specify a different stage-in setup than for stage-out
  • COPYTOOL - The (inbound) outbound copy tool for (in)output files. Examples: cp, lcg-cp, storm, xcp. If COPYTOOLIN is None, this covers all copies in and out of the queue.
    • lcg-cp ( lcgcp ): pilots use lcg-cr for upload to SE and registration to LFC
    • lcg-cp2 ( lcgcp2 ): pilots use lcg-cp for upload to SE and no registration to LFC by pilot. To make the registration done the panda server, the other parameter needs to be set (LFCREGISTER='server').
  • COPYTOOLIN - The inbound copy tool for input files. Examples: cp, lcg-cp, storm, xcp. If set to None, COPYTOOL will be used for both stage-in and stage-out
  • CORECOUNT - Number of cores per machine for AthenaMP use
  • COUNTRYGROUP - Allows for national federations
  • DATADIR - Directory in which data are stored
  • DDM - A comma-separated list of DDM endpoints used for input data. For a T1 the order should match the setoken order, because this is used to translate the output file space token into the final ddm destination. It is ok for the ddm list to be longer than setokens, e.g. to have an extra input location. For a T2 this is typically just local PRODDISK. The first value in the list has some special meanings. 1) The free space in this endpoint ends up in via curl). 2) If the site is a T1, then it also ends up in the cloud table space, which is used for task assignment, 3) Bamboo may choose to subscribe EVNT input between clouds, to aid brokerage - the destination space token is this one.
  • DEPTHBOOST - Maximum number of pilotDepth block submissions when a site has enough jobs
  • DN - Grid DN of the most recent modifier
  • DQ2URL - DQ2 Access URL
  • EMAIL - The site admin's contact email(s)
  • ENVIRON - List of environment variables to be set. Example: APP=/home/osgstore/app TMP=/home/osgstore/tmp DATA=/home/osgstore/data
  • ENVSETUP - Command that is run (can be variable setting or script) to set up copy for (in)out transfers. If ENVSETUPIN is None, this covers all copies in and out of the queue.
  • ENVSETUPIN - Command that is run (can be variable setting or script) to set up copy for out transfers.
  • ERRINFO - Not used.
  • FAXREDIRECTOR - The FAX redirector (e.g. root://
  • GATEKEEPER - The physical hostname of the headnode for the queue. For virtual queues, it is
  • GLEXEC - Is GLEXEC set? (No, for the most part)
  • GLOBUSADD - A part of the JDL for some circumstances. ??
  • GOCNAME - Set from AGIS rc_name
  • GSTAT - Derived from AGIS' atlas_site variable Derived from BDII GSTAT variable. Site identity.
  • IDLEPILOTSUPRESSION - Only send an idling pilot every N cycles
  • JDL - The NAME of the entry in the jdllist table that provides the basic JDL for the jobs being submitted
  • JDLADD - Any additional JDL needed for the queue. Must end with Queue, on its own line.
  • JDLTXT - ??
  • JOBMANAGER - condor, lsf, cream, arc, pbs, etc. For virtual queues, it is
  • LASTMOD - Timestamp of last modification. Not stored in SVN.
  • LAST_STATUS - Status historical state (helps with bulk changes online/offline). Not stored in SVN.
  • LFCHOST - LFC (LHC File Catalog) to use with this queue.
  • LFCPATH - LFC path on LFCHOST to use with this queue.
  • LFCPRODPATH - LFC path on LFCHOST to use with this queue for PROD jobs.
  • LFCREGISTER - LFC registration is done by the pilot ( 'None' ) or by the panda server ( 'server' ). should be used with COPYTOOL=lcgcp2 . All the queues in the panda siteid MUST have the same value.
  • LOCALQUEUE - local queue name to use in the queue's batch system. If unset, this is not specified.
  • MAXINPUTSIZE - 14336 is the default (14 GB), but can be higher. Maximum copy-to-scratch input data total. To be obsolete after MAXWDIR being set properly.
  • MAXMEMORY - Memory available (MB) per job. Zero means infinite and should not be used in queue config. Used in Panda Brokerage. The pilot will set this as a virtual memory limit for the payload to cause a clean malloc() failure.
  • MAXTIME - Wall time. Zero means infinite.
  • MAXWDIR - Maximum available workdir space per job. The initial value has been set as MAXINPUTSIZE + 2GB
  • MINMEMORY - Minimum memory (MB) requirement for the jobs to be assigned to the queue. To be set only for the "hi-mem" queues, and zero for standard ("lo-mem") queues. Used in Panda Brokerage.
  • MINTIME - Wall time (s) requirement for the jobs to be submitted to the queue. To be set only for the "long" queues, and zero for standard ("short") queues. Used in Panda Brokerage.
  • MULTICLOUD - The queue contributes to the tasks with the destination clouds specified here. NOT changeable in AGIS. NOT stored in the SVN.
  • NAME - Unused. Set to default.
  • NICKNAME - The unique PRIMARY KEY of the table. The queue is named by this parameter. Should match the config file name.
  • NODES - Number of compute elements in the queue's possession.
  • NQUEUE - Baseline number of pilots to queue. APFv2 does not use this value, though the APFv2 factories at CERN are automatically configured with this value taken into account. NOT changeable in AGIS. Not stored in SVN.
  • PCACHE - pCache enabled ( None or 1)
  • PILOTLIMIT - Maximum number of pilots to have (in any state) at a site
  • PLEDGEDCPU - CPU Pledges (MoU) in HS06, contributes to ration with AVAILABLECPU. If PLEDGEDCPU=0, the 'weight' AVAILABLECPU/PLEDGEDCPU will be set to 1.
  • PLEDGEDSTORAGE - Storage Pledges (MoU), contributes to ration with AVAILABLESTORAGE
  • PROXY - donothide or noimport. Currently not used by the pilot
  • PYTHON_PATH - Sets the PYTHONPATH variable at run start
  • QUEUEHOURS - Max job lifetime in hours
  • RECOVERDIR - Local env. variable for recovery data. Example: $ATLAS_RECOVERDIR
  • REGION - Region. Example: us
  • RELEASES - Becoming obsolete. List of releases available, separated by pipes. Autogenerated for BDII sites. Example: 0.1.31|0.1.32|0.1.33|0.1.34|14.2.21|14.2.25|15.0.0|15.3.1|15.5.4|15.6.1|15.6.10|15.6.11|15.6.3|15.6.5|15.6.6|15.6.7|15.6.8|15.6.9|15.8.0
  • RETRY - (TRUE or FALSE) Turns on or off job recovery
  • SE - space token and full endpoint of default output destination. The path is extracted from the matching token in seprodpath If output files do not have spacetoken set, then the token in this entry is used. This applies to direct storage from T1 jobs, but not to subscriptions from T2(see setokens).
  • SEIN - Comma separated list of SEs, same length as DDM and SEPATH/SEPRODPATH for input transfers
  • SEINOPT - Currently not used by the pilot
  • SEOPT - Comma separated list. seopt is used in combination with SETOKENS to select the proper SE corresponding to a given space token descriptor. The order of SE's listed in SEOPT has to match the order of tokens in SETOKENS
  • SEPATH - Same as SEPRODPATH but for analysis sites
  • SEPRODPATH - list of destination paths, to append to endpoint in 'se'. Compressed notation is supported /blah/[tok1,tok2]/more/. Must be the same length as setokens.
  • SETOKENS - List of destination space tokens. The first token is the default destination(for subscription from T2) if an output file does not have spacetoken set. Must match length of SEPRODPATH
  • SITE - Site name
  • SITEID - Name of similar queues in site (shared gatekeepers)
  • SPACE - Available disk space on the site. Refers to first token in 'ddm'. NOT changeable in AGIS. NOT stored in the SVN.
  • SPECIAL_PAR - Additional JDL to add to job
  • STATUS - Site online/offline/test/brokeroff status. NOT changeable in AGIS. Not stored in the SVN.
  • STATUSOVERRIDE - (offline, notonline, None) to manage changing of site status through cloud curl commands. Default - offline, site will not change status. notonline - site cannot be set online. None - all changes are possible. NOT changeable in AGIS.
  • SYSCONFIG - (None or manual) - Prevent BDII updating of parameters
  • TAGS - Obsolete
  • TIER - ?
  • TIMEFLOOR - The pilot will keep downloading jobs during the limit set by timefloor (minutes). If at the end of a job, more time has passed than that of timefloor since the beginning of the first job, the pilot will end
  • TMPDIR - Temp directory
  • TRANSFERRINGLIMIT - Maximum number of transferring jobs to allow at a site
  • TSPACE - ? -- NOT changeable in AGIS.
  • VALIDATEDRELEASES - Reprocessing jobs are brokered only to the queues with this value set to 'True'
  • VERSION - Unused
  • WNTMPDIR - Worker node temp directory

Major updates:
-- AldenStradling - 30-Nov-2010

Responsible: AldenStradling

Never reviewed

Edit | Attach | Watch | Print version | History: r23 < r22 < r21 < r20 < r19 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r23 - 2016-10-14 - PaulNilsson
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    PanDA All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback