WLCG Workload Recommendations

Scope of recommendations

These recommendations are intended as a guide for new sites or sites looking to make changes to their workload management systems, as in batch scheduler or grid CE.

Picking one of the recommended solutions is not mandatory. Sites are free to choose other solutions if they wish, as long as they make their users (LHC experiments in WLCG) and funders happy (for pledged resources this very likely means having working accounting and monitoring).

The recommendations below has some rationale for which to choose, but other factors might be more relevant for your site, like what do nearby sites run or what local expertise do you have access to.

Recommended batch systems

  • HTCondor - for sites with HTC loads, lots of jobs each up to one node in size. Especially for sites with a very large number of jobs. A typical dedicated WLCG site looks like this.
  • SLURM - for site with HPC loads, as in multi-node MPI.

Recommended CEs

  • HTCondor-CE - makes most sense when connected to HTCondor
    • Caveats:
      • There is no standard solution for accounting yet
        • The code developed by PIC may be reusable
      • This CE type can be used by all LHC experiment frameworks, but check with other communities your site needs to support
      • A generic GLUE-2 info provider provided by CERN-IT has been accepted upstream and will be included in future releases
  • ARC-CE - works with SLURM and HTCondor.
    • Also used for lightweight CEs with file staging to local filesystem instead of a close SE

Stakeholder statements

Here we will list further statements from experiments or infrastructures that could help sites make informed choices.

For example:

  • In the Nordic countries, ARC-CE has good local expertise. -- Mattias Wadenstein for NDGF & NorduGrid collaboration.

Editors

The editors of the recommendations are AlessandraForti, HelgeMeinhard, MaartenLitmaath, MattiasWadenstein - please send any stakeholder statements or issues to them.

-- MattiasWadenstein - 2018-01-10

Edit | Attach | Watch | Print version | History: r4 < r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r4 - 2018-02-19 - MaartenLitmaath
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback