WLCG Containers Working Group

Mandate

The mandate discussed at the first meeting and agreed at the 11 July 2017 GDB is:

To understand and track the use of containers within WLCG for compute deployment, and to make recommendations to sites on potential deployment patterns that they may wish to adopt. To ensure that Singularity and its support services are deployable for the LHC experiments.

We agreed two main goals:

  1. Singularity is sensibly deployable in WLCG and used by all experiments, and we’ve documented and understand the operational aspects.
  2. We should have surveyed and made recommendations on potential deployment patterns (and technologies) for container-based compute deployment and provisioning, with an aim to reduce operational effort on the sites, and to meet the needs of the ongoing analysis reproducibility work going on in the experiments. At the end, we should have clearly documented a few reasonable paths for a site (should it wish to) to migrate their compute to these technologies.

Work areas

Singularity deployment

The group's current focus is coming up with a baseline recommendation for the deployment of Singularity for sites, while trying to ensure as much commonality as possible between the experiments in the way they use it.

Container-based compute deployment recommendations

The final baseline document for Singularity is attached below.

Actions

Action Who What Status
WC10 Ben CERN to enable unprivileged namespaces and disable SUID after tests Done on Tier-0 resources, to-do on public share.

DONE Who What Status
WC1 Maarten, Alessandra Contact David Love in Manchester (the current Singularity EPEL maintainer) to discuss our community's needs Done. Brian now maintains EPEL.
WC2 Andrew Describe more fully the potential workaround for ATLAS' PWD bind-mounting Done / Superseded with WC6.
WC3 Andrej Understand whether WC6 solution would work better for ATLAS Yes, thought that underlay should help a lot.
WC4 Dave, Jakob Understand the issue with bind-mount overlay on CVMFS unpacked images, and whether a CVMFS workaround is appropriate Understood: CVM-1434. Workaround is to use underlay.
WC5 Gavin Start google doc to track emerging baseline recommendation and issues Done, see http://cern.ch/go/Q6rP
WC6 Vincent, Dave Track feature request with upstream Singularity for unprivileged overlay trick Now supported by EPEL 2.6 & upstream 3.0,
WC7 Alessandra / Andrej Test the new underlay feature for ATLAS and comment on pull-request Reported https://docs.google.com/spreadsheets/d/1SGKyja47Veu_8IUXlXWOOEferuFoD62O4m64pgTNgSk/edit
WC8 Olga / Gavin Test the new underlay feature for CERN and comment on pull-request Running in production now
WC9 Olga / Gavin CERN to setup HammerCloud based CI test for epel-testing including Singularity based test Not HC, but our batchtest resources run this in production now
WC11 Gavin, Maarten Write down initial baseline recommendation based on 3 July meeting and iterate on list FINAL Done

Meetings

The Indico event area is https://indico.cern.ch/category/9769/.

Reports

Mailing lists

The mailing list is wlcg-containers@cernNOSPAMPLEASE.ch - you can subscribe from egroups.cern.ch.

Edit | Attach | Watch | Print version | History: r15 < r14 < r13 < r12 < r11 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r15 - 2019-10-16 - GavinMcCance
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback