WLCG Operations Coordination Minutes, July 4, 2019





  • local:
  • remote:
  • apologies:

Operations News

Special topics

First look at site survey results

Middleware News

Tier 0 News

Tier 1 Feedback

Tier 2 Feedback

Experiments Reports


  • Normal activity levels on average.
  • No major issues.


  • Smooth Grid production over the last weeks with ~400k concurrently running grid job slots with the usual mix of MC generation, simulation, reconstruction, derivation production and user analysis and a small fraction of dedicated data reprocessing. This includes ~100k job slots from the HLT/Sim@P1 farm. Some periods of additional HPC contributions with peaks of ~100k concurrently running job slots running simulation using EventService.
  • Commissioning of a new PanDA worker node pilot version on-going. We are continuing to slowly roll out the new version out to the sites and enabling singularity.
  • CentOS7 migration still not finished - about 20-30% of Tier2s still in the process of migration: link


  • smooth running, compute systems busy at about 230k cores
    • usual production/analysis mix, i.e. about 50k cores used by analysis
    • 2.7 billion events reconstructed so far of the B-physics data that could not be processed during Run 2 but were parked instead
    • heavy-ion data re-reconstruction 80% complete
    • Monte Carlo simulation of the 2018 detector configuration and to match the re-reconstructed 2017 data (ultra-legacy reconstruction)
  • high-level trigger farm becoming available again after UPS fire


Task Forces and Working Groups

GDPR and WLCG services

Accounting TF

  • WLCG Operations Team works on the new display which will combine CPU accounting and Storage Space Accounting plots. There will be also a possibility for site admins to validate auto-generated accounting data and only validated/corrected data would be included in the monthly accounting reports

Archival Storage WG

Information System Evolution TF

IPv6 Validation and Deployment TF

Detailed status here.

Machine/Job Features TF


MW Readiness WG

Network Throughput WG

Squid Monitoring and HTTP Proxy Discovery TFs

  • wlcg-wpad is now including squids that are registered with shoal.
    • While testing there was only one registered in shoal that wasn't also statically registered in GOCDB or OIM
    • Thinking about including shoal-agent in the frontier-squid rpm and making it very easy to auto-register with a config variable or two

Traceability WG

Two documents finalized within the WG, external feedback welcome:

These results and the potential next steps will be presented and discussed at the GDB next week: https://indico.cern.ch/event/739880/

Container WG

Action list

Creation date Description Responsible Status Comments

Specific actions for experiments

Creation date Description Affected VO Affected TF/WG Deadline Completion Comments

Specific actions for sites

Creation date Description Affected VO Affected TF/WG Deadline Completion Comments


This topic: LCG > WebHome > WLCGCommonComputingReadinessChallenges > WLCGOperationsWeb > WLCGOpsCoordination > WLCGOpsMinutes190704
Topic revision: r8 - 2019-07-04 - JuliaAndreeva
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback