Week of 200525

WLCG Operations Call details

  • For remote participation we use the Vidyo system. Instructions can be found here.

General Information

  • The purpose of the meeting is:
    • to report significant operational issues (i.e. issues which can or did degrade experiment or site operations) which are ongoing or were resolved after the previous meeting;
    • to announce or schedule interventions at Tier-1 sites;
    • to inform about recent or upcoming changes in the experiment activities or systems having a visible impact on sites;
    • to provide important news about the middleware;
    • to communicate any other information considered interesting for WLCG operations.
  • The meeting should run from 15:00 Geneva time until 15:20, exceptionally to 15:30.
  • The SCOD rota for the next few weeks is at ScodRota
  • Whenever a particular topic needs to be discussed at the operations meeting requiring information from sites or experiments, it is highly recommended to announce it by email to wlcg-scod@cernSPAMNOTNOSPAMPLEASE.ch to allow the SCOD to make sure that the relevant parties have the time to collect the required information, or invite the right people at the meeting.

Best practices for scheduled downtimes



  • local:
  • remote: Miro (Chair, DB), Christoph (CMS), Maarten (ALICE, GGUS), Dave (FNAL), Onno (NL-T1), Vincent (Security), Julia (WLCG), Borja (Monitoring), Elena (CNAF), David (IN2P3), Andrew (TRIUMF), Aritz (Storage), Concezio (LHCb), Nils (Compute), Pavlo (ATLAS)

Experiments round table:

  • ALICE -
    • NTR

  • LHCb reports ( raw view) -
    • Activity:
      • Usual MC and working group productions
    • Issues: Nothing new to report.

Sites / Services round table:

  • ASGC: NC
  • BNL: NC
  • EGI: NC
  • IN2P3: NTR
  • KIT: NC
  • NL-T1:
    • SARA-MATRIX had two dCache pool nodes that were not running properly. After a boot they're OK.
    • New releases of dCache (5.2.21, and also the 6.1.4 we have since last week's downtime) seem to check xrootd usernames in a different way. Some usernames are now regarded as bad, for example: adrian.c, atlaspro.3872:30. This issue is discussed on the dCache mailing list. There is also this ticket: GGUS:147117
  • NRC-KI: NC
  • OSG: NC
  • PIC: (J.Flix) Sorry, I was not able to connect. Last week, PIC Disk for CMS was full, and it took some time until files where deleted from the CMS p.o.v. The CMS tape usage is as well at maximum, and we received a few days ago a massive deletion of files. Hope this congested situation improves soon.
  • RAL: NC

  • CERN computing services:
    • OTG:0056642 - 40% of public batch capacity was lost on Thursday morning (21st May) due to power problem at point 8. Recovered during the day.
    • OTG:0056457 - Wed 27th May - migration to new schedd mapping tool for local users (should be transparent, and not affecting CEs)
  • CERN storage services: NTR
  • CERN databases: NTR
  • GGUS:
    • A new release is planned for Wed this week
      • Release notes
      • A downtime has been scheduled for 06:30-08:30 UTC
      • Test alarms will be submitted as usual
  • Monitoring:
    • Final reports for the April 2020 availability sent around
  • MW Officer: NC
  • Networks: NTR
  • Security: Incident within HPC sites still ongoing.


  • NOTE: the operations meeting next Mon will be virtual .
  • You may provide relevant incidents, announcements etc. for the operations record.
Edit | Attach | Watch | Print version | History: r19 < r18 < r17 < r16 < r15 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r19 - 2020-05-25 - MaartenLitmaath
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback