Week of 130923

WLCG Operations Call details

To join the call, at 15.00 CE(S)T, by default on Monday and Thursday (at CERN in 513 R-068), do one of the following:

  1. Dial +41227676000 (Main) and enter access code 0119168, or
  2. To have the system call you, click here

The scod rota for the next few weeks is at ScodRota

WLCG Availability, Service Incidents, Broadcasts, Operations Web

VO Summaries of Site Usability SIRs Broadcasts Operations Web
ALICE ATLAS CMS LHCb WLCG Service Incident Reports Broadcast archive Operations Web

General Information

General Information GGUS Information LHC Machine Information
CERN IT status board WLCG Baseline Versions WLCG Blogs GgusInformation Sharepoint site - LHC Page 1



  • local: AndreaV/SCOD, Vitor/Grid, Ken/CMS, MariaD/GGUS, Belinda/Storage, Ivan/Dashboard
  • remote: Michael/BNL, Onno/NLT1, John/RAL, Sang-Un/KISTI, Wei-Jen/ASGC, Christian/NDGF, Lisa/FNAL, Rob/OSG, Rolf/IN2P3, Salvatore/CNAF, Doug/ATLAS

Experiments round table:

  • ATLAS reports (raw view) -
    • Central Services
      • FT3-Pilot (GGUS:97359) Problem with a cached proxy affected functional tests to all sites. (solved)
      • AFS - ~13:03 on 19-Sept spurious rm process on /afs/cern.ch/atlas/offline/* removed RW areas including panda client areas needed by Hammer Cloud. Computing operations restored the needed area from tape promptly when alerted 20-Sept. Exact cause of rm is unknown.(INC:388802) ATLAS investigation continues.
    • T0
      • NTR
    • T1
      • NTR

  • CMS reports (raw view) -
    • It has been fairly quiet until just a few hours ago!
    • Appears to be a CVMFS problem at KIT, see GGUS:97505 and GGUS:97506.
    • SAV:139882 and SAV:139885 appear to indicate EOS transition teething pains at FNAL.
    • No one seems to be complaining about network connectivity to Russia, nor has anyone given me any update on the status.
    • Why does CERN external networking show as degraded to 60% when it appears that all services are green? The service has been degraded for a week; I think the usual culprit is the (irrelevant) link to the Wigner center, but at the moment that looks green too. [MariaD: this may be due to the many interventions last week. AndreaV: will follow up - discussed this with Edoardo after the meeting]

  • ALICE -
    • NTR

  • LHCb reports (raw view) -
    • Main activity are MC productions
    • Fall incremental stripping campaign will be launched coming Monday 30 Sep (see also WLCG Operations Coordination meeting, 19 Sep)
    • T0: ntr
    • T1:
      • IN2P3: Currently in DT for sl6 upgrade [Rolf/IN2P3: sorry we did a mistake, IN2P3 is marked in DT today since yesterday evening, but actually the downtime and interventions will start this evening]
      • SARA: Currently in DT
      • GRIDKA: In DT as of 14.00 today

Sites / Services round table:

  • Michael/BNL: ntr
  • Onno/NLT1:
    • today's maintenance was successful
    • announcement, on October 8 we will move from LHCOPN to LHCONE
  • John/RAL: disk server issue for ALICE, being fixed
  • Sang-Un/KISTI: ntr
  • Wei-Jen/ASGC: ntr
  • Christian/NDGF: ntr
  • Lisa/FNAL: ntr
  • Rob/OSG: ntr
  • Rolf/IN2P3: nta
  • Salvatore/CNAF: ntr
  • Pavel/KIT [via email]: CVMFS problems are not yet quite understood and under investigation

  • Belinda/Storage:
    • issue with non-resolving IP address for EOS ALICE over the weekend, now fixed but a more robust fix is being developed (this is a known issue)
    • EOS ATLAS crashed this morning, fixed by restarting, investigations ongoing
  • Vitor/Grid: ntr
  • Ivan/Dashboard: ntr
  • MariaD/GGUS: Due to personal circumstances in the GGUS team, GGUS release will be postponed from this Wednesday 2013/09/25. You shall be informed about the new release date. It will be decided a.s.a.p.

AOB: none



  • local:
  • remote:

Experiments round table:

  • ALICE -
    • various T2 moved or moving to CVMFS

Sites / Services round table:

  • AndreaV/Network: Edoardo Martelli clarified the issue with external network SLS reported by CMS on Monday. USLHCNet has decommissioned some devices in the US that were no longer relevant to the network for CERN and the SLS status. IT-CS is working with USLHCnet to define the exact devices to be monitored: until this is clarified, a (fake) degradation of external network connectivity is expected to show up in SLS.


Edit | Attach | Watch | Print version | History: r13 | r11 < r10 < r9 < r8 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r9 - 2013-09-25 - MaartenLitmaath
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2023 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback