Week of 110425

Daily WLCG Operations Call details

To join the call, at 15.00 CE(S)T Monday to Friday inclusive (in CERN 513 R-068) do one of the following:

  1. Dial +41227676000 (Main) and enter access code 0119168, or
  2. To have the system call you, click here
  3. The scod rota for the next few weeks is at ScodRota

WLCG Service Incidents, Interventions and Availability, Change / Risk Assessments

VO Summaries of Site Usability SIRs, Open Issues & Broadcasts Change assessments
ALICE ATLAS CMS LHCb WLCG Service Incident Reports WLCG Service Open Issues Broadcast archive CASTOR Change Assessments

General Information

General Information GGUS Information LHC Machine Information
CERN IT status board M/W PPSCoordinationWorkLog WLCG Baseline Versions WLCG Blogs   GgusInformation Sharepoint site - Cooldown Status - News


Monday:

  • No meeting - CERN closed.

Tuesday:

Attendance: local();remote().

Experiments round table:

  • ATLAS reports -
    • In a nutshell: Physics all day (data11_7TeV) with short calibration periods and few interruptions
    • Peak luminosity record broken for a hadron collider
    • T0
      • Tier0 manager hung and no heartbeat in T0 monitor (twice over the Easter break). Seems to be related to bpeek option in the task lister. Experts are aware and investigating the problem.
      • Massive problems reading and writing to ATLAS svcclass/pool T0merge (twice over the Easter break). Caused by a backup job that should have been migrated to a different node.
    • T1s
      • IN2P3-CC: Transfers to/from the site were failing heavily (including T0 export) on Friday between 4AM-8:30AM. Presumably dCache problem - only known solution is to restart the pool manager.
      • Staging from tape broken for TRIUMF-LCG2_MCTAPE since Monday morning: "Pinning failed: finding read pool failed".
    • Central Services
      • Downtime collector stuck - quattor template of the machine had been incorrectly modified and was preventing any activity of the user under which the collectors run. This is why RAL was not automatically re-included in DDM activity for a couple of hours after coming back from their downtime on Thursday ~12:00AM.

  • ALICE reports -
    • T0 site
      • Nothing to report
    • T1 sites
      • Nothing to report
    • T2 sites
      • Usual operations

  • LHCb reports -
    • RAW data distribution and their FULL reconstruction is going on at most Tier-1s.
    • A lot of MC continues to run.
    • T0
      • Problem SOLVED: Problems staging files out of Tape (72 files). Files requested to be staged yesterday we would have expected them online.
      • Yesterday, there were files missing to the OFF-LINE due to a hardware failure. Now (earlier) files start to move to OFF-LINE.
    • T1
      • IN2P3: Problems with software installation, "share" set to zero. Solution is in progress.
      • RAL: Storage Elements full (RAW and RDST, which use the same Space Token). It was reported that "some tape drives becoming stuck and not working", which seems to be fixed. However, there still a big backlog.
    • T2

Sites / Services round table:

  • CERN VOMS service The certificate for the LHC voms services on voms.cern.ch will be updated tomorrow on Wednesday around 10:00 CEST April 27th. The current version of lcg-vomscerts is 6.4.0 and was released 2 weeks ago. It should certainly be applied to gLite 3.1 WMS and FTS services. [ Has been put into release for those services a few weeks ago. T1s running FTS services should be sure that they have latest version of RPM.

AOB:

Wednesday

Attendance: local();remote().

Experiments round table:

Sites / Services round table:

AOB:

Thursday

Attendance: local();remote().

Experiments round table:

Sites / Services round table:

AOB:

Friday

Attendance: local();remote().

Experiments round table:

Sites / Services round table:

AOB:

-- JamieShiers - 19-Apr-2011

Edit | Attach | Watch | Print version | History: r21 | r6 < r5 < r4 < r3 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r4 - 2011-04-26 - DirkDuellmann
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback