Week of 120716

Daily WLCG Operations Call details

To join the call, at 15.00 CE(S)T Monday to Friday inclusive (in CERN 513 R-068) do one of the following:

  1. Dial +41227676000 (Main) and enter access code 0119168, or
  2. To have the system call you, click here
  3. The scod rota for the next few weeks is at ScodRota

WLCG Service Incidents, Interventions and Availability, Change / Risk Assessments

VO Summaries of Site Usability SIRs, Open Issues & Broadcasts Change assessments
ALICE ATLAS CMS LHCb WLCG Service Incident Reports WLCG Service Open Issues Broadcast archive CASTOR Change Assessments

General Information

General Information GGUS Information LHC Machine Information
CERN IT status board M/W PPSCoordinationWorkLog WLCG Baseline Versions WLCG Blogs   GgusInformation Sharepoint site - Cooldown Status - News


Monday

Attendance: local(Massimo, Luc, Maarten, Giuseppe, Ulrich, Edward, Alexandre);remote(Michael, Saerda, Gonzalo, JhenWei, Lisa, Ronald, Paolo, Tiju, Vladimir, Rolf, Rob).

Experiments round table:

  • ATLAS reports -
    • CERN CENTRAL SERVICES, T0
    • T1
      • PIC transfer failures after migration to Chimera. Alarm ticket GGUS:84217. PIC stable now & back in T0 export.
    • CALIB_T2

  • CMS reports -
    • LHC machine / CMS detector
      • Taking data during the week-end
      • Van der Meer scan for CMS is foreseen on Tuesday morning
    • CERN / central services and T0
      • NTR
    • Tier-1/2:
      • PIC recovered almost completely, some RUN where not transferred from T0 to PIC due to a problem which was fixed in the morning
      • GGUS:83486 (FTS delegation problem): currently no problems but keeping here until sw is fixed
      • GGUS:84229 : CMSSW_5_3_2_patch4 missing at PIC. Will be installed by SW deployment team ASAP
      • T2_DE_DESY had a power cut today. Site is recovering, GRID services may be affected until tomorrow
    • Other:
      • NTR

  • LHCb reports -
    • Users analysis and Reconstruction at at T1s
      • MC production at T2s
    • New GGUS (or RT) tickets

Sites / Services round table:

  • ASGC: ntr
  • BNL: ntr
  • CNAF: ntr
  • FNAL: ntr
  • IN2P3: ntr
  • NDGF: ntr
  • NLT1: In response to to the question GGUS:84223 (ticket solved after the weekend) Roland pointed out this is their service level (weekend on best effort)
  • PIC: CMS_SW ticket: due to the "sgm" worker node misconfiguration. CMS should trigger another sw install. The GGUS:84217 was due to the fact that after the upgrade the system was overloading (regitrations + new transfers). The latter had to be cancelled in order to finish registration (situation recovered on Saturday around noon).
  • RAL: ntr
  • OSG: ntr

  • CASTOR/EOS: ntr
  • Central Services: One CE got /var full. Now it is back in production but the root cause is under investigation. The LHCb ticket is also under investigation
  • Dashboard: ntr

AOB:

Tuesday

Attendance: local(Massimo, JhenWei, Oliver, Guido, Alexandre, Edward, Ulrich, Eva, Maarten);remote(Michael, Saerda, Paolo, Lisa, Tiju, Gonzalo, Jeremy, Rolf, Vladimir, Rob).

Experiments round table:

  • ATLAS reports -
    • CERN CENTRAL SERVICES, T0
      • CERN-PROD: FTS problem GGUS:84154 still open, no major news, not a showstopper
      • atlt3 Castor pool being erased, will be discarded by ATLAS in the next few days * T1
      • PIC downtime finished yesterday
      • NDGF-T1 transfer failures to MCTAPE due to staging problem GGUS:84207 solved (files lost) * CALIB_T2
      • INFN-NAPOLI still in downtime after power cut in the week end

  • CMS reports -
    • LHC machine / CMS detector
      • Van der Meer scan for CMS is now foreseen on Tuesday afternoon/night
      • Tomorrow, Wednesday, back to physics
    • CERN / central services and T0
      • NTR
    • Tier-1/2:
      • KIT: high load situation on Frontier squids, maybe related to large number of running jobs yesterday? Peaked at close to 5k running jobs in parallel.
      • GGUS:83486 (FTS delegation problem): currently no problems but keeping here until sw is fixed
      • T2_DE_DESY had a power cut yesterday. Site is recovering, network and basic services are working again, queues have been opened
    • Other: * NTR

  • LHCb reports -
    • Nothing new to report

Sites / Services round table:

  • ASGC: ntr
  • BNL: ntr
  • CNAF: ntr
  • FNAL: ntr
  • IN2P3: ntr
  • KIT: ntr
  • NDGF: ntr
  • NLT1: ntr
  • PIC: ntr
  • RAL: ntr
  • OSG: ntr

  • CASTOR/EOS: ntr
  • Central Services: ntr
  • Data bases: ntr
  • Dashboard: ntr

AOB:

Wednesday

Attendance: local();remote().

Experiments round table:

Sites / Services round table:

  • ASGC:
  • BNL:
  • CNAF:
  • FNAL:
  • IN2P3:
  • KIT:
  • NDGF:
  • NLT1:
  • PIC:
  • RAL:
  • OSG:

  • CASTOR/EOS:
  • Central Services:
  • Data bases:
  • Dashboard:

AOB:

Thursday

Attendance: local();remote().

Experiments round table:

Sites / Services round table:

  • ASGC:
  • BNL:
  • CNAF:
  • FNAL:
  • IN2P3:
  • KIT:
  • NDGF:
  • NLT1:
  • PIC:
  • RAL:
  • OSG:

  • CASTOR/EOS:
  • Central Services:
  • Data bases:
  • Dashboard:

AOB:

Friday

Attendance: local();remote().

Experiments round table:

Sites / Services round table:

  • ASGC:
  • BNL:
  • CNAF:
  • FNAL:
  • IN2P3:
  • KIT:
  • NDGF:
  • NLT1:
  • PIC:
  • RAL:
  • OSG:

  • CASTOR/EOS:
  • Central Services:
  • Data bases:
  • Dashboard:

AOB:

-- JamieShiers - 09-Jul-2012

Edit | Attach | Watch | Print version | History: r11 | r5 < r4 < r3 < r2 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r3 - 2012-07-17 - MassimoLamanna
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback