Week of 091221

LHC Operations

• Sharepoint site - Cooldown Status - News

WLCG Service Incidents, Interventions and Availability

VO Summaries of Site Availability				SIRs & Broadcasts
ALICE	ATLAS	CMS	LHCb	WLCG Service Incident Reports	Broadcast archive

GGUS section

• GGUS Escalation reports every Monday (used for WLCG Service Report to MB)
  • Full search: https://gus.fzk.de/ws/ticket_search.php and select "VO" - this will give all tickets, including team & alarm
• LHCOPN Tickets in GGUS: see https://gus.fzk.de/pages/all_lhcopn.php and change your selection criteria. Future actions are also listed.
  • If network group participation is necessary, please invite them in time.
• OSG items selected from the GGUS escalation reports.
• Procedure to become a LHC Experiment VO TEAM member
• Other recent GGUS FAQs

Daily WLCG Operations Call details

To join the call, at 15.00 CE(S)T Monday to Friday inclusive (in CERN 513 R-068) do one of the following:

1. Dial +41227676000 (Main) and enter access code 0119168, or
2. To have the system call you, click here
3. The scod rota for the next few weeks is at ScodRota

General Information
CERN IT status board	M/W PPSCoordinationWorkLog	WLCG Baseline Versions	Weekly joint operations meeting minutes

Additional Material:

STEP09	ATLAS	ATLAS logbook	CMS WLCG Blogs

Monday:

Attendance: local();remote().

Experiments round table:

• ATLAS -

• CMS reports -

• ALICE -

• LHCb reports -

Sites / Services round table:

• NL-T1: Xen issue at Nikhef: Xen virtual machines cannot be restarted since a recent update of the Xen rpms. The solution appears to be a reboot of the Xen host. We are doing that this afternoon to prevent such situations happening over the Christmas break. All grid services will briefly be unavailable between 14:30 en 15:30 CET.

AOB:

Tuesday:

Attendance: local();remote().

Experiments round table:

• ATLAS -

• CMS reports -

• ALICE -

• LHCb reports -

Sites / Services round table:

• NL-T1: SARA SRM was stuck. Restarted and then OK. Probably PNFS had crashed. Migration planned from PNFS to Chimera in the week of January 11, so we don't expect this to happen again.

• ES-PIC: There was a cooling incident last Saturday 19th Dec at PIC. We had to shutdown the services and start them back up when the temperature problem had been solved. Overall around 4 hours of outage for most of the tier-1 services. A SIR is being attached to the WLCGServiceIncidents page

AOB:

Wednesday

Attendance: local();remote().

Experiments round table:

• ATLAS -

• CMS reports -

• ALICE -

• LHCb reports -

Sites / Services round table:

• ES-PIC: After the site cooling problem on Saturday 19th December the streams replication of the LHCb LFC does not happen in real-time for PIC. This is due to a configuration of the streams CERN-PIC for this service which needs to be corrected by CERN 3D experts. We agreed with them to wait until 4-Jan-2010 to do this intervention. Until then, the service is running ok but the DB is replicated from CERN with some delay.

Release report: deployment status wiki page

AOB:

Thursday

Attendance: local();remote().

Experiments round table:

• ATLAS -

• CMS reports -

• ALICE -

• LHCb reports -

Sites / Services round table:

AOB:

Friday

Attendance: local();remote().

Experiments round table:

• ATLAS -

• CMS reports -

• ALICE -

• LHCb reports -

Sites / Services round table:

AOB:

-- JamieShiers - 15-Dec-2009