Week of 091221
LHC Operations
WLCG Service Incidents, Interventions and Availability
GGUS section
Daily WLCG Operations Call details
To join the call, at 15.00 CE(S)T Monday to Friday inclusive (in CERN 513 R-068) do one of the following:
- Dial +41227676000 (Main) and enter access code 0119168, or
- To have the system call you, click here
- The scod rota for the next few weeks is at ScodRota
Additional Material:
Monday:
Attendance: local();remote().
Experiments round table:
Sites / Services round table:
- NL-T1: Xen issue at Nikhef: Xen virtual machines cannot be restarted since a recent update of the Xen rpms. The solution appears to be a reboot of the Xen host. We are doing that this afternoon to prevent such situations happening over the Christmas break. All grid services will briefly be unavailable between 14:30 en 15:30 CET.
AOB:
Tuesday:
Attendance: local();remote().
Experiments round table:
Sites / Services round table:
- NL-T1: SARA SRM was stuck. Restarted and then OK. Probably PNFS had crashed. Migration planned from PNFS to Chimera in the week of January 11, so we don't expect this to happen again.
- ES-PIC: There was a cooling incident last Saturday 19th Dec at PIC. We had to shutdown the services and start them back up when the temperature problem had been solved. Overall around 4 hours of outage for most of the tier-1 services. A SIR is being attached to the WLCGServiceIncidents page
AOB:
Wednesday
Attendance: local();remote().
Experiments round table:
Sites / Services round table:
- ES-PIC: After the site cooling problem on Saturday 19th December the streams replication of the LHCb LFC does not happen in real-time for PIC. This is due to a configuration of the streams CERN-PIC for this service which needs to be corrected by CERN 3D experts. We agreed with them to wait until 4-Jan-2010 to do this intervention. Until then, the service is running ok but the DB is replicated from CERN with some delay.
Release report:
deployment status wiki page
AOB:
Thursday
Attendance: local();remote().
Experiments round table:
Sites / Services round table:
AOB:
Friday
Attendance: local();remote().
Experiments round table:
Sites / Services round table:
AOB:
--
JamieShiers - 15-Dec-2009