WLCG Experiment and Site plans for the end of 2009 holidays

These are as reported to the WLCG Service Coordinator on Duty




  • ALICE does not require any additional support from that provided during the last Christmas time.
  • Due to the reinforcement of the gLite WMS as service it is expected to be a more stable period while using this service this time.
  • ALICE is expecting to perform reconstruction operations of the real data which are being recorded these days using both T0 and T1 sites.
  • Eventually, if this is required by the physics groups, new bunches of MC productions might be expected during this time, but this still needs to be confirmed. In any case this was the operation performed during the last Christmas period and should not change the support requirements at all.



A detailed planning meeting was held on Dec 10 - the slides are at http://indico.cern.ch/conferenceDisplay.py?confId=76171 There will be a 'Fast' (reduced validation - to be ready by 22 December) reprocessing of recent 2009 good runs (between 30 to 150 million events) spread over Tier-0 and Tier-1 that should be finished by 1 January. There will be a limited amount of Monte-Carlo production starting now with reconstruction after the 22 December validation. Extensive user analysis activity is expected at all Tier levels. An executive summary (shown below) was given by K.Bos:

The run will end on Wednesday and we will prepare for reprocessing all good data as soon as possible afterwards. We would like to do this reprocessing with the same software release that we will also use to reconstruct the corresponding sample of simulated data at 900 GeV. The preparation and validation of this release including the right calibration is a real challenge. But ,if all goes well, we could start the reprocessing even before Christmas, to be finished still this year. For this re-processing we would like to use all Tier-1's but we have nearly 4 copies of the data on the grid on disk so we have some redundancy. Depending on the results of this analysis we are already preparing for a possible second re-processing in January. The results should be ready for publication in February and to be presented at the spring conferences.



Operation with beams will most probably end on Dec 16th. Decisions are being taken as of opening the detector or not and when. Ops teams are supposed to get new conditions on Dec 17th, and then get a request for rereco with CMSSW 33X and 34X for the Collisions09 runs list. To prepare that, we are pushing to deploy CMSSW out asap, and also some pre-releases to check how they run on SLC5 (please note that 34X will be the first CMSSW release to be SLC5 only). We are following up on the sites status as usual, after a busy and productive CMS week. Over Xmas - provided we can get the rereco's through before - we do not expect anything special, and people are encouraged to take a (deserved) break.



LHCb do not anticipate massive re-processings during the Xmas break and January. We have no particular MC requests pending. Even if some may come up during the week, but unlikely to fill our share during 2 weeks. Analysis will go on (MC09 and real data) at Tier1s.



We don't have intervention planning except for the final delivery of 2009 pledged resources (storage). the acceptance will be complete in one week and setup/deployment will be done in two weeks, till end of the year. fyi


At BNL normal operations of all services will be maintained over the period from 12/24/2009 until 01/04/2010.


Most of the services provided by the IT Department - including WLCG (EGEE) production services - will remain available during the CERN annual closure. No interruptions are scheduled but in the event of a failure, the restoration of services cannot be guaranteed.

Problems will be dealt with on a best effort basis only. However, for the following services:

Network, Databases, Mail, Web, Windows & Terminal Services, Authentication Services, Indico, CDS, Grid (SAM, FTS, VOMS, GridView, LFC (new)): experts should be reachable to start investigations within about half a day except on Christmas Eve and Christmas Day (24 and 25 December) and New Year’s Eve and New Year’s Day (31 December and 1 January). Incidents will be documented at http://cern.ch/ssb.

The backup service will remain operational, but backups cannot be guaranteed and file restores may not be possible.

Castor: damaged tapes will not be processed.

Please note that the Helpdesk will be closed but an operator service will be maintained throughout and can be reached on telephone: 75011 or email: computer.operations@cernNOSPAMPLEASE.ch in the event of urgent problems.

Potential computer security incidents should be reported to Computer.Security@cernNOSPAMPLEASE.ch or 70500 as usual.


INFN T1 will be operational with reduced staff between December 24 and January 6. The festive days, besides Saturdays and Sundays, are: December 25,26, January 1,6. The on-call service will be active as usual to cover any emergency that could arise


From December, 24th, to January 3rd, nearly normal operations, except that overall staff coverage will be comparable to the lowest levels during the summer vacation period, which means that expert interventions might take longer than usual, especially on week ends. Holidays: December 25th, January 1st.


During the X-Mas and New Year holiday period, we run essentially in the same "mode" than during weekends and other public holidays. We have on-call engineers (OCEs) on duty who will react on Nagios alarms and alarm tickets. Problems that can not be resolved by the OCEs themselves will be dealt with on a best effort basis. Experts for all systems and services can be reached by the OCEs. However, the short-term availability of experts is not guaranteed, especially on certain days like Dec 24+25 and Dec 31+Jan 1.


Until 24/12 normal operations. The Dutch Tier-1 will provide Best Effort service from Thursday 24/12/2009 18:00 until Monday 4/1/2010 9:00. Grid services and storage services at the SARA site of the NL-T1 will be done by a minimal number of admins onsite on 28/12, 29/12 and 30/12 and 31/12 (until 15:00). Due to limited staffing, response times and the solving of problems might take longer than usual.


During the period 24-Dec until 7-Jan there will be normal operations at PIC. The holiday days will be 25-Dec, 1-Jan and 6-Jan. During those days the coverage will be the same as during weekends: there is always a person (Manager on Duty) in charge of monitoring the site and that receives critical alarm notifications to a mobile phone. The level of staff on holiday during this period will be higher than average (comparable to mid August) but there will always be a reachable expert for each of the critical services.


RAL will be closed from Friday 25th December and re-open on Monday 4th January. During this time we plan for services at the RAL Tier1 to remain up. The usual on-call cover will be in place (as per nights and weekends). This cover will be enhanced by daily checks of key systems. Some hardware interventions, such as to swap out faulty disks will also take place over this time. However, we have relaxed our expectation that the on-call person will respond within two hours, particularly on 25/26 December and 1st January.

During the holiday will check for tickets in the usual manner. However, only service critical issues will be dealt with.


TRIUMF laboratory is closed from December 25 and re-opens on January 4th, however Tier-1 operations will continue and services will remain monitored. The usual on-call coverage will still be in place and we will respond to problems within a reasonable time.

Topic attachments
I Attachment History Action Size Date Who Comment
JPEGjpeg alice.jpeg r1 manage 3.8 K 2009-12-16 - 12:40 JamieShiers  
JPEGjpeg atlas.jpeg r1 manage 3.9 K 2009-12-16 - 12:40 JamieShiers  
PNGpng cms.png r1 manage 9.2 K 2009-12-16 - 12:41 JamieShiers  
GIFgif lhcblogo.gif r1 manage 8.3 K 2009-12-16 - 12:41 JamieShiers  
Edit | Attach | Watch | Print version | History: r15 < r14 < r13 < r12 < r11 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r15 - 2009-12-17 - JamieShiers
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback