Week of 110509

Daily WLCG Operations Call details

To join the call, at 15.00 CE(S)T Monday to Friday inclusive (in CERN 513 R-068) do one of the following:

  1. Dial +41227676000 (Main) and enter access code 0119168, or
  2. To have the system call you, click here
  3. The scod rota for the next few weeks is at ScodRota

WLCG Service Incidents, Interventions and Availability, Change / Risk Assessments

VO Summaries of Site Usability SIRs, Open Issues & Broadcasts Change assessments
ALICE ATLAS CMS LHCb WLCG Service Incident Reports WLCG Service Open Issues Broadcast archive CASTOR Change Assessments

General Information

General Information GGUS Information LHC Machine Information
CERN IT status board M/W PPSCoordinationWorkLog WLCG Baseline Versions WLCG Blogs   GgusInformation Sharepoint site - Cooldown Status - News


Monday:

Attendance: local(Massimo, Eva, Nilo, Jamie, Maria, Jarka, Ueda, Felix, Gavin, Mattia, Dirk, Maarten);remote(Jon, Michael, Joel, Ulf, Rolf, Chiara, Tiju, Rob, Andrea, Ron, Dimitri, Gonzalo, Ian).

Experiments round table:

  • ATLAS reports -
  • Report to WLCGOperationsMeetings
    • Information :
      • LHC : Technical Stop
      • ATLAS : No data taking, no data export
    • Issues :
      • RAL reported disk crash (ELOG:25117) May 06, 2011, 18:06 +0200
        • This machine was put back in production on Saturday(7th May) afternoon (Mon, 9 May 2011 10:54:17 +0100)
        • ATLAS has not been affected much
      • SARA, failed to contact SRM srm (elog:25152,GGUS:70360) 2011-05-08 04:00 UTC
        • One of the dCache head nodes had a full /var file system (2011-05-09 07:49)
        • The site was excluded from ATLAS production and DDM transfers on Sunday, and being put back today. Transfers going fine.
      • TW slow transfers : the site reported 10Gb link is back
        • ATLAS is watching the transfers

  • CMS reports -
  • LHC / CMS detector
    • Technical stop

  • CERN / central services
    • NTR
  • Tier-0 / CAF
    • Replays of Tier-0 workflows have begun. Trying to find a way to test the new code under realistic conditions before we have new data.
  • Tier-1
    • 2010 re-reco announced. Skims had a configuration issue and are being resubmitted
    • Problems over the weekend with pile-up production traced to large prestaging requested by all CMS application for the pile-up sample. Problematic jobs are killed and a patch is in preparation. Resubmitting workflows tomorrow.
      • Running jobs at CNAF will be allowed to finish.
  • Tier-2
    • MC production and analysis in progress (summer11 production)

New CRC tomorrow: Oliver Gutsche


  • ALICE reports -
    • T0 site
      • User activity backlog drained over the weekend. Users still have priority over production.
    • T1 sites
      • KIT: cream-5-kit.gridka.de failed during the weekend and was taken out of the VOBOX job submission configuration.
    • T2 sites
      • Usual operations


Experiment activities:

  • Technical stop : no data taking
  • MC productions on most T1/T2 sites

New GGUS (or RT) tickets:

  • T0: 1
  • T1: 0
  • T2: 0

Issues at the sites and services

  • T0
  • T1
    • SARA problem with aborted pilots for reconstruction jobs (GGUS:70170). [ Ron - still looking at this issue. 1-2 weeks ago implemented memory limits on batch jobs to 4GB / job slot. Same limit at NIKHEF yet LHCb has problem at SARA but not NIKHEF. Still looking into it and contacting NIKHEF colleagues
  • T2
  • AOB - GOCDB at risk. Did not receive e-mail directly. Is it normal? Rolf - don't know what really happened but if this is a GOCDB downtime announcement you might need to subscribe to it.

De : EGI BROADCAST <cic-information@in2p3.fr>
Date : 9 mai 2011 12:43:24 HAEC
 : NGI managers/global_email <noc-managers@mailman.egi.eu>
Objet : [Noc-managers] [EGI BROADCAST] GOCDB At Risk Period


---------------------------------------------------------------------------------------------------------------
EGI BROADCAST TOOL : https://operations-portal.egi.eu/broadcast

---------------------------------------------------------------------------------------------------------------
Publication from : john casson <John.Casson@stfc.ac.uk>
Targets : NGI managers/global_email <noc-managers@mailman.egi.eu>
----------------------------------------------------------------------------------------------------------------



GOCDB should be considered at risk from 08:00 - 09:00 UTC on 10/5/2011. This 
is due to security updates being applied to our back end database.

https://goc.gridops.org/portal/index.php?Page_Type=View_Object&object_id=27615&grid_id=0

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------
link to this broadcast : 
https://operations-portal.egi.eu/broadcast/archive/id/380
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------

_______________________________________________
Noc-managers mailing list
Noc-managers@mailman.egi.eu
https://mailman.egi.eu/mailman/listinfo/noc-managers

Sites / Services round table:

  • FNAL - ntr
  • BNL - ntr
  • NDGF - in two weeks we will probably have some severe network problems - some fibres will be moved, 12h downtime but don't know more at the moment. Probably affect some pools in Denmark
  • IN2P3 - for info we got back machines hosted for several months near Montpellier. Now installed in new machine room. If ever a new power cut will now loose all WNs (and not only 85%). Now can schedule all jobs of all LHC expts to all WNs. The distance to Montpellier implied network latencies so decided not to schedule ATLAS or LHCb jobs there because of I/O problems. Now back to 100 %
  • RAL - tomorrow will upgrade SRM for ALICE, Will also include drain of FTS channels.
  • CNAF - ntr
  • NL-T1 - problem with SRM which started Sunday night. Fixed this morning. Also a problem with a CREAM CE - one service crashed so m/c rebooted. Also issues of LHCb jobs as above.
  • KIT - one issue with CMS dCache instance currently under investigation. Not really clear why it crashed at w/e. A ticket about this.
  • PIC - tomorrow morning we will upgrade firmware in some core switches. Will try and make it transparent by stopping running jobs. Scheduled < 4h but if all goes well should be much faster.
  • ASGC - 10Gbit link should be back as from Sunday. Configured BDII for DPM and put into Taiwan LCG2. Now all transfers from FTS will go to Taiwan LCG2 channel. Ueda- checksum schedule? Should be available.
  • OSG - ntr

  • CERN VOMS Last reminder, CERN VOMS services will be unavailable tomorrow morning from 09:00 CEST for couple of hours. Any updates during the intervention will be posted to http://cern.ch/ssb and at this meeting tomorrow. Details: VomsInterventions

  • CERN DB - we had a problem with COMPASS DB yesterday night. Caused by high load of applications on DB following request from them to increase # sessions. Being followed up. Tomorrow upgrade of LCGR (LHC integration DB) to 11g. Problems of patching of production DB.
  • CERN Storage - reminder of intervention tomorrow for CASTOR n/s at same time as VOMS. Meant to be transparent.

AOB:

Tuesday:

Attendance: local(Felix, Jarka, Maria, Jamie, Maarten, Miguel, Ollie, Michal, Gavin);remote(Gonzalo, Ulf, Joel, Michael, Jon, Xavier, Tiju, Rob, Rolf, Paolo, JT).

Experiments round table:

  • ATLAS reports -
    • RAL: LFC/FTS Oracle patching "at risk" was not transparent, ALARM GGUS:70435 created. Now fixed. SIR requested
    • Network link (10 Gbps) between CERN and TAIWAN-LCG2 is back. GGUS:70228 proposed to be closed.
    • CERN had series of transparent DB interventions today. Intervention is over, everything seems to be working fine. Thank you!
    • CERN CASTOR namespace server upgraded successfully. Thank you!


  • CMS reports -
  • LHC / CMS detector
    • Technical stop
  • CERN / central services
    • During today VOMS outage some users got upset! Was fully announced and visible on SSB etc.
  • Tier-0 / CAF
    • Testing new software release at high scale (filling the whole T0 farm) with replays of older runs. Since 08:30 not able to get any jobs through as cannot read jobs with xrootd. Opened an ALARM ticket GGUS:70434 . Ticket still open - doesn't seem to be reproducible using xrdcp - maybe higher up in the stack.
  • Tier-1
    • 2010 data and skims in tails and should be cleared out of the system soon.
    • First pass of pileup MC reprocessing at Tier-1 sites canceled except CNAF (StoRM/GPFS stayed up!). Jobs have been removed from sites and sites should have recovered. For today, expecting new workflows. Will test first at single site (ramping up to use all cores, chose FNAL because it is the largest)
    • Also expected 2011 data re-reconstruction pass to start, pre-staging should start today (most of the data should be on disk anyway, but pre-staging will be triggered nevertheless)
  • Tier-2
    • MC production and analysis in progress (summer11 production)

  • ALICE reports -
    • T0 site
      • Activity drained to very low level. Users still have priority over production.
    • T1 sites
      • KIT: yesterday evening a user reported a problem with accessing files on the xrootd SE; seems to have been a temporary issue.
      • RAL: VOBOX ran out of space on the ALICE SW area; OK again after a big cleanup.
    • T2 sites
      • Usual operations

Experiment activities:

  • Technical stop : no data taking
  • MC productions on most T1/T2 sites

New GGUS (or RT) tickets:

  • T0: 1
  • T1: 0
  • T2: 0

Issues at the sites and services

  • T0
    • VOMS intervention
    • Replication stream : Oracle intervention
  • T1
    • NIKHEF jobs kill by memory. Could we increase the size limit to 5Gb ?. Problem in our code - we are working on it but a temp increase could help us to run at this site which is the only one at which we can't run at today. JT - code is increasing memory use with each event. Limit was set at twice request in VOID card. We'll investigate to see how far we can push it up before we run into problems but clearly this is more than what machines are dimensioned for. Joel - clear. Maarten - might be that most of the memory would get paged out so if swap is big enough would hardly notice.
    • SARA : SRM intervention
    • PIC : interventions in network equipment and firmware updates
  • T2
  • AOB - GOCDB announce will be raised with EGI

Sites / Services round table:

  • PIC - ntr; intervention on network elements was fine
  • NDGF - ntr
  • BNL - currently a network intervention in progress, upgrading code on various swiches and integrating new 80Gbit switch block which requires switch reboots and short interruptions with which applications can cope - hence transparent
  • FNAL - had trouble again with CVMFS which required WN reboot to clear. Close to point to backing this product out and reverting back to previous way of distributing code. Previously system was absolutely reliable and if I now have to be dependent on developers seems a step back. Maarten - you might want to report to the developers. Jon - don't have a clue of what issue is. Steve - but they may have some advice. Maarten - good that they know that a big site has serious problems.
  • KIT - maintenance 10:00 - 12:00 tomorrow on 1 tape library. Tapes not available for reading but writing will use other libs.
  • RAL - intervention to patch Oracle DBs. Some issues on DB after patching - something wrong with ACLs so had to remove them. LFC and FTS were not available for about 1 h during AT RISK
  • IN2P3 - announcement: in switch to new batch system new CREAM CE will be able tomorrow for OPS. Thursday latest also LHC. Those using WMS will be transparent for others actions will be needed to integrate this. Additional CEs for GridEngine will be made available depending on how things go
  • CNAF - ntr
  • NL-T1 - intervention that took a little longer than planned. Some dark data. Problem with farm - ATLAS had been limited to small piece of farm but can now use all WNs.
  • OSG - ntr

  • CERN VOMS - upgrade went ok in terms of migrating to new h/w, o/s etc. A bit of instability in VOMS - VOMS-proxy-init fails sometimes for CMS and ALICE. Some counter-measures until we fix. Continue to investigate - not a roll-back situation yet. Maarten - first VOMS-proxy-init fails with a DB error but this fails over automatically and you still get proxy. Joel - next time try to avoid to send fake mail to user. Subject was "test ignore" but in middle of real subject which confused users! Steve - will change mail now and reminders will be ok.

  • CERN storage - DB group will continue to patch CASTOR DBs in next 2 days. Tomorrow CASTOR ATLAS and CASTOR CERNT3. Thursday CASTOR ALICE, CMS and LHCb. For stagers and SRMs.

AOB:

Wednesday

Attendance: local(Steve, Jarka, Jamie, Michal, Chiara, Alessandro, Massimo, Luca, Gavin, MariaDZ, Pedro, Lola, Edoardo);remote(Michael, Jon, Oliver, Rolf, Ulf, Gonzalo, Onno, Tiju).

Experiments round table:

  • ATLAS reports -
    • CERN: Massive failures at Tier-0 due to lost AFS token on ATLAS Tier-0 LSF batch nodes, ALARM GGUS:70450 ticket. Fixed for now. Can Gavin present follow-up, please? [ Gav - preparing a SIR with CASTOR and Kerberos team ]
    • SARA-MATRIX: yesterday there was a scheduled downtime of SRM. During this downtime a GGUS:70442 was filed. After the SD site was performing well.
    • SARA-MATRIX: Locality unavailable errors, GGUS:70469 opened.
    • FZK-LCG2: Stager issues on FZK-LCG2_MCTAPE. GGUS:70471 ticket in progress.
    • PIC: Failed to contact SRM errors, GGUS:70470 opened.

  • CMS reports -
  • LHC / CMS detector
    • Technical stop
  • CERN / central services
    • NTR
  • Tier-0 / CAF
    • GGUS Alarm ticket 70434 about not being able to read from xrootd was traced back to LSF that didn't foreard anymore afs tokens, under investigation [ Massimo - small clarification: two distinct tickets, one with xrootd access in title which we picked up. Other is probably correlated but slightly different. One of calls to start authenticated jobs probably has a bug. Probably just one report. ] Ollie - running with work-around but will backout workaround tonight as apparently not needed any more. ]
    • Continuing testing program
  • Tier-1
    • Summer11 MC processing started again on all T1 sites
    • Also expected 2011 data re-reconstruction pass to start, pre-staging should start today (most of the data should be on disk anyway, but pre-staging will be triggered nevertheless)
  • Tier-2
    • MC production and analysis in progress (summer11 production)


  • ALICE reports -
    • General Information: Users still have priority over production until 23rd of May.
    • T0 site
      • Nothing to report
    • T1 sites
      • FZK: all CREAM-CEs back in production
      • NIKHEF: Cluster Monitor stopped running yesterday evening, restarted and jobs already running at the site
    • T2 sites
      • Usual operations

Experiment activities:

  • Technical stop : no data taking
  • GGUS ticket against GGUS because we were not able to submit TEAM ticket (GGUS:70459 fixed )

New GGUS (or RT) tickets:

  • T0: 1
  • T1: 0
  • T2: 0

Issues at the sites and services

  • T0
  • T1
    • NIKHEF jobs kill by memory. Memory will increase to 5Gb
    • IN2P3 : Pilot jobs aborted during the nigth but now back to normal.
    • RAL : diskserver migration on going
    • CERN : diskserver migration on going
  • T2 *


Sites / Services round table:

  • BNL - intervention on network mentioned yesterday completed successfully for no impact on applications
  • FNAL - ntr
  • IN2P3 - ntr
  • NDGF - we will have network connection from NDGF - CERN cut on 17 May at 22:00 UTC and will be down for 6h but backup link through SARA should be up. Reduced redundancy for that time and at risk in GOCDB.
  • PIC - report that as ATLAS said we have currently an issue with SRM - currently failing and experts looking into it - overload. Not clear where from,
  • NL-T1 - as ATLAS reported we have issue on SRM - some ATLAS disk files currently unavallable GGUS:70469. Dark data cleanup on ATLAS disk pools. in dCache cleaner service is supposed to cleanup deleted files but problems. We need to cleanup dark data manually - dCache procedure to do this is pnfs unregister command followed by register for each pool separately to make each pool cross-check with namespace. dCache team working on a fix so that this procedure won't be needed in the future. After ATLAS ticket entered warning in GOCDB. Hope that when cleanup over files will be accessible again. 165TB of dark data cleaned up! [ Jarka - how long do you expect manual deletion will ask so we can ask shifters not to file another ticket? A: started first pools early this morning and they have already finished ~2h ago. Probably during afternoon most pools will finish. GOCDB downtime until 12:00 tomorrow. ]
  • RAL - ntr
  • CNAF - ntr
  • ASGC - ntr
  • OSG - ntr

  • CERN DB - we are patching (rolling) WLCG DB at the moment will finish in 1h as announced.

  • CERN Grid services - VOMS: potential problem with ALICE and CMS - still happening but being avoided with cron jobs. Oracle error message being followed up. After yesterday's meeting email thread: all ALICE, ATLAS, LHCb corrected previous incorrect emails about expiry. Sorry - message on SSB:
Update at 9:00 am 11th May:
All voms services were restored around 14:00 on May 10th. Unfortunately a large number of incorrect email notifications were sent to members of LHCb, ATLAS and ALICE with a false expiry date in one weeks time. Subsequent notifications have now been sent with the correct date for each member.

  • CERN network - intervention tomorrow in one of routers in front of CERN firewall. Traffic to sites other than T1s may be affected. Replace a router. When we switch off old one might be a short glitch.

AOB:

Thursday

Attendance: local(Jarka, Jamie, Maarten, Ollie, Michal, Eva, Simone, Alessandro, Gavin);remote(Ulf, Michael, Dimitri, Weijen, Jon, Rolf, Jeremy, John, Todd, Joel, Gonzalo, Foued, Rob, Chiara).

Experiments round table:

  • ATLAS reports -
  • PIC: Failed to contact SRM errors, GGUS:70470 in progress. ATLAS DDM ops investigating.
  • NDGF-T1: SRMV2STAGER error "Not enough space" on stager machine, GGUS:70495 in progress.
  • RAL: 1 disk server (gdss212) lost in ATLASSCRATCHDISK. Any news from the site?
  • SARA-MATRIX: Locality unavailable errors GGUS:70469 due to cleanup of dark data -- fixed. Thank you!
  • INFN-T1: No space left on ATLASDATATAPE. GGUS:70482, site deployed another 80 TB of disk space to disk buffer. Thank you!
  • TRIUMF: Yesterday LFC downtime, cloud was set offline for ATLAS activities in advance, everything was smooth.
  • New ATLAS Dataset Project Tags:
    • mc11_7TeV
    • mc11_valid
  • CERN SNOW: In case of ticket RQF0008106 originator of the ticket has not been notified by email about the resolution of the request. Can CERN SNOW please investigate?


  • CMS reports -
  • LHC / CMS detector
    • Technical stop
  • CERN / central services
    • NTR
  • Tier-0 / CAF
    • Continuing testing program
  • Tier-1
    • lots of activity, MC and 2011 data re-reconstruction ongoing
  • Tier-2
    • MC production and analysis in progress (summer11 production)


  • ALICE reports - ntr; everything smooth and activity very low

Experiment activities:

  • Technical stop : no data taking

New GGUS (or RT) tickets:

  • T0: 1
  • T1: 0
  • T2: 0

Issues at the sites and services

  • T0
  • T1
    • RAL : diskserver migration finished
    • CERN : diskserver migration on going
  • T2 *

Sites / Services round table:

  • NDGF On 26.5 there will be a total outage of the Danish resources. All DCSC/KU pools under srm.ndgf.org will be out for up to 12h 26.5 19:00 - 27.5 07:00 UTC. This will also offline all Danish compute nodes. A minor upgrade will be made to srm.ndgf.org on monday, it is filed as AT_RISK in GOCDB.

  • BNL -
  • KIT - ntr
  • ASGC - incident report for network cut has been uploaded to WLCG SIR page. https://twiki.cern.ch/twiki/pub/LCG/WLCGServiceIncidents/20110501_SIR_ASGC_10GbLINKDOWN.pdf
  • FNAL - ntr
  • IN2P3 - have to postpone the CREAM CE for GridEngine as we discovered as last minute bug in interface. There is already a fix but not tested. Will give news when sure fix is valid
  • RAL - ATLAS q about m/c (gdss212) failed last night, failed RAID card, replaced and running tests. Informed local rep and if things go well expect back today
  • PIC - issue with SRM overload seems fixed. Cause traced to wrong parameter in bringonlines issues from ATLAS VO box. Message from Cederic in VO box around noon when fixed. Since then things look fine. [ SImone - please do some further invesigation - this SRM timeout parameter was too quick (60") but we don't see how this resulted in overload. ] ok - will keep investigating
  • CNAF - ntr
  • GridPP - ntr
  • OSG - ntr

  • CERN BDII: Top Level and SAM BDIIs will be upgraded to the latest version early next week (Mon-Tue), test nodes are available (bdii206 (top) and bdii209 (sam)). The SAM BDII upgrade is pending agreement from the SAM team. The Site BDII will also be upgraded on Wednesday to the latest version, a test node available (bdii-test). [ Maarten - I am checking it and see some non-trivial differences between current production and this test node. ] [ Alessandro - from ATLAS point of view, SAM BDII no longer critical and hence OSG can remove checks on this. Rob - we haven't done this yet but expect to in next few weeks ]

  • CERN: The current test lcg-CE that submits to the "whole nodes" test resources needs to be retired (out of warranty). Given the very limited use of this resource (tests seem to have stopped now), we'd like to know the plans of the experiments in this regard - should this facility be replaced? Should the batch resources behind it be increased? Alessandro - yes, we are using the q!

  • CERN DB - all production DBs have been patched with latest security and recommended patches. COMPASS running back on the 3 nodes again.

  • CERN Dashboard - there will be an upgrade of ATLAS DDM dashboard on Friday 13 May and 10:00 CERN time and will take approx 40'

AOB: (MariaDZ) The problem to submit GGUS TEAM tickets reported by LHCb yesterday in GGUS:70459 is related to a change of the voms-admin interface following the recent upgrade. VOMS and GGUS developers are working on adapting the TEAM and ALARM members' extract script. Savannah:120905

  • CVMFS developers are interested in learning of FNAL problems and assisting with resolution

Friday

Attendance: local(Riccardo, Jarka, Jamie, Maarten, Stefan, Michal, Eva, Alessandro, Simone, Miguel);remote(Jon, Xavier, Gonzalo, Joel, Onno, Jhen-Wei, Rolf, Rob, Gareth, Chiara).

Experiments round table:

  • ATLAS reports -
    • RAL: 1 disk server (gdss212) of ATLASSCRATCHDISK is back. Thank you!
    • NDGF-T1: stager issues continue GGUS:70511, issue to read files: Looking closer the two cases might be related, symptoms of something I just found now, one of the stagers is having issues delivering files to the read pools. Seems to be a networking problem which has given rise to very non-obvious error messages. Not all of the attempts are failures though, so repeated tries will probably eventually get the files properly online. Resolution by pool operators will probably not be done until tomorrow. But at least this was an error message that pointed us in the right direction (rather than the very specific stager script error which was somewhat of a red herring).
    • DDM dashboard upgraded today. Thank you! Will there be another upgrade on Monday? A - no this was today. Mistake.

  • CMS reports -
  • LHC / CMS detector
    • Recovery from technical stop
  • CERN / central services
    • BDII update next week: in general CMS is happy about the update but would be interested in details of the discrepancies seen by Maarten reported yesterday [ Maarten - still have to continue investigation but expectation is that things will be fine - other sites are running this version and have not complained but would still like to understand the issues. Riccardo - one of reasons for upgrading is that new BDII is stricter. Some sites have not upgraded pending CERN. ]
  • Tier-0 / CAF
    • data taking resumes
    • Saturday VDM scans during 12 hour period, expect 2 kHz rate for 3 hours, empty events so short reco times
  • Tier-1
    • lots of activity, MC and 2011 data re-reconstruction ongoing
  • Tier-2
    • MC production and analysis in progress (summer11 production)


  • ALICE reports - NTR; low activity in preparation for quark matter conference in a week for now. Users still have priority for last minute checks - up to conference so all of next week. Some production but low priority background activity. After conference backlog for pp reconstruction will be handled.

  • LHCb reports - Technical stop : no data taking
    • T1
      • SARA : where are we with GGUS:70170 [ GGUS ticket has been updated - memory increased and restarted activity]
      • KIT - ticket about CREAM2 which was down and brought back around 11:00

Sites / Services round table:

  • FNAL - ntr
  • KIT - due to GPFS problems 3 of CMS diskonly pools down overnight. Some files lost due to corruption in f/s. Running complete check - will inform CMS Germany about lost files. Ollie - only user files? Not sure...
  • PIC - ntr
  • NL-T1 - ntr
  • ASGC - ntr
  • IN2P3 - ntr
  • RAL - nta
  • CNAF - ntr
  • BNL - ntr
  • OSG - would like to understand BDII upgrade. Early in week SAM BDIIs will be updated. Will top level BDIIs then follow? Riccardo - yes, this is the idea. SAM Monday, top level Tuesday and site Wed. Any issues expected wrt OSG? Please check - we have a test m/c (BDII206) please connect directly to that which has the new version running. Ticket exchange with GGUS and upgrade on 25th - exchange will be broken until OSG puts in a patch to allow OSG to communicate with new version. All tickets should make there way after patch is applied. Will check any tickets that had activity that night to check all is ok. Will still getting paging and if there is an alarm in this period we will have to go to GGUS to check.

AOB:

-- JamieShiers - 04-May-2011

Edit | Attach | Watch | Print version | History: r21 < r20 < r19 < r18 < r17 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r21 - 2011-05-13 - JamieShiers
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback