LHC VO User Support

Maria Dimou (CERN/IT/GD/OPS) since March 1st 2007 is the contact person on Grid User Support questions for the LHC experiment VOs.

Objectives

Provide and record via the ESC meeting:
  1. Information flow in tickets or procedures and documentation improvement.
  2. Ticket reviews with the 1st line supporters within the LHC VOs.
  3. Concrete proposals in the GGUS tickets aiming to reduce the ticket turn-around time.
  4. Awareness increase of users, supporters and sites of the Grid's main information sources, i.e. GGUS, GOCdb, gocwiki, Operations' portal.

Practices

Ticket assignment analysis

As a Ticket Process Management (TPM) monitor, analyse chunks of GGUS tickets to ensure that:
  • 95% of the tickets pass to status ASSIGNED in less than one hour (within reason).
  • 95% of the tickets assigned to the correct Support Unit (SU) the first time.

Remarks from this analysis are in the tickets themselves and in a TPM report discussed at the ROC managers' meetings every two weeks.

Open ticket review per VO

Check regularly the latest list of tickets per LHC Experiment VO in Status=_open_ . Discuss difficult cases with the relevant SU, EIS colleagues and 1st-line supporters in the VO. We could establish a session at the ESC meeting to do that.

User Supporters for non-LHC VOs who wish to construct such a list at all times, should use this URL (substitute VOname with its value):

https://gus.fzk.de/ws/ticket_search.php?supportunit=all&vo=VOname&status=open&radiotf=1&timeframe=no

Meetings

  • With Grid Sites weekly on Monday at 16:30hrs at the Operations' meetings.
  • With Grid Service Managers weekly on Wednesday at 10:00am at the LCGSCM (Service Coordination Meeting).
  • With Experiment Supporters fortnightly on Monday at 14:30hrs at the ECM (Experiment Coordication Meeting).
  • With ROC and GGUS developers monthly on Thursday at 11:00am at the ESC (Executive Support Committee).
  • With experiments, ticketing system experts, IT service managers on 2011/03/21 https://twiki.cern.ch/twiki/pub/LCG/WLCGVOCCoordination/VObox_application_support_by_WLCG_VOCs.txt

ALARM tickets' quality
The meeting took place on 2010/05/10. It was an action from the 2010/04/29 Tier1 Coordination meeting. Agenda http://indico.cern.ch/conferenceDisplay.py?confId=93347

Purpose: To discuss with experts from GGUS, GOCDB, Operations' portal, CERN PRMS, Tier0, Tier1 service managers and LHC VO Authorised ALARMers, analyse issues we encountered in production or during regular tests like: the monthly tests after each GGUS Release.

Present: [Peter Kreuzer (CMS), Maite Barroso (T0 services), Maria Dimou, Maria Girone, Jamie Shiers (Experiment Services)](CERN), Gareth Smith (RAL), Jon Bakker (FNAL), Cyril L'Orphelin (IN2P3), Joel Closier (LHCb), Gilles Matthieu (GOCDB), Xavier Mol (KIT), [Guenter Grein, Helmut Dres] (GGUS), Di Qing (Triumf), Josep Flix (PIC).

What must work at all times - decisions per agenda item:

  1. GGUS front-end and database. So that shifters are always able to submit or update a ticket, See savannah #113831 for front-end and savannah #101122 for the database.. The latter is in service since April 2010. GGUS, with German Grid funds will make the fail-safe front-end available during the 2nd half of 2010. ACTION_1: Torsten should enter a detailed plan in the ticket #113831 and ACTION_2 the wlcg-smod should regularly emphasise this requerement at the MB.

  2. GGUS extract of Authorised Alarmers from VOMS. So that Authorised Alarmers can always be recognised as such. Service in place since June 2009. savannah #104835. If the nightly update returns an empty list the script used for the synchronization stops and the Alarmers of the previous day remain valid for GGUS. No action.

  3. GGUS extract of Site names and contact/emergency emails from GOCDB. Tier1 emergency emails were found empty in GOCDB twice in October 2009 and April 2010. See savannah #15009 . GGUS extracts by script the agreed GOCDB fields every night. Gilles decided to introduce a work-around directly at the database level to not allow a NULL value in the 'Emergency Email' field of Tier0/1 sites. The reason was that GOCDB4 will be out at the end of June 2010. ACTION_3 Gilles and Guenter to test GGUS-GOCDB4 to avoid surpises at release time. Maite asked whether LHCVOname-operator-alarm@cern.ch is still needed or if we can have one common address, as in GOCDB for the whole Tier0. ACTION_4 MariaD will investigate current needs and discuss at the next TIer1 coordination meeting on 2010/05/20.

  4. Correct and timely Site downtimes' notification from the Operations' Portal. So that no ALARMs are opened against sites on scheduled downtime. Cyril proposes not to use downtime notifications to discover the site status but the Operations' portal web interface for current downtimes. CMS and LHCb are happy with notifications, are open to the web interface evolution but for now they report that they use the google calendar (CMS example). Ideally they would like to see downtimes taken into consideration by the dashboards. This request will also be discussed at the next Tier1 coord.meeting. ACTION_5 MariaG will prepare with Julia Andreeva.

  5. Clear operators' instructions for all relevant WLCG services at the Tier0 and all Tier1s. The list of critical services must be checked for updates. The review of instructions per service will be done in one of the coming Tier1 coordination meetings. ACTION_6 MariaG to include this in the agenda when appropriate.

  6. Smooth and quick assignment to the Tier0/Tier1 local ticketing system, e.g. GGUS-PRMS for Tier0. See Tier0 presentation at the Tier1 coordination meeting of March 25th 2010. Every ALARM ticket to the Tier0 is automatically assigned to CERN_ROC. This now means that CERN/IT/PES members dispatch the relevant (automatically created) Remedy PRMS ticket to the appropriate category. 3rd level support is only involved when the incident reveals a middleware bug. In such cases a savannah bug is opened and linked from/to the relevant GGUS ticket. MariaG asked Maite to check https://gus.fzk.de/ws/ticket_info.php?ticket=57053 which needed to be re-classified and required a special 3rd level intervention. Guenter said that CNAF, PIC, FZK and CERN have local ticketing systems so possible delays in correct dispatching can happen there as well. ACTION_7 Should we review each of these sites at the Tier1 coordination meeting?

  7. Should we continue the periodic tests 3-4 times/year before a GDB according to these testing rules ? Yes! ACTION_8 Should we run the tests in time for the June 9th GDB? If yes, a reminder should be announced at the daily meeting during the last week of May and Authorised ALARMers should run tests during the 1st week of June.

  8. A.O.B. by Peter:
    1. It is at the discretion of the VO to decide the number of Authorised ALARMers by putting them in the right VOMS Group. See here HowTo instructions.
    2. There is no way to convert a usual or TEAM ticket to an ALARM one. One has to re-open and make reference to existing unsatisfied tickets for emphasis.
    3. A 'critical' service doesn't automatically assign the ticket priority level. The expert will decide if opening an ALARM ticket is the safe way to obtain the solution in the time required. Abuses will be shown by the drills we perform for every WLCG MB. Example from the 2010/05/11 MB, slides 3 to 10.

Per VO
  • With 1st-line supporters within the Experiment in ad-hoc meetings.

Documentation review

As part of the it-dep-gd-ops-us@cernNOSPAMPLEASE.ch team improve the online documentation for supporters linked from https://gus.fzk.de/pages/info_for_supporters.php
VO FAQs
Reviewed. FAQs defining VOSupport(VOName) Support Units became obsolete with the introduction of VO-ID cards. Details in savannah bug #102661
Document 1300
Reviewed. Updated version in savannah bug #102727
Proposal for VO documentation changes
The following concerns mostly the GGUS Documentation section for VO Users, link https://gus.fzk.de/pages/docu.php#2

Row number Current text Proposed change Who will do what
1 VO Tools description (2005-09-16) Verify that this link is all that exists about VO Tools.Remove date and link recent SAM/FCR Critical tests' FAQs Maria/Judit/VO contacts
2 VO BOX description This LCG User Guide chapter looks outdated Maria/Patricia/other ALICE experts to give new pointer
3 VOBOX How TO Verify that this link is the best and of interest to all VOs Maria/Patricia/VO Contacts
4 LCG Experiment Integration Support This group and website no more exists. This entire row should be changed to LHC Experiment VO Support and point to this page for the moment GGUS developers
5 LCG EIS Documentation and tools This group and website no more exists. This entire row should be removed GGUS developers
6 Experiment Computing Sites, including Documentation Contains 8 VOs only. Very outdated. All individual VO names should be removed and the VO-ID cards' list should only appear. VO web pages appear there. GGUS developers
7 A list of VOs supported by EIS can be found... This group and website no more exists. This entire row should be removed GGUS developers
8 All links anywhere on the page https://gus.fzk.de/pages/docu.php pointing to the LCG-2 User Guide (2 occurences) Obsolete. Should be replaced by the glite User Guide GGUS developers
9 The link (3rd row from page top pointing to the LCG-2 User Scenario Obsolete. Should be removed GGUS developers
10 VOMS Administration and User Guide (Date: 2005-03-17) Replace link by this for VOMS core and {http://edms.cern.ch/document/572406][this for VOMS Admin]] until further notice Maria will check with VOMS developers but GGUS developers should make the change now
11 Instructions for installing and configuring an LCG-2 site Obsolete link. Replace by the glite entry point GGUS developers
12 Instructions for installing and configuring VOMS Obsolete link. Replace by http://edms.cern.ch/document/818502 GGUS developers
13 Information System Trouble Shouting Guide Grammar wrong (shooting) and link invalid. Point to https://twiki.cern.ch/twiki/bin/view/EGEE/InfoTrouble GGUS developers

GGUS enhancement requests

Create and follow up savannah tickets with new functionality asked by the 1st line LHC VO supporters.

Follow the acceptance and implementation status via GGUS plans and releases.

Newly registered VO wants to provide support via GGUS

You have recently completed your VO-ID card via http://operations-portal.egi.eu/vo . The GGUS development team was notified that you wish to provide support to the users of your VO via http://ggus.org. In order to do that your VO experts will have to become part of a Support Unit (SU). In practice this means:

How to set-up the mailing list(e-group) for ticket notifications: This mailing list will be either set up at CERN or at the site SU requester him(her)self.

If the e-group is set up at CERN it will be labelled yourVOname-grid-support@cernNOSPAMPLEASE.ch. Owner of the e-group will be the SU requester. If the SU requester (who should preferably be the VO Admin) has no CERN login account, the e-group should NOT be created at CERN as external e-group owners are not allowed (see Response in https://remedy01.cern.ch/cgi-bin/consult.cgi?caseid=CT0000000661882&email=maria.dimou@cern.ch ).

The e-group yourVOname-grid-support-admin@cernNOSPAMPLEASE.ch should also be created to contain the administrators of the SU e-group. These will be the SU requester, one or more backup people and the GGUS team via the inclusion of the existing e-group 'ggus-lists-admins'.

If the requester provides a mailing list for receiving ticket notification emails he has to make sure that GGUS is allowed to post to this list. GGUS uses two different mail addresses: helpdesk@ggus.org is used for ticket related emails, support@ggus.org is used for contacting the GGUS developers.

It is your responsibility to maintain your VO-ID card on the Operations' portal up-to-date with valid information about the supporters of your VO.

The GGUS development team will open, on your behalf a request in the savannah project at https://savannah.cern.ch/support/?func=additem&group=esc to plan the effective date of your VO integration as a production GGUS SU. You will receive an automatic notification with the savannah ticket corresponding to your case. If you have questions about this process please contact ggus-info@cernNOSPAMPLEASE.ch

All this information is also included in https://wiki.egi.eu/wiki/FAQ_GGUS-New-Support-Unit Thank you!

ATLAS-IT meeting on ALARMs from SNOW 20121001

This was the start of multiple meetings and presentations on this issue. The idea to raise ALARMs from SNOW was abandoned, in favour of the well-tested GGUS entry point. Matter is on-going. Progress is recorded in Savannah:132582

SIRdeDNSfailure20100512

Description

Partial Failure of the Name Service for .de domains on 20100512 between 13:30 and up to 17:45 (CEST) for some sites, due to caching by the ISPs, even if the problem was solved by 15:45 according to DENIC, the company hosting the .de Top Level Domain. Here is a DENIC press release.

Impact

This incident prevented access to GGUS as the http://ggus.org domain is today a re-direct to https://gus.fzk.de. This means, like in any incident causing GGUS unavailability, that ticket submission/update and site notification was impossible. This is particularly critical in case of ALARMs. It also presumably, prevented data transfer to the GridKa Tier1 (did it?).

Analysis

Currently http://www.ggus.org, the 'promoted' URL is just a re-direct to https://gus.fzk.de. As the institute is now renamed to kit.edu, GGUS plans to fully move to http://www.ggus.eu soon (no redirect anymore). It is suggested that, as a fallback, http://ggus.org be also registered and remain in operation at all times.This would produce a certificate warning in the browser, but it would still work. This solution would bypass the problem of new DNS propagation delay (up to one day) when the same domain name mapping is changed to correspond to different IP addresses. If both top level domains fail at the same time, using the IP address would be the solution. Emergency broadcast procedures should be rehearsed to propagate such information to all interested parties. A detailed plan on how to implement, configure, test and rehearse such a fall-back must be documented by the GGUS developers in https://savannah.cern.ch/support/?113831 . Similar solutions must be put in place for the Operations' portal, GOCDB and OSG OIM.

Timeline

WHEN CEST WHAT
2010/05/12 13:30 : Users sometimes receive the incorrect reply “domain does not exist” for domains under .de.
2010/05/12 14:20 : DENIC technicians diagnose an incomplete update of the name service data (zone file) at 12 of the 16 service locations and switch off those locations to avoid faulty responses.
2010/05/12 14:30 : The switched-off locations are successively provided with a complete zone file and re-integrated into the name server network.
2010/05/12 15:45 : Worldwide DNS data re-distribution of these locations completed by DENIC.
2010/05/12 17:45 : End of all possible disturbances perceived by Internet users due to caching by the ISPs. This end-of-incident timestamp is approximate.

SIRGGUSfailure20101116

Description

Total unavailability of the GGUS web interface between ~08:15 and ~11am CET. This SIR will be presented at the 20101202 T1SCM.

Impact

This incident prevented access to GGUS. Every page behind http://ggus.org was hanging. This means, like in any incident causing GGUS unavailability, that ticket submission/update and site notification was impossible. This is particularly critical in case an ALARM ticket submission is needed during the incident.

Analysis

A problem with the mail parser was already observed the day before. GGUS developer Guenter Grein notified the TPM e-group on 2010/11/15 at 16:53. It is unclear if this is related to the 20101116 incident (web pages hanging). A possible cause may be the simultaneous submission of multiple tickets to the spanish ticketing system, done by the TPM on 2010/11/16 around 8:30am which caused the SOAP component to freeze and entail a blockage of all webservices. GGUS developers tried to reproduce the problem on the training system by doing multiple updates to the Spanish test system without provoking the problem. The logs nor on GGUS side neither on the Spanish side contain any hint about the outage. Hence the outage isn't understood up to now.

Timeline

WHEN CET WHAT
2010/11/16 8:46 Multiple reports by Atlas shifters (Jaroslava) and supporters (Alessandra F.)that they can't open GGUS web pages. Developers, already aware of the problem investigating.
2010/11/16 9:20 As the problem persists, Guenter emails the ggus-info e-group.
2010/11/16 9:33 MariaD forwards the information to relevant WLCG e-groups.
2010/11/16 9:54 MariaA forwards the information to the middleware developers' e-group.
2010/11/16 10:10 Guenter announces the web interface is back, mail parser still down.
2010/11/16 11:59-12:22 IN2P3 ticketing system interface developer reports still an error requesting GGUS webservice => Server returned HTTP response code: 503 for URL: https://gusiwr.fzk.de/arsys/services/ARService?server=gusiwr&webService=CIC_HelpDesk. Problem disappeared 20 minutes later.

SIRGGUSfailure20101126

Description

GGUS database unavailability between ~13:00 and 14:30 CET. This SIR will be presented at the 20101202 T1SCM.

Impact

This incident prevented access to GGUS (homepage and tickets' view, submission). This, like all GGUS unavailability is particularly critical in case of ALARMs.

Analysis

As soon as Atlas (Jaroslava) reported the unaccessibility to the GGUS pages, Guenter responded within 5 minutes and started investigation. The problem was quickly identified as database-related. 1.5 hours later the following diagnostic was published via email and at the WLCG daily meeting on the same day: _The reason for the outage was an unscheduled reboot of the Oracle DB cluster failing for all nodes. The DB reboot was caused by a combination of hardware failure and miss-configuration. The hardware failure made the DB cluster losing some essential control files. The miss-configuration prevented the cluster reboot.

Timeline

WHEN CET WHAT
2010/11/26 13:11 Jaroslava reports one can't open ggus web pages.
2010/11/26 13:17 Guenter starts investigation.
2010/11/26 13:22 Guenter informs the tpm and ggus-info e-groups that GGUS is down. First estimate: database problem.
2010/11/26 13:26 David Crooks from the Glasgow Grid site submits the error message he gets: No DB connect ...please try later or contact webmaster....
2010/11/26 14:37 Guenter announces GGUS service available again.

SIRGGUSproblemALARMs20110727

Submitted by Helmut, now attached to https://twiki.cern.ch/twiki/pub/LCG/WLCGServiceIncidents/20110727GGUS_Service_Incident_Report.pdf

SIRDNSfailure20111006

Description

Partial Failure of the Name Service at KIT on 20111006 11:15 up to 20111007 12:00 (CEST) for some SOAP interfaces, due to an update of the KIT IPS system (intrusion prevention system).

Impact

This incident breaks some SOAP based interfaces of GGUS e.g. CERN, NGI_IT, NGI_GRNET, i.e. for ~24 hours GGUS tickets couldn't create/update their peers in the those other local ticketing systems to which they interface using web services.

Analysis

The administrators of the IPS didn't broadcast the update as they didn't expect impacts on any system. Hence it took the GGUS developers some hours finding out the reason for the SOAP problems.

SIRSNOWinterfacefailure20111104

Description

Failure of the wrapper tool used for interfacing GGUS with SNOW on 20111104 16:15 up to 2011107 11:00 (CET).

Impact

This incident breaks the interfaces to SNOW, NGI_France and ROC_Russia.GGUS tickets couldn't create/update their peers in the other ticketing systems.

Analysis

There was an issue with the outgoing mail server at KIT last Thursday (2011-11-04) and Friday (2011-11-05). Not being aware of the mail server issue the GGUS team tried to fix the mail problems by restarting the Remedy system. Unfortunately the Remedy server didn’t load the correct environment during the reboot due to not using the root account for the reboot. This lead to a failure of all scripts used in GGUS and triggered by the Remedy server such as the SNOW wrapper and some others. The startup script of the Remedy server has now been changed forcing using the environment settings of the root account for rebooting the server.

SIRGGUSunreachable20120320

Description

The DNS update was not propagated to all DNS servers in a timely manner on 20120320 13:00 up to 20120321 11:00 (CET).

Impact

GGUS was not reachable for some regions depending on the frequency of DNS cache updates.

Analysis

With the release on March, 20th 2012 GGUS changed the server infrastructure for the web front-end from running only one server to running two servers in parallel. The IP address of the previous standalone server was disabled while the new parallel servers IPs have been enabled during the release.
The network team at KIT wasn't aware of the appropriate way to set the TTL value for the external DNS service. Therefore some regions could access GGUS as usual. Others like CERN could not.
For those who couldn't access GGUS after the release a DNS update forced by their local network administrators solved the problem.

SIRGGUSunreachable20120522

Again a DNS update related problem. In case the WLCG SCOD doesn't require a formal SIR, here is the info about what happened for the record: https://savannah.cern.ch/support/?113831#comment40

SIRGGUSunreachable20120701

One of the troubled services by the leap second of June 30th. Reported in https://twiki.cern.ch/twiki/bin/view/LCG/WLCGDailyMeetingsWeek120701#Tuesday (within the ATLAS report): GUUS issue Sun to Mon due to leap second (Full explanation by GGUS developer Guenter Grein: "The problem started after introducing a leap second in the night 20120630 to 20120701. It ended Monday morning 09:30 after restarting not only the Remedy server but also the underlying OS. Especially the Java based Remedy mid-tier was facing problems which increased with the rising number of accesses to the GGUS web portal this morning. Several publications in the www report problems of Linux kernels and Java applications related to the leap second. We believe this has caused the GGUS problems too. The GGUS server is running on a Linux OS and a major part of Remedy is Java-based.")

Aggregation of monitoring information

Notes from the 2010/09/28 CERN/IT/ES meeting http://indico.cern.ch/event/Monitoring20100928. In this table 'Y' = Yes, the experiment has it. Comments by Julia, Federico, Nicolo', Patricia included. Last update 20101013.

Tool\VO Alice ATLAS CMS LHCb Is the tool private to the VO? Does it have a programmatic interface? Comments
Lemon->SLS (GridMap) Y Y Y Y N Y  
SAM->Nagios Y Y Y Y N Y Nagios only works for the CREAM CE with a special submission mode not used by most VOs
MonALISA Y N N N full suite used by Alice only but framework used by other tools too Y Discrepancy with GridView!
GridView Y Y Y Y Y Y Discrepancy with MonALISA
Database tools N Y N N ? ?
DDM (Data Distribution Monitor) N Y N N Y, made by IT/ES/DNG for ATLAS Y Many DDM tools for data replication and more
SSB (Site Status Board) Y Y Y Y N, made by IT/ES/DNG Y Pending discussion for the best use of SSB by Alice
Dashboard job monitoring N Starting Y N N Y Provides job monitoring data for all 4 VOs, not used by Alice and LHCb now
Dashboard SAM usability monitor Y Y Y Y N Y Will be replaced when SAM will become NAGIOS only
Downtime Calendar N but need one and wishes to discuss its required features Y (google) Y (google) A calendar tool based on downtime notifications and google now being replaced by a GOCDB local cache of LHCb -supporting-sites only with their DIRAC names Implementation differences are normal due to topology (GOCDB and OIM imports) Wish for common info collector across VOs
Panda Analysis Monitor N Y N N Y (ATLAS) Y N
Hammercloud (site stress testing) N Y Y Y N (tool made by IT/ES/VOS) Y Talk to the developers for use by other tools
PhEDEx (transfer monitoring) N N Y N Y (developed by CMS with support from IT-ES-VOS) Y N
RSS (Resource Status System) N N N Y Y, LHCb Y N, a DIRAC service
VO comments/wishes Solve the MonALISA/GridView issue, identify more common tools Need FTS monitor, need network monitor, wishes common downtime info collector across experiments Same as ATLAS + Storage accounting Happy with RSS but also need FTS and network monitoring tools N/A N/A N/A

EGI SA3 Metrics

Metric ID: M.SA3.16 / Task: TSA3.3 Project Year 1

Definitions as per Savannah:118984#comment0 Total number of tickets provided for the 4 LHC VOs.

VO\Period 2010/05/01-2010/07/31 2010/08/01-2010/10/31 2010/11/01-2011/01/31 2011/02/01-2011/04/31
Alice 38 22 13 This being end of the project year 1
Atlas 689 775 699 there was no quarterly report
CMS 75 68 108 but a yearly report
LHCb 200 158 109 where these metrics weren't required

Metric ID: M.SA3.16 / Task: TSA3.3 Project Year 2

Definitions as per Savannah:118984#comment5 Only True ALARM ticket totals now required for the 4 LHC VOs as a result of the reviewers' report for the 1st year of the EGI project.

Period Total number of true ALARMS for all 4 LHC VOs Mean time to resolution of true ALARMS for all 4 LHC VOs HH:MM
2011/05/01-2011/07/31 45 22:56
2011/08/01-2011/10/31 47 11:37
2011/11/01-2012/01/31 20 Under review in Savannah:118984
2012/02/01-2012/04/30 17 18:23
2012/05/01-2012/07/31 28 43:18
2012/08/01-2012/10/31 15 47:46
2012/11/01-2013/01/31 16 46:06
2013/02/01-2013/04/30 11 36:33 (Last report for SA3)

Grid Information Sources

As 1st-line VO supporters, please consult these links which might contain an announcement that explains a problem your users are facing:

-- Main.dimou - 23 Apr 2007

Topic attachments
I Attachment History Action Size Date Who Comment
PDFpdf 20110727GGUS_Service_Incident_Report.pdf r1 manage 51.0 K 2011-08-22 - 15:33 MariaDimou SIR GGUS problem with ALARM email notification of 20110727
Unknown file formatpptx ALARM_ToP.pptx r1 manage 70.2 K 2012-03-19 - 15:54 MariaDimou Real ALARM totals with ToP value Sep 2011 - March 2012
Unknown file formatpptx CERN_ALARM_ToP.pptx r1 manage 69.8 K 2012-03-20 - 10:24 MariaDimou Real ALARM totals (to CERN only) with ToP value Sep 2011 - March 2012
PDFpdf GGUS-SNOW-20110125.pdf r1 manage 979.4 K 2011-01-25 - 12:28 MariaDimou GGUS-SNOW interface presentation to CERN IT Service Managers
PowerPointppt GGUS-SNOW-20110125.ppt r1 manage 1618.5 K 2011-01-25 - 12:29 MariaDimou The above in another format
Unknown file formatpptx GGUS-SNOW-20110125.pptx r1 manage 1309.4 K 2011-01-25 - 12:29 MariaDimou GGUS-SNOW interface presentation to CERN IT Service Managers - another format
Unknown file formatpptx GGUS_ALARMs_notifications.pptx r1 manage 2290.9 K 2013-02-14 - 14:32 MariaDimou GGUS ALARMs notifications for the Tier0 for the 2013/02/19 WLCG MB - More experiment critical services.
PDFpdf IT-CMS-Tickets-20110907.pdf r3 r2 r1 manage 86.7 K 2011-09-06 - 14:11 MariaDimou Ticket handling procedures' presentation at the IT/CMS meeting (.pdf)
PowerPointppt IT-CMS-Tickets-20110907.ppt r4 r3 r2 r1 manage 662.0 K 2011-09-06 - 14:12 MariaDimou Ticket handling procedures' presentation at the IT/CMS meeting (.ppt)
Texttxt MB_20120320_ALARM_ToP.txt r1 manage 5.0 K 2012-03-19 - 15:55 MariaDimou Real ALARM details with ToP value Sep 2011 - March 2012
Microsoft Excel Spreadsheetxls closingtime_lhcvo_20100501-20100731.xls r1 manage 11.0 K 2011-02-01 - 16:54 MariaDimou  
Microsoft Excel Spreadsheetxls closingtime_lhcvo_20100801-20101031.xls r1 manage 11.0 K 2011-02-01 - 16:57 MariaDimou EGI metric "Mean time to resolution of HEP VO Support Units"
Microsoft Excel Spreadsheetxls closingtime_lhcvo_20101101_20110131.xls r1 manage 8.5 K 2011-02-01 - 16:52 MariaDimou  
Microsoft Excel Spreadsheetxls closingtime_lhcvo_20110501-20110531.xls r1 manage 11.0 K 2011-08-18 - 17:31 MariaDimou EGI metric "Total number of tickets and Mean time to resolution of HEP VO Support Units" in May 2011
Microsoft Excel Spreadsheetxls closingtime_lhcvo_20110601-20110631.xls r1 manage 8.5 K 2011-08-18 - 17:25 MariaDimou EGI metric "Total number of tickets and Mean time to resolution of HEP VO Support Units" in June 2011
Microsoft Excel Spreadsheetxls closingtime_lhcvo_20110701-20110731.xls r1 manage 8.5 K 2011-08-18 - 17:24 MariaDimou EGI metric "Total number of tickets and Mean time to resolution of HEP VO Support Units" in July 2011
Microsoft Excel Spreadsheetxls closingtime_lhcvo_alarm_20110501-20110531.xls r1 manage 8.5 K 2011-08-18 - 17:27 MariaDimou EGI metric "Total real ALARM tickets and Mean time to resolution of HEP VO Support Units" in May 2011
Microsoft Excel Spreadsheetxls closingtime_lhcvo_alarm_20110601-20110630.xls r1 manage 8.5 K 2011-08-18 - 17:27 MariaDimou EGI metric "Total real ALARM tickets and Mean time to resolution of HEP VO Support Units" in June 2011
Microsoft Excel Spreadsheetxls closingtime_lhcvo_alarm_20110701-20110731.xls r1 manage 8.5 K 2011-08-18 - 17:26 MariaDimou EGI metric "Total real ALARM tickets and Mean time to resolution of HEP VO Support Units" in July 2011
Edit | Attach | Watch | Print version | History: r69 < r68 < r67 < r66 < r65 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r69 - 2013-05-14 - MariaDimou
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback