ProductionProceduresDailyShifterChecklist < LHCb

LHCb Web>LHCbComputing>ProductionProcedures>ProductionProceduresDailyShifterChecklist (2008-12-09, GreigCowan) (raw view)

EditAttachPDF

---+ Shifter Checklist

There are some common tasks that shifters should be regularly checking.

%TOC%

---+++ eLogger

   * Read the operations [[http://lblogbook.cern.ch/Operations][elogger]] from the previous day to find out what has been happening.

---+++ DIRAC monitoring pages

   * Production summary [[http://lhcbweb.pic.es/DIRAC/LHCb-Production/visitor/jobs/ProductionMonitor/display][webpage]] for the number of Active productions and the number of waiting, running and failed jobs in each production.
   * The job plots from the accounting page, looking in particular at the number of failed jobs per site over the past day for the running productions. If one site has a large number of failed jobs then this is the one that you should start investigating first of all. See image below for the Failed jobs.
   * The pilot summary page to check the number of pilots that are being submitted, running and aborted. If there are a lot of aborted jobs or no pilots being submitted/running then this indicates a problem which requires immediate investigation. Contact the developers.
   * The site downtime [[https://lhcbweb.pic.es/DIRAC/LHCb-Production/lhcb_admin/web/External/display?site=calendar][calendar]]. The sites that are in downtime should remain banned and those that have successfully come back from maintenance (SAM jobs passing etc) should be unbanned.

<center>
<img src="%ATTACHURLPATH%/lhcb_failed_jobs_cumulative_daypng" width='550' height='350'>
</center>

---+++ GGUS tickets

   * The [[https://gus.fzk.de/pages/rss/rss_open_lhcb.php][RSS feed]] of open LHCb tickets in GGUS. This should always be checked before submitting a new ticket.

---+++ email lists

   * Postings to lhcb-dirac@cern.ch to see what the latest developments are and when new releases of DIRAC are made.
   * Postings to lhcb-distributed-analysis@cern.ch to help debug problems with user jobs on the Grid.



-- Main.GreigCowan - 09 Dec 2008

Attachments

Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
ext	lhcb_failed_jobs_cumulative_daypng	r1	manage	117.3 K	2008-12-09 - 19:49	GreigCowan

Topic revision: r1 - 2008-12-09 - GreigCowan

LHCb

Public webs

Welcome Guest

- Cern Search
- TWiki Search
- Google Search
LHCb All webs