------------------------------------- LAST UPDATED: April 2018 -------------------------------------
twikiBanner dailyShifter.001.png

Getting started
This page describes the workflow of each day of the shift.
There will be links throughout that give more details on each topic. We advise reading through this page first, and then going back to look at the details.
Note that this page shows the workflow of a daily shift but NOT the individual requirements of how each subdetector requires the run to be processed. That information is given on Page 3, which comes later in the tutorial.

As always, if something is unclear or confusing: contact your expert shifter!


CaloCombined is actually 5 mini-shifts in one! As a CaloCombined shifter you are responsible for looking at plots from:

  • Calo Global (often just called Calo)
  • Jets
  • MET
  • Tau
  • EGamma
Each of these mini-shifts is sorted into a different folder in the DQ webdisplay, and each has a different strategy and requirements for how the defects should be flagged and reported. These details are covered on Page 3, which comes later in the tutorial.
It may seem like a lot of information, but after a few shifts there is a clear rhyme and reason to each folder and you'll quickly get used to chatting to the subdetector shifters about your findings.

Let's start the shift!

Check your email

Assuming you subscribed to the email lists you were told to on the Front Page, each day of your shift you will recieve an email from the address shown in the picture, below.
These daily emails are your first port of call, particularly on the first day of the shift where the previous shifter may have some leftover runs that you'll have to take care of.
Let's look at the example email, in the screenshot below:

twiki email.001.png

The first thing to note is that this email was sent during collision runs. You may end up with a shift during collisions or cosmics, be sure to check the email for both.
Highlighted in yellow are each of the subdetector systems you are responsible for as a shifter: first for the express (ES1) stream, and then for the bulk (BULK) stream.

If you got this email on your first day of the weekly shift, you'd be a bit surprised
Although the previous shifter kept up-to-date with the express stream (there's only 1 outstanding run - probably from today), there are 16 outstanding bulk runs.
Yes, a few of them may have finished processing today but not all of them. Typically 3-4 can come through in a day. I've never seen 10+, unless each run was very small. It's not impossible but it is unlikely.
The point is, no one likes the person who doesn't sign off their bulk runs and leaves them for the next shifter. Don't be that person! smile

So, now we look at the next email:

DQ runsToConsider email.001.png

This email shows you which express runs have finished processing since this email was sent.
These express runs MUST be processed before 16:00 CERN-time, i.e. EDT/EST.
If you're taking this shift from the USA or East Asia, you know that it's not always possible to catch every express run. As a US-based shifter, my strategy is to check for newly processed express runs using the Tier0 as well as this email in the morning and the evening to try and catch anything I might have missed.


In this email there were three express runs ready for review.
So let's review them!

Tier0 Processing Page

There are several ways to get to the the information we want, but I'll just cover two here. The first is the tier0 webdisplay page: https://atlasdqm.cern.ch/webdisplay/tier0/


or, formally, Tier 0 Processing DQ Monitoring.
The page is pretty self explanatory, it's arranged by run number, whether the processing being done is on Express (ES1) or Bulk (BLK) and then broken down further into the various physics streams. Some things to note:

  • Run Number: Clicking on the run number will take you to the run query page for that run.
    But hovering over the run number will give you a summary of information about that run, for example:
    runQuery hover.png

  • Iteration: Other than whether something is ES1 or BLK, note that there are a bunch of f, h, and sometimes x-tags. These indicate different software versions used in the masking and reconstruction.
    Some runs may be reprocessed if a bug is found. It is important to read the emails from the DQ conveners as they may tell you to re-process a run with a different software tag.
    For example, "Dear DQ Shifter, please take another look at run 123456, we found an issue with the foo algorithm and have reprocessed the run. Please only present results with h-tag "j123" at next week's DQ meeting. etc."
    If you want to know what a tag corresponds to, clicking the tag in this column will take you to an AMI page that lists a description.

  • Streams: As a CaloCombined DQ shifter during standard operation you'll mostly be interacting with either:
    • ES1: express_express, or
    • BLK: physics_main

When opening this page for the first time, it can be helpful to know which runs have already been signed off (we'll get to that at the end of this page), so you can see how many runs have been processed, since.
Some of these new runs might have finished processing but some might still be in progress, you can tell by the stars appending the stream name, for example, look at the express_express stream in ES1, circled below:
unfinished tier0.001.png
This stream is not yet ready to be reviewed and you should check back in, later in the day to see if it has finished.

When you are ready to begin reviewing the run, simply click the name of relevant stream which will take you to the webdisplay page containing folders of histograms. We'll cover these histograms in the next section.

DQ Run table

The other way to access the runs and see which streams are ready is the DQ Run Table: https://atlasdqm.cern.ch/dqruntable/
Which is pretty self-explanatory. It has some cool features that aren't on the Tier0 page, including:
  • Missing sign-off: So you can see if all 5 of the CaloCombined subgroups have been signed off or not.
  • Tier 0 Reco Status: This is similar to the tier0 stream column but now with c o l o r s .
The idea is that there is a block for each stream (physics_main, cosmics_calo, etc.), that has a color. Each color represents the state of readiness:
  • Grey : Tier0 has not started processing the stream yet.
  • Yellow : Processing is in progress, but not finished.
  • Green : Tier0 has finished processing the stream!
When a stream is finished, simply click the green box and it will take you to the webdisplay folders.
Many people do use the DQ run table as their front page when starting a shift. However, something to note is that the run table takes longer to update than the Tier0 page, so a run that might not even have shown up on the DQ Run Table page could have finished processing and therefore be visible on the Tier 0 page.


We've now checked and there is a run that's ES1 express_express stream has finished processing. It's time to start the shift proper and review some histograms!

Tier0 webdisplay: plots!

After clicking on the relevant stream, either on the Tier0 or DQ Run Table pages, you'll be brought to a stream summary page like this one:

entryPage webdisplay2.png

This page contains links to the logbook and database, where you'll write your report summarizing the run, and enter the flags for defects and sign-off the run, respectively.
Then, there is a link that takes you to the histogram webdisplay for both the entire run, and individual lumi blocks.
You'll want to start off looking at the entire run for defects, and then, if you see a defect, you may need to go back to this page and establish whih lumi blocks the defect affects.
After clicking on the 'entire run' link, you'll be brought to the webdisplay page:


The toolbar at the top has two rows which serve distinct functions. The first row shows you the run number and stream and allows you to select a different run number or stream without having to go back to the tier0 page or change the URL.
The second row gives you the option to display the histogram you're currently looking side-by-side with the same histogram from a different run. This is very useful when trying to establish whether a feature you're seeing in a plot is new or has shown up in other plots before.
At the bottom of the page there are some more useful links: the logbook and defect DB that were also available on the entry page, and links to the commit tags and git commit hash. Clicking the git commit hash will take you to the gitlab repository, at the commit that was used to process this run. You, likely, will not need to use this.
What you will spend most of your time in this shift with, however, is the folders. Below is one of the 5 major folders you will look at, CaloMonitoring, expanded:


Within the CaloMonitoring folder there are two subfolders: one for a shifter and one for the expert. As a shifter, you just need to focus on this first folder.
Within the shifter folder are 3 more subfolders, one for each part of the calorimeter: barrel, endcap A, and endcap C. Within each of those folders are two plots: one for the EM calorimter and one for the hadronic calorimeter.
We're not going to focus on how to interpret the plots in this part of the twiki, or which folders to look at, that will be the focus of parts 4 and 5.


For now, let's assume you've reviewed, and taken note of, an interesting feature that shows up across one or many plots in the CaloCombined folders.
The next step of your daily shift will be to document these features and inform the relevant parties of your findings.

Report and document your findings

To document your findings, first go to the logbook by clicking one of the links, either on the histogram webdisplay page or the entry page, to the DQ Logbook.
The link will take you to a new logbook entry with some of the details already filled in:

TODO: no runs available at the moment so can't get the screenshots

Ensure that you've ticked off the relevant subsystems:
If you are signing off the ES1 stream instead of the BLK stream you'll also have to tick the 'processing pass' box.
The, write a sensible subject line. A good example might be: "CaloComb ES1 sign-off: run 12344".
Finally, you'll enter some text summarising the run. Part 4 of this twiki will suggest some things to include in your log and also provide a template.


TODO: also need screenshots for this part.

Reaching out

Now that you've clearly documented your findings and signed off the run, it's time to get in touch with the expert shifter. Remember that only runs that have been BLK processed are reviewed at the weekly DQ meeting. So, if a defect shows up, or persists from the express stream, in a bulk run you should inform your expert shifter and the subdectector shifters that the defect might be associated with. Your expert shifter can help you with this.

TODO: more

-- RebeccaCarney - 2018-04-01

Topic attachments
I Attachment History Action Size Date Who Comment
PNGpng DQRunTable.001.png r1 manage 108.6 K 2018-04-08 - 22:05 RebeccaCarney  
PNGpng DQ_MissingSignOff_email.png r1 manage 171.8 K 2018-04-02 - 02:18 RebeccaCarney  
PNGpng DQ_runsToConsider_email.001.png r1 manage 161.2 K 2018-04-02 - 03:14 RebeccaCarney  
PNGpng entryPage_webdisplay2.png r1 manage 138.4 K 2018-04-08 - 23:42 RebeccaCarney  
PNGpng folders.png r1 manage 40.5 K 2018-04-08 - 23:42 RebeccaCarney  
PNGpng runQuery_hover.png r1 manage 60.5 K 2018-04-08 - 21:13 RebeccaCarney  
PNGpng systemsAffected.png r1 manage 34.3 K 2018-04-09 - 00:42 RebeccaCarney  
PNGpng tier0Page.png r1 manage 187.3 K 2018-04-08 - 21:02 RebeccaCarney  
PNGpng twikiBanner_dailyShifter.001.png r1 manage 46.5 K 2018-04-02 - 02:06 RebeccaCarney  
PNGpng twiki_email.001.png r1 manage 201.8 K 2018-04-02 - 02:27 RebeccaCarney  
PNGpng unfinished_DQgruntable.001.png r1 manage 62.1 K 2018-04-08 - 21:45 RebeccaCarney  
PNGpng unfinished_tier0.001.png r1 manage 66.9 K 2018-04-08 - 21:45 RebeccaCarney  
PNGpng webdisplay.png r1 manage 174.4 K 2018-04-08 - 23:42 RebeccaCarney  
Edit | Attach | Watch | Print version | History: r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r3 - 2018-04-09 - RebeccaCarney
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Sandbox All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback