5.5 Data Quality Monitoring (DQM) for physics analysis

Complete:

Detailed Review status

Goals of this page:

Learn how the DQM is operated in CMS to check the data and how to use the basic tools provided by the DQM task

N.B. DQM can also be used to check MC production with new CMSSW releases. This is NOT the topic of this page. If you are interested in that, you should instead read DQMforMC.

Introduction

The DQM is performed online and offline.

1. Online DQM: A subset of the data are reconstructed and monitored at P5 to give immediate feedback about the detector status.
   
2. Offline DQM is operated in various steps:
   1. A subset of the data (express stream) are promptly reconstructed and monitored with ~1h latency with the aim of certifying the goodness of the data regarding also the reconstruction software and the alignment and calibration constants.
   2. The full set of data taken are promptly reconstructed and monitored with ~48h latency; same aim as 2a, but typically better alignment and calibration constants are available.
      
   3. At each reprocessing the data are again monitored and certified; the aim is the same as 2a and 2b, but typically better reconstruction software and better alignment and calibration constants are available.

For all these steps automatic certification is in place; for online and offline express stream a shifter is in charge of monitoring the results and delivering a manual certification (as explained below).

The DQM involve different levels:

1. DPG level monitors the status and behavior of each subsystem up to the local reconstruction (DPG DQM runs online and offline).
2. POG level monitors the quality of the reconstructed physics objects (muons, jets, electrons,...) (POG DQM runs only in the offline stage).
3. PAG level monitors higher level quantities (e.g., dilepton mass peaks), correlation between different physics objects or simple kinematic distributions with more analysis oriented cuts (PAG DQM runs only in the offline stage).

The DQM provides histograms produced in each of the steps previous listed. It provides also the results of tests on these histograms to quantify the goodness of the data monitored (a.k.a. data certification).
Two kinds of data certification are available:

• Automatic data certification: Algorithmic tests implemented in the DQM code run automatically (available for all the DQM steps online and offline but only for the DPG and POG histograms). A detailed set of flags is delivered to cope with the different parts of the detector (e.g. barrel, endcaps) and for different detector problems (DAQ, DCS, event data).
• Manual data certification: A shifter checks the most important histograms and defines the goodness of the data on the basis of the instructions provided by the experts (available only for steps 1, 2a, and 2c and again only for DPG and POG histograms). A single boolean flag per subsystem is set.

The manual data certification is visible in the Run Registry; both the automatic and the manual certifications are stored in DBS. The automatic data certification is presently issued by run; there are plans to deliver it by luminosity section. Manual certification is delivered by run.
A general description of the DQM system can be also found here.

How to visualize the DQM histograms

You need to have your GRID certificate installed in your browser (instructions).
Then you can directly access the offline and online DQM Graphical User Interface (GUI) where all the histograms are shown.

How to chose dataset/run

• Click on "Run #"; you will be redirected to a page where if you click on "Any" you see the full list of available datasets and runs.
  
• You can choose to order the list by run number or by dataset name.
• You can click on a given dataset/run or you can search for it using the tool bar (for instance, if you type CRAFT09 in the tool bar you will see all the runs for all the datasets with the string CRAFT09 in their name).

Once you click on a given run you will access the "Summary" page for that run, where one summary histogram for each subsystem is shown (if available).

• The same GUI can also be used to view validation histograms for MC, as explained here.

How to navigate between the various histograms

If you click on "Workspace" you will access a list of possible views.

• "Summary": one summary histogram for each subsystem is shown (if available).
• "Report": the main information for each subsystem is listed (percentage of FED in the DAQ, Event rate, etc.).
• "Shift": view for the DQM shifter where you can access only the histograms (divided into directories, one for each subsystem) that the shifter is supposed to look at in order to certify the goodness of the run.
• "Everything": all the histograms are available organized in a directory tree that you can navigate by clicking on each directory icon (if you want to go back, click on "Top").

Features available for each histogram

Once you get a list of histograms visualized on the page you can:

• clicking on "Describe" you can see a description of the histogram (if available). This will appear in a pop-up window on the upper, right part of the page
• clicking on "Customise" you can change the scale (logarithmic or linear and the range), change the draw options, add (if available) or remove the reference distribution chosen by the experts to describe the expected good behavior of that particular histogram
• clicking on "View details" or on the CMS logo you can also display trend plots (e.g. in the "Strip Chart" section, chose "X Mean" in the Display field for last 100 runs), you can compare histograms from other runs/datasets (e.g. in the "Reference" section you can decide to "Show reference: For All", "Position: On side" then chose "Other" in the first field, put the run number and dataset name of the histogram you want to compare, you can compare up to 4 histograms at once!)

How to exploit the data certification issued by DQM

The automatic certification provides a detailed set of flags to address all the possible problems of the subsystem with full granularity.
The manual certification issued by the shifters is based on a boolean flag (GREEN=GOOD, RED=BAD) for each subsystem. Notice that the following policy will be established:

• GOOD runs are runs where everything was good; they can be used for physics analysis without any further check
• BAD runs include runs where only part of the subsystem was problematic. These runs can be possibly used but you must check yourself (e.g., accessing the automatic certification) if the part of the subsystem relevant for your analysis (e.g., RPC barrel vs RPC endcaps) is in good or bad status. A dedicated keyword in the comment stored in the Run Registry will identify these not completely bad runs.

Visualization of data certification flags (Run Registry)

Run Registry service (details).

In the Run Registry all the runs judged "relevant" by the online shifter are stored with all their details (run number, number of events, rate, etc.). The offline shifter analyzes only those particular runs. Runs are consider relevant if they have considerable statistics and not completely crazy hardware settings.
The manual certification of the online and offline shifters are stored in the Run Registry: each shifter assign a quality flag (GOOD, BAD, or EXCL, i.e., not in the DAQ) for each subsystem.

• You can click on a single run and see all the details for that particular run (online and offline data certification flags included)
• You can click on "Filter"->"Columns" and decide which information you want to have shown in the your Run Registry view
• You can also view only a subset of runs, filtering on the basis of the value of a given column. For instance, to visualize only runs where the DT system was GOOD, click on "Filter"->"Show", type GOOD in the DT column, and then press "Enter"
• You can also see some monitoring plots: click on "Data"->"Generate", this will generate the plots for the runs that are currently in your view. Then click on "data" again and chose the kind of plot you want to see (e.g. the magnetic field value or the number of event recorded as a function of the run number or the percentage of runs where each subsystem was BAD, GOOD, or EXCL)

Full access to data certification flags (DBS)

All the automatic and manual (online and offline) certification flags are stored permanently in the Data Bookkeeping Service (DBS).
This twiki describe in full details how to access the quality flags from DBS.
Also, you can profit from a help page on the DBS web site. Many more details are in the DBS Users' Guide.

DBS Data Discovery page is depricated now. One must use the Data Aggregation Service (DAS)

Review status

Reviewer/Editor and Date (copy from screen)	Comments
JohnStupak - 4-June-2013	Review, minor revisions
NitishDhingra - 01-Apr-2012	Complete review, minor changes.