Offline DQM short term instructions

General offline shift Instructions -- Shift histogram description by subsystem

Goals of this page:

Here are some important short-term notes/instructions for current reconstruction conditions. Notes here are supposed to represent temporary changes with respect to the normal instructions the shifter was given. In case of doubt, contact the subsystem expert.

  • Shifters
    • The instructions in this page supersede the Offline Shift instructions.
    • If the dates in which the instructions were written are very old, suggest to the subsystem expert to move them to the general Offline Shift instructions or remove them.
  • Experts
    • When adding/updating instructions, always update the date and your name.


General short term instructions
(last updated: Aron Soha/Kaori Maeshima 10th August 2012)

Attention :

Current Offline DQM shifts mode is back to Normal = come in on time and stay for entire shift unless instructed otherwise.

Total of 304 runs from 13July2012AB ReReco data are ready to be certified. Following is the certification procedure for the offline DQM shifts for all locations (CC, DESY, FNALROC).

  • Certify any Prompt dataset collision runs (current runs) awaiting for you when you arrive at your shift, first.
  • Certify any available Cosmic runs, next.
  • Then, certify 13July2012AB ReReco runs, until the end of your shift.
    • DESY shifts: Start with the first run (run 190456) of 2012A data and certify the runs in assending order.
    • CERN-CC-shifts: Start with the first run (run 193833) of 2012B data and certify the runs in assending order.
    • FNALROC-shifts: Start with the last run (run 196531) of 2012B data and certify the runs in descending order.
    • Be sure to write in e-log, clearly which runs are certified, in the shift summary.
    • More instruction regarding ReReco data certification can be found just below, in this page.
  • Any collisions and cosmic runs which became available during your shift will be certified by the next offline DQM shift person.

Collision data 2012

For Collision12 runs use ONLY the Prompt Reconstruction data. Please follow the relevant documentation here. Please keep following points in mind

  • The dataset name in Run Registry /PromptReco/Collisions2012C/DQM
  • Collision (and Cosmic) runs will now automatically move from the waiting list to the datasets table, so you no longer need to monitor the waiting list
  • Certify the runs and SIGNOFF them when you are done
  • Consider Collision runs with highest priority wrt the Cosmic runs
  • There is no need to certify Express dataset, they will no longer be automatically moved from waiting list table to datasets table
  • No longer certify Commissioning runs

Instructions for Certification of 2012A&B ReReco

  • The GUI to be used is here
    • Please do not confuse this with the usual Prompt DQM GUI for collision data or the DQM GUI that you use for cosmics.
  • To log the certification results use the Run Registry
    • Filter in the dataset name column by entering " like %2012AB-13Jul% " and locate the next run that is still OPEN in the sequence for you locaiton:
    • Full dataset name of 13July2012ReReco in RR is called: /Run2012AB-13Jul2012ReReco/DQM
    • There are 109 2012A runs and 195 2012B runs, total of 304 runs.
  • Run assignments in each locations are as follows:
    • DESY shifts: Start with the first run (run 190456) of 2012A data and certify the runs in assending order.
    • CERN-CC-shifts: Start with the first run (run 193833) of 2012B data and certify the runs in assending order.
    • FNALROC-shifts: Start with the last run (run 196531) of 2012B data and certify the runs in descending order.
  • Below are complete list of 13July2012ReReco runs:

2012A runs: 190456 190459 190462 190465 190482 190490 190491 190492 190538 190539 190591 190592 190593 190595 190645 190646 190659 190661 190662 190663 190678 190679 190688 190702 190703 190704 190705 190706 190707 190708 190710 190733 190735 190736 190738 190782 190895 190906 190945 190949 191043 191046 191056 191057 191062 191086 191090 191201 191202 191226 191247 191248 191264 191271 191276 191277 191367 191387 191391 191397 191401 191404 191406 191411 191419 191421 191424 191426 191691 191692 191694 191695 191697 191700 191701 191718 191720 191721 191726 191800 191810 191811 191830 191833 191834 191837 191839 191842 191845 191849 191856 191857 191858 191859 193093 193112 193116 193123 193124 193192 193193 193207 193334 193336 193541 193556 193557 193575 193621

2012B runs: 193833 193834 193835 193836 193878 193898 193913 193917 193919 193922 193925 193928 193998 193999 194027 194050 194051 194052 194075 194076 194108 194115 194117 194119 194120 194150 194151 194153 194199 194210 194223 194224 194225 194270 194303 194304 194305 194314 194315 194317 194424 194428 194429 194439 194455 194464 194479 194480 194533 194619 194631 194643 194644 194691 194699 194702 194704 194711 194712 194735 194778 194788 194789 194790 194825 194896 194897 194912 194914 194915 195013 195014 195015 195016 195099 195109 195110 195111 195112 195113 195114 195115 195147 195163 195164 195165 195251 195265 195266 195303 195304 195378 195379 195385 195387 195390 195396 195397 195398 195399 195522 195523 195527 195528 195529 195530 195540 195551 195552 195633 195634 195644 195645 195647 195649 195655 195656 195658 195749 195757 195758 195774 195775 195776 195841 195868 195913 195915 195916 195917 195918 195919 195923 195925 195926 195927 195929 195930 195937 195947 195948 195950 195963 195970 196019 196023 196027 196046 196047 196048 196090 196096 196097 196098 196099 196103 196107 196122 196131 196135 196197 196199 196200 196202 196203 196218 196239 196249 196250 196252 196334 196349 196350 196351 196352 196353 196357 196359 196362 196363 196364 196430 196431 196432 196433 196437 196438 196452 196453 196458 196495 196497 196501 196509 196531


  • There has been some confusion about dataset names. Please note that the correct dataset name is now automatically assigned when a new offline dataset is automatically created in the RunRegistry. That dataset name should match the information in the table here. Also note that this dataset name is NOT the same as the input "Primary Dataset" name that was used for Prompt or Express reconstruction and appears in the DQM GUI.

  • Place holders for the components Tau, BTag, and Lumi were recently added to Run Registry but instructions are not yet in place, so please ignore these for now.

  • There are four runs (191306, 191350, 191393, 191394) taken with Vdm scan with special HLT menu and corresponding dataset names will be different. These runs would require slightly different prompt reco certification procedure. So we propose to certify these two runs here at CERN tomorrow morning. Prompt Reconstruction for these two runs will be faster.

  • We expect to have some Collision12 runs taken with special HLT menu. Those runs will be certified at Meyrin-CC. The list will be posted here. Currently there there just two runs
    • Fill 2574 : Runs 192989, 192990 (taken with VdM menu and Stream A rate was zero)
    • Fill 2576 : Runs 193091 (with Vdm Menu), 193092 (taken with Low Pile Up menu)
    • NB : Run 193093 taken with physics menu 5e33 and can be certified with standard procedure
  • Collision runs taken with lower magnetic field (3.0Tesla wrt 3.8Tesla) are listed bellow
    • Fill # 2608 : Run 193878
    • Fill # 2609 : Runs 193928 193925 193922 193919 193917 193913 193912 193905 193904 193898
    • Histograms (specially the Tracking ones) may look different
    • CSC and HCAL were out most of the times during these runs
  • VDM scan runs taken on 19th July
    • Fill # 2853 : 199275 199276
    • Fill #2855 : 199282
    • Fill # 2856 : 199306
    • These runs will be certified at Meyrin-CC using the offline GUI with data set /ZeroBias1/Run2012C-PromptReco-v2/DQM

Automatic Run Registration in the Run Registry Is Again Working
  • However, if runs do appear for the Prompt data in the DQM GUI but do not automatically appear in the list of runs to be certified in Run Registry, please follow the below instructions:
    • Collision Runs : if histograms exist for all sub-systems in the PromptRecoGUI register the run using Create Dataset button of the "Waiting List" table in Run Registry
    • Cosmic Runs : if the run is available in the GUI for the dataset /Cosmics/Run2012C-PromptReco-v1/DQM register the run using Create Dataset button of the "Waiting List" table in Run Registry
    • Proceed with run certification after the registration
    • At the end of your shift, make an elog entry of Type Problem Report and a subject of "Manual registration of runs", listing in the message body all runs you had to register by hand.

CASTOR instructions
(last updated: Virginia Azzolini/Ilaria Segoni - 12 Mar 2010)

No Offline CASTOR Certification possible yet, copy the Online quality flag.

CSC instructions
(last updated: Greg Rakness - 27-July-2011)

THE most important thing to evaluate CSC is the recHit occupancy plots. These are the crucial cross-check that the CSC Summary plot is correct. To access them...

  • Workspace = CSC

An example of good distributions can be seen below. Use your brain. If you see problems with this distribution, CONTACT THE CSC DOC.

Error: (3) can't find 167913_streamexpress_rechits.png in Sandbox

Descriptions of other plots in the CSC summary section can be found at DQMShiftCSC...

DT instructions
(last updated: Antonio Branca - 03-August-2012)

Expert: Antonio Branca

  • NEW: the '03-EfficiencySummary' map is debugged and available again in the shift workspace, so it has to be used for validation.


  • Please check the DT read-out status in the 'ROChannelSummary' histo (Workspace DT->00-ROChannels-> ROChannelSummary); if you see many (> ~5) sectors RED, then mark the run BAD for the DT and inform the expert.

  • If you are validating runs taken with particular beam conditions, as e.g. low beam current, small number of bunches or so, but also Heavy Ions runs , this can affect some DT plots (e.g 00-SegmentOccupancySummary and 03-EfficiencySummary could show a higher number of red sectors, or 01-SegmentReso-Mean ans 02-Sigma peaks can be broadened and heavily shifted). In these peculiar conditions, please relax the requirements to judge the plots, applying looser criteria. In any case please check the Read Out and the HV status (see below)

  • In very short runs, be aware that occupancy map can suffer for low statistics, and likely to appear with many sectors red or empty; please DO NOT set DT as BAD for that. In the statistics is really small and many histos are empty, you could suspect the DT HV were not even ON; in order to verify that you can:

  • check the DT HV status through the plot HVGlbSummary in the DT->EventInfo->DCSContents folder, or althernatively in the by-LumiSection table in the run registry. If DT were off for most part of the run, mark the run BAD for the DT. The status can be monitored also on the 'reportSummaryMap' histo (Everything->Info->EventInfo->reportSummaryMap) for the HV. In case of mismatching, trust the HV DQM plot or the run registry, but report the case to the expert. If the online flag is BAD because DT were OFF(standby), you can replicate it, but please perform these checks anyway.

  • If you are validating Cosmic runs, the histograms of mean and sigma of segments' residuals could appear quite out of place, not fulfilling the requirements about the range; DO NOT set DT as BAD because of them, and rely only on Read-Out test.

ECAL barrel / endcap instructions
(last updated: Emanuele Di Marco - 7 May 2011)

  • ECAL Barrel:
    • The ECAL Barrel Summary Map will have an occasional red tower, but the majority will be green with a few orange towers. There is also normally a little extra noise in the ECAL Barrel in EB+4 to EB+12. Below is a typical image of the ECAL Barrel Summary Map and ECAL Barrel Pedestals.
    • Due to pileup in 2011 runs, the high eta region in barrel shows inclreased noise. Consider the following situation as normal until thresholds are updated.

  • ECAL Endcap:
    • EE reportSummaryMap is currently blacklisted. The situation shall be fixed ASAP. To certify the EE plots in the mean time, please visit the shifter layout of EcalEndcap (workspace Shift > EcalEndcap) and check that the plots 01, 02, and 03 are all green or dark red (like the example below).

    • Due to the pileup, the noise is very increased in the high eta regions. Consider the following situation as normal until the thresholds are updated.

ES instructions
(last updated: - )

No major difference wrt previous round of certification due to reconstruction changes. Keep the same flags as before

HCAL instructions
(last updated: Abideh Jafari - 17 May 2012)

Important when certifying HCAL use only Collisions datasets (promptReco or ReReco), HCAL does not certify Cosmics datasets. For Cosmic Data Sets please copy the online flag.

Important The "Problem Digi Rate " histograms are RED for HF. The run should still be marked as "GOOD". Data format has been recently changed in HF, and change is being implemented in DQM code.

The following shape is seen for ProblemDigis plot (noticed since run 194050):

Important As indicated on the picture, these red spots are known, confirmed with Hcal experts. If other things are fine, DON'T mark the run as BAD until furthur notice. Talk to the expert if you are in doubt.

* HCAL-DeadChannels-194050.png:

To certify recent HCAL runs, the following summary values in "HCAL Report Summary" map (shown below on the left) must be true to judge a run as "GOOD", otherwise bad:

  • HB>0.98;
  • HE>0.98;
  • HF>0.98;

If any of the above values is bad, look at Certification Map (shown below on the right) in DQM to identify the cause of the failure.

reportSummaryMap.png CertificationMap.png

Below: The Raw Data, Hot Cell, and Dead Cell HCAL monitors show only low-level (green) or known (white) problems. No problems are seen in HF (|ieta|>=29).


HLT instructions
(last updated:Ben Kilminster - 16 March 2012)

  • (16 Mar 2012) Please copy the flag from the online DQM.

L1T instructions
(last updated: Dimitrije Maletic - March 2010 )

  • (24th of March) Don't look at "L1T Report Summary Map" until further notice.The plot in not refreshing for Offline shifters.
  • (Feb. 22nd) Please check the status of Muon Detector (which part is excluded in a run). For example, if the bottom of the GMT is empty, the part would be excluded in the run. Check trigger configuration, DAQ setup, Detector status.
  • Until further notice, please ignore "RED" in FED Report Summary Map in Summary layout. It is caused by excluding some muon subsystem.
  • All emulator bins (right-hand column of L1T Report Summary Map) should be empty (white) until further notice.
  • For the data bins, Jets, TauJets, IsoEM, NonIsoEM, and MET should all be empty (white) until further notice.
  • Also, check the L1T histograms 04 Global Muon Trigger DT-CSC BX, 05 Global Muon Trigger DT-RPC BX, and 06 Global Muon Trigger CSC-RPC BX. The red maximum should be at 0 for DT and CSC. For RPC, it should also be at 0 for cosmics but at -1 for collision data (this is due to the HSCP triggering algorithm for RPC, by which the RPC trigger is advanced by 1 bunch crossing and widened to 2 bunchcrossings; this algorithm is activated only for collisions, not for cosmics). If it is shifted (usually on two histograms, because of the timing problem in CSC, DT, or RPC) the TGMT (muon trigger) group should be notified, and L1T should be marked BAD .
  • If in doubt, call the L1_DOC (this is the 24/7 TPG on call expert) at 161958.

No major difference wrt previous round of certification due to reconstruction changes. Keep the same flags as before

L1T emulator instructions
(last updated: Nuno Leonardo - Feb 2010 )

The emulator based dqm results are not available offline temporarily, given the offline reco+dqm workflow currently continues excluding the required emulator sequence. A resolution for this is awaited, Trigger and Offline management advised.

Pixel instructions
(last updated: Abideh Jafari 17 May 2012)

The low fraction reported for Barrel Pixels on the Report page as well as the red bins (Ndigis, DigiCharge, NClusters) for the Barrel in the reportSummaryMap plot on the same page are because the new cut values based on 8 TeV collisions used for the automatic certification have not yet been picked up offline. The current report summary map, shown below, is Ok for pixels, so do not mark the run BAD for Pixels because of these. Additionally, if a category besides Barrel Ndigis, Barrel DigiCharge and Barrel NClusters do show up as red, please check if this is due to a systematic shift in the distribution, or caused by a subset of channels. Due to the rapidly-changing luminosity conditions, we may see shifts in the overall value of some distributions. As long as the behavior exhibits the same trends as the good distributions, these should be still marked as GOOD for pixels. In case of questions, please email Dan Duggan ( Error: (3) can't find PixelreportSummaryMapOffline.png in Sandbox

Also, please do not mark runs bad due to the following issue:

  • Three blades showing higher than expected charge (and ADC counts) in the endcap. This does not affect the quality of the run.
Error: (3) can't find BAD_SUMOFF_charge_OnTrack_Endcap.png in Sandbox
  • Another blade (around 20) is added. Don't mark the run as bad.

RPC instructions
(last updated: Anna Cimmino - 19th April 2012)

The RPC histogram on the Summary Work space IS NOT A MEASUREMENT OF DETECTOR EFFICIENCY, nor is the percentage written on top of it. Do not refer to this number with the term efficiency. When referring to this number in an ELog or in the RR always call it Report Summary.

For RPC data certification, the preferred dataset is SingleMu.

Oncall phone is NOT available. In case of problems, please send an email to:

SiStrip and Tracking instructions
(last updated: Tomas Hreus - 03 Apr 2012)

  • Collisions: Reference run (177790) is taken with Tracker in deconvolution mode. On Wednesday (4th April) the first stable-beams fill will be taken with tracker in a peak mode. Because of this, on-track cluster signal-to-noise plots might be shifted wrt reference run, this is expected.
  • Cosmic runs: SiStrip Report Summary map and the Good Detector Fraction map can show yellow or orange areas in some layers. Please do not mark the run bad for strip or tracking just because of this (but please put anyway a comment in the RunRegistry)

E/Gamma POG instructions instructions
(last updated: Suchandra Dutta - Apr22nd 2012)

  • There is problem in the layout and many of the histograms are not visible in the GUI. So for the time being the E/Gamma flag should be set in terms of the Pizero Invariant Mass distribution and other subsystem flags as follows:

ECAL & SiSTRIP & Pixel = GOOD ==> EGamma = GOOD
ECAL || SiSTrip || Pixel = BAD/EXCL ==> EGamma BAD (which is in anycase a standard rule for BAD runs)

  • The E/Gamma POG offline DQM is not as vital for cosmics running as it will be for collisions.
    For cosmics running, BAD should be applied for runs with high statistics (>50M) in which the shift histograms differ greatly from the reference histograms. GOOD should be applied in any other case.

Jet/MET POG instructions
(last updated: June 19 2012)

  • IMPORTANT: JetMET only certifies collision runs, Cosmic runs do not need to be certified.

Instructions for pp collisions. Basically follow the standard instruction at this twiki.

Important : please note that the JetMET histograms are filled only for those lumisections where ECAL and HCAL are all ON (for CaloMET and CaloJets); and ECAL/ES/HCAL/PIXEL, STRIPS, and MUON are ON for JPT, TcMET and PFJets and PFMET. Additionally, all events are required to contain at least one good primary vertex.

  • IMPORTANT: If your certification results change wrt to previous rerecos or express stream, PUT a comment why (even if GOOD ) and send an email to the DQMShiftOfflineJetMET experts, so they are aware.
  • If plots are missing: send an email to the DQMShiftOfflineJetMET experts, so they are aware.

If you see something weird there and you want to investigate further, you can also check JetMon (for low pT jets less prescaled distributions) or SingleMu PD (for event with less intrinsic MET).
  • If ECAL = EXCL || HCAL = EXCL, mark the run BAD.
  • If ECAL = BAD || HCAL = BAD: we do not automatically mark runs BAD even if ECAL, HCAL etc consider runs as bad. Check the JetMET plots! But, runs with test such as B-field off, Time scans should be marked as BAD. It's too risky to use them for physics analysis.
  • If you see a Hot Spot in the Calo Tower plot, mark the run only BAD if you see this effect also in other shift plots, otherwise GOOD. The same applies if there are white holes (HCAL or ECAL parts missing in the read out) or cold regions in the CT plot, and you see a significant effect on the jets/MEt distributions, mark the run BAD (an effect visible in the CT plots only is not enough to mark a run BAD). We realize "significant effect" is sort of a subjective statement, let's say it is something (a hole or a spike) clearly visible in a Jets or MET distribution, see the JetMET histogram description for some examples.
  • Some discrepancies with the reference histograms can be observed since a new release is used for data processing. The typical example is the reduction of the MET tail.
  • You may observe discontinuity in jet pt distributions, but they may be due to different triggers with different prescales. Do not mark those runs bad automatically..
  • You will find that tcmet and pfmet plots are empty in some runs or spikey, but do not mark those runs bad automatically. If the calomet plots look ok, please keep the runs GOOD.

temporary instructions FOR HEAVY ION RUNNING, for the moment (Nov 15) they superseed all the others

* perform the certification on the HiHighPt PromptReco PD

* if ECAL or HCAL are EXCLUDED from data taking, mark the run BAD

* you only have to check two things: a) CaloTower plots and b) CaloJet plots

a) you can check the CaloTower plots as usual in the "Shift" workspace, directory JetMET. We should mark the run BAD only if we see evident and significant spikes or holes in these distributions.

b) to check the CaloJets plots, you have to go into "JetMET" workspace (not "Shift"), under the POG workspaces, click on "Jet", then "AntiKtJets". Check especially the Eta, Phi and Eta VS Phi distribution. We should mark the run BAD only if we see evident and significant spikes or holes in these distributions.

end of instructions for heavy ion running.

Muon POG instructions
(last updated: Suchandra Dutta - 15 March 2012)

Track POG instructions
(last updated: Mia Tosi - 6 April 2012)

Shor term instructions:
  • If the SummaryMap is showing the RecHits part in YELLOW but the tracking distributions are as expected, sign the run as GOOD and ADD IN THE COMMENT THIS BEHAVIOUR
  • In the Tracking layout there are 3 histograms without reference, it will be fixed soon. The involved distributions are:

General remarks
  • COSMIC RUN: check the report summary map to define the flag GOOD or BAD
  • COSMIC RUN: some difference can be shown in the ETA and/or PHi track distributions: do not flag the run as bad since this can be related to different trigger condition during MWGR
  • CRAFT12: reference run has B-field ON. Major difference in track pT distribution (such as the one shown below) means B-field value was not correctly propagated into offline reconstruction and B=0 T was wrongly assumed. Please nofity ORM.

Review Status
(last updated: -- VirginiaAzzolini - 24-Feb-2009)

Reviewer/Editor and Date Comments
Paolo Bellan - 29 Jun 2010 DT instructions reviewed
ValdasRapsevicius - 15-Mar-2010 Removed outdated entry - CSC offline DQM is fine now.
VirginiaAzzolini - 24-Feb-2010 Reviewed for outdated entries before start up.
Ilaria Segoni - 27 Jan 2010 updated instructions relative to Tracker HV, Jan23rd rereco, cleaned up obsolete part of General section
Virginia Azzolini - 28 Oct 2009 added instruction for Reprocessed Runs certification
Volker Adler - 14 Aug 2009 Created new page and entered first entries for switch from /Cosmics to /ExpressMuon
Volker Adler - 19 Aug 2009 Reviewed for outdated entries before general clean-up.

  • EESFT_EE_-_front-end_status_summary.png:

-- KaoriMaeshima - 10-Aug-2012

Edit | Attach | Watch | Print version | History: r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r1 - 2012-08-10 - KaoriMaeshima
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Sandbox All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback