Muon DOC3: Data Certification Shift Instructions

Introduction

The task of the Muon DOC3 shifter is to certify:

• Collision runs for both PromptReco and Express datasets
• Cosmic runs for PromptReco dataset (unless CMS explicitely asks to certify the Express dataset also for Cosmics)

• Runs from the Express dataset must be certified day by day for the whole shift period, as soon as they become available
• PromptReco runs instead, have to be certified before an internal deadline set, each week, on Saturday night ^[1][2]

Muon DOC3-certification tutorial

^[1] Please note that the Muon POG certification starts from the outcome of the Muon subsystems one. Therefore the deadline on Saturday is set to give Muon POG enough time to complete the certification task before the final deadline from the CMS Data Certification team (Monday morning).

^[2] The central list of PromptReco runs to be certified is circulated by the DC team on Friday, so you will have just one day to complete the certification. Therefore we strongly suggest to certify the PromptReco runs day by day as soon as they become available.

Please take some time to look at the slides that were presented at the last Muon DOC3-certification tutorial for further details about the workflow of the certification.

Before your first shift

• In order to access the online tools, you need a GRID certificate installed in your browser (instructions about how to install it are available in this TWIKI).
• The first time you connect to the Run Registry (RR), you need to request the shifter role for the CSC, DT, RPC, GEM workspaces if you already don't have one:
  • If you don't see "(SHIFT)" next to your name at the top right of the RR page, then:
    1. go to Tools -> My Roles and, if the table is empty, then:
    2. click on New role,
    3. in the Role field select SHIFT,
    4. enter something in Location (it doesn't seem to matter much what that is),
    5. enter a Reason (e.g. Muon DOC3 shifter),
    6. enter an Expire date,
    7. check the Receive emails flag,
    8. click Submit.
  • You must repeat the procedure from within each of the CSC, DT, RPC, GEM workspaces.
  • The experts are contacted automatically and will grant you SHIFT permissions ^[1].
• Please subscribe to central data certification team hypernews email notifications hn-cms-data-certification@cernNOSPAMPLEASE.ch as well as hn-cms-commissioning@cernNOSPAMPLEASE.ch (through hypernews.cern.ch).
• Subscribe also to the Shift Leader Elog ( CMSOnline-->Elog-->Config-->General > Shift leader).
• Please attend the Muon DOC3 certification reports at the General Muon Meeting at least 1 month in advance before the beginning of your shift, in order to be aware of the latest findings.
• During your training shift, contact the main Muon DOC3 Shifter and certify a subset of runs under her/his supervision [2]. Try to get as familiar as possible with the tools. If you have any doubts contact the experts.

^[2] You can retrieve the name of the main shifter from past elogs or from CMSOnline-->Shiftlist-->Monthly view (be sure that you selected "Muon" as System and "Certification" as Shift Type).

Subsystem contacts

Here you can find the list of people to contact in case of doubt or if you need assistance:

DOC3 manager

1. Caterina Aruta, email: caterina.aruta@cernNOSPAMPLEASE.ch

RPC experts

1. Andres Leonardo Cabrera Mora (CERN phone: 160947), email: a.cSPAMNOT@cernNOSPAMPLEASE.ch
2. Mehar Ali Shah (CERN phone: 165251), email: mehar.ali.shahSPAMNOT@cernNOSPAMPLEASE.ch
3. Roumyana Hadjiiska, email: roumyana.mileva.hadjiiskaSPAMNOT@cernNOSPAMPLEASE.ch

DT experts

1. (Offline DQM & Certification) email: juan.rodrigo.gonzalez.fernandezSPAMNOT@cernNOSPAMPLEASE.ch
2. (Offline DQM) email: javier.fernandezSPAMNOT@cernNOSPAMPLEASE.ch
3. (DPG Coordinators) email: cms-dpg-conveners-dtSPAMNOT@cernNOSPAMPLEASE.ch
4. (Run Coordinator) email: gianni.masettiSPAMNOT@cernNOSPAMPLEASE.ch

CSC experts

1. CSC DPG Coordinators, email: cms-dpg-conveners-cscSPAMNOT@cernNOSPAMPLEASE.ch
2. CSC General Hypernews, hn-cms-csc-opsSPAMNOT@cernNOSPAMPLEASE.ch

The CSC hypernews is the general one for all CSC issues, and reaches all experts including those for DQM, DC, DPG, and Operations. It is the best place to post for a fast response for any CSC-related issue at all.

GEM experts

1. GEM DPG Coordinators, email: cms-dpg-conveners-gemSPAMNOT@cernNOSPAMPLEASE.ch

Daily tasks

The central certification team communicates through the hypernews hn-cms-data-certification@cernNOSPAMPLEASE.ch. Please check this HN daily and keep in mind that the list of PromptReco runs to be certified is circulated in this HN on Friday. Please use this HN only to gather input from the CMS Data Certification team, in case of doubts, contact the experts from the list above.

The workflow of the Muon DOC3 certification shifter is:

1. Check what runs are available for the certification (both in Express and PromptReco), through the web pages listed below and the Offline RR.
2. Inspect the quality of the data for each run by looking at the histograms in the Offline DQM GUI.
3. Set the quality flag for all the muon subsystems in the RR for each inspected run (both in Express and PromptReco)
   • As a general rule, never mark a run as BAD before receiveing a feedback from subsystem experts.
4. Once per week, for the list PromptReco runs already certified, check the list of lumisections flagged as BAD for the muon subsystems following the instructions below.
5. Upload an elog (daily and weekly) with the list of runs certified and a short description of problems/observations (if any).^[1]
6. Upload an elog (weekly) with the list of the BAD lumisection (if any).

Step 1,2,3 should be performed for DT, CSC, RPC, GEM.

^[1] Try to be as detailed as possible. This e-log will be carefully read by the Muon DPGs and used by Muon POG for the data certification. Your input to them is really important.

Details for each step are given below:

Runs to be certified

Daily, look at runs to be certified appearing in these pages:

• Summary of the Certification Status - PromptReco Collisions18
• Summary of the Certification Status - Cosmics18
• Summary of the Certification Status - Express Collisions18

Setting the quality flag in the Run Registry

For each system workspace (CSC, DT, RPC, GEM), and within the Run Registry:

• use the Dataset Name filter to select the Express or PromptReco dataset for a given run, according to the type of certification that you are performing,
• use the Dataset State filter to select runs in OPEN state,
• once you select one line:
  • Lumi Sections allows you to check the detector status during the data taking,
  • you can use Manage to change/set the quality flag for a given run,
    • As a general rule, never mark a run as BAD before receiveing a feedback from subsystem experts.
  • you can use Move to change the status of a run from OPEN to SIGNOFF once you're done.

Inspecting the run quality with the Offline DQM GUI

Open the Offline DQM , and follow the instructions for the subsystems ( see below) to inspect the histograms. The following statements hold for the certification of each of the 3 Muon subsystems:

• To certify a run belonging to the Express dataset, choose the /StreamExpress/RunYEARXX-Express-vY/DQMIO dataset in the Offline DQM GUI
• To certify a run belonging to the PromptReco dataset, choose the /SingleMuon/RunYEARXX-PromptReco-vY/DQMIO dataset in the Offline DQM GUI
• Carefully inspect the histograms for each run (listed in the webpages above), subsystem (CSC, RPC, DT, GEM) and dataset (Express, PromptReco)
• The goal is to assign the quality flag in the RR and move the status to SIGNOFF for each inspected run.
• Be aware that the instructions and guidelines for the inspection of the distributions given in next sections, mainly refer to PromptReco.
  • They are valid also for the Express dataset, but in this case some distributions can slightly change from the reference ones.
• As a general rule, never mark a run as BAD before receiveing a feedback from subsystem experts.

Check the list of BAD lumisections

Even if, from the inspection of the run-based plots in the Offline DQM, a given muon subsystem seems performs properly, it could happen that a subset of the lumisections of a given run were affected by problems. Single lumisections are marked as BAD on the basis of status flags computed exploiting information about the subsystem HV accessed from the DCS. These flags are propagated automaticaly to the RR and used for the generation of JSON files.

It is important to monitor that the automatic procedure was performed correctly and to identify false positives/negatives in the workflow in case they happen. To accomplish this task, you are requested to run a script that retrieves information about lumisection that are BAD, for each of the muon subsystems, in a given run range.

To install the script and its environment, follow the instructions below.

Show installation instructions. Hide installation instructions.

cmsrel CMSSW_10_1_5
cd CMSSW_10_1_5/src
cmsenv
git clone https://github.com/battibass/dpgByLumiFlag.git
cd dpgByLumiFlag

python dpgByLumiFlag.py --minRun=315506 --maxRun=315801

[... several lines from JSON creation ...]

***************************************
List of luminosity losses exclusively due to muon DPGs DCS flags:

DT :
{}

CSC :
{"315713": [[360, 373], [386, 399]]}

RPC :
{}
***************************************

The script provides you a list of bad lumisections for DT, CSC, RPC.

Once per week, when the JSON files for PromptReco are announced (in the hn-cms-data-certification@cernNOSPAMPLEASE.ch HN) you have to:

• Run the script, adjusting the --minRun and --maxRun options according to the range of runs in the announcment.
• Append the last part of the script output (the one between the asterisks) to the weekly elog entry corresponding to that run range (it can be from you or from the revious shifter).

Muon subsystems DPGs responsibles will check this list and take actions if needed.

Elog usage

Please use regularly the Muon Certification ELOG to summarize the results at the end of the day/week, report problems and ask questions to the experts.

• At the end of each day of shift, compile a daily report.
  • Please indicate clearly in the elog the "flavour" of runs certified: Collisions-PromptReco, Collisions-Express, Cosmics.
  • This elog will be checked by Muon DPGs experts and used as a starting point by POG certifiers.
    • It is important to compile this report every day as POG certification won't start until DPG one is performed.
  • Try to follow the skeleton given below. Always include all the relevant information and observation.

Daily Summary 
Collision Runs:Prompt Reco , Express 
Cosmic Runs: 

Bad Runs: none / see the list of bad runs

BAD Collision Runs:
    Run - reason / link to elog
BAD Cosmic Runs:
    Run - reason / link to elog

Other Comments:

• At the end of the each week of shift compile a weekly summary report with the list of runs certified.
  • For runs marked as bad, include a small description or put the short link of the e-log where you describe the problem.

• When the JSON files for PromptReco are announced in the hn-cms-data-certification@cernNOSPAMPLEASE.ch HN:
  • Append to the appropriate weekly report (from you or from the previous shifter) the status of the BAD lumisections following the instructions above.

• In case of problems or doubts, create an elog entry describing the issue. Choose the appropriate elog type from the menu and assign a relevant subject. Then, in order to receive prompt feedback, contact privately the experts by email. Don't forget to update the elog with the outcome of your discussion with the experts.

Setting the CSC quality flag

Please see the CSC instructions for minimal data certification twiki page.

Setting the DT quality flag

Please see the DT instructions for data certification.

Setting the RPC quality flag

Please see the RPC instructions for the data certification

Setting the GEM quality flag

Please see the GEM instructions for the data certification twiki page - IN PROGRESS.

Reporting shift results

At the end of your 2 weeks of shift, you have to give a report to the General Muon Meeting (GMM), summarizing the list of certified runs and the problems you faced (if any). Please always look critically at the distibutions in the offline GUI and report any discrepancy (even if small or apparently negligible) with respect to the expected trend reported in the twiki.

What you have noticed could be relevant for the experts!

Please indicate clearly in the elog the "flavour" of runs certified: Collisions-PromptReco, Collisions-Express, Cosmics-PromptReco.

In addition, you are requested to include in your presentation:

• The downtime plots for the period covered by the PromptReco runs that you have certified.^[1]
• The output of the script listing the BAD lumisections for the run ranges corresponding to the JSONs annouced while you were on shift.

The first DOC3-certification report is linked here to give you an example of report.

^[1] In order to produce the downtime plots please follow the instructions below:

• Go to the fill analysis page on WBM.
• Fill the Begin date and the End date fields with the corrensponding information for the first and the last run that you have to certify.
  • You can retrieve those information using the RunSummary page on WBM.
  • Just insert the run number in the corrensponding field and press the Submit button.
  • All the infos of that run will be displayed, just take note of the start time of the first run and stop time of the last run.
• Include in your presentation the Downtime and LumiLost charts by category
• What we want to infer from these plots is if the muon system is responsible to the loss of some data.
  • If you scroll down the page, you will find a table with more details.
  • If any of the muon detectors is present in the Category column, you can click on the corrensponding Group (the column on the left).
  • Another table will appear. In the column category details of such table you can search for the muon detectors and find the error/problem that affected a given subystem.

Frequently Asked Question

In case of doubts, please check first the FAQ twiki page

Useful Tools

The CMS WBM run summary interface gives you the possibility to retrieve information about a given run that you provide as an input. From here you can retrieve information about the

• total number of events collected in that run, from the " L1 Triggers" raw of the table. This number is usually different from the number of events that appear on top of the DQM GUI that shows instead the events passing the Single Muon triggers
• the total duration of a run
• the run flavor (cosmic, pp collision, special run, ..)
• the number of events in the Single Muon dataset, from the Dataset Summary link on the right of the page. Also accessible from here

In addition, for a problematic run, could be useful to check the information provided by the shift leader and/or trigger shifter during the data taking. In order to do this, you can search the run number in the shift-leader and/or trigger shifter elogs.

-- CaterinaAruta - 2021-04-30