Validation of HLT code

trigger validation e-space

Latest: HLT All
Releases up to 32x documented in CMS.HLTTriggerMenuValidation.
Summaries for Trigger L1 in SWGuideValidationTableL1.
Summaries of other CMS systems in SWGuideValidationTable.

Subsystems

HLT validation subsystem responsibles in ValidationContacts.

List of subsystems performing validation checks: Electron, Photon, Muon, Jet, Met, Tau, Top, Higgs, Heavy flavor, Susy, Exotica.

All subsystems: Electron, Photon, Muon, Jet, Met, Tau, B-jet, Alignment, Calibration, Forward physics, Top, Higgs, Heavy flavor, Susy, Exotica, Electroweak, QCD.

Automated validation

Guidelines for developers are described in SWGuideTriggerSoftwareValidation.
Including: (i) dqm sources, (ii) dqm clients, (iii) dqm gui: layouts, quality tests, renders.

Validation results are shown in the DQM Offline GUI. To view them:

  1. Install grid certificates (one time)
  2. Open the GUI
  3. Click on Run #
  4. In the search box, enter relval ttbar 3_3_0
  5. Click on Any and confirm the dataset
  6. In Workspace, select HLT
  7. Navigate to the subsystems' report layout
  8. Compare releases, by clicking on the cms logo and selecting references

Validation GUI

system:

dataset:

folder:

Reference

disable enable

position:

dataset:

Legend

  • No : Validation done, result is "Not Validated". Report the resons below
  • Yes / Done : Validation done, result is "Ok, Validated"
  • Work in progress, under construction : Validation still in progress or results need to be followed up before final green/red light
  • Minus : Validation not (yet) done
  • n.s. : Needed validation (relval) samples not available
  • report : click on icon in table for link to the associated detailed report in the hlt forum
  • fastsim references: corresponding fullsim results in the same release
[feedback on possible alternative table formats]

Readiness of Validation Code

To be considered "at full readiness," the validation code needs to either

  • Reliably produce filled histograms that accurately reflect the trigger behavior in the DQM GUI OR
  • Reliably produce results that can be quickly analyzed using a standalone package
To be considered "not at full readiness," the validation code is present, but either
  • The histograms are not reliably filled in the DQM GUI, and/or they do not accurately reflect the trigger behavior OR
  • The results are produced by standalone packages, but these packages take a non-negligible amount of time to run
There is a further category for packages that currently lacking active development work

Legend for Readiness Table

  • No : Package not at full readiness AND no development work underway
  • Yes / Done : Package is at full readiness
  • Work in progress, under construction : Package not at full readiness
  • Yes / Done/Work in progress, under construction : Package generally at full readiness and can be used for results, but some technical issues being worked on

  Group
Electron Photon Muon Jet MET Tau Top Higgs HeavyFlavor Susy Exotica
Work in progress, under construction Work in progress, under construction Yes / Done Yes / Done/Work in progress, under construction Yes / Done/Work in progress, under construction Yes / Done/Work in progress, under construction No No Yes / Done Yes / Done Work in progress, under construction

Details of Readiness Table

  • Electron/!Photon: Validation currently relies on standalone analysis to produce results, results take some time to obtain.
  • Muon: Upcoming work on L1 propagator, protection of module labels in configuration for automatic path recognition. Results can be used in the meantime
  • Jet: Some empty histograms, active development to address this
  • MET: Same issue as jet. Stadalone code exists as backup to provide validation results
  • Tau: Generally ready; most recent RelVals only in DQM tier, RECO needed (Looks like it might be an event content issue)
  • Top, Higgs: Validation contact still needed to develop these packages
  • SUSY: Basic path efficiency counts for MC selection for each RA analysis
  • Exotica: Done by standalone analysis; validation package not yet integrated in HLTriggerOffline

Release 5_2_X

  • Set B = No
  • Set O = Yes / Done
  • Set W = Work in progress, under construction
  • Set N = Minus
  • Set S = Stop

Moved to PdmVTableHLT52X No permission to view CMS.PdmVTableHLT52X

5.2.0.prex details

</>

Release 5_1_X

  Release Group
    Electron Photon Muon Jet MET Tau Top Higgs HeavyFlavor Susy Exotica
  510pre1 fullsim Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Yes / Done Minus
  fastsim Yes / Done Yes / Done Minus n.s. Minus n.s. Minus n.s. n.s. n.s. n.s.
  510pre2 fullsim Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Yes / Done Minus
  fastsim Yes / Done Yes / Done Minus n.s. Minus n.s. Minus n.s. n.s. n.s. n.s.
  51X fullsim Minus Minus Minus Minus Minus Minus Minus Minus Minus Minus Minus
  fastsim Minus Minus Minus Minus Minus n.s. Minus n.s. n.s. n.s. n.s.

5.1.0.prex details

Release 5_0_X

  Release Group
    Electron Photon Muon Jet MET Tau Top Higgs HeavyFlavor Susy Exotica
  500pre2 fullsim Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Yes / Done Minus
  fastsim n.s. n.s. Yes / Done n.s. Minus n.s. Minus n.s. n.s. n.s. n.s.
  500pre3 fullsim Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Yes / Done Minus
  fastsim No No Yes / Done Yes / Done Minus n.s. Minus n.s. n.s. n.s. n.s.
  500pre4 fullsim Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Yes / Done Minus
  fastsim No No Yes / Done Yes / Done Minus n.s. Minus n.s. n.s. n.s. n.s.
  500pre6 fullsim Yes / Done Yes / Done Yes / Done n.s. Minus Yes / Done Minus Minus Minus Yes / Done Minus
  fastsim n.s. Yes / Done Yes / Done n.s. Minus n.s. Minus n.s. n.s. n.s. n.s.
  500pre7 fullsim Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Yes / Done Minus
  fastsim Yes / Done Yes / Done Yes / Done n.s. Minus n.s. Minus n.s. n.s. n.s. n.s.
  500 fullsim Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Yes / Done Minus
  fastsim Yes / Done Yes / Done Yes / Done n.s. Minus n.s. Minus n.s. n.s. n.s. n.s.
  50X fullsim Minus Minus Minus Minus Minus Minus Minus Minus Minus Minus Minus
  fastsim Minus Minus Minus Minus Minus n.s. Minus n.s. n.s. n.s. n.s.

5.0.0.pre2 details

5.0.0.pre3 details
  • Electron FastSim: Fastsim vs. fullsim shows lower efficiency (-3%) due to the PixelMatch filters. Fastsim vs. fastsim 4.4.1 shows lower efficiency (-1%) for the PixelMatch filters as well.
  • Photon FastSim: Fastsim vs. fullsim shows higher efficiency (+2%). These issues are found in 4.4.x versions as well.

5.0.0.pre4 details
  • Electron FastSim: Fastsim vs. fullsim shows lower overall efficiency (-1%) caused by the PixelMatch filters which have about -3%.
  • Photon FastSim: Fastsim vs. fullsim shows higher efficiency (+ 1-2%). The issue is the same as in the previous versions. See this HN thread for more information: https://hypernews.cern.ch/HyperNews/CMS/get/hlt/2618/1/1/1/1/1/1/1/1.html

Fall11 preproduction

  • pvt request post
  • release: 440pre8 (reco), 429HLT1 (HLT menu)
  • preproduction sequence: code
    • restrictions on memory and event content
    • packages enabled: fourvector, tau, top, heavyflavor, susy/exo
  • validation request post
  • plots empty due to wrong sequence (production instead of preproduction) employed
  • muon discrepancies found consistent with pileup effect: muon Yes / Done

Release 4_4_X

  Release Group
    Electron Photon Muon Jet MET Tau Top Higgs HeavyFlavor Susy Exotica
  440pre1 fullsim No Yes / Done No Yes / Done Yes / Done Yes / Done Minus Minus Minus No Minus
  fastsim Minus Minus No No Minus n.s. Minus n.s. n.s. n.s. n.s.
  440pre2 fullsim No Yes / Done Minus Yes / Done Yes / Done Yes / Done Minus Minus Minus No Minus
  fastsim Minus Minus Minus No Minus n.s. Minus n.s. n.s. n.s. n.s.
  440pre4 fullsim No Yes / Done No Yes / Done Yes / Done Yes / Done Minus Minus Minus No Minus
  fastsim Minus Minus No No Minus n.s. Minus n.s. n.s. n.s. n.s.
  440pre5 fullsim No Yes / Done Minus Yes / Done Yes / Done Yes / Done Minus Minus Minus No Minus
  fastsim Minus Minus Minus No Minus n.s. Minus n.s. n.s. n.s. n.s.
  440pre7 fullsim No No Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus No Minus
  fastsim Minus Minus Yes / Done No Minus n.s. Minus n.s. n.s. n.s. n.s.
  440pre8 fullsim No Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Yes / Done Minus
  fastsim Minus Minus Yes / Done No Minus n.s. Minus n.s. n.s. n.s. n.s.
  440pre9 fullsim No Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Yes / Done Minus
  fastsim Minus Minus Yes / Done No Minus n.s. Minus n.s. n.s. n.s. n.s.
  440pre10 fullsim Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Yes / Done Minus
  fastsim No No Yes / Done No Minus n.s. Minus n.s. n.s. n.s. n.s.
  440 fullsim Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Yes / Done Minus
  fastsim No No Yes / Done Yes / Done Minus n.s. Minus n.s. n.s. n.s. n.s.
  441 fullsim No Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Yes / Done Minus
  fastsim No n.s. Yes / Done Yes / Done Minus n.s. Minus n.s. n.s. n.s. n.s.
  442 fullsim No Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Yes / Done Minus
  fastsim n.s. No Yes / Done Yes / Done Minus n.s. Minus n.s. n.s. n.s. n.s.
  442 GT fullsim Minus Minus Minus Minus Minus Minus Minus Minus Minus Minus Minus
  44X fullsim Minus Minus Minus Minus Minus Minus Minus Minus Minus Minus Minus
  fastsim Minus Minus Minus Minus Minus n.s. Minus n.s. n.s. n.s. n.s.

4.4.0.pre1 details

Samples produced with MC conditions only; limited validation expected.

  • Muon: efficiency dips at L1 in the central region (hn)
  • Electron: hltEGRegionalL1EG5HTT75 Filter drops from 14.6% in 430pre7 to 5.2% for the Zee sample.
  • Jets: fastisim: same bug as in 430pre6

4.4.0.pre2 details

  • Electron: Efficiency for hltEGRegionalL1EG5HTT75 filter remains at much lower level than in 430pre7 for Zee sample.
  • Jets: fastisim: same bug as in 430pre6

4.4.0.pre4 details

  • Muon: same problem as in pre1; efficiency dips at L1 in the central region
  • Electron: Efficiency for hltEGRegionalL1EG5HTT75 filter remains at much lower level than in 430pre7 for Zee sample. PixelMatch filters seem less efficient in the highest eta bins.
  • Jets: fastisim: same bug as in 430pre6

4.4.0.pre5 details

  • Electron: Efficiency HLT_Ele15_CaloIdT_CaloIsoVL_TrkIdT_TrkIsoVL_HT250* remains lower than in 430pre7 probably due to the hltEGRegionalL1EG5HTT75 filter. There has also been a change in the L1 seed in that paths which could explain a drop in the L1 efficiency. The PixelMatch filters seem less efficient in the highest eta bins. HLT_Ele17_CaloIdL_CaloIsoVL_Ele15_HF* show drop in efficiency probably caused by changed L1 seeds. Method to calculate efficiencies has changed which results in lower efficiencies for some double electron and double photon paths.
  • Jets: fastisim: same bug as in 430pre6

4.4.0.pre6 details
DQM harvesting not run

4.4.0.pre7 details

  • Electron/Photon: Lower efficiency of hltEGxxEtFilter especially for higher thresholds. Problems with the preshower sim->digi step leaves endcaps bad. This is solved in 4.4.0.pre8. Apart from that the same issues as in 4.4.0.pre5.
  • Muon: Small decrease in L1 efficiency.
  • Jets: fastisim: same bug as in 430pre6

4.4.0.pre8 details

  • Electron: The same issues as in 4.4.0.pre5 are present.
  • FastSim: Egamma/Jet: L1 issues; cause detected in L1/EcalTPG Emulator SimCalorimetry/EcalTrigPrimAlgos, change adopted since 430pre6; developers contacted, no answer in time for 440pre9 however;
V02-03-03 SimCalorimetry/EcalTrigPrimAlgos restores for fastsim the same behavior it had before the introduction of the spike killing additions. Should inspect results in 4_4_0 release.

4.4.0.pre9 details

  • Electron: The same issues as in 4.4.0.pre8 are present. Status update, no resolution yet for L1, posted to L1 hn; re efficiency loss of the pixelMatch filters at high eta there is a tentative but yet yet verified explanation (change from START43 to START44) post,
  • FastSim: same as 440pre8

4.4.0.pre10 details

  • FastSim: same as before; code fix committed to cvs, but not picked in release yet post
  • Electron: Issue with high eta efficiency loss in pixelMatch filters improved. Behaviour of RegionalFilter in Ele15*HT* paths still unchanged but declared unproblematic.

4.4.0 details

  • FastSim: previous major issue fixed!
  • FastSim: higher (2-3%) efficiency wrt fullsim, using h130gg. comment by A.Perrotta There are indeed some (a few %) differences, not only in the photon, but also in the electron triggers. I see that the L1 part seems to behave correctly. For the electron paths those filters are tracker seeded, and the several approximations we had initially put in the first fastsim implementation of those filters was indeed one of the main reason why I was asking since longtime to get a detailed validation of the HLT performance in fastsim. We are actually working with the fastsim specific customization of the seeding of the HLT tracking algorithm, and try to see whether some fix/tuning can be applied there. Re the photon paths, which do not contain tracking modules, same unmodified "fullsim" HLT modules are also used in fastsim, and thus harder to understand. FastSim experts are looking into it.

4.4.1 details

  • Electron: DoubleEle45 path seems to have a slightly lower efficiency for positive eta than for negative eta.
  • Electron FastSim: Generally 2-3% lower efficiency wrt fullsim with ZEE sample. This is mainly coming from PixelMatch filters but also OneOEMinusOneOP filters have a generaly lower efficiency. See also comments for 4.4.0 for more information.

4.4.2 details

  • Electron: DoubleEle45 path still seems to have a slightly lower efficiency for positive eta as already reported for 4.4.1.
  • Photon FastSim: Still generally 2-3% higher efficiency wrt fullsim. Only the path HLT_Photon20_CaloIdVT_IsoT_Ele8_CaloIdL_CaloIsoVL which contains a PixelMatch filter has 3% lower efficiency.

Release 4_3_X

  Release Group
    Electron Photon Muon Jet MET Tau Top Higgs HeavyFlavor Susy Exotica
  430pre1 fullsim No Yes / Done Yes / Done Yes / Done Yes / Done n.s. Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. Minus n.s. n.s. n.s. n.s.
  430pre2 fullsim No Yes / Done Yes / Done Yes / Done Yes / Done n.s. Minus Minus Minus Yes / Done Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. Minus n.s. n.s. n.s. n.s.
  430pre3 fullsim No Yes / Done Minus n.s. Yes / Done n.s. Minus Minus Minus Minus Minus
  fastsim Minus Minus Minus n.s. Minus n.s. Minus n.s. n.s. n.s. n.s.
  430pre4 fullsim n.s. n.s. Minus Yes / Done Yes / Done Yes / Done Minus Minus Yes / Done Yes / Done Minus
  fastsim Minus Minus Minus Yes / Done Minus n.s. Minus n.s. n.s. n.s. n.s.
  430pre5 fullsim Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Yes / Done No Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. Minus n.s. n.s. n.s. n.s.
  430pre6 fullsim Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Yes / Done Minus
  fastsim Minus Minus Yes / Done No Minus n.s. Minus n.s. n.s. n.s. n.s.
  430pre7 fullsim Yes / Done Yes / Done No Yes / Done Yes / Done Yes / Done Minus Minus Minus No Minus
  fastsim Minus Minus Minus No Minus n.s. Minus n.s. n.s. n.s. n.s.
  430 fullsim Yes / Done Yes / Done No Yes / Done Yes / Done Yes / Done Minus Minus Minus No Minus
  fastsim Minus Minus Yes / Done No Minus n.s. Minus n.s. n.s. n.s. n.s.
  43X fullsim Minus Minus Minus Minus Minus Minus Minus Minus Minus Minus Minus
  fastsim Minus Minus Minus Minus Minus n.s. Minus n.s. n.s. n.s. n.s.

4.3.0.pre1 details

Samples produced with MC conditions only; limited validation expected.

  • Electron: HLT_Ele10_CaloIdT_CaloIsoVL_TrkIdT_TrkIsoVL_HT200_v2 contains a filter with gets its input from an other path.
  • Tau: Missing needed event content

4.3.0.pre2 details

  • Electron: Same issues as in 4.3.0.pre1 but with version v3 of the path.
  • Tau: Missing needed event content

4.3.0.pre3 details

  • Electron: Same issues as in 4.3.0.pre2.

4.3.0.pre5 details

  • Egamma: Most of the double electron and double photon paths could not be fully validated due to a dropped collection.

4.3.0.pre6 details

  • Jets: New JEC (Jec11 v1). The turn-on is a little shifted to higher pT. cf: here
  • Jets: fastsim: a bug has been introduced. The bug fix is in V02-03-03 SimCalorimetry/EcalTrigPrimAlgos, cf here

4.3.0.pre7 details

  • Muon: Dips in efficiency at L1 in the central region.
  • Jets: fastisim: same as 430pre6

4.3.0 details

  • Muon: Same problem as in pre7; dips in efficiency at L1 in the central region.
  • Susy: alphaT triggers:
check by B.Mathias: post: a dedicated test based on relvals from CMSSW_4_3_0 / 430_pre6 / 430_pre7 was performed, and the decrease in efficiencies could not be reproduced.
  • Jets: fastisim: same as 430pre6

Release 4_2_X

  Release Group
    Electron Photon Muon Jet MET Tau Top Higgs HeavyFlavor Susy Exotica
  4.2.0.pre1 fullsim Minus Minus Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Yes / Done Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. Minus n.s. n.s. n.s. n.s.
  420pre2 fullsim Minus Minus Minus n.s. Minus Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Minus Yes / Done Minus n.s. Minus n.s. n.s. n.s. n.s.
  420pre4 fullsim Minus Minus Minus Yes / Done Minus n.s. Minus Minus Minus Minus Minus
  fastsim Minus Minus Minus Yes / Done Minus n.s. Minus n.s. n.s. n.s. n.s.
  420pre5 fullsim No Yes / Done Minus Yes / Done Minus n.s. Minus Minus Minus Minus Minus
  fastsim Minus Minus Minus Yes / Done Minus n.s. Minus n.s. n.s. n.s. n.s.
  420pre6 fullsim No Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. Minus n.s. n.s. n.s. n.s.
  420pre7 fullsim Minus Minus Minus Yes / Done Yes / Done Minus Minus Minus Minus Minus Minus
  fastsim Minus Minus Minus Yes / Done Minus n.s. Minus n.s. n.s. n.s. n.s.
  420pre8 fullsim No Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. Minus n.s. n.s. n.s. n.s.
  420 fullsim No Yes / Done Yes / Done Yes / Done Yes / Done n.s. Minus Minus Minus Yes / Done Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. Minus n.s. n.s. n.s. n.s.
  421 fullsim No Yes / Done Yes / Done Yes / Done Yes / Done n.s. Minus Minus Minus Yes / Done Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. Minus n.s. n.s. n.s. n.s.
  422 fullsim No Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Yes / Done Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. Minus n.s. n.s. n.s. n.s.
  423 fullsim No Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus No Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. Minus n.s. n.s. n.s. n.s.
  424 fullsim No Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Yes / Done Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. Minus n.s. n.s. n.s. n.s.
  424 ECAL Spike cleaning fullsim Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Yes / Done Minus
  fastsim Minus Minus Yes / Done n.s. Minus n.s. Minus n.s. n.s. n.s. n.s.
  425 fullsim No Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Yes / Done Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. Minus n.s. n.s. n.s. n.s.
  426 fullsim No Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Yes / Done Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. Minus n.s. n.s. n.s. n.s.
  427 fullsim n.s. n.s. Minus n.s. n.s. Minus Minus Minus Minus Minus Minus
  fastsim Minus Minus Minus n.s. n.s. n.s. Minus n.s. n.s. n.s. n.s.
  428 fullsim No Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Yes / Done Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. Minus n.s. n.s. n.s. n.s.
  429HLT1 fullsim No Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Yes / Done Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. Minus n.s. n.s. n.s. n.s.
  429HLT1_hltpatch1 fullsim No Yes / Done Yes / Done Yes / Done No Work in progress, under construction Minus Minus Minus Yes / Done Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. Minus n.s. n.s. n.s. n.s.
  42X fullsim Minus Minus Minus Minus Minus Minus Minus Minus Minus Minus Minus
  fastsim Minus Minus Minus Minus Minus n.s. Minus n.s. n.s. n.s. n.s.

4.2.X general comments
  • BPhysics: Except 4.2.0.pre1, for all other pre/releases in the series the plots were not filled properly due to a bug that was fixed but was incorporated only in 43x

4.2.0.pre1 details
  • Muon: none
  • Tau: The tau validation cannot be done currently because the relvals are neither made with the old menu or the new menu, they appear to be made with some early version of 5E32 which the code is not ready to handle.
4.2.0.pre4 details
  • Jet: new trigger menu (jet energy corrections, new thresholds, etc)
4.2.0.pre5 details
  • Electron: HLT_Ele10_CaloIdL_TrkIdVL_CaloIsoVL_TrkIsoVL_HT220_v1 and HLT_Ele10_CaloIdT_TrkIdT_CaloIsoVL_TrkIsoVL_HT220_v1 do not fire at all.
4.2.0.pre6 details
  • Electron: HLT_Ele10_CaloIdT_CaloIsoVL_TrkIdT_TrkIsoVL_HT200_v2 contains a filter with gets its input from an other path.
4.2.0.pre8 details
  • Electron: The same faulty path as in 4.2.0.pre6.
4.2.0 details
  • Tau: Needed Event content is missing.
  • Electron: The same faulty path as in 4.2.0.pre6.
4.2.1 details
  • Electron: The same problems as in 4.2.0.
  • Tau: Missing needed event content
4.2.2 details
  • Electron: The same problems as in 4.2.1.
  • Tau: Increase in lead track finding efficiency due to use of online beamspot location instead of 0,0,0
4.2.3 details
  • Electron: The same problems as in 4.2.2.
  • Jets: New JEC (Jec11 v1). The turn-on is a little shifted to higher pT. cf: here
4.2.4 details
  • Electron: The same problems as in 4.2.3.
4.2.5 details
  • Electron: The same problems as in 4.2.4.
4.2.6 details
  • Electron: The same problems as in 4.2.5.
4.2.8 details
  • Electron: The same problems as in 4.2.6. Compared to 4.2.6 there is a loss in efficiency in the highest eta bin of the PixelMatch filters. This is observed for the WE sample and also to a smaller extent in the ZEE sample.
4.2.9.HLT1 details
  • Muon: small L1 inefficiency at low-pT and high eta. Checks performed by L1/Muon experts, summary (by A.Perrotta): (i) in 4_2_9_HLT1 there is a slightly lower L2 efficiency for low pt muons in the endcaps wrt 4_2_8, because of the tightened L2 muon quality requirements entered in the 3e33 menu, (ii) in 4_4_X we have a slightly larger L1 efficiency wrt 4_2_9_HLT1, because of modified LUT's

  • Electron: hltEGRegionalL1EG5HTT75 has lower efficiency compared to 430pre7 as all the 440preX versions as well. Also the highest eta bin in the PixelMatch filter remains at a lower efficiency.

4.2.9.HLT1_hltpatch1 details
  • Electron: hltEGRegionalL1EG5HTT75 has lower efficiency compared to 430pre7 as all the 440preX versions as well. Also the highest eta bin in the PixelMatch filter remains at a lower efficiency just like in 4.2.9.HLT1.
  • MET: slower turn on curve wrt 429HLT1; investigating

4_1_X vs 3_11_1_hltpatch1

  • Yes / Done - HLT accepts agree to per cent.
  • Comparison of HLT results in RelVal 4_1_1 vs 4_1_2 are given: This file shows the difference in accept rate between 4.1.1 and 4.1.2. The differences are most likely due to the additional HLT-related tags needed to run the same menu configuration in 4.1.1 tag list found here. The path-by-path differences broken down by the modules within the paths are contained in this file.

Dependence of accept rates on tower thresholds in 3_11_1_hltpatch1 - No

This file shows how the accept rates per path differ in ttbar MC for normal tower thresholds and for increased tower thresholds. Accept rates can be affected by over 10% for the hadronic paths, with the higher thresholds (right arrows in the log file) leading to lower accept rates across the whole pT spectrum for jets, MET, and HT.

Comparing 3_11_1_hltpatch1 to 4_1_1 with tags -Yes / Done

This file shows the difference in accept rates for 4_11_1 + aforementioned tags compared to the 3_11_1_hltpatch1 values (default tower thresholds). Though the HLT-report has changed between releases, the path accept rates are identical.

Comparing 4_1_3 to 4_1_1 + tags

4_1_3 includes v6 of the 5e32 menu. Compared to the previous version of the menu from above, the version of many cross-channel and egamma paths changed. For the paths that can be directly compared:

  • HLT_JetX paths have a slightly different accept rate (few/10k events)
  • HLT_DijetAveU have an identical accept rate
  • HLT_DoubleJet60_ForwardBackward_v2 accepts an event
  • Very small difference in accept rate for HLT_CentralJet_MET paths
  • A few more events are taken by the HLT_DiJetX_PTY, DijetX_METY paths
  • The accept rates for the paths HLT_QuadJet40_IsoPFTau40_v1, HLT_QuadJet40_BTagIP both higher by ~20%
  • HLT_QuadJet path accept rates generally lower by ~1%
  • HLT_Mu paths take a few more events
  • Newer versions of electron paths have significantly higher accept rate (~ 250/6000)

The list of path accept rates is here; while this is the full list of modules within the paths.

Release 4_1_X

Validation on Data: New Calibration Constants

Reprocessing jobs were run on a recent run (161311) with both the nominal GlobalTag (GR_H_V16), and a GlobalTag with updated constants (GR_H_V16T). Details of the reprocessing can be found here.

Group            
Electron Photon Muon Jet MET Tau HeavyFlavor
Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done n.s.

RelVal MC Validation

  Release Group
    Electron Photon Muon Jet MET Tau Top Higgs HeavyFlavor Susy/Exotica
  410pre1 fullsim Minus Minus Minus Yes / Done Minus Yes / Done Minus Minus Minus Minus
  fastsim Minus Minus Minus Yes / Done Minus n.s. Minus n.s. n.s. n.s.
  410pre2 fullsim Minus Minus Minus Yes / Done Minus Yes / Done Minus Minus Minus Minus
  fastsim Minus Minus Minus Yes / Done Minus n.s. Minus n.s. n.s. n.s.
  412 fullsim Minus Minus Yes / Done Yes / Done Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Minus Yes / Done Minus n.s. Minus n.s. n.s. n.s.
  413 fullsim Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Minus Minus
  fastsim n.s. n.s. Minus Yes / Done Minus n.s. Minus n.s. n.s. n.s.
  414 fullsim Yes / Done Yes / Done Yes / Done n.s. Yes / Done n.s. Minus Minus Minus Yes / Done/Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. Minus n.s. n.s. n.s.
  415 fullsim Yes / Done Yes / Done Yes / Done Minus Yes / Done n.s. Minus Minus Minus Yes / Done/Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. Minus n.s. n.s. n.s.
  416 fullsim Minus Minus Yes / Done Minus Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. Minus n.s. n.s. n.s.
  41X fullsim Minus Minus Minus Minus Minus Minus Minus Minus Minus Minus
  fastsim Minus Minus Minus Minus Minus n.s. Minus n.s. n.s. n.s.

4.1.5 details
  • Jet: 41X is still using the uncorrected jet paths.

Release 3_11_X

  Release Group
    Electron Photon Muon Jet MET Tau Calibration Top Higgs HeavyFlavor Susy/Exotica
  3.11.0-pre2 fullsim Minus Minus Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Minus n.s. Minus n.s. n.s. Minus n.s. n.s. n.s.
  3.11.0-pre3 fullsim Minus Minus Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done n.s. Minus n.s. n.s. Minus n.s. n.s. n.s.
  3.11.0-pre5 64-bit Minus Minus Yes / Done Yes / Done Yes / Done Yes / Done Minus Yes / Done Yes / Done Minus Minus
    32-bit Minus Minus Yes / Done Yes / Done Yes / Done Yes / Done Minus Yes / Done Yes / Done Minus Minus
  3.11.0 64-bit Minus Minus Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Minus Minus
    32-bit Minus Minus Yes / Done Yes / Done Yes / Done Minus Minus Minus Minus Minus Minus
  3.11.1 fullsim Minus Minus Minus n.s. Minus Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Minus n.s. Minus n.s. n.s. Minus n.s. n.s. n.s.
  3.11.1 hltpatch1 Yes / Done Minus Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Yes / Done Yes / Done/Yes / Done
  3.11.2 fullsim Minus Minus Yes / Done Yes / Done Yes / Done Minus Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Minus n.s. Yes / Done n.s.
  3.11.3 fullsim Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Minus Minus Minus
  fastsim n.s. n.s. n.s. Yes / Done Minus n.s. n.s. Minus n.s. Yes / Done n.s.
  3.11.X fullsim Minus Minus Minus Minus Minus Minus Minus Minus Minus Minus Minus
  fastsim Minus Minus Minus Minus Minus n.s. n.s. Minus n.s. n.s. n.s.

3.11.1.hltpatch1

  • Tau: Put as pass but known issue with beam spot location in pixel seeding creates low efficiency in lead track finding for all tau triggers if beams spot is displaced.

  • SUSY/Exo: Separate validation reports from each group.

3.11.0.pre5 details

  • Jets: 32 v 64 ok. Only fulssim has been validated. No change wrt pre2.

3.11.0.pre4 details

3.11.0.pre2 details

  • Tau: Differences seen in L1 but seems to be related to change in GCT Emulator. (see post)

  • Note sample generation failed for the following fastsim samples: ttbar, singlemupt(10,100), zee, h130gg.

  • Jets: small differences with 3_10 results, related to change in GCT Emulator.

3.11.0.pre1 details
Sample generation failed

Release 3_10_X

  Release Group
    Electron Photon Muon Jet MET Tau Calibration Top Higgs HeavyFlavor Susy/Exotica
  3.10.0.pre1 fullsim Minus Minus Yes / Done Yes / Done Minus Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Minus n.s. n.s. n.s.
  3.10.0.pre2 fullsim Minus Minus Yes / Done Yes / Done Minus Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Minus n.s. n.s. n.s.
  3.10.0.pre3 fullsim Minus Minus Yes / Done n.s. Minus n.s. Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Minus n.s. n.s. n.s.
  3.10.0.pre4 fullsim Minus Minus Yes / Done n.s. Minus n.s. Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Minus n.s. n.s. n.s.
  3.10.0.pre5 fullsim Minus Minus Yes / Done Yes / Done Minus Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Minus n.s. n.s. n.s.
  3.10.0.pre7 fullsim Minus Minus Yes / Done Yes / Done Minus Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Minus n.s. n.s. n.s.
  3.10.0.pre8 fullsim Minus Minus Yes / Done Yes / Done Minus Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Minus n.s. n.s. n.s.
  3.10.0.pre9 fullsim Minus Minus Yes / Done Yes / Done Minus Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Minus n.s. n.s. n.s.
  3.10.0 fullsim Minus Minus Yes / Done Yes / Done Minus Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Minus n.s. n.s. n.s.
  3.10.1 fullsim Minus Minus Yes / Done Yes / Done Minus Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Minus n.s. n.s. n.s.
  3.10.X fullsim Minus Minus Minus Minus Minus Minus Minus Minus Minus Minus Minus
  fastsim Minus Minus Minus Minus Minus n.s. n.s. Minus n.s. n.s. n.s.

3.10.0pre3 details
Many of the harvesting jobs failed for this cycle, so the number of reports expected is limited.

3.10.0pre1 details

? Tau: There is an increase in the level 2.5 efficiency from 20% to 40% which is believed to be due to a change in the TauRegionalPixelSeedGenerator which is meant to recover the inefficiency due to a displaced beam spot.

Release 39X

Validation using data

Dec20th re-reco
  • production details: HLT validation is done in CMSSW_3_9_5_patch2 with updated calibration constants. Harvested DQM results from running old (38X) and new constants are compared within the same release. full details of request.

Results

  Release Group
    Egamma Muon Jet MET Tau
  395p2 Yes / Done Yes / Done   Yes / Done Yes / Done

Full validation details are included here.

RelVal Validation

  Release Group
    Electron Photon Muon Jet MET Tau Calibration Top Higgs HeavyFlavor Susy/Exotica
  390pre1 fullsim Minus Yes / Done Yes / Done Yes / Done Minus Yes / Done Minus Minus Minus Minus Yes / Done
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Minus n.s. n.s. Minus
  390pre2 fullsim Minus Minus Yes / Done Yes / Done Minus Yes / Done Minus Yes / Done Yes / Done Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Yes / Done n.s. n.s. n.s.
  390pre3 fullsim Minus Minus Yes / Done Yes / Done Minus Yes / Done Minus Minus Minus Minus No
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Minus n.s. n.s. n.s.
  390pre4 fullsim Minus Minus Yes / Done Yes / Done Minus Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Minus n.s. n.s. n.s.
  390pre5 fullsim Minus Minus Yes / Done Yes / Done Minus Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Minus n.s. n.s. n.s.
  390pre6 fullsim Minus Minus Minus No Minus Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Minus Yes / Done Minus n.s. n.s. Minus n.s. n.s. n.s.
  390pre7 fullsim Minus Minus Yes / Done Yes / Done Minus n.s. Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Minus n.s. n.s. n.s.
  390 fullsim Minus Minus Yes / Done Yes / Done Minus Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Minus n.s. n.s. n.s.
  391 fullsim Minus Minus Yes / Done Yes / Done Minus Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Minus n.s. n.s. n.s.
  392 fullsim Minus Minus Yes / Done Yes / Done Minus Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Minus n.s. n.s. n.s.
  393 fullsim Minus Minus Yes / Done Yes / Done Minus Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done n.s. Minus n.s. n.s. Minus n.s. n.s. n.s.
  394 fullsim Minus Minus Yes / Done Yes / Done Minus Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Minus n.s. n.s. n.s.
  395 fullsim Minus Minus Yes / Done Yes / Done Minus Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Minus n.s. n.s. n.s.
  396 fullsim Minus Minus Yes / Done Yes / Done Minus Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Minus n.s. n.s. n.s.
  397 fullsim Minus Minus Yes / Done No Minus Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Minus n.s. n.s. n.s.
  399 fullsim Minus Minus Minus Yes / Done Minus Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Minus Yes / Done Minus n.s. n.s. Minus n.s. n.s. n.s.
  39X fullsim Minus Minus Minus Minus Minus Minus Minus Minus Minus Minus Minus
  fastsim Minus Minus Minus Minus Minus n.s. n.s. Minus n.s. n.s. n.s.

397 details

? Jet: some little discrepancies in fullsim. Fastsim is OK.

391 details

? Tau: There is an increase in the level 2.5 efficiency from 20% to 40% which is believed to be due to a change in the TauRegionalPixelSeedGenerator which is meant to recover the inefficiency due to a displaced beam spot.

390pre7 details

? Tau: Validation could not be done by standard procedures due to changes in the code and harvesting failures.

390pre6 details

? Tau: Could only validate L1,L2,L2.5,L3 plots due to change in the trigger menu, update to the code has been submitted.
? Jets: Fastsim is OK, but the jet eta distribution has changed in fullsim. Probably because Pre6 is broken in EE.

390pre5 details

? Tau: Could only validate L1,L2 plots due to change in the trigger menu.

390pre4 details

? Tau: Could only validate L1,L2 plots due to change in the trigger menu.

390pre3 details

? Tau: Some increased efficiency in L2 double tau but final efficiency is within the error. Individual L2 efficiencies are stable.
? Susy: L1 EG: many bits have a lower efficiency in 390pre3 than in 384. L1 cross triggers: decreas in L1_Mu5_Jet6 (42% to 38%).

390pre2 details

? Jets: change from corrected paths to uncorrected paths in MC_38Y_VX MC.

390pre1 details

? Susy: 390pre1 w.r.t. 380: perfect agreement.

Release 38X

Validation using data

Sep23rdReHLTReReco, Sep24thReHLTReReco
  • production details: HLT validation is done on the new menu for 384, reflecting the 2E32 lumi scenario. Two samples were produced using 384patch2; the samples in the Sep24th reprocessing were done with the CMS.GlobalTag GR_R_38X_V12, which includes new EB alignment constants. Full details linked below:

Results

  Release Group
    Egamma Muon Jet MET Tau
  384 Yes / Done Yes / Done Yes / Done Minus Yes / Done

Full validation details are included here.

Sep3rdReHLTReReco
  • production details: Reprocessing of samples used in 381 patch 2 with updated globalTag (GR10_H_V8_T2) that includes changes of conditions for Muon alignment and Global Alignment, Ecal Endcap and preshower alignment, DT Ttrig calibration, CSC timing. Full details are here

Results

  Release Group
    Egamma Muon Jet MET Tau
  382patch1 Yes / Done Yes / Done Yes / Done Minus Yes / Done

Aug13thReHLTReReco
  • Production details: HLT validation is done comparing the HLT resutls from 36X to data reprocessed (re-reco, re-HLT) in 381. The full details of the production request are listed here.

Results

  Release Group
    Egamma Muon Jet MET Tau FourVector
  381 Yes / Done Yes / Done Yes / Done Minus Yes / Done Yes / Done

CMS.RelVal Validation

  Release Group
    Electron Photon Muon Jet MET Tau Calibration Top Higgs HeavyFlavor Susy/Exotica
  380pre1 fullsim Minus Yes / Done Yes / Done Minus Minus Minus Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Minus Minus n.s. n.s. Minus n.s. n.s. n.s.
  380pre2 fullsim Minus Minus Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Yes / Done n.s. n.s. Minus n.s. n.s. n.s.
  380pre3 fullsim Minus Minus Minus Minus Minus Minus Minus Minus Minus Minus Minus
  fastsim Minus Minus Minus Minus Minus n.s. n.s. Minus n.s. n.s. n.s.
  380pre4 fullsim Minus Minus Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Yes / Done n.s. n.s. Minus n.s. n.s. n.s.
  380pre5 fullsim Minus Minus Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Yes / Done n.s. n.s. Minus n.s. n.s. n.s.
  380pre6 fullsim Minus Minus Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Yes / Done n.s. n.s. Minus n.s. n.s. n.s.
  380pre7 fullsim Minus Minus Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Minus No
  fastsim Minus Minus Yes / Done Yes / Done Yes / Done n.s. n.s. Minus n.s. n.s. n.s.
  380pre8 fullsim Minus Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Yes / Done n.s. n.s. Minus n.s. n.s. n.s.
  380 fullsim Minus Yes / Done Yes / Done Yes / Done Yes / Done n.s. Minus Yes / Done Yes / Done Minus Yes / Done
  fastsim Minus Minus Yes / Done Yes / Done Yes / Done n.s. n.s. Yes / Done n.s. n.s. n.s.
  381 fullsim Minus Minus Yes / Done Yes / Done Minus Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Minus n.s. n.s. n.s.
  382 fullsim Minus Minus Yes / Done Yes / Done Minus Yes / Done Minus Yes / Done Yes / Done Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Yes / Done n.s. n.s. n.s.
  383 fullsim Minus Minus Yes / Done Yes / Done Minus Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Minus n.s. n.s. n.s.
  384 fullsim Minus Minus Yes / Done Yes / Done Minus Yes / Done Minus Minus Minus Minus No
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Minus n.s. n.s. n.s.
  385 fullsim Minus Minus Yes / Done Yes / Done Minus Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Minus n.s. n.s. n.s.
  386 fullsim Minus Minus Yes / Done Yes / Done Minus Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Minus n.s. n.s. n.s.
  387 fullsim Minus Minus Minus Yes / Done Minus Yes / Done Minus Minus Minus Minus Minus
  fastsim Minus Minus Minus Yes / Done Minus n.s. n.s. Minus n.s. n.s. n.s.
  38X fullsim Minus Minus Minus Minus Minus Minus Minus Minus Minus Minus Minus
  fastsim Minus Minus Minus Minus Minus n.s. n.s. Minus n.s. n.s. n.s.

385 details

? Tau: Could only validate L1,L2,L2.5,L3 plots due to change in the trigger menu, update to the code has been submitted.

384 details
For 384, the MC-based menus HLT_8E29 and HLT_1E31 were deprecated and replaced with HLT_GRun. POGs and PAGs should examine their paths in the MC-based HLT_GRun menu and adapt their validation code accordingly.
? Tau: Since the trigger paths in the current DQM are not available in GRUN, tau HLT was validated using the L1 and L2 turn on curves.
? Susy: we still have to adapt the validation code. Using the old validation code: L1 EG: better in 384 than in 380 for EG triggers (lower efficiency bits are just swapped w.r.t. 380, they are not real lowe efficiency bits). L1 Jets: L1SingleJet30 decreases (from more than 90% to ~87%). Same for L1SingleJet50. L1 Tau: L1TauJet30 decreased (from >60% to 50%). HLT Electron: Ele10_SW_L1R: 70% in 380, 40% in 384. HLT Photon: Photon10_Cleaned: 80% in 380, 50% in 384. DoublePhoton10: 50% in 380, 20% in 384. DoublePhoton15: 40% in 380, 12% in 384. HLT Jet: Jet30 slightly decreased (>90% to <90%). Same for Jet50 and Jet110.

383 details

382 details

Jet: empty histograms due to uncorrected vs corrected path names (also affects 3.9.X). Fix has been queued for at least 3.9.X, may be back-ported to 3.8 (?)

381 details

380 details

? Muon: Same drops in efficiency for JpsiMM as seen since 380pre1, but this is now considered a real effect. Previous versions showed a higher efficiency due to unrealistic ghost cancellation windows.
? Tau: Results for ZTT not available this cycle (harvesting failure).
? Susy: validation w.r.t. 363. The Tau triggers look as for 363, so no drop of efficiency, but I have to report that in 363 I observed a big drop for tau and btag triggers.

380pre8 details

? Muon: Same drops in efficiency for JpsiMM as seen since 380pre1.

380pre7 details

? Muon: Same drops in efficiency for JpsiMM as seen since 380pre1.
? Susy: Validation w.r.t. 370. Much lower efficiency for all the L1Mu (about 10% less); slightly lower efficiency (few percent) for the L1TauJets; Drop in efficiency for HLT_L1Mu (37% to 31%), HLT_L1Mu20HQ (20% to 13%), HLT_L1Mu30 (22% to 13%), HLT_SingleIsoTau30 (25% to 17%), HLT_DoubleLooseIsoTau (25% to 10%), HLT_BTagIp_Jet80, HLT_BTagIp_Jet120 (25% to 7%); the three HLT_L1Mu14 cross triggers loose 5-10% in efficiency. The effects are seen also in TTbar samples and not only in LM1 ones.

380pre6 details

? Tau: Efficiency increase seen in lvl1 followed by a decrease in lvl 2, final result is a flat efficiency in the HLT
? Muon: Same drops in efficiency for JpsiMM as seen since 380pre1.

380pre5 details

? Muon: Same drops in efficiency for JpsiMM as seen since 380pre1.

380pre4 details

? Muon: Same drops in efficiency for JpsiMM as seen since 380pre1.

380pre3 details

380pre2 details

? Muon: Same drops in efficiency for JpsiMM as seen in 380pre1.
? Tau: There is an increase in efficiency attributed to a change in the RCT activity cut from 3 GeV to 4 GeV.

380pre1 details

? Muon: Drop in efficiency of ~10% in endcaps at L1 for JPsiMM; smaller drops at L2, L3.

Release 37X

  Release Group
    Electron Photon Muon Jet MET Tau Calibration Top Higgs HeavyFlavor Susy/Exotica
  370pre1 n.s. fullsim Minus Work in progress, under construction Yes / Done Minus Yes / Done Minus Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Minus Minus n.s. n.s. Minus n.s. n.s. n.s.
  370pre2 fullsim Minus Work in progress, under construction Yes / Done Yes / Done Yes / Done No Minus Yes / Done Yes / Done Minus No
  fastsim Minus Minus Yes / Done Yes / Done Yes / Done n.s. n.s. Yes / Done n.s. n.s. n.s.
  370pre3 fullsim Minus Minus Yes / Done No Yes / Done Minus Minus No Yes / Done Minus Minus
  fastsim Minus Minus Yes / Done No Yes / Done n.s. n.s. No n.s. n.s. n.s.
  370pre4 fullsim Minus Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Yes / Done Yes / Done Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Yes / Done n.s. n.s. Yes / Done n.s. n.s. n.s.
  370pre5 fullsim Minus Minus Yes / Done Yes / Done Yes / Done No Minus Yes / Done Yes / Done Minus No
  fastsim Minus Minus Yes / Done Yes / Done Yes / Done n.s. n.s. Yes / Done n.s. n.s. n.s.
  370 fullsim Minus Minus Yes / Done Yes / Done Yes / Done Yes / Done Minus Yes / Done Yes / Done Minus No
  fastsim Minus Minus Yes / Done Yes / Done Yes / Done n.s. n.s. Yes / Done n.s. n.s. n.s.
  371 fullsim Minus Minus Yes / Done Yes / Done Minus Minus Minus Minus Minus Minus Minus
  fastsim Minus Minus Yes / Done Yes / Done Minus n.s. n.s. Minus n.s. n.s. n.s.
  37X fullsim Minus Minus Minus Minus Minus Minus Minus Minus Minus Minus Minus
  fastsim Minus Minus Minus Minus Minus n.s. n.s. Minus n.s. n.s. n.s.

370 details

? Tau: The efficiency is higher than in 3_6_0 but this has been resolved by the TSG
? Susy: validation w.r.t. 360. Big drop (up to 25% less) in L1SingleTauJet and L1DoubleTauJet efficiencies. Decrease in efficiency for HLT_tau paths.

370pre5 details


? Susy: validation w.r.t. 360. Big drop (up to 25% less) in L1SingleTauJet and L1DoubleTauJet efficiencies. Decrease in efficiency for HLT_tau paths.

370pre4 details

370pre3 details

370pre2 details

? Susy: comparion vs 360pre1. much lower efficiencies for L1_TauJet triggers. Slightly lower efficiencies for L1 EG, Jet, ETM-ETT-HTT triggers. Lower efficiency for L1 EG cross triggers. Much lower efficiency for HLT_Ele15_SC15_SW (30% vs 65%). Smaller efficiencies for many other Electron, Photon, Jet, MET paths. Much smaller efficiency for BTagIP_Jet* paths and for HLT_Ele10 cross triggers. Discrepancies in distributions of reco objects. Reco issue, not dependent on HLT.

370pre1 details

? Central results absent (harvesting failure)

Release 36X

  Release Group
    Electron Photon Muon Jet MET Tau Calibration Top Higgs HeavyFlavor Susy/Exotica
  360pre1 fullsim Minus Minus No Yes / Done Yes / Done Minus Yes / Done No No No Yes / Done
  fastsim Minus Minus No Yes / Done Minus n.s. n.s. No n.s. n.s. n.s.
  360pre2 fullsim Minus Yes / Done No Yes / Done Yes / Done Minus Yes / Done No No No No
  fastsim Minus Minus No Yes / Done Yes / Done n.s. n.s. No n.s. n.s. n.s.
  360pre3 fullsim Minus Minus No Yes / Done Minus Minus Yes / Done Minus Minus n.s. No
  fastsim Minus Minus No Yes / Done Yes / Done n.s. n.s. Minus n.s. n.s. n.s.
  360pre4 fullsim Minus Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Minus Yes / Done Yes / Done Yes / Done Minus
  fastsim Minus Minus Yes / Done Yes / Done Yes / Done n.s. n.s. Yes / Done n.s. n.s. n.s.
  360pre5 fullsim Minus Minus Yes / Done No Yes / Done Yes / Done Minus No Yes / Done Yes / Done Minus
  fastsim Minus Minus Yes / Done No Yes / Done n.s. n.s. No n.s. n.s. n.s.
  360pre6 fullsim Minus Minus Minus No Yes / Done Yes / Done Minus No Yes / Done Yes / Done Minus
  fastsim Minus Minus Minus No Yes / Done Minus n.s. No n.s. n.s. n.s.
  360 fullsim Minus Minus Yes / Done Yes / Done Yes / Done Yes / Done Minus Yes / Done Yes / Done Yes / Done No
  fastsim Minus Minus Yes / Done Yes / Done Yes / Done n.s. n.s. Yes / Done n.s. n.s. n.s.
  361 fullsim Minus No Yes / Done Yes / Done Yes / Done Yes / Done Minus Yes / Done Yes / Done Minus No
  fastsim Minus Minus Yes / Done Yes / Done Yes / Done n.s. n.s. Yes / Done n.s. n.s. n.s.
  361patch1 fullsim Minus Minus Yes / Done Yes / Done Yes / Done Minus Minus Yes / Done Yes / Done Minus Minus
  362 fullsim Minus Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Minus Yes / Done Yes / Done Minus No
  fastsim Minus Minus Minus Yes / Done Yes / Done n.s. n.s. Yes / Done n.s. n.s. n.s.
  363 fullsim Minus Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Minus Yes / Done
  fastsim Minus Minus Yes / Done Yes / Done Yes / Done n.s. n.s. Minus n.s. n.s. n.s.
  36X fullsim Minus Minus Minus Minus Minus Minus Minus Minus Minus Minus Minus
  fastsim Minus Minus Minus Minus Minus n.s. n.s. Minus n.s. n.s. n.s.

363 details


? Susy: good agreement between 363 and 362. Anyway, the validation of 363 w.r.t. 361 still shows the big discrepancies in Btag and Tau HLT paths as reported for 362.

362 details


? Susy: validation w.r.t. 361. Efficiency decrease for Tau and BTag paths. Seen also in TTbar sample. Checked also the Tau package plots for LM1, seen some inconsistencies, asked the tau experts.
? Tau: Some differences when compared to 361, this is due to changes in RCT GCT to have the correct settings in the emulator.

361patch1 details

361 details


? Susy: validation w.r.t. 360. Big drop (up to 50% less) in L1SingleTauJet and L1DoubleTauJet efficiencies. Decrease in efficiency (5%) in HLT_SingleIsoTau and HLT_DoubleLooseIsoTau. In E/g HLT, the preshower requires a different setting in ESUnpackerWorkerESProducer.RHAlgo.ESBaseline, for data it is 0, for MC it is 1000. In the menu, only one value can be set so its set correctly for data. Therefore in the MC, the preshower energy is too high, which caused the validation to fail. This is a problem of the preshower code and not the E/g HLT
? Tau: After much discussion, validation status changed to "ok." I (T.D.) made the edit in the table to reflect the outcome of the discussion.

360 details

? Susy: comparison vs 360pre1. Slightly lower efficiencies in some L1 EG, Jet, TauJet, ETT-ETM-HTT paths. Much lower efficiency for HLT_Ele15_SC15_SW (30% vs 65%). Smaller efficiencies for many other HLT Electron, Photon, Jet, MET paths (similar to 357 vs 350). Much smaller efficiency for BTagIP_Jet* paths and for HLT_Ele10 cross triggers. Discrepancies in distributions of reco objects. Reco issue, not dependent on HLT.

360pre6 details

? Jets: smaller number of events in the high pt jet paths compared to 360pre1.
? Heavy flavor: According to the release report, JPsiMM RelVal production failed for this release. SingleMuPt10 sample was used instead.
? Heavy flavor: Increase of 5% in L2 efficiency at eta above 1.8 in the high Pt paths due to the improved handling of L1 ghosts

360pre5 details

? Jets: smaller number of events in the high pt jet paths compared to 360pre1.

360pre4 details

? Muon: L1 has reverted to previous version of their algorithm while L2 works on a fix that can accommodate the new ghost L1 objects.
? BPAG: L2 performance is back to that of CMSSW_3_5_0_pre3 by removing the wide patterns from RPCs and therefore L1 ghosts; 5-10% increase of L1 efficiency in eta-region above ~2.1 compared to 360pre2

360pre3 details

? Muon: same issue with an L1 inefficiency as seen since 3.5.0.pre5
? Update on Muons for 35X and 36X: The issue that has been seen at L1 in the validation package is really an issue of seeding at L2. In 35X, the L1 pattern recognition was changed to take more L1s, resulting an expected higher ghost rate. The L2 seed cleaner would occasionally keep the L2 seeded by the ghost and remove the L2 seeded by the good L1. The temporary fix is to revert the L1 code, which should be done for 3.6.0.pre4 or pre5, and maybe for 3.5.5. The permanent fix, being worked on by the L2 people is to improve or remove the seed cleaner at L2.
? Susy: Comparison with 350. Decrease of efficiency in many L1 bits: taus, xtriggers, etm, ett, ht, egamma. Decrease in efficiency in many HLT paths: electrons, photons, jets, met and ht, btag, xtriggers. Discrepancies in many distributions of reco objects for many trigger paths

360pre2 details

? Muon: same issue with an L1 inefficiency as seen since 3.5.0.pre5
? BPAG: drop of ~10% efficiency in the barrel at L2 in HLT_Mu5, HLT_Mu9 and HLT_DoubleMu3 paths due to the presence of L1 ghosts.
? Susy: Comparison with 350. Small reduction of L1 mu efficiency as for 360 pre1. Big reduction of efficiency for HLT_BTag paths (seen in 351 and 352 but not seen in 360pre1). Dicrepany in Jet multiplicity for HLT_BTag paths.

360pre1 details

? Muon: same issue with an L1 inefficiency as seen since 3.5.0.pre5
? B physics: muon issues at L1 RPC/GMT detected in 35X remain in 36X; awaiting status clarification from L1/RPC experts in l1 hn
? Top: improvement in muon trigger efficiencies, drop wrt 3.5.0.pre3 reduced to ~2%.
? Higgs: HLTMu9 and HLTIsoMu3 eff decrease wrt 3.5.0.pre3 remains.
? BPAG: drop of ~10% efficiency in the barrel at L2 in HLT_Mu5, HLT_Mu9 and HLT_DoubleMu3 paths due to the presence of L1 ghosts.
? Susy: small but systematic lower efficiency for L1 muons.

Release 35X

  Release Group
    Electron Photon Muon Jet MET Tau Calibration Top Higgs HeavyFlavor Susy/Exotica
  350pre1 No Minus Yes / Done Yes / Done Yes / Done Minus Yes / Done No No Yes / Done Minus
  350pre2 No Minus Yes / Done Yes / Done Yes / Done Minus Yes / Done No No Yes / Done Minus
  350pre3v1 No Minus Yes / Done Yes / Done Yes / Done Minus Minus Yes / Done No Yes / Done Minus
  350pre3v2 No Minus Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Minus Minus Minus
  350pre5 No Minus No Yes / Done Yes / Done Minus Yes / Done No No No Minus
  350pre5v2 Minus Minus No Yes / Done Yes / Done Minus Minus Minus Minus Minus Minus
  350pre5g493 Minus Minus No Minus Yes / Done Minus Minus Minus Minus Minus Minus
  350 No Minus No Yes / Done Yes / Done Minus Yes / Done No Yes / Done No Yes / Done
  351 Minus Minus No Yes / Done Yes / Done Minus Yes / Done No No No No
  352 Minus Minus No Yes / Done Yes / Done Yes / Done Yes / Done No No No No
  353 Minus Minus No Yes / Done Yes / Done Minus Yes / Done No No No Minus
  354 Minus Yes / Done Minus Yes / Done Yes / Done Minus Yes / Done Minus Minus No Minus
  355 bad Minus Yes / Done No Yes / Done Minus Yes / Done Minus Minus Minus No Minus
  357 Minus Work in progress, under construction Yes / Done Yes / Done Yes / Done Yes / Done Minus Yes / Done Yes / Done Yes / Done No
  358 Minus Minus Yes / Done Minus Yes / Done Yes / Done Minus Yes / Done Yes / Done Minus Minus
  35x Minus Minus Minus Minus Minus Minus Minus Minus Minus Minus Minus
note: v1 (slc5_ia32_gcc434) v2 (slc5_amd64_gcc434) ; bad release has been deprecated (see)

358 details

357 details

? HeavyFlavor: L2 performance is back to that of CMSSW_3_5_0_pre3 by removing the wide patterns from RPCs and therefore L1 ghosts and due to the improved handling of L1 ghosts at L2
? Jets: The efficiency is better than in the previous release. Turn-on curves are sharper. It is due to the fact that scheme6 is now used instead of the old SchemeB.
? Photons: see discussion in the trigval forum, outcome: performance as expected in 357
? Susy: Lower efficiencies for almost all the L1 triggers. Lower efficiencies in many HLT paths. For Electron paths, much larger efficiency for HLT_Electron_SiStrip paths, but lower for the others. For photon paths, larger efficiency for DoublePhoton paths, lower for the others. Lower efficiency for almost all the jet, MET and HT paths. Much lower efficiency for BTagIP_Jet* paths. Lower efficiency for cross triggers Ele10_TripleJet and Ele10_HT. Discrepancies in distributions for reco electrons, photons, muons, jets and MET, but this is s reco issue: they are not trigger dependent.

355,356 details

? release 355 has been deprecated and replaced by 356; the difference between the two is unrelated to the trigger; the 355 samples should be used for validation (no 356 samples have been produced)

354 details

353 details

? Muon: same issue with an L1 inefficiency as seen since 3.5.0.pre5

353 details

? Muon: same issue with an L1 inefficiency as seen since 3.5.0.pre5
? BPAG: drop of ~10% efficiency in the barrel at L2 in HLT_Mu5, HLT_Mu9 and HLT_DoubleMu3 paths due to the presence of L1 ghosts.

351/2 details

? Top: identical to 350
? Tau: slightly broader distributions for the ECAL isolation (mean value 1.48 vs 1.38 in 351)
? Muon: same issue with an L1 inefficiency as seen since 3.5.0.pre5
? Susy: ~20% lower efficiency for HLT_BTagIp_Jet80 and HLT_BTagJet120. Difference in reco electron distributions for these two triggers. (wrt 342)
? Higgs: small decrease in HLT_Mu9 and HLT_IsoMu3 efficiencies.
? BPAG: drop of ~10% efficiency in the barrel at L2 in HLT_Mu5, HLT_Mu9 and HLT_DoubleMu3 paths due to the presence of L1 ghosts.

350 details

? Muons: Drop in L1 efficiency of ~5% for most paths that use an L1 quality cut of 5 or 6.
? Bphysics: Drop of ~10% efficiency in the barrel at L2 in HLT_Mu5, HLT_Mu9 and HLT_DoubleMu3; these paths have pT cuts at L2 and presumably a degradation in pT resolution is the cause of the problem.
? Electrons in fastsim: underestimate of the pixel-matching efficiency compared to the full simulation at the edge of the pixel-acceptance, reduced wrt 350pre5 but still remains; small underestimation of efficiency in the track matching step in the barrel/disk transition region
? Top: overall muon trigger efficiency drop wrt 350pre3 reduced but sill visible
? Top fastsim: slightly higher muon efficiencies(2-3%) wrt Fullsim, otherwise agreement with fullsim is good
? Higgs: muon efficiencies now compatible with 3.5.0.pre3; HLT_Ele10_LW_EleId_L1R still a bit lower as expected
? updates on muon issues in 350: (i) L2 inefficiencies are found to be due to L1: modified spectrum of the seeding L1 muons; run 350 HLT on 350pre3 sample using the L1 object available there and observed no problems (Zoltan); reco inefficiencies, see thread in muon obj hn concludes diffs verified wrt 350pre5 are not due to changes after the raw stage; no clear explanation available in terms of modified tags introduced (Andrea P.). Updates by RPC/GMT experts on the L1 DPG meeting 4.3.2010, issue continues to be studied (NL).
? update on fastsim issues: in 351/2 expect better agreement Fast/Full in the Tracking in endcaps, and in the Jets/MET energy scale (Andrea P.), see famos-hn
? BPAG: drop of ~10% efficiency in the barrel at L2 in HLT_Mu5, HLT_Mu9 and HLT_DoubleMu3 paths due to the presence of L1 ghosts.

350pre5 details

? Muons: Drop in L1 efficiency of ~10% for most paths that use an L1 quality cut of 5 or 6.
? BPAG: Drop in L1 efficiency of ~10% in the barrel part for HLTLevel1GTSeeds that require a pT cut
? Electron, fastsim: overall good agreement with full-sim, except for minor issues: in pixel end-caps, and track matching step, which perform below full-sim expectations
? Electron, updates on pixel-less seeding: looking at the map of the SiStrip dead channels, the efficiency loss looks compatible with the damages in the phi coordinate; whereas for the eta coordinate the loss appears mainly in the TIB+ while the majority of dead channels are in the TIB-; this could be explained, though, if the SiStrip seeding does not look for all layers, but gives priority (or is exclusive) to TIB inner layers; in this case the leaking cooling loop of TIB+ L2 could explain the fact. (from R.Covarelli) confirmed (Avishek) that SiStrip seeding does not look in all layers (use first two rings of TIB, all rings of TID, and the first two rings of the first three wheels of TEC), and does give priority to TIB rings 1 and 2 through various cuts. This is done to reduce execution time and seed multiplicity, but makes the algorithm suffer when there are such hardware problems.
? Muon, update of L1 inefficiency: the source of the problems appears to have been identified as related to introduction of RPC cosmic patterns which affected the GMT muons (from Vasile); the temporary solution being implemented involves updating a GMT record from the global tag (requiring a new global tag for 35X); a more persistent solution should also be envisaged in the future.
? BPAG: drop of ~10% efficiency in the barrel at L2 in HLT_Mu5, HLT_Mu9 and HLT_DoubleMu3 paths due to the presence of L1 ghosts.

350pre3 details

? Electrons: In addition to the track matching problem seen in recent releases there is now an additional problem with the pixelless seeding (as seen for HLT_Ele15_SiStrip_L1RD). Checked not to be related to 32 vs 64 bit archs. Avishek traced these problem to the transition from 'STARTUP3X_V15::All' to 'STARTUP3X_V14::All'.

350pre1,2 details

? Electrons: track matching has efficiency problems
? Top: HLT_IsoMu9 efficiency seems to be slightly lower (~2-3%) wrt 340.
? Higgs: HLT_Ele10_LW_EleId_L1R efficiency not yet recovered.

Release 34X

  Release Group
    Electron Photon Muon Jet MET Tau Calibration Top Higgs HeavyFlavor Susy/Exotica
  340pre1 Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done Yes / Done n.s. No
  340pre2 Yes / Done Yes / Done No Yes / Done Yes / Done Yes / Done Yes / Done No No n.s. No
  340pre3 Minus Minus Minus Minus Minus Minus Minus Minus Minus n.s. Minus
  340pre4 Minus Minus No No Minus Minus Minus Minus Minus No Minus
  340pre5 Yes / Done Yes / Done No No Yes / Done Minus Yes / Done No No No Minus
  340pre6 No Yes / Done Yes / Done Yes / Done Yes / Done Minus Minus Yes / Done No Yes / Done Minus
  340pre7