Purpose of the page

Collect the validation archive of SWGuideValidationTableReco for release not in use anymore.

Release 41X

SWGuideValidationTableReco41X

Release 311X

SWGuideValidationTableReco311X

Release 310X

SWGuideValidationTableReco310X

Release 39X

SWGuideValidationTableReco39X

Release 38X

SWGuideValidationTableReco38X

Release 37X

Release

Tk

Ecal

Hcal

DT

CSC

RPC

Tracking

Electron

Photon

Muon

Jet

MET

BTag

Tau

PartFlow

370pre1

370pre2

370pre3

370pre4

370pre5

370

371

370pre1 details:

• 20-April-2010 (J. Goh) RPCRecHit : High noise rate due to the digitizer bugs which was already reported from 360pre3, bugfix will be presented in the next pre-release.
• 23-Apr-2010 - (A. Perrotta) Muon POG: same as since 360pre3. It is quite likely linked to the too high noise level mistakenly inserted in the RPC simulation, so all those releases are probably OK for the muon reco: but let wait for the RPC fix become effective before removing the BAD sign...
• 28-May-2010 (Tim Cox) In 370pre1 the CSC cond data indexer is broken due to release of incorrect version. (Not due to CSC developers.)

370pre2 details:

• 14-May-2010 (J. Goh) RPCRecHit : Bugfix applied and noise rate reduced to reasonable value.
• 28-May-2010 (Tim Cox) In 370pre2 the CSC cond data indexer is broken due to release of incorrect version. AND the incorrect gas gains for CSCs escaped again due to HEAD version of CSCDigitizer being released. (Not due to CSC developers.)

370pre5 details:

• 24-May-2010 (P. Janot) Particle Flow reconstruction : tested with newest tags for PFTracking and PFClusterProducer (to be included in 370) with no problem whatsoever. Jets, MET, electrons and muons all ok.
• 27th-May-2010 (A. Vartak) Tracking: Decrease in fake rate in 370pre5 as compared to 370pre4. This can be explained by changes in tracking as listed at https://twiki.cern.ch/twiki/bin/viewauth/CMS/TrackerDPGRelIntegration37X#3_7_0_pre5

370 details:

• 28-May-2010 (Tim Cox) In offline DQM gui there are already samples for real min bi /MinimumBias/CMSSW_3_7_0-GR_R_37X_V4_CMS.RelVal_valskim-v1/RECO. CSC local reco looks fine.

Release 36X

Release

Group

Tk

Ecal

Hcal

DT

CSC

RPC

Tracking

Electron

Photon

Muon

Jet

MET

BTag

Tau

PartFlow

360pre1

360pre2

360pre3

360pre4

360pre5

skip

360pre6

360

361

361_patch1

362

363

360pre1 details:

• 5-Mar-2010 - (A.Giammanco) Tracker DPG: A small systematic shift is present for RecHit charges in TEC and TID; no such effect observed at digi level.

• 8-Mar-2010 - (B.Mangano) Tracking POG:

1. significant increase of the fake rates for "loose" tracks. Fake rates is higher (compared to 35X release) in particular around |eta| ~ 1.5.
2. Fake rate increases both for perfect-alignment and startup-alignment samples.
3. Iteration2 tracks have slightly lower efficiency in the endcaps. Iteration3 tracks have higher fake rate at 1<|eta|<1.5 . Iteration5 tracks have significantly higher fake rate in the whole eta range.
4. Fake rate is almost unchanged for "highPurity" tracks, expect for the QCD_Pt_3000_3500 sample, which has an higher fake rate too.
5. link to results link

360pre2 details:

• 8-Mar-2010 - (B.Mangano) Tracking POG: Tracking performance in 360pre2 is fully compatible with 360pre1. However, the performance is still much worse (in particular the fake rate) than in 35X. Differences are still to be understood.
• 3-Mar-2010 - (P. Janot, C. Bernet, M. Gouzevitch) Particle-Flow POG: Tracking fake rate generates lots of tails in jet pt spectrum (for pt above 200-300 GeV/c and eta ~ 1.3), especially in the third iteration which now reconstructs many fake tracks when a nuclear interaction occurs . Talked to Tracking POG conveners (K. Burkett) who says that K. Stensson is working on it, and that a solution will appear in pre4.

360pre3 details:

• 16-Mar-2010 - (C. Charlot) Higher number of fakes on QCD jets by ~10% (average fraction of electrons matched to a reco jet moving from 0.086 to 0.096). Slight changes in the number of collected hits per gsf track in the barrel, the efficiency is not affected.
• 17-Mar-2010 - (B.Mangano) Tracking POG:

1. Most striking differences in tracking performance are visible for the QCD_3TeV sample (plot). The differences can be explained by the change in the LHC energy (from 10TeV to 7TeV), as seen for 353 release (link), except for the efficiency_vs_pt plot: this has to be followed up.
2. Performance for all the other samples look very similar to 360pre2. In general, slight improvement in the fake rate.

• 20-Mar-2010 - (A. Perrotta) Muon POG:
  - STA muon inefficiency at |eta|~0.8 in multiparticle CMS.RelVal samples
  
• 01-Apr-2010 - (T. Cox) In 360pre3 the CSC cond data indexer is broken due to release of incorrect version. And the CSC gas gain in CSCDigitizer was incorrect due release of HEAD version. (Not due to CSC developers.)
• 04-Apr-2010 - (J. Goh) RPCRecHits validation : High noise rate due to the bug of RPCDigitizer. Otherwise OK.

360pre4 details:

• 26-Mar-2010 - (M. Sani) Validation ok for Photons, waiting for results from Electrons.
• 29-Mar-2010 - (B.Mangano) Tracking POG: Tracking performance in 360pre4 would be OK according to the validation plots used usually by the tracking group. However, the PF POG sees a degradation in PF performance which seems to be related to a larger number of fake tracks in 36X releases. No consensus on the source of this effect has been achieved yet.
• 29-Mar-2010 - (A. Perrotta) Muon POG: Same issues as in 360pre3, namely:
  - STA muon inefficiency at |eta|~0.8 in multiparticle CMS.RelVal samples
  (Details in https://hypernews.cern.ch/HyperNews/CMS/get/muon-object-validation/118.html)
• 25-Mar-2010 - (Patrick Janot) Particle Flow POG. The tracking fake rate in the third iteration (eta ~ 1.0->1.6 and eta ~ 2.0) has not decreased. Working in collaboration with tracking POG to solve this issue (solved on March 30th (k. Stenson) by a tuning of the number of shared hits allowed to create a new track.). A tag from the tracking POG will appear in pre6.
• 5-Apr-2010 - (A.Giammanco) huge discrepancies in the low-charge region of RecHits, but no impact on their multiplicity (apart TID where Nhits is slightly different); slight differences in charge distributions for TrackingRecHits too, but small effect on multiplicity, pulls and resolutions unaffected.
• 28-May-2010 (Tim Cox) In 360pre4 the CSC cond data indexer is broken due to release of incorrect version. (Not due to CSC developers.)

360pre6 details:

• 12-Apr-2010 - (A. Perrotta) Muon POG: same as in pre3.
• 12-Apr-2010 - (P. Janot) Validation of jets, MET, electrons, taus and muons ok for Particle Flow. All tracking outliers disappeared. There is one event with two generated muons reconstructed with three muons, one being clearly a duplication. In contact with muon POG. Fast Sim tracking is now off for all iterations and needs retuning. Finally, the largest contributor for MET and jet tails is now the many (many) 5xt5 crystal arrays masked in the electromagnetic calorimeter. An attempt to compensate for this energy loss was made in pre4, but is shown to create high energy tails as well and needs to be removed from 360.
• 13-Apr-2010 - (Nancy-Claude) Conversions: ECAL seeded conversions NOT succesfull Much lower reconstrcution efficiency (problem identified and fixed). Electrons: there is a very significant change in the # of ambiguous tracks found attached to the electron track, toward a decrease of ambiguous tracks. It also reflect in a slightly decreased charge misID. All other distributions are found very much compatible with previous prerelease. Problem understood and related to the changes in tracking parameters (# shared hit in the cleaning). No action needs to be taken.
• 28-May-2010 (Tim Cox) In 360pre6 the CSC cond data indexer is broken due to release of incorrect version. (Not due to CSC developers.)

360 details:

• 23-Apr-2010 - (A. Perrotta) Muon POG: same as since 360pre3. It is quite likely linked to the too high noise level mistakenly inserted in the RPC simulation, so all those releases are probably OK for the muon reco: but let wait for the RPC fix become effective before removing the BAD sign...
• 28-May-2010 (Tim Cox) In 360 the CSC cond data indexer is broken due to release of incorrect version. (Not due to CSC developers.)

361 details:

• 13-May-2010 - (Nancy - David) Wrong SC energy scale in Endcap, drop in efficiency in photons in EE. This is probably related to the Ecal/ES simulation change which was back-ported from 370pre1->4 into 361. Either a condition mismatch in MC->RECO, or an effect of the PRE-shower (now known) SIM / RECO inconsistency in 361.
• 15-May-2010 - (J. Goh) RPCRecHits : Bugfix applied and the noise rate reduced to the reasonable values.

361patch1 details:

• 02-Jun-2010 (Tim Cox) Offline dqm gui contains RelvalTTBar samples - OK for CSC local reco.

362 details:

• 14-Jun-2010 - (A.Perrotta) Muon POG (all the same also holds for 363):
  • StandAlone muon resolutions (pT and i.p.) for 1 TeV pT single muons are slightly worse. To be checked
  • StandAlone SET muon resolutions are screwed up at large muon momenta. To be checked and fixed
  • Change of i.p. resolutions of the updated-at-vertex STA and L2 (in both fast and full simulation). To be checked

Release 35X

Release

Group

Tk

Ecal

Hcal

DT

CSC

RPC

Tracking

Egamma

Muon

Jet

MET

BTag

Tau

PartFlow

350pre1

350pre2

350pre3

350pre4

350pre5

350

351

352

352patch2

353

354

355

357

358

350pre3 details:

• 20-Jan-2010 - (J. Keller) bTag: The combined secondary vertex MVA tagger shows degraded performance compared to 350pre2. C. Saout traced this back to the conditions global tag grabbing an old version of the MVA calibration.
• 30-Jan-2010 - Electrons (vs pre1) (C. Charlot): SingleElectron and Zee: pT spectrum shifted toward lower values, effect is localised in the endcaps, differences in # of collected track hits in the barrel. QCD: changes in pT distribution, shift in E/p ditsribution, differences in # of collected track hits in the barrel (h_ele_foundHits, h_ele_lostHits).
• 01-Feb-2010 -- Tracking POG (Kelley): Strange drop in efficiency observed in step zero tracks around -1.5 < φ < -1.0. hypernews. Apparently there was an additional deal layer in pixels in barrel, layer 1, (BPix_BpI_SEC8_LYR1_LDR9_MOD2), and at Z = 10.0516 and Phi = -1.22175. This is the reason for the drop in efficiency in this phi region.

350pre5 details:

• 02-Feb-2010 - Tracking (vs pre3): only change is a small improvement in chi2 together with a very small decrease in the number of valid hits on track (mostly visible in single mu pt 10 under ideal conditions); unexpected and being investigated, probably related to the global tag. in any case, not unwelcome.
• 04-Feb-2010 -- Egamma: confirmed differences in #hits for electron tracks (even if less w.r.t. to pre3 vs pre1) and in chi2 for both electrons and convertions.
• 09-Mar-2010 - (B.Mangano) Tracking POG: the difference in the tracks chi2 has been proven to be a consequence of the changes in the SiStrip Cross-coupling parametrization and other modifications to Pixel and Strip digitization code. The flag for 350pre5 tracking validation has then been changed to "OK".

350 details:

• 8-Mar-2010 - (B.Mangano) Tracking POG: small difference (wrt to 350pre5) in the chi2 distribution link. Likely due to the changes implemented in the new GlobalTag:new SiPixel templates, new SiStrip APV gains and noise for simulation.

353 details:

• 8-Mar-2010 - (B.Mangano) Tracking POG: efficiency and fake rate for QCD_Pt_3000_3500 sample are different wrt 350 release. The difference is due to the change in the c.m. LHC energy (7 TeV instead of 10 TeV).
• 11-Mar-2010 - (C. Charlot) Important loss of efficiency observed on di-electron gun events, ~4% for |eta|<~1.5 and ~6% for |eta|>2.2. Being investigated with tracking guys. In any case it is not observed in 36X. Problem found and fixed. Validation OK for photons.

Release 34X

Release

Group

Tk

Ecal

Hcal

DT

CSC

RPC

Tracking

Egamma

Muon

Jet

MET

BTag

Tau

PartFlow

340pre1

340pre2

340pre3

340pre4

340pre5

340pre6

340pre7

340

341

342

340pre2 details:

• 22-Oct-2009 - (A. Perrotta) Muons: StandAlone (and, consequently, Global and TeV muon) inefficiency in the barrel showed up in 340pre2, only for the 1000 GeV pt SingleMuon sample.
• 23-Oct-2009 - (R. Kelley) Tracking: Better resolution and fake rate in the barrel due to the recent change to the one dimensional rec hits (SiStripRecHit1D).
• 04-Nov-2009 - (R. Remington) MET: Huge discrepancies were observed in the tcMet distributions. This is due to a bug in the energy component in the Lorentz vector that stores the missing energy components. Thanks to Frank, Slava, and Dayong for spotting this.

340pre4 details:

• 5-Nov-2009 - (A. Perrotta) Muons: all muon collections are fully inefficient above |eta|~1.6.

Release 33X

Release

Group

Tk

Ecal

Hcal

DT

CSC

RPC

Tracking

Egamma

Muon

Jet

MET

BTag

Tau

PartFlow

330pre1

330pre2

330pre3

330pre4

330pre5

330pre6

330

331

331 gcc345 vs gcc432

331 slc4 vs slc5

332

333

334

335

336

331 gcc345 vs gcc432 details

• 28-Oct-2009 (A. Perrotta) Muons: StandAlone (and, consequently, Global and TeV muon) inefficiency in the barrel showed up, only for the 1000 GeV pt SingleMuon sample. The very same inefficiency already observed in 3_4_0_pre2 and reported below. It is not present in the plain 3_3_1, nor in the samples processed with slc5_ia32_gcc434.
• 22-Nov-2009 - (R. Kelley ) Tracking: Drop in efficiency vs phi in step0 in gcc432 w.r.t gcc345. Also noticed in non-pileup events. This is due to the difference in compilers; however, not related to tracking -- see hn posting

330pre4 details:

• 22-Sep-2009, 15:00 - Particle Flow sees no HCAL RecHits in HB/HE, a reduction of 40% of the photon energy and of 5% of the charged energy, in full simulation. Fast Simulation is OK in all respects, which means that the problem(s) come(s) from parts not emulated in fast simulation (digis ? packers/unpackers ? full pattern recognition ? ...). Finally tracked down to Geometry/HcalTowerAlgo detID initialization, fixed in 330pre5 with tags provided by Brian Heltsley.
• Tracking: different pulls due to wrong recomaterial (fixed for pre5).
• BTag: A significant increase in the electron tagger efficiencies was observed (possibly due to the HCAL bug reported by Particle Flow). A significant decrease in the b-jet efficiency was observed for the muon taggers. Re-reconstruction of 330pre3 samples with 330pre4 reconstruction did not signal any reconstruction issues with pre4.

Release 32X

Release

Group

Tk

Ecal

Hcal

DT

CSC

RPC

Tracking

Egamma

Muon

Jet

MET

BTag

Tau

PartFlow

322

326

327

328

322 details:

Release 31X

Release

Group

Tk

Ecal

Hcal

DT

CSC

RPC

Tracking

Egamma

Muon

Jet

MET

BTag

Tau

PartFlow

311

312

313

314

315

316

311 details:

• 09-Jul-2009, 13:30 (Patrick Janot): PFlow reported too low a jet response in end-caps. The problem appeared between 310 and 311. It is due to the absence of ECAL RecHits for |eta| between 1.5 and 1.8 (roughly) and between 2.4 and 3.0 (roughly). Might be related to the otherwise reported bug in L1 (trigger primitives affect the selective readout), or in recent changes in the ECAL code. Hits are otherwise normally found in ECAL barrel, in HCAL , in HF, and in preshower, with the proper energy. The absence of ECAL RecHits in the End-Caps is not seen in the Fast Simulation, only the full simulation is affected. Energy losses in EE confirmed by single electron validation at 15:40 (B. Gobbo) and calotowers validation with QCD, TTbar and private single-pion events at 17:40 (S. Abdoulline). The latter is fine when ECAL packer-unpacker is not used, which points to a bug in this domain.
• 09-Jul-2009 - N. Marinelli - Confirm reduced photon reconstruction efficiency at eta 1.5-1.8 and 2.4-2.5. Observed in single photons with pt=10,35GeV and H->2photons.
• 10-Jul-2009 - C. Charlot - Confirm efficiency problem in region 1.5 < asb(eta) < 1.8 with CMSSW_3_1_1 on all signal samples (SingleElectronPt10, SingleElectronPt35, ZEE). See validation results at http://cmsdoc.cern.ch/cms/Physics/egamma/www/validation/311/vs310/ and in particular efficiency plot vs eta, see eg for Pt35 . It is also correlated with some particular regions in phi.
• 10-Jul-2009 - T. Boccali - After the other comments I looked more deeply at electron but within the big stat errors we have we cannot claim to see any problem. It just means btaggign is not a good test for electrons!
• 14-Jul-2009 - S. Rappoccio - JetMET confirms the same results as the PF, electron, and photon groups, with a reduced efficiency in the same eta gap regions.

312 details:

• 27-Jul-2009 (N.Marinelli) RelVal single photons pt=10, pt=35 and H->2photons show that the reconstruction efficiency is back to normal level and the problem reported in 311 is fixed However monitoring of the photon isolation in ECAL shows an asymmetry in eta: the average et sum of rec hits in the negative endcap is higher than in the positive. Cause under investigation.
• 27-Jul-2009 (B.Gobbo) RelVal Single Electron pt=35 validated against 310, as 311 was plagued by an unpacker problem.
• 28-Jul-2009 (A. Perrotta) POG muons marked as good, even though there is still a pending issue with the RPC geometry, that seems not to affect reconstruction but still has to be fully understood and fixed by the experts.
• 29-Jul-2009 (T.Boccali): Jason Keller (Univ. of Nebraska) found a possible issue with muon btagging (fewer muons in the event) for qcd pt 80-120 and startup conditions. Please have a look at here. COULD be stat effect, but needs to be understood. Also, we see the same 322 vs 321 (here). Addendum: after careful checking of both 312 and 322, the differences in the soft muon taggers appear to be due to monte carlo fluctuations due to the low statistics of the QCD_Pt_80_120 data samples.