Validation of Fast Simulation code

Legend:

• : Validation done, result is "Not Validated". Report the reasons below
• : Validation done, result is "Ok, Validated"
• : Validation still in progress or results need to be followed up before final green/red light
• : Validation not (yet) done
• : Validation done, in general results are OK, but there are some reservations/caveats

Release 4_2_X

	Release	Group
		Tracking	Electron	Photon	Muon	Jet	MET	BTag	Tau	PartFlow	HLT
	4_2_0_pre8
	4_2_0
	4_2_1
	4_2_2

4_2_0_pre8 details:

• Electron/Photons: tuning of ECAL noise still to be finalized. Fix submitted for 4_2_0
• JET/MET: final HCAL tuning will enter in 4_2_0; wrong energy distribution in HO, fix available for 4_2_0
• Particle Flow: issue with deficit of neutral PF hadrons in FastSim still there.

Release 4_1_X

	Release	Group
		Tracking	Electron	Photon	Muon	Jet	MET	BTag	Tau	PartFlow	HLT
	4_1_3
	4_1_4
	4_1_5

4_1_3 details:

• Electron/Photons: tuning of ECAL noise still to be finalized. Fix submitted for 4_1_4
• Muons: bug in the event vertex generator visible as broader distributions in the STA/L2 muon updated at vertex. Fix submitted for 4_1_4
• Particle Flow: issue with deficit of neutral PF hadrons in FastSim still there.

Release 3_11_X

	Release	Group
		Tracking	Electron	Photon	Muon	Jet	MET	BTag	Tau	PartFlow	HLT
	3_11_0_pre5
	3_11_0

3_11_0_pre5 details:

• Electron: due to modifications intervened at the FullSIm level, efficiencies of the Sigma_etaeta cuts are diverging from FastSim to FullSim, especially in the endcaps. Some tuning is foreseen shortly.
• JET/MET: HF tuning completed in 3_11_0_pre5 (and 3_10_1 in the 3_10_X serie), and after it response and noise are back in agreement with FullSim

Release 3_10_X

	Release	Group
		Tracking	Electron	Photon	Muon	Jet	MET	BTag	Tau	PartFlow	HLT
	3_10_0_pre4
	3_10_0_pre5
	3_10_0_pre7
	3_10_0_pre8
	3_10_0
	3_10_1

3_10_0_pre7 details:

• Electron, Photon, Jet, MET: changes in Zero Suppression algorithm and thresholds in ECAL not propagated to FastSim. (They were however also wrongly implemented even in FullSim, therefore one has to wai for the next pre-release before evaluating the possible effect on FastSIm distributions.

3_10_X details:

• Muons (Andrea): while the caveats for muons already reported in 38X still hold, we consider them for the moment as a "feature" of muons in FastSim, and we signal with the green light that muons in the release as good "modulo that feature"

Release 3_9_X

	Release	Group
		Tracking	Egamma	Muon	JetMET	BTag	Tau	PartFlow	HLT
	3_9_1
	3_9_2
	3_9_3
	3_9_5
	3_9_7

3_9_X details:

Releases in the cycle older than CMSSW_3_9_1 are not recommended, and therefore deprecated.

As there are not significant changes affecting the final observables in FastSim with respect to 3_8_6 (as most of the latest additions had been already backported in 3_8_x, whenever possible), the same summary as in the previous release cycle can be considered. Every single validator is however invited to modify the entries of his/her pertinence if needed.

• Muons (Andrea): while the caveats for muons already reported in 38X still hold, we consider them for the moment as a "feature" of muons in FastSim, and we signal with the green light that muons in the release as good "modulo that feature"

Release 3_8_X

	Release	Group
		Tracking	Egamma	Muon	JetMET	BTag	Tau	PartFlow	HLT
	3_8_6
	3_8_7

3_8_X details:

Releases in the cycle older than CMSSW_3_8_6 are not recommended, and therefore deprecated.

CMSSW_3_8_6

The main outcomes of the release validation for Fast Simulation are listed below. In general, the comparisons with previous releases do not show any drawbacks; below the results of the comparisons with the corresponding distributions in the Full Simulation are summarized.

• Tracking: regular validation of Tracking in FastSim missing. For a reference with some comparisons in the former releases you can have a look here

• Electrons (Florian and David): report on hn, where you should look for details and possible caveats.

• Photons (Nancy): 3_8_6 specific validation missing, but you can see this report on hn related to 3_9_0, which is basically the same as 3_8_6 for photons in FastSim. The open issue here is on the lack of ECAL seeded conversions in FastSim, while distribution from calorimetric photons and track seeded conversions are in good shape.

• Muons (Andrea): report on hn. Usual caveats are that FastSim lack decays in flight out of the tracker volume: effects on the muon distributions and collections are summarized here.

• Jet-MET (Salavat, Keng, Dayong): report on hn, where you should look for details and possible caveats.

• BTag (Aram): report on hn, where you should look for details and possible caveats.

• Tau: regular validation of Tau related objects in FastSim missing.

• Particle Flow: although no official release validation was produced recently, there are reports since longtime that the Neutral Hadron content of PF showers is underestimated in FastSim, and this leads PF people declaring FastSim not validated for PF. A summary of the issue and of its effects can be found here

• HLT: all results, also for FastSIm whenever prepared, are available here: click on the validation's icons to get a link to the specific reports.

Release 37X

Release

Group

Tk

Ecal

Hcal

DT

CSC

RPC

Tracking

Egamma

Muon

JetMET

BTag

Tau

PartFlow

HLT

370

370patch4

37X details:

370 and following patches

• Goes in synchro with 36
• Particle Flow still "bad" because of the bad description of the hadronic neutral energy in their algorithm (due do some not optimal modeling of the lateral shape of the hadronic showers in the HCAL in the Fast Simulation)

Release 36X

Release

Group

Tk

Ecal

Hcal

DT

CSC

RPC

Tracking

Egamma

Muon

JetMET

BTag

Tau

PartFlow

HLT

360

363

36X details:

362

• Tuning of shower shapes in the endcaps (behind the preshower)
• Particle Flow still "bad" because of the bad description of the hadronic neutral energy in their algorithm (due do some not optimal modeling of the lateral shape of the hadronic showers in the HCAL in the Fast Simulation)

361

• Changed default pileup from 10 TeV to 7 TeV (sqrts)

360

• Improved pixel parametrization

Release 35X

Release

Group

Tk

Ecal

Hcal

DT

CSC

RPC

Tracking

Egamma

Muon

JetMET

BTag

Tau

PartFlow

HLT

350pre5

350

354

355/356

357

358

Status of 351/2/3, 360pre1/2 stays the same as for 350 as there is no major changes in these subsequent versions.

35X details:

358

• Fix: now the TrackExtra collection is saved in the event for pixelTracks

357

• Fix for very high-energy (>1 TeV) particles energy losses in case of big energy deposits in ECAL

355/356

• Improvements in the iterative tracking algorithms;
• Improvements in muon hits multiplicities and in the L1/L2 muon efficiencies

354 plots and materials can be found via the agenda of the Fast Simulation meeting on March, 16 here

350pre5 plots and materials can be found via the agenda of the Fast Simulation meeting on February, 9 here

350pre5 (MUONS):

• Release specific issues:
  • Problem with L1 efficiency and spectra (not a FastSim issue, being also observed in the FullSim validation, and to be dealt with by L1 muon group)
• Remaining issues in the Fast/Full sim comparison:
  • L1,L2,L3 muons slightly overefficient in FastSim
  • Calo deposits not correctly associated to muons
  • muon segments (used in muon Id) too close to the extrapolation of the tracker track

350pre5 (CMS.JetMET):

• Observed discrepancies (known since some time)
  • Slightly lower energy scale in HB/HE for low-to-medium Jet pT (since chnage in ECAL part of hadron shower in 350pre1)
  • Soft part of the HO spectrum is now wll reproduced for very high-pT jets
  • HF energy scale is OK, but not energy sharing between L and S fibers (even though improved since 331)
  • As a consequence of the above issue, CaloEmEtInHF in MET is softer, while other MET-related distributions demonstrate quite a good agreement
  • Number of jet constituents is lower than in FullSim since long time due to the discrepancy in the hadron transverse shower shape
  • FastSim PFlow Jets has substantially lower "neutral hadron fraction", which makes quite low the overall FastSim PFlow jet energy scale, while regular an agreement in CMS.CaloJets is quite satisfactory

350pre5 (HLT):

• Good overall performance
• Observed discrepancies
  • Drop of Electron efficiency at the edge of pixels
  • The track-matching step performs below full sim expectations in the barre/endcap transition region
  • TTBar sample HLT evaluation confirms electron and muon observations (FastSim Muon over- and Electron under-efficiency respectively)

350pre5 (Tracking):

• Validation with single pions and single muons: still showing large endcap inefficiencies in iterations 4 and 5. Failed.

350pre5 (Electron):

• Overall good performance
• Few problems observed:
  • GSF track outermost stat: discrepancies visible in the pout and fbrem distributions; to some extent in the deltaeta (SC-track_out) as well. The inefficiency of the pattern recognition should be emulated
  • Shower shape in the endcap: Showers too narrow in the Fast Sim. Long standing issue. Need dedicated tuning (unconverted photons)
  • ECAL mis-calibration absent in startup scenario: Problem in the cmsDriver. Fixed by Andrea
  • Delta-eta vs eta distribution inverted wrt full sim. Beam position problem ?

350 (Tracking) * SUCCESS: endcap inefficiency solved. Note: FullSim tracking changed, small tuning still needed. Higher efficiency still observed in the barrel.

-- PatriziaAzzi - 2009-09-22