Validation of Fast Simulation code

Legend:
  • No : Validation done, result is "Not Validated". Report the reasons below
  • Yes / Done : Validation done, result is "Ok, Validated"
  • Work in progress, under construction : Validation still in progress or results need to be followed up before final green/red light
  • Minus : Validation not (yet) done
  • Warning, important: Validation done, in general results are OK, but there are some reservations/caveats

Release 4_2_X

  Release Group
    Tracking Electron Photon Muon Jet MET BTag Tau PartFlow HLT
  4_2_0_pre8 Yes / Done Work in progress, under construction Work in progress, under construction Yes / Done Work in progress, under construction Work in progress, under construction Yes / Done Minus No Minus
  4_2_0 Yes / Done Yes / Done Minus Yes / Done Yes / Done Minus Yes / Done Minus No Minus
  4_2_1 Minus Minus Minus Yes / Done Yes / Done Minus Yes / Done Minus No Minus
  4_2_2 Minus Minus Minus Yes / Done Minus Minus Minus Minus No Minus

4_2_0_pre8 details:
  • Electron/Photons: tuning of ECAL noise still to be finalized. Fix submitted for 4_2_0
  • JET/MET: final HCAL tuning will enter in 4_2_0; wrong energy distribution in HO, fix available for 4_2_0
  • Particle Flow: issue with deficit of neutral PF hadrons in FastSim still there.

Release 4_1_X

  Release Group
    Tracking Electron Photon Muon Jet MET BTag Tau PartFlow HLT
  4_1_3 Yes / Done Work in progress, under construction Work in progress, under construction Work in progress, under construction Yes / Done Yes / Done Yes / Done Minus No Minus
  4_1_4 Yes / Done Minus Minus Yes / Done Minus Minus Yes / Done Minus No Minus
  4_1_5 Minus Minus Minus Yes / Done Minus Minus Minus Minus No Minus

4_1_3 details:
  • Electron/Photons: tuning of ECAL noise still to be finalized. Fix submitted for 4_1_4
  • Muons: bug in the event vertex generator visible as broader distributions in the STA/L2 muon updated at vertex. Fix submitted for 4_1_4
  • Particle Flow: issue with deficit of neutral PF hadrons in FastSim still there.

Release 3_11_X

  Release Group
    Tracking Electron Photon Muon Jet MET BTag Tau PartFlow HLT
  3_11_0_pre5 Minus Work in progress, under construction Minus Yes / Done Yes / Done Yes / Done Minus Minus No Minus
  3_11_0 Minus Work in progress, under construction Minus Yes / Done Minus Minus Minus Minus No Minus

3_11_0_pre5 details:
  • Electron: due to modifications intervened at the FullSIm level, efficiencies of the Sigma_etaeta cuts are diverging from FastSim to FullSim, especially in the endcaps. Some tuning is foreseen shortly.
  • JET/MET: HF tuning completed in 3_11_0_pre5 (and 3_10_1 in the 3_10_X serie), and after it response and noise are back in agreement with FullSim

Release 3_10_X

  Release Group
    Tracking Electron Photon Muon Jet MET BTag Tau PartFlow HLT
  3_10_0_pre4 Minus Warning, important Warning, important Yes / Done Yes / Done Yes / Done Minus Minus No Warning, important
  3_10_0_pre5 Minus Warning, important Warning, important Yes / Done Yes / Done Yes / Done Minus Minus No Warning, important
  3_10_0_pre7 Minus No Minus Yes / Done No No Minus Minus No Minus
  3_10_0_pre8 Minus Minus Minus Yes / Done Minus Minus Minus Minus Minus Minus
  3_10_0 Minus Minus Minus Yes / Done Minus Minus Minus Minus Minus Minus
  3_10_1 Minus Work in progress, under construction Minus Yes / Done Yes / Done Yes / Done Minus Minus No Minus

3_10_0_pre7 details:

  • Electron, Photon, Jet, MET: changes in Zero Suppression algorithm and thresholds in ECAL not propagated to FastSim. (They were however also wrongly implemented even in FullSim, therefore one has to wai for the next pre-release before evaluating the possible effect on FastSIm distributions.

3_10_X details:

  • Muons (Andrea): while the caveats for muons already reported in 38X still hold, we consider them for the moment as a "feature" of muons in FastSim, and we signal with the green light that muons in the release as good "modulo that feature"

Release 3_9_X

  Release Group
    Tracking Egamma Muon JetMET BTag Tau PartFlow HLT
  3_9_1 Minus Warning, important Yes / Done Yes / Done Warning, important Minus No Warning, important
  3_9_2 Minus Warning, important Yes / Done Yes / Done Warning, important Minus No Warning, important
  3_9_3 Minus Warning, important Yes / Done Yes / Done Minus Minus No Warning, important
  3_9_5 Minus Warning, important Yes / Done Yes / Done Minus Minus No Warning, important
  3_9_7 Minus Minus Yes / Done Minus Minus Minus No Minus

3_9_X details:

Releases in the cycle older than CMSSW_3_9_1 are not recommended, and therefore deprecated.

As there are not significant changes affecting the final observables in FastSim with respect to 3_8_6 (as most of the latest additions had been already backported in 3_8_x, whenever possible), the same summary as in the previous release cycle can be considered. Every single validator is however invited to modify the entries of his/her pertinence if needed.

  • Muons (Andrea): while the caveats for muons already reported in 38X still hold, we consider them for the moment as a "feature" of muons in FastSim, and we signal with the green light that muons in the release as good "modulo that feature"

Release 3_8_X

  Release Group
    Tracking Egamma Muon JetMET BTag Tau PartFlow HLT
  3_8_6 Minus Warning, important Warning, important Yes / Done Warning, important Minus No Warning, important
  3_8_7 Minus Warning, important Warning, important Yes / Done Warning, important Minus No Warning, important

3_8_X details:

Releases in the cycle older than CMSSW_3_8_6 are not recommended, and therefore deprecated.

CMSSW_3_8_6

The main outcomes of the release validation for Fast Simulation are listed below. In general, the comparisons with previous releases do not show any drawbacks; below the results of the comparisons with the corresponding distributions in the Full Simulation are summarized.

  • Tracking: regular validation of Tracking in FastSim missing. For a reference with some comparisons in the former releases you can have a look here

  • Electrons (Florian and David): report on hn, where you should look for details and possible caveats.

  • Photons (Nancy): 3_8_6 specific validation missing, but you can see this report on hn related to 3_9_0, which is basically the same as 3_8_6 for photons in FastSim. The open issue here is on the lack of ECAL seeded conversions in FastSim, while distribution from calorimetric photons and track seeded conversions are in good shape.

  • Muons (Andrea): report on hn. Usual caveats are that FastSim lack decays in flight out of the tracker volume: effects on the muon distributions and collections are summarized here.

  • Jet-MET (Salavat, Keng, Dayong): report on hn, where you should look for details and possible caveats.

  • BTag (Aram): report on hn, where you should look for details and possible caveats.

  • Tau: regular validation of Tau related objects in FastSim missing.

  • Particle Flow: although no official release validation was produced recently, there are reports since longtime that the Neutral Hadron content of PF showers is underestimated in FastSim, and this leads PF people declaring FastSim not validated for PF. A summary of the issue and of its effects can be found here

  • HLT: all results, also for FastSIm whenever prepared, are available here: click on the validation's icons to get a link to the specific reports.

Release 37X

  Release Group
    Tk Ecal Hcal DT CSC RPC Tracking Egamma Muon JetMET BTag Tau PartFlow HLT
  370 Minus Minus Minus Minus Minus Minus Yes / Done Warning, important Warning, important Yes / Done Warning, important Minus No Warning, important
  370patch4 Minus Minus Minus Minus Minus Minus Yes / Done Warning, important Warning, important Yes / Done Warning, important Minus No Warning, important

37X details:

370 and following patches

  • Goes in synchro with 36
  • Particle Flow still "bad" because of the bad description of the hadronic neutral energy in their algorithm (due do some not optimal modeling of the lateral shape of the hadronic showers in the HCAL in the Fast Simulation)

Release 36X

  Release Group
    Tk Ecal Hcal DT CSC RPC Tracking Egamma Muon JetMET BTag Tau PartFlow HLT
  360 Minus Minus Minus Minus Minus Minus Yes / Done Warning, important Warning, important Yes / Done Warning, important Minus No Warning, important
  363 Minus Minus Minus Minus Minus Minus Yes / Done Warning, important Warning, important Yes / Done Warning, important Minus No Warning, important

36X details:

362

  • Tuning of shower shapes in the endcaps (behind the preshower)
  • Particle Flow still "bad" because of the bad description of the hadronic neutral energy in their algorithm (due do some not optimal modeling of the lateral shape of the hadronic showers in the HCAL in the Fast Simulation)

361

  • Changed default pileup from 10 TeV to 7 TeV (sqrts)

360

  • Improved pixel parametrization

Release 35X

  Release Group
    Tk Ecal Hcal DT CSC RPC Tracking Egamma Muon JetMET BTag Tau PartFlow HLT
  350pre5 Minus Minus Minus Minus Minus Minus No Work in progress, under construction Warning, important Yes / Done Minus Minus No Warning, important
  350 Minus Minus Minus Minus Minus Minus Yes / Done Work in progress, under construction Warning, important Yes / Done Minus Minus No Warning, important
  354 Minus Minus Minus Minus Minus Minus Yes / Done Work in progress, under construction Warning, important Yes / Done Warning, important Minus No Warning, important
  355/356 Minus Minus Minus Minus Minus Minus Yes / Done Work in progress, under construction Warning, important Yes / Done Warning, important Minus No Warning, important
  357 Minus Minus Minus Minus Minus Minus Yes / Done Warning, important Warning, important Yes / Done Warning, important Minus No Warning, important
  358 Minus Minus Minus Minus Minus Minus Yes / Done Warning, important Warning, important Yes / Done Warning, important Minus No Warning, important

Status of 351/2/3, 360pre1/2 stays the same as for 350 as there is no major changes in these subsequent versions.

35X details:

358

  • Fix: now the TrackExtra collection is saved in the event for pixelTracks

357

  • Fix for very high-energy (>1 TeV) particles energy losses in case of big energy deposits in ECAL

355/356

  • Improvements in the iterative tracking algorithms;
  • Improvements in muon hits multiplicities and in the L1/L2 muon efficiencies

354 plots and materials can be found via the agenda of the Fast Simulation meeting on March, 16 here

350pre5 plots and materials can be found via the agenda of the Fast Simulation meeting on February, 9 here

350pre5 (MUONS):

  • Release specific issues:
    • Problem with L1 efficiency and spectra (not a FastSim issue, being also observed in the FullSim validation, and to be dealt with by L1 muon group)
  • Remaining issues in the Fast/Full sim comparison:
    • L1,L2,L3 muons slightly overefficient in FastSim
    • Calo deposits not correctly associated to muons
    • muon segments (used in muon Id) too close to the extrapolation of the tracker track

350pre5 (CMS.JetMET):

  • Observed discrepancies (known since some time)
    • Slightly lower energy scale in HB/HE for low-to-medium Jet pT (since chnage in ECAL part of hadron shower in 350pre1)
    • Soft part of the HO spectrum is now wll reproduced for very high-pT jets
    • HF energy scale is OK, but not energy sharing between L and S fibers (even though improved since 331)
    • As a consequence of the above issue, CaloEmEtInHF in MET is softer, while other MET-related distributions demonstrate quite a good agreement
    • Number of jet constituents is lower than in FullSim since long time due to the discrepancy in the hadron transverse shower shape
    • FastSim PFlow Jets has substantially lower "neutral hadron fraction", which makes quite low the overall FastSim PFlow jet energy scale, while regular an agreement in CMS.CaloJets is quite satisfactory

350pre5 (HLT):

  • Good overall performance
  • Observed discrepancies
    • Drop of Electron efficiency at the edge of pixels
    • The track-matching step performs below full sim expectations in the barre/endcap transition region
    • TTBar sample HLT evaluation confirms electron and muon observations (FastSim Muon over- and Electron under-efficiency respectively)

350pre5 (Tracking):

  • Validation with single pions and single muons: still showing large endcap inefficiencies in iterations 4 and 5. Failed.

350pre5 (Electron):

  • Overall good performance
  • Few problems observed:
    • GSF track outermost stat: discrepancies visible in the pout and fbrem distributions; to some extent in the deltaeta (SC-track_out) as well. The inefficiency of the pattern recognition should be emulated
    • Shower shape in the endcap: Showers too narrow in the Fast Sim. Long standing issue. Need dedicated tuning (unconverted photons)
    • ECAL mis-calibration absent in startup scenario: Problem in the cmsDriver. Fixed by Andrea
    • Delta-eta vs eta distribution inverted wrt full sim. Beam position problem ?

350 (Tracking) * SUCCESS: endcap inefficiency solved. Note: FullSim tracking changed, small tuning still needed. Higher efficiency still observed in the barrel.

-- PatriziaAzzi - 2009-09-22

Edit | Attach | Watch | Print version | History: r34 < r33 < r32 < r31 < r30 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r34 - 2011-04-30 - AndreaPerrotta
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback