Legacy-2017 Analysis (Private)

Purpose of this page.

This page collects useful information to be aware of when performing analysis or producing ntuples for analysis. Guidelines are collected on which release, detector conditions, and datasets. We also keep track of special filters or tools useful to remove atypical and problematic events.

Last updated: 7 June 2020

Analysis Recipe

Release

Recommended release: CMSSW_10_6_20
  • This release is good for analysis and also any private production for all Legacy-16/17/18.

*Important note

  • With CMSSW_10_6_20, Run2-Legacy MiniAODv2
  • With CMSSW_10_6_20, NanoAODv8
  • Missing MuonReducedTrackExtra in data AOD, MiniAOD
    • Available in MC (Summer20 campaign)

Conditions

  • Data
    • recommended GT for analysis: 106X_dataRun2_v33
      • As of 2020-04-29: JEC/JER, please use text and database files provided by JERC (see link to JetMET below)
      • v28 GT is good for all Legacy analysis, including special runs.
    • GT used in AOD production: 106X_dataRun2_v20
    • GT used in (Re) MiniAODv2 production: 106X_dataRun2_v33

  • MC
    • recommended GT for analysis: 106X_mc2017_realistic_v9
      • include UL17 JEC (v8) and low-pT EGM tags (v9)
    • GT used in production: 106X_mc2017_realistic_v9
    • GT used in production ReNanoAOD (v9): To be updated

Trigger (MC)

  • CMSSW_9_4_14_UL_patch1 with HLT:2e34v40 is used for trigger simulation.

Validation

Target (CMSSW_10_6_1)

Dataset name PU statistics link
QCD_FlatPt_15_3000 no PU 1000K https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_1__fullsim_noPU_2017_HS-1561737890-QCD_FlatPt_15_3000HS_13
ZMM no PU 100K https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_1__fullsim_noPU_2017_HS-1563190560-ZMM_13
ZEE no PU 100K https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_1__fullsim_noPU_2017_HS-1563190560-ZEE_13
NuGun PU statistics https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_1__fullsim_PU_2017_HS_highIO-1561739485-NuGun
QCD_FlatPt_15_3000 PU 1000K https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_1__fullsim_PU_2017_HS_highIO-1561739485-QCD_FlatPt_15_3000HS_13
TTbar PU 100K https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_1__fullsim_PU_2017_HS_highIO-1561739485-TTbar_13
ZEE PU 100K https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_1__fullsim_PU_2017_HS_highIO-1561739485-ZEE_13
ZMM PU 100K https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_1__fullsim_PU_2017_HS_highIO-1561739485-ZMM_13

With special Ecal GT: 106X_mc2017_realistic_forECAL_v6 (CMSSW_10_6_1)

Dataset name PU statistics link
QCD_FlatPt_15_3000 no PU 1000K http://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_1__fullsim_noPU_2017_HS_forEcal-1563190857-QCD_FlatPt_15_3000HS_13
ZEE no PU 100K http://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_1__fullsim_noPU_2017_HS_forEcal-1563190857-ZEE_13
ZMM no PU 100K http://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_1__fullsim_noPU_2017_HS_forEcal-1563190857-ZMM_13
QCD_FlatPt_15_3000 PU 1000K http://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_1__fullsim_PU_2017_HS_forEcal_highIO-1563191466-QCD_FlatPt_15_3000HS_13
ZEE PU 100K http://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_1__fullsim_PU_2017_HS_forEcal_highIO-1563191466-ZEE_13
ZMM PU 100K http://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_1__fullsim_PU_2017_HS_forEcal_highIO-1563191466-ZMM_13

Reference (CMSSW_9_4_14_UL)

Dataset name PU statistics link
NuGun PU 100K http://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_9_4_14_UL__fullsim_PU_2017_HS_rsb_highIO-1562678425-NuGun
QCD_FlatPt_15_3000 PU 1000K http://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_9_4_14_UL__fullsim_PU_2017_HS_rsb_highIO-1562678425-QCD_FlatPt_15_3000HS_13
TTbar PU 100K http://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_9_4_14_UL__fullsim_PU_2017_HS_rsb_highIO-1562678425-TTbar_13
ZEE PU 100K http://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_9_4_14_UL__fullsim_PU_2017_HS_rsb_highIO-1562678425-ZEE_13
ZMM PU 100K http://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_9_4_14_UL__fullsim_PU_2017_HS_rsb_highIO-1562678425-ZMM_13

Target (CMSSW_10_6_0) for HLT (trigger simulation in 94X)

Dataset name PU statistics link
MonoJet realistic PU 100k https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_0__ulhlt17hs_pmx-1558542948-ADDMonoJet_d3MD3_13UP17
BuMixing realistic PU 100k https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_0__ulhlt17hs_pmx-1558542948-BuMixing_13UP17
DisplacedJet realistic PU 100k https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_0__ulhlt17hs_pmx-1558542948-DispSUSY_M300_1000mm_13UP17
GunProdFiveEl realistic PU 100k https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_0__ulhlt17hs_pmx-1558542948-GunProdFiveEl_13UP17
GunProdFiveMu realistic PU 100k https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_0__ulhlt17hs_pmx-1558542948-GunProdFiveMu_13UP17
Higgs200ChargedTaus realistic PU 100k https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_0__ulhlt17hs_pmx-1558542948-Higgs200ChargedTaus_13UP17
QCD_FlatPt_15_3000HS realistic PU 100k https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_0__ulhlt17hs_pmx-1558542948-QCD_FlatPt_15_3000HS_13UP17
QCD_Pt_3000_3500_13 realistic PU 100k https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_0__ulhlt17hs_pmx-1558542948-QCD_Pt_3000_3500_13UP17
QQH1252T realistic PU 100k https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_0__ulhlt17hs_pmx-1558542948-QQH1252T_13UP17
QQH1352T realistic PU 100k https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_0__ulhlt17hs_pmx-1558542948-QQH1352T_13UP17
SingleMuPt100 realistic PU 100k https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_0__ulhlt17hs_pmx-1558542948-SingleMuPt100_UP17
TTbarLepton realistic PU 100k https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_0__ulhlt17hs_pmx-1558542948-TTbarLepton_13UP17
TTbar realistic PU 100k https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_0__ulhlt17hs_pmx-1558542948-TTbar_13UP17
Upsilon1SToMuMu realistic PU 100k https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_0__ulhlt17hs_pmx-1558542948-Upsilon1SToMuMu_13UP17
VBF_H_NNPDF30_13TeV realistic PU 100k https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_0__ulhlt17hs_pmx-1558542948-VBF_H_NNPDF30_13TeV
WpM realistic PU 100k https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_0__ulhlt17hs_pmx-1558542948-WpM_13UP17
Zee realistic PU 100k https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_0__ulhlt17hs_pmx-1558542948-ZEE_13UP17
Zmumu realistic PU 100k https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_0__ulhlt17hs_pmx-1558542948-ZMM_13UP17
ZpEE realistic PU 100k https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_0__ulhlt17hs_pmx-1558542948-ZpEE_13UP17
ZpTT realistic PU 100k https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_0__ulhlt17hs_pmx-1558542948-ZpTT_13UP17
ZTT realistic PU 100k https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_10_6_0__ulhlt17hs_pmx-1558542948-ZTT_13UP17

Data High Statistics rereco: datasets and Conditions

ZeroBias
DoubleMuon
MET
JetHT
SingleElectron


2017C (299481,299480,299479,299478,299477, 299593, 299594, 299595, 299597),
2017F (306459,306456,305064, 306125)

DQM, AlCa and SKIM following UL2017 proposal

CMSSW_10_6_1 with GT 106X_dataRun2_v15

https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=ReReco-Run2017C-ZeroBias-29Jun2019_UL2017validation-0001

https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=ReReco-Run2017C-ZeroBias-29Jun2019_UL2017validation-0001

https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=ReReco-Run2017C-MET-29Jun2019_UL2017validation-0004

https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=ReReco-Run2017C-JetHT-29Jun2019_UL2017validation-0002

https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=ReReco-Run2017C-DoubleMuon-29Jun2019_UL2017validation-0003

https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=ReReco-Run2017F-ZeroBias-29Jun2019_UL2017validation-0002

https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=ReReco-Run2017F-SingleElectron-29Jun2019_UL2017validation-0001

https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=ReReco-Run2017F-MET-29Jun2019_UL2017validation-0003

https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=ReReco-Run2017F-JetHT-29Jun2019_UL2017validation-0005

https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=ReReco-Run2017F-DoubleMuon-29Jun2019_UL2017validation-0004

Available Datasets in DAS (DQMIO)

https://cmsweb.cern.ch/das/request?view=list&limit=50&instance=prod%2Fglobal&input=dataset+dataset%3D%2F*%2F*29Jun2019*%2FDQMIO

Physics objects

Datasets: Data

Example of cmsDriver

  • AOD, MiniAOD: CMSSW_10_6_20
cmsDriver.py RECO -s RAW2DIGI,L1Reco,RECO,SKIM:JetHTJetPlusHOFilter,ALCA:HcalCalIsoTrkFilter+HcalCalIsolatedBunchFilter+TkAlMinBias,EI,PAT,DQM:@rerecoCommon --runUnscheduled --nThreads 8 --data --era Run2_2017 --scenario pp --conditions 106X_dataRun2_v28 --eventcontent AOD,MINIAOD,DQM --datatier AOD,MINIAOD,DQMIO --customise Configuration/DataProcessing/RecoTLR.customisePostEra_Run2_2017 --filein file:pippo.root -n 100 --python_filename=recoskim_Run2017F_JetHT.py --no_exec

cmsDriver.py NANO -s NANO --nThreads 2 --data --era Run2_2017 --conditions 106X_dataRun2_v28 --eventcontent NANOEDMAOD --datatier NANOAOD --filein file:pippo.root -n 100 --python_filename=nano_Run2017.py --no_exec

Data Certification

Detail for Legacy-2017 DC can be found in CMS.DQMULReRecoCertificationRules2017; Presentation
  • Golden JSON (41.48 /fb)
/afs/cern.ch/cms/CAF/CMSCOMM/COMM_DQM/certification/Collisions17/13TeV/Legacy_2017/Cert_294927-306462_13TeV_UL2017_Collisions17_GoldenJSON.txt 
  • Muon JSON (42.02 /fb)
/afs/cern.ch/cms/CAF/CMSCOMM/COMM_DQM/certification/Collisions17/13TeV/Legacy_2017/Cert_294927-306462_13TeV_UL2017_Collisions17_MuonJSON.txt

Datasets: Monte Carlo

Example of cmsDriver

Quick MC without MC trigger information

  • From GEN to RAW: CMSSW_10_6_20
    • Please consider to use the last version of CMSSW_10_6 for this step if you need to use the latest generator setting
    • Grid environment is needed to pick up list of file. It will take 10-15 min to pick up the list of PU
cmsDriver.py SingleNuE10_cfi.py --fileout file:RAW.root --mc --eventcontent PREMIXRAW --datatier GEN-SIM-RAW --conditions 106X_mc2017_realistic_v6 --step GEN,SIM,DIGI,DATAMIX,L1,DIGI2RAW,HLT:@fake2 --beamspot Realistic25ns13TeVEarly2017Collision --procModifiers premix_stage2 --datamix PreMix --era Run2_2017 --nThreads 8 --geometry DB:Extended --pileup_input "dbs:/Neutrino_E-10_gun/RunIISummer19ULPrePremix-UL17_106X_mc2017_realistic_v6-v1/PREMIX" --python_filename GENToHLT_2017_cfg.py -n 10 --no_exec

  • From RAW to MiniAOD: CMSSW_10_6_20
cmsDriver.py step2 --filein file:RAW.root --fileout file:MiniAODSIM.root --mc --eventcontent MINIAODSIM --datatier MINIAODSIM --runUnscheduled --conditions 106X_mc2017_realistic_v6 --step RAW2DIGI,L1Reco,RECO,RECOSIM,PAT --nThreads 8 --geometry DB:Extended --era Run2_2017 --python_filename RECOToPAT_2017_cfg.py -n 10 --no_exec

  • To produce NanoAOD: CMSSW_10_6_20 (NanoAODv6)
    • To create private flat ntuple, please use --eventcontent NANOAODSIM --datatier NANOAODSIM --customise_commands="process.add_(cms.Service('InitRootHandlers', EnableIMT = cms.untracked.bool(False)))" to create flat ntuple. See instruction here.
cmsDriver.py step3 --filein file:MiniAODSIM.root --fileout file:NanoAODSIM.root --mc --eventcontent NANOEDMAODSIM --datatier NANOAODSIM --conditions 106X_mc2017_realistic_v6 --step NANO --nThreads 8 --era Run2_2017 --python_filename NanoAOD_2017_cfg.py -n 10 --no_exec

Using production recipe (with MC HLT information)

Example: McM - TTJets_TuneCP5_13TeV-amcatnloFXFX-pythia8

One can consider to merge (i) GEN, SIM, DIGI and/or (ii) RECO, MiniAOD as above recipe.

  • GEN: CMSSW_10_6_20
    • Please consider to use the last version of CMSSW_10_6 for generator
cmsDriver.py SingleNuE10_cfi.py --fileout file:GEN.root --mc --eventcontent RAWSIM --datatier GEN --conditions 106X_mc2017_realistic_v6 --beamspot Realistic25ns13TeVEarly2017Collision --step GEN --geometry DB:Extended --era Run2_2017 --python_filename GEN_2017_cfg.py -n 10 --no_exec
  • SIM: CMSSW_10_6_20
cmsDriver.py step2 --filein file:GEN.root --fileout file:SIM.root --mc --eventcontent RAWSIM --runUnscheduled --datatier GEN-SIM --conditions 106X_mc2017_realistic_v6 --beamspot Realistic25ns13TeVEarly2017Collision --step SIM --nThreads 8 --geometry DB:Extended --era Run2_2017 --python_filename SIM_2017_cfg.py -n 10 --no_exec
  • DIGI (premix): CMSSW_10_6_20
    • Grid environment is needed to pick up list of file. It will take 10-15 min to pick up the list of PU
    • For standard premixing setting, you can get the cfg file from /afs/cern.ch/user/s/srimanob/public/ForRun2LegacyProduction/DIGIPremix_2017_cfg.py to save time.
cmsDriver.py step3 --filein file:SIM.root --fileout file:DIGIPremix.root  --pileup_input "dbs:/Neutrino_E-10_gun/RunIISummer19ULPrePremix-UL17_106X_mc2017_realistic_v6-v1/PREMIX" --mc --eventcontent PREMIXRAW --runUnscheduled --datatier GEN-SIM-DIGI --conditions 106X_mc2017_realistic_v6 --step DIGI,DATAMIX,L1,DIGI2RAW --procModifiers premix_stage2 --nThreads 8 --geometry DB:Extended --datamix PreMix --era Run2_2017 --python_filename DIGIPremix_2017_cfg.py -n 10 --no_exec
  • HLT: CMSSW_9_4_14_UL_patch1
    • runUnscheduled is not supported in the HLT step
cmsDriver.py step4 --filein file:DIGIPremix.root --fileout file:HLT.root --mc --eventcontent RAWSIM --datatier GEN-SIM-RAW --conditions 94X_mc2017_realistic_v15 --customise_commands 'process.source.bypassVersionCheck = cms.untracked.bool(True)' --step HLT:2e34v40 --nThreads 8 --geometry DB:Extended --era Run2_2017 --python_filename HLT_2017_cfg.py -n 10 --no_exec
  • RECO: CMSSW_10_6_20
cmsDriver.py step5 --filein file:HLT.root --fileout file:RECO.root --mc --eventcontent AODSIM --runUnscheduled --datatier AODSIM --conditions 106X_mc2017_realistic_v6 --step RAW2DIGI,L1Reco,RECO,RECOSIM --nThreads 8 --geometry DB:Extended --era Run2_2017 --python_filename RECO_2017_cfg.py -n 10 --no_exec
  • MiniAOD: CMSSW_10_6_20
cmsDriver.py step6 --filein file:RECO.root --fileout file:MiniAOD.root --mc --eventcontent MINIAODSIM --runUnscheduled --datatier MINIAODSIM --conditions 106X_mc2017_realistic_v6 --step PAT --nThreads 8 --geometry DB:Extended --era Run2_2017 --python_filename MINIAOD_2017_cfg.py -n 10 --no_exec
  • NanoAOD: CMSSW_10_6_20 (NanoAODv6)
    • To create private flat ntuple, please use --eventcontent NANOAODSIM --datatier NANOAODSIM --customise_commands="process.add_(cms.Service('InitRootHandlers', EnableIMT = cms.untracked.bool(False)))" to create flat ntuple. See instruction here.
cmsDriver.py step7 --filein file:MiniAOD.root --fileout file:NanoAOD.root --mc --eventcontent NANOEDMAODSIM --datatier NANOAODSIM --conditions 106X_mc2017_realistic_v6 --step NANO --nThreads 8 --era Run2_2017 --python_filename NANOAOD_2017_cfg.py -n 10 --no_exec

-- PhatSrimanobhas - 2020-12-24

Edit | Attach | Watch | Print version | History: r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r1 - 2020-12-24 - PhatSrimanobhas
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Main All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback