The NanoAOD-Tools
NanoAOD -Tools provide a simple and lightweight python post-processing framework for fast skimming, analysis and computation of derived quantities.
NanoAOD -Tools should contain code implementation of NanoAOD ready "POG recipes" so that no one has to translate from twiki to code and the code is already available to access the data in NanoAOD format.
For more details please refer to the dedicated
page
We anticipate that many analysis would run skimming and computation of derived quantities or systematic variations in a single step.
Such step can be easily implemented with the NanoAOD -Tools postprocessors.
Existing modules in NanoAOD -Tools
Modules are defined under
NanoAOD-Tools modules
- PU weighting: code
- Prefiring weights: code
POG specific
BTV: tool to compute scale factors
-
- Beginning with PR #233 TBU
, the names of branches change. The default stored in 'Jet_btagSF' corresponds to CSVv2, medium WP, and is now more clearly named 'Jet_btagSF_csvv2_M'. Similarly, the shape corrected version which was previously 'Jet_btagSF_shape' now becomes 'Jet_btagSF_csvv2_shape'. The names of branches corresponding to variations of these are modified similarly.
MUO: momentum calibration (to be implemented)
JME: jetmetHelperRun2
The recommended jetmet correction module is called
jetmetHelperRun2.py
and is located in python/postprocessing/modules/jme/jetmetHelperRun2. It can be used to correct AK4 and AK8 jets and MET in both data and Monte Carlo.
Jet energy corrections
- Already applied to data and MC in nanoAOD (default versions listed here)
- Updating to a new JEC version with jetmetHelperRun2:
- Done by default for AK4 jets (pt is updated)
- AK8 jets: corrections can be updated by setting the "redoJEC" flag to "True". Note: This should be done to be consistent with the JEC uncertainties.
- Uncertainties (applied to MC jet branches):
- Default: total JEC uncertainty - only get one uncertainty, which is the sum of uncertainty sources summed in quadrature
- Access all uncertainty sources by setting the "jesUncert" flag to "All" (alternatively, the reduced set of variations is also available by setting the flag to "Merged")
Jet energy resolution smearing (MC)
- Not previously applied in nanoAOD.
- In jetmetHelperRun2. MC JEC-corrected jets are smeared by default, using the hybrid method. See below for the MET.
- It is possible to split the JER uncertainties into different sources (based on the pt and eta range of the jet) by setting the "splitJER" to "True".
Jet mass scale (AK8 MC)
- Not previously applied in nanoAOD.
- Correction for data/MC jet mass scale difference after tau21 cuts applied.
- In jetmetHelperRun2, the MC AK8 jet mass is scaled by default, using the PUPPI+softdrop W-tagging JMS values.
Jet mass resolution smearing ( AK8 MC)
- Not previously applied in nanoAOD.
- Correction for data/MC jet mass resolution difference after tau21 cuts applied.
- In jetmetHelperRun2, the MC AK8 jet mass is smeared by default, using the W-tagging JMR values.
W Jet Mass Scale (W-JMS) softdrop corrections (AK8) (formely known as puppi corrections)
- Not previously applied in nanoAOD.
- Correction for PUPPI softdrop mass shift at the W mass peak, as a function of pt and eta. (Effect seen without any W-tagging cuts applied). More information here.
- In jetmetHelperRun2, W-JMS corrections are applied to AK8 jets (data and MC).
- Corrections can be turned off by setting the flag "noGroom" to "True." (This will turn off all softdrop mass branches, including e.g. jet energy correction uncertainties.)
MET
- In jetmetHelperRun2, PF MET is recorrected by default, using the type-1 algorithm, starting from raw (uncorrected) MET, and AK4 jets. It can handle both MC and data. (NOTE: Puppi MET can not be recorrected since the AK4PuppiJets are not stored in NanoAOD currently)
- It is recommended to use jetmetHelperRun2 to obtain the uncertainties (JES, JER, Unclustered) for PF MET. (NOTE: This should not be used for Puppi MET, a set of uncertanities for Puppi MET are stored in the central NanoAOD production for the latest version)
Branches added:
AK8 jets
- Data and MC:
- FatJet _pt_raw - raw ungroomed jet pt, no JECs applied
- FatJet _mass_raw - raw ungroomed jet mass, no JECs applied
- FatJet _pt_nom - corrected ungroomed jet pt: JEC corrections applied to data and MC; JER corrections also applied to MC
- FatJet _mass_nom - corrected ungroomed mass: JEC corrections applied to data and MC; JER, JMS, and JMR corrections also applied to MC
- FatJet _corr_JEC - nominal JEC jet correction factor
- FatJet _corr_JER - nominal JER jet correction factor
- MC only:
- FatJet _corr_JMS - nominal JMS AK8 jet correction factor
- FatJet _corr_JMR - nominal JMR AK8 jet correction factor
- FatJet _[pt or mass]_jer[uncertainty source][shift Up or Down] - jet pt or mass JER up/down uncertainty shift
- FatJet _mass_jmr[shift Up or Down] - jet mass JMR up/down uncertainty shift
- FatJet _mass_jms[shift Up or Down] - jet mass JMS up/down uncertainty shift
- FatJet _[pt or mass]_jes[uncertainty source][shift Up or Down]- jet pt or mass variation for a given uncertainty source
AK8 softdrop mass (not added when "noGroom" is set to "True")
- Data and MC:
- FatJet _msoftdrop_raw - raw ungroomed jet mass (Warning: JECs applied!)
- FatJet _msoftdrop_nom - corrected groomed mass: JEC and W-JMS corrections applied to data and MC; JER, JMS and JMR corrections also applied to MC. (Warning: these set of corrections are NOT recommended in all cases, see more above and here)
- FatJet _corr_PUPPI - nominal W-JMS softdrop correction factor (formerly known as puppi correction)
- MC only:
- FatJet _softdrop_corr_JMS - nominal JMS AK8 softdrop jet correction factor
- FatJet _softdrop_corr_JMR - nominal JMR AK8 softdrop jet correction factor
- FatJet _msoftdrop_jer[shift Up or Down] - softdrop mass JER up/down uncertainty shift
- FatJet _msoftdrop_jmr[shift Up or Down] - softdrop mass JMR up/down uncertainty shift
- FatJet _msoftdrop_jms[shift Up or Down] - softdrop mass JMS up/down uncertainty shift
- FatJet _msoftdrop_jes[uncertainty source][shift Up or Down]- softdrop mass variation for a given uncertainty source
- FatJet _msoftdrop_tau21DDT_nom - corrected groomed mass: JEC and JER corrections applied, tau21ddt JMS and JMR corrections also applied
- FatJet _msoftdrop_tau21DDT_jer[shift Up or Down] - softdrop mass JER up/down uncertainty shift (tau21ddt JMS and JMR corrections applied)
- FatJet _msoftdrop_tau21DDT_jmr[shift Up or Down] - softdrop mass JMR up/down uncertainty shift (tau21ddt JMS and JMR corrections applied)
- FatJet _msoftdrop_tau21DDT_jms[shift Up or Down] - softdrop mass JMS up/down uncertainty shift (tau21ddt JMS and JMR corrections applied)
AK4 jets
- Data and MC:
- Jet_[pt or mass]_raw - raw ungroomed jet pt or mass, no JECs applied
- Jet_[pt or mass]_nom - corrected ungroomed jet pt or mass: JEC corrections applied to data and MC; JER corrections also applied to MC
- Jet_corr_JEC - nominal JEC jet correction factor
- Jet_corr_JER - nominal JER jet correction factor
- MC only:
- Jet_[pt or mass]_jer[uncertainty source][shift Up or Down] - jet pt or mass JER up/down uncertainty shift
- Jet_[pt or mass]_jes[uncertainty source][shift Up or Down]- jet pt or mass variation for a given JEC uncertainty source
MET
- Data and MC:
- [MET branch]_T1_pt - JEC-recorrected MET (no JER smearing in MC)
- [MET branch]_T1_phi - JEC-recorrected MET phi (no JER smearing in MC) * MC only:
- [MET branch]_T1smear_pt - JEC-recorrected and JER-corrected MET
- [MET branch]_T1smear_phi - JEC-recorrected and JER-corrected MET phi
- [MET branch]_T1smear_pt_jer[uncertainty source][shift Up or Down] - MET JER up/down uncertainty shift (with nominal JEC correction)
- [MET branch]_T1smear_phi_jer[uncertainty source][shift Up or Down] - MET phi JER up/down uncertainty shift (with nominal JEC correction)
- [MET branch]_T1smear_pt_jes[uncertainty source][shift Up or Down] - MET variation for a given JEC uncertainty source (with nominal JER smearing applied)
- [MET branch]_T1smear_phi_jes[uncertainty source][shift Up or Down] - MET phi variation for a given JEC uncertainty source (with nominal JER smearing applied)
- [MET branch]_T1_pt_jes[uncertainty source][shift Up or Down] - MET variation for a given JEC uncertainty source (no JER smearing applied)
- [MET branch]_T1_phi_jes[uncertainty source][shift Up or Down] - MET phi variation for a given JEC uncertainty source (no JER smearing applied)
- [MET branch]_pt_unclustEn[shift Up or Down] - MET variation due to unclustered energy uncertainty (with nominal JEC correction, but no JER smearing applied)
- [MET branch]_phi_unclustEn[shift Up or Down] - MET phi variation due to unclustered energy uncertainty (with nominal JEC correction, but no JER smearing applied)
Note:
- [MET branch] is the metBranchName set in the initialization of the module ("MET" by default).
- [uncertainty source] is the name of the JEC uncertainty source, e.g. "Total" or AbsoluteScale. See the JEC uncertainty sources twiki for the complete list of uncertainty sources.
- [shift Up or Down] is either "Up" or "Down"
2017 Data example, using MET v2
from PhysicsTools.NanoAODTools.postprocessing.modules.jme.jetmetHelperRun2 import *
- Get the corrector for MET and the AK4 jets:
jetmetCorrector = createJMECorrector(isMC=False, dataYear=2017, runPeriod="E", metBranchName="METFixEE2017")
fatJetCorrector = createJMECorrector(isMC=False, dataYear=2017, runPeriod="E", jetType = "AK8PFPuppi")
2018 MC example (redo JECs and get all uncertainty sources; add extra JEC uncertainty for the HEM issue)
from PhysicsTools.NanoAODTools.postprocessing.modules.jme.jetmetHelperRun2 import *
- Get the corrector for MET and the AK4 jets:
jetmetCorrector = createJMECorrector(isMC=True, dataYear=2018, jesUncert="All", applyHEMfix=True)
fatJetCorrector = createJMECorrector(isMC=True, dataYear=2018, jesUncert="All", jetType = "AK8PFPuppi", applyHEMfix=True)
How to run via CRAB on NanoAOD using NanoAODTools
Such step can also be run on crab using the example give in
NanoAODTools/crab/
. It is sufficient to edit
crab_script.py
replacing the settings of the
PostProcessor with the ones you want.
Quickly make plots with NanoAODTools postprocessor
The postprocessor can be used also to simply make histograms. Quick recipe for impatient user:
cmsrel CMSSW_10_6_19_patch2
cd CMSSW_10_6_19_patch2/src
cmsenv
git cms-init #not really needed unless you later want to add some other cmssw stuff
git clone https://github.com/cms-nanoAOD/nanoAOD-tools.git PhysicsTools/NanoAODTools
scram b
voms-proxy-init -voms cms
cd PhysicsTools/NanoAODTools/python/postprocessing/examples/
python exampleAnalysis.py
More information
Now the post-processing engine allows you to recalculate and overwrite an already existing branch/variable, but be careful if you managed to recalculate an already existing variable in the Events tree, you have to keep the same type of the variable, for examples if you would like to overwrite the variable "Muon_mediumId" which is of type "Bool" you should do that by assigning the recomputed variable to the type Bool as well.
This is well discussed in :
issue 81, and solved by
PR 84
On going effort
Unify POG calibration payloads - avoid duplication of effort/code.
- First develop a common format for all POGs to collect the correction factors and uncertainties. This will be using a centralized json schema. More details are in this talk
- The second step is to develop a common set of tools/libraries/functions to propagate corrections and uncertainties from the POGs.
Llightweight C++ based and decoupled from CMSSW. Ideally these tools should have flexibility for other frameworks, e.g., R-DataFrame, Coffee, etc..
--
LoukasGouskos - 2020-12-29