Difference: WorkBookNanoAOD (1 vs. 57)

Revision 572019-09-13 - AndreyPopov

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"
Changed:
<
<

NanoAOD Documentation

NanoAOD format

>
>

NanoAOD Documentation

NanoAOD format

 
Changed:
<
<
NanoAOD format consists of an Ntuple like format, readable with bare root and containing the per-event information that is needed in most generic analyses. A NanoAOD is not a CMSSW EDM file but several EDM features are available in this simplified format too.
>
>
NanoAOD format consists of an Ntuple like format, readable with bare root and containing the per-event information that is needed in most generic analyses. A NanoAOD is not a CMSSW EDM file but several EDM features are available in this simplified format too.
 
Changed:
<
<
The size per event of NanoAOD is order of 1Kb, they can be centrally produced in ~1-2 week time, central production can be triggered often if many new features are made available in "hot periods". Users can easily extend NanoAOD for their specific study making a private production when needed (suggested for specific studies on samples that are not too large or for some special needs) or can ask for new features inclusion in the central format (suggested when need to run on multi billion events)
>
>
The size per event of NanoAOD is order of 1Kb, they can be centrally produced in ~1-2 week time, central production can be triggered often if many new features are made available in "hot periods". Users can easily extend NanoAOD for their specific study making a private production when needed (suggested for specific studies on samples that are not too large or for some special needs) or can ask for new features inclusion in the central format (suggested when need to run on multi billion events)
 
Changed:
<
<
A NanoAOD file contains a main TTree named Events and some additional TTrees for run, lumi and metadata.
>
>
A NanoAOD file contains a main TTree named Events and some additional TTrees for run, lumi and metadata.
 
Deleted:
<
<

The Events TTree

 
Changed:
<
<
The main TTree contains only scalar branches or simple array branches. No special dataformat is needed to read NanoAODs. Physics objects features are grouped via naming conventions and by sharing the same array dimensionality.
>
>

The Events TTree

 
Changed:
<
<
As an example there is a branch named Muon_pt[nMuon] and a branch named Muon_eta[nMuon], both are arrays of length nMuon. Access as whole objects can be obtained with light frameworks (see e.g. the NanoAOD -Tools below)
>
>
The main TTree contains only scalar branches or simple array branches. No special dataformat is needed to read NanoAODs. Physics objects features are grouped via naming conventions and by sharing the same array dimensionality.

As an example there is a branch named Muon_pt[nMuon] and a branch named Muon_eta[nMuon], both are arrays of length nMuon. Access as whole objects can be obtained with light frameworks (see e.g. the NanoAOD -Tools below)

  References across objects are implemented via branches with idx suffix that are just bare indices that contains information about the collection they point to, e.g. Electron_jetIdx contains information about the jet, if available that contains a given electron. A value of -1 means the ref is null, a value i >= 0 means that given Electron is associated to the i-th object in the Jet collection. Another example Jet_genJetIdx is the index to the GenJet collection. The pointer could also point outside the boundary of the collection if some elements in the tail of the collection are dropped. For example before accessing the GenJet associated to a Jet one should verify that 0<= Jet_genJetIdx < nGenJet.
Line: 26 to 27
  No skimming is applied at production level.
Changed:
<
<
Some linking to gen and trigger objects is available. No cleaning is applied but objects are cross referenced (with indices, see above). The cross referencing is based on common PF constituents, not on DeltaR. DeltaR matching is less accurate and can be reproduced from NanoAOD if needed, while PF matching can only be done while reading MiniAOD. Predefined clean bitmasks can be used to store persistently a given cleaning choice (e.g. specific of a whole PAG), so that the user can clean by simply checking the proper bit.
>
>
Some linking to gen and trigger objects is available. No cleaning is applied but objects are cross referenced (with indices, see above). The cross referencing is based on common PF constituents, not on DeltaR. DeltaR matching is less accurate and can be reproduced from NanoAOD if needed, while PF matching can only be done while reading MiniAOD. Predefined clean bitmasks can be used to store persistently a given cleaning choice (e.g. specific of a whole PAG), so that the user can clean by simply checking the proper bit.
 
Changed:
<
<
Specific information on some branches/objects
>
>

Specific information on some branches/objects

 
Changed:
<
<
Jets
>
>
Jets
 
  • Jets are corrected to latest available JECs and provided with latest available JetID, PU ID, b-tag and QGL.
  • No systematic variations of JECs are stored, but the quantities (rho, area, pt, eta) needed to recompute JECs or JEC systematic variations are available. Dedicated nanoAOD-tools modules exist to compute those quantities.
  • b-jet energy regression, developed for final states with one or more H->bb, is also available.
  • Jets of MC samples are not smeared (dedicated nanoAOD-tools modules exist to apply the smearing).
Changed:
<
<
Muons
>
>
Muons
 
Changed:
<
<
Electrons
>
>
Electrons
  ..e.g. explain the bitmask...
Changed:
<
<
MET
Trigger
>
>
MET
Trigger
  Trigger bits information is available for all trigger in the input files. If multiple files are given in input with different trigger bits, the bits not available in some events are filled with zeros. When merging two nanoaod file with haddnano.py the same algorithm to fill missing trigger bits is used (i.e. the output file will have zeros for events that had a given trigger bit not available)

Trigger objects are stored in nanoaod for some filters as defined here, if you need some additional object p4 to be stored please open an issue.

Changed:
<
<
Generator, dressed leptons, LHE
>
>
Generator, dressed leptons, LHE
  Several collections are available for gen level information:
Changed:
<
<
  • dressed lepton are also available as configured [[https://github.com/cms-sw/cmssw/blob/master/PhysicsTools/NanoAOD/python/particlelevel_cff.py][here]
  • gen jets with pt > 10 are stored. Parton and Hadron flavour as well as link to gen jet is available in Jets
  • Some LHE information is stored (see here
Weigths
>
>
  • dressed lepton are also available as configured here
  • gen jets with pt > 10 are stored. Parton and Hadron flavour as well as link to gen jet is available in jets
  • Some LHE information is stored (see here)

Weigths

The nominal generator-level weight is stored in branches genWeight and Generator_weight, which both contain the same value obtained from GenEventInfoProduct::weight() (here and here). This is typically the product of the nominal weight from LHE (if available) and the nominal weight from Pythia. The nominal LHE weight is available as LHEWeight_originalXWGTUP, filled from LHEEventProduct::originalXWGTUP() (here).

  ..e.g. explain the PDF weights...
Changed:
<
<

The Other TTrees

>
>

The Other TTrees

 

Content auto-documentation

Line: 75 to 81
 
Changed:
<
<

NanoAOD -Tools Documentation

>
>

NanoAOD -Tools Documentation

 
Changed:
<
<
NanoAOD -Tools provide a simple and lightweight python post-processing framework for fast skimming, analysis and computation of derived quantities.
>
>
NanoAOD -Tools provide a simple and lightweight python post-processing framework for fast skimming, analysis and computation of derived quantities.
 
Changed:
<
<
NanoAOD -Tools should contain code implementation of NanoAOD ready "POG recipes" so that no one has to translate from twiki to code and the code is already available to access the data in NanoAOD format.
>
>
NanoAOD -Tools should contain code implementation of NanoAOD ready "POG recipes" so that no one has to translate from twiki to code and the code is already available to access the data in NanoAOD format.
 
Changed:
<
<
We anticipate that many analysis would run skimming and computation of derived quantities or systematic variations in a single step. Such step can be easily implemented with the NanoAOD -Tools postprocessors.
>
>
We anticipate that many analysis would run skimming and computation of derived quantities or systematic variations in a single step. Such step can be easily implemented with the NanoAOD -Tools postprocessors.
 
Changed:
<
<

Existing modules in NanoAOD -Tools

>
>

Existing modules in NanoAOD -Tools

 

Generic

  • PU weighting
Line: 92 to 98
 
  • JME: JEC uncertainties, JER smearing with MET propagation
  • MUO: momentum calibration (to be implemented)
Changed:
<
<

How to run via crab on NanoAOD using NanoAOD -Tools

>
>

How to run via crab on NanoAOD using NanoAOD -Tools

  Such step can also be run on crab using the example give in PhysicsTools /NanoAODTools/crab/
Line: 100 to 106
  replacing the settings of the PostProcessor with the ones you want
Changed:
<
<

Quickly make plots with NanoAODTools postprocessor

>
>

Quickly make plots with NanoAODTools postprocessor

  The postprocessor can be used also to simply make histograms. Quick recipe for impatient user:
cmsrel CMSSW_9_4_11_cand1

Line: 122 to 128
 This is well discussed in :

issue 81, and solved by PR 84

Changed:
<
<

Presentations on NanoAOD format

>
>

Presentations on NanoAOD format

  see presentation in the PPD general meeting of 21/09/2017 and PPD workshop

Centrally produced samples

Changed:
<
<
NanoAOD are centrally produced. Available campaigns are
>
>
NanoAOD are centrally produced. Available campaigns are
 
  • 05Feb2018 - 80X samples (2016 run, data and MC)
  • 12Apr2018 - 94X miniAODV2 samples (MC for 2017 run)
Line: 140 to 148
  You may want to make ntuples yourself for testing new features, for signal samples with special additional variables or for other particular workflows.
Changed:
<
<

Recipe for the current HEAD of NanoAOD:

>
>

Recipe for the current HEAD of NanoAOD:

 
Changed:
<
<
From CMSSW_9_4_X a version of NanoAOD is included in CMSSW.
>
>
From CMSSW_9_4_X a version of NanoAOD is included in CMSSW.
 For a basic test you can use that one, for the latest version and developments please merge the HEAD of the repository as shown below.

If you want to integrate a new feature, please start with the master-102X branch. The new feature will then be rebased (if code functionality allows that) on the master-94X branch during the review of the pull request.

Changed:
<
<
NanoAOD is supposed to run on all supported data (e.g. MINIAOD files created with CMSSW_9_4_X) from the same CMSSW_10_2_X area. You will just have to configure the correct era in the cmsDriver command.
>
>
* NanoAOD is supposed to run on all supported data (e.g. MINIAOD files created with CMSSW_9_4_X) from the same CMSSW_10_2_X area. You will just have to configure the correct era in the cmsDriver command.*
 
Changed:
<
<
Support of running NanoAOD production from a 9_4_X area has been dropped. The currently supported releases are late 10_2_X and 10_4/5_X ones.
>
>
* Support of running NanoAOD production from a 9_4_X area has been dropped. The currently supported releases are late 10_2_X and 10_4/5_X ones.*
 
Line: 204 to 212
  as it is done in the merge jobs in central production (Configuration/DataProcessing/python/Merge.py).
Changed:
<
<

Instructions for NanoAODv5 production

>
>

Instructions for NanoAODv5 production

  THIS SET OF INSTRUCTIONS IS IN PREPARATION, AND WAS NOT YET VALIDATED
Line: 248 to 256
 
Changed:
<
<
NB: this is not the way NanoAOD are meant to be produced. Production will soon be handled centrally by computing
>
>
NB: this is not the way NanoAOD are meant to be produced. Production will soon be handled centrally by computing
 
Changed:
<
<

How to make NanoAOD via crab

>
>

How to make NanoAOD via crab

  While central production will take care soon of making nanoAOD for all CMS samples there are two use cases for NanoAOD made with crab:
Changed:
<
<
  • test NanoAOD format before production is ready to handle them (expected late 2017)
>
>
  • test NanoAOD format before production is ready to handle them (expected late 2017)
 
  • add analysis specific information in the format for a few samples (e.g. signal samples), i.e. run your customized version of NanoAOD

Example config is available here: PhysicsTools /NanoAOD/test/crab_mc.py

Changed:
<
<
It is important to modify the input configuration so that it runs a single NanoAODOutputModule and add the following parameter as configuraion of the outputmodule.
>
>
It is important to modify the input configuration so that it runs a single NanoAODOutputModule and add the following parameter as configuraion of the outputmodule.
 
    fakeNameForCrab =cms.untracked.bool(True),
Changed:
<
<

How to run via crab on NanoAOD using NanoAOD -Tools

>
>

How to run via crab on NanoAOD using NanoAOD -Tools

 
Changed:
<
<
NanoAOD -tools provide a post processing framework for simple and fast skimming and computation of derived quantities.
>
>
NanoAOD -tools provide a post processing framework for simple and fast skimming and computation of derived quantities.
  Such step can also be run on crab using the example give in PhysicsTools /NanoAODTools/crab/
Line: 289 to 297
 

How to contribute and request changes

Changed:
<
<
You can contribute to NanoAOD the same way you contribute to the rest of CMSSW via github. The only difference is that NanoAOD related PR should be first submited to github.com/cms-nanoAOD/cmssw fork and then from there they will be collected, tested and pushed in main CMSSW via central Pull Requests.
>
>
You can contribute to NanoAOD the same way you contribute to the rest of CMSSW via github. The only difference is that NanoAOD related PR should be first submited to github.com/cms-nanoAOD/cmssw fork and then from there they will be collected, tested and pushed in main CMSSW via central Pull Requests.
 You should setup your area as indicated here.

If you want to help but do not know what you can do, have a look at the issues for NanoAOD and NanoAOD-Tools and tell us there that you are going to implement some of the missing features or fix some of the open issues.

Revision 552019-06-05 - MarcoPeruzzi

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

NanoAOD Documentation

Line: 232 to 232
 cmsDriver.py myNanoProdMc2018 -s NANO --mc --eventcontent NANOAODSIM --datatier NANOAODSIM --no_exec --conditions 102X_upgrade2018_realistic_v19 --era Run2_2018,run2_nanoAOD_102Xv1 --customise_commands="process.add_(cms.Service('InitRootHandlers', EnableIMT = cms.untracked.bool(False)))"

2018 ABC data:

Changed:
<
<
cmsDriver.py myNanoProdData2018 -s NANO --data --eventcontent NANOAOD --datatier NANOAOD --no_exec --conditions 102X_dataRun2_v11 --era Run2_2018,run2_nanoAOD_102Xv1 --customise_commands="process.add_(cms.Service('InitRootHandlers', EnableIMT = cms.untracked.bool(False)))"
>
>
cmsDriver.py myNanoProdData2018ABC -s NANO --data --eventcontent NANOAOD --datatier NANOAOD --no_exec --conditions 102X_dataRun2_v11 --era Run2_2018,run2_nanoAOD_102Xv1 --customise_commands="process.add_(cms.Service('InitRootHandlers', EnableIMT = cms.untracked.bool(False)))"
  2018 D data:
Changed:
<
<
cmsDriver.py myNanoProdData2018 -s NANO --data --eventcontent NANOAOD --datatier NANOAOD --no_exec --conditions 102X_dataRun2_Prompt_v14 --era Run2_2018,run2_nanoAOD_102Xv1 --customise_commands="process.add_(cms.Service('InitRootHandlers', EnableIMT = cms.untracked.bool(False)))"
>
>
cmsDriver.py myNanoProdData2018D -s NANO --data --eventcontent NANOAOD --datatier NANOAOD --no_exec --conditions 102X_dataRun2_Prompt_v14 --era Run2_2018,run2_nanoAOD_102Xv1 --customise_commands="process.add_(cms.Service('InitRootHandlers', EnableIMT = cms.untracked.bool(False)))"
 

Revision 542019-06-05 - MarcoPeruzzi

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

NanoAOD Documentation

Line: 231 to 231
 2018 MC: cmsDriver.py myNanoProdMc2018 -s NANO --mc --eventcontent NANOAODSIM --datatier NANOAODSIM --no_exec --conditions 102X_upgrade2018_realistic_v19 --era Run2_2018,run2_nanoAOD_102Xv1 --customise_commands="process.add_(cms.Service('InitRootHandlers', EnableIMT = cms.untracked.bool(False)))"
Changed:
<
<
2018 data:
>
>
2018 ABC data:
 cmsDriver.py myNanoProdData2018 -s NANO --data --eventcontent NANOAOD --datatier NANOAOD --no_exec --conditions 102X_dataRun2_v11 --era Run2_2018,run2_nanoAOD_102Xv1 --customise_commands="process.add_(cms.Service('InitRootHandlers', EnableIMT = cms.untracked.bool(False)))"
Added:
>
>
2018 D data: cmsDriver.py myNanoProdData2018 -s NANO --data --eventcontent NANOAOD --datatier NANOAOD --no_exec --conditions 102X_dataRun2_Prompt_v14 --era Run2_2018,run2_nanoAOD_102Xv1 --customise_commands="process.add_(cms.Service('InitRootHandlers', EnableIMT = cms.untracked.bool(False)))"
 

Revision 532019-06-05 - MarcoPeruzzi

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

NanoAOD Documentation

Line: 204 to 204
  as it is done in the merge jobs in central production (Configuration/DataProcessing/python/Merge.py).
Changed:
<
<
Also, for the moment, please do not run on MC samples with multi-threading (--nThreads option). This should however not be the typical case for private production, even for data.
>
>

Instructions for NanoAODv5 production

THIS SET OF INSTRUCTIONS IS IN PREPARATION, AND WAS NOT YET VALIDATED

Run from a CMSSW_10_2_15 vanilla area, without merging any newer version of the NanoAOD package (e.g. master-102X continues to evolve with time!)

JEC in use are:

  • 2016: Summer16_07Aug2017_V11
  • 2017: Fall17_17Nov2017_V32
  • 2018: Autumn18_V8

2016 RunIISummer16MiniAODv3 MC: (change era modifier if you run on RunIISummer16MiniAODv2 80X MiniAOD)
cmsDriver.py myNanoProdMc2016 -s NANO --mc --eventcontent NANOAODSIM --datatier NANOAODSIM  --no_exec  --conditions 102X_mcRun2_asymptotic_v7 --era  Run2_2016,run2_nanoAOD_94X2016 --customise_commands="process.add_(cms.Service('InitRootHandlers', EnableIMT = cms.untracked.bool(False)))"

2016 data (Run2016*-17Jul2018*):
cmsDriver.py myNanoProdData2016 -s NANO --data --eventcontent NANOAOD --datatier NANOAOD  --no_exec  --conditions 102X_dataRun2_v11 --era Run2_2016,run2_nanoAOD_94X2016 --customise_commands="process.add_(cms.Service('InitRootHandlers', EnableIMT = cms.untracked.bool(False)))"

2017 MC: (change era modifier if you run on RunIIFall17MiniAOD v1 samples)
cmsDriver.py myNanoProdMc2017 -s NANO --mc --eventcontent NANOAODSIM --datatier NANOAODSIM  --no_exec  --conditions 102X_mc2017_realistic_v7 --era Run2_2017,run2_nanoAOD_94XMiniAODv2 --customise_commands="process.add_(cms.Service('InitRootHandlers', EnableIMT = cms.untracked.bool(False)))"

2017 data (Run2017*-31Mar2018*):
cmsDriver.py myNanoProdData2017 -s NANO --data --eventcontent NANOAOD --datatier NANOAOD  --no_exec  --conditions 102X_dataRun2_v11 --era Run2_2017,run2_nanoAOD_94XMiniAODv2 --customise_commands="process.add_(cms.Service('InitRootHandlers', EnableIMT = cms.untracked.bool(False)))"

2018 MC:
cmsDriver.py myNanoProdMc2018 -s NANO --mc --eventcontent NANOAODSIM --datatier NANOAODSIM  --no_exec  --conditions 102X_upgrade2018_realistic_v19 --era Run2_2018,run2_nanoAOD_102Xv1 --customise_commands="process.add_(cms.Service('InitRootHandlers', EnableIMT = cms.untracked.bool(False)))"

2018 data:
cmsDriver.py myNanoProdData2018 -s NANO --data --eventcontent NANOAOD --datatier NANOAOD  --no_exec  --conditions 102X_dataRun2_v11 --era Run2_2018,run2_nanoAOD_102Xv1 --customise_commands="process.add_(cms.Service('InitRootHandlers', EnableIMT = cms.untracked.bool(False)))"
 

Where to get the ntuples

Revision 462019-02-18 - MarcoPeruzzi

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

NanoAOD Documentation

Line: 183 to 183
 If unsure, as a starting point, you can use the same one used in MINIAOD production, that is present in the sample name. Please refer to https://twiki.cern.ch/twiki/bin/view/CMSPublic/SWGuideFrontierConditions and https://twiki.cern.ch/twiki/bin/viewauth/CMS/PdmV for more updated versions.
Changed:
<
<
MyEraModifiers has to be replaced by one of the following strings (you must provide one of these):
>
>
MyEraModifiers has to be replaced by one of the following strings (choose the line that fits your use case):
 
--era Run2_2016,run2_miniAOD_80XLegacy: for 2016 80X data (07Aug17) and RunIISummer16MiniAODv2 MC
Line: 193 to 193
  --era Run2_2017,run2_nanoAOD_94XMiniAODv2: for 2017 94X MiniAODv2 samples (including 31Mar2018 data)
Changed:
<
<
--era Run2_2018,run2_nanoAOD_102Xv1: for all 2018 10XY samples
>
>
--era Run2_2018,run2_nanoAOD_102Xv1: for all 2018 10XY samples
 

Important:

Revision 452019-01-23 - MarcoPeruzzi

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

NanoAOD Documentation

Line: 257 to 257
 

How to contribute and request changes

You can contribute to NanoAOD the same way you contribute to the rest of CMSSW via github. The only difference is that NanoAOD related PR should be first submited to github.com/cms-nanoAOD/cmssw fork and then from there they will be collected, tested and pushed in main CMSSW via central Pull Requests.

Added:
>
>
You should setup your area as indicated here.
  If you want to help but do not know what you can do, have a look at the issues for NanoAOD and NanoAOD-Tools and tell us there that you are going to implement some of the missing features or fix some of the open issues.

If you want to add something first create an issue under https://github.com/cms-nanoAOD/cmssw/issues (unless a similar issue doesn't exist already, in that case comment in the existing issue) clearly telling what you want to add/change and what is the use case.

So the workflow to contribute is something like:

Changed:
<
<
  • set up a recent CMSSW release and take the master branch of cms-nanoAOD/cmssw (as explained in the top part of this twiki)
>
>
  • set up a recent CMSSW release and take the appropriate branch of cms-nanoAOD/cmssw (as explained earlier in this twiki)
 
  • add/modify your content, mostly editing PhysicsTools /NanoAOD/python/ and PhysicsTools /NanoAOD/plugins/
  • if your code adds some variable, run the prepareDQM.py script on a file having the new variable to update the DQM configuration
    • prepareDQM.py -d -u nano_file_mc.root

Revision 442019-01-22 - MarcoPeruzzi

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

NanoAOD Documentation

Line: 150 to 150
  NanoAOD is supposed to run on all supported data (e.g. MINIAOD files created with CMSSW_9_4_X) from the same CMSSW_10_2_X area. You will just have to configure the correct era in the cmsDriver command.
Added:
>
>
Support of running NanoAOD production from a 9_4_X area has been dropped. The currently supported releases are late 10_2_X and 10_4/5_X ones.
 

# run the commands below in the src/ directory of a
Changed:
<
<
# CMSSW_9_4_11_cand1 resp. CMSSW_10_2_6 release area
>
>
# CMSSW_10_2_10 resp. CMSSW_10_4_0 release area
 
Changed:
<
<
git cms-merge-topic cms-nanoAOD:master-94X resp. cms-nanoAOD:master-102X
>
>
git cms-merge-topic cms-nanoAOD:master-102X resp. cms-nanoAOD:master-cmsswmaster
 # create a branch directly on the nanoAOD branch, skipping the unnecessary merge commit from cms-merge-topic
Changed:
<
<
git checkout -b nanoAOD cms-nanoAOD/master-94X resp. cms-nanoAOD/master-102X
>
>
git checkout -b nanoAOD cms-nanoAOD/master-102X resp. cms-nanoAOD/master-cmsswmaster
  # get also the support library if you want to use it git clone https://github.com/cms-nanoAOD/nanoAOD-tools.git PhysicsTools/NanoAODTools

Revision 432019-01-07 - MarcoPeruzzi

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

NanoAOD Documentation

Line: 194 to 194
  --era Run2_2018,run2_nanoAOD_102Xv1: for all 2018 10XY samples
Added:
>
>
Important:

Moreover, in order to avoid segfault errors in the output module when producing flat ntuples, the following options should be added to the cmsDriver command:

--customise_commands="process.add_(cms.Service('InitRootHandlers', EnableIMT = cms.untracked.bool(False)))"
as it is done in the merge jobs in central production (Configuration/DataProcessing/python/Merge.py).

Also, for the moment, please do not run on MC samples with multi-threading (--nThreads option). This should however not be the typical case for private production, even for data.

 

Where to get the ntuples

New test ntuples (CMSSW_9_4_X)

Revision 412018-12-03 - MarcoPeruzzi

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

NanoAOD Documentation

Line: 169 to 169
 

Running on various datasets from past campaigns

Changed:
<
<
The following example cmsDriver commands provide the correct config for different Eras (you need a valid GRID Proxy as data is served from remote locations)
>
>
In order to run NANOAOD production, this are the base cmsDriver command for MC and data:
 
Changed:
<
<
cmsDriver.py test80X -s NANO --mc --eventcontent NANOAODSIM --datatier NANOAODSIM --filein /store/mc/RunIISummer16MiniAODv2/DYJetsToLL_M-50_TuneCUETP8M1_13TeV-madgraphMLM-pythia8/MINIAODSIM/PUMoriond17_80X_mcRun2_asymptotic_2016_TrancheIV_v6_ext1-v2/120000/02A210D6-F5C3-E611-B570-008CFA197BD4.root --no_exec --conditions auto:run2_mc -n 1000 --era Run2_2016,run2_miniAOD_80XLegacy
>
>
cmsDriver.py myNanoProdMc -s NANO --mc --eventcontent NANOAODSIM --datatier NANOAODSIM --no_exec --conditions MyGlobalTag --era MyEraModifiers

cmsDriver.py myNanoProdData -s NANO --data --eventcontent NANOAOD --datatier NANOAOD --no_exec --conditions MyGlobalTag --era MyEraModifiers

 
Changed:
<
<
cmsDriver.py test92X -s NANO --mc --eventcontent NANOAODSIM --datatier NANOAODSIM --filein /store/relval/CMSSW_9_2_12/RelValTTbar_13/MINIAODSIM/PU25ns_92X_upgrade2017_realistic_v11-v1/00000/080E2624-F59D-E711-ACEE-0CC47A7C35A4.root --no_exec --conditions auto:phase1_2017_realistic -n 1000 --era Run2_2017,run2_nanoAOD_92X
>
>
You should replace MyGlobalTag with the GT you want to use for production (it is used, for instance, to correct jets). You must provide this! If unsure, as a starting point, you can use the same one used in MINIAOD production, that is present in the sample name. Please refer to https://twiki.cern.ch/twiki/bin/view/CMSPublic/SWGuideFrontierConditions and https://twiki.cern.ch/twiki/bin/viewauth/CMS/PdmV for more updated versions.

MyEraModifiers has to be replaced by one of the following strings (you must provide one of these):

--era Run2_2016,run2_miniAOD_80XLegacy: for 2016 80X data (07Aug17) and RunIISummer16MiniAODv2 MC
 
Changed:
<
<
cmsDriver.py test94X -s NANO --mc --eventcontent NANOAODSIM --datatier NANOAODSIM --filein /store/mc/RunIIFall17MiniAOD/TTToSemiLeptonic_TuneCP5_PSweights_13TeV-powheg-pythia8/MINIAODSIM/94X_mc2017_realistic_v10-v1/60000/A0D71AEE-13E1-E711-B3C9-FA163E629498.root --no_exec --conditions auto:phase1_2017_realistic -n 1000 --era Run2_2017,run2_nanoAOD_94XMiniAODv1
>
>
--era Run2_2016,run2_nanoAOD_94X2016: for 2016 94X data (17Jul2018) and RunIISummer16MiniAODv3 MC
 
Changed:
<
<
## FOR DATA: cmsDriver.py test_data_92X -s NANO --data --eventcontent NANOAOD --datatier NANOAOD --filein /store/data/Run2017C/JetHT/MINIAOD/PromptReco-v1/000/299/649/00000/1A4E4E38-E871-E711-A5E3-02163E01A2A6.root --conditions auto:run2_data_relval -n 100 --era Run2_2017,run2_nanoAOD_92X
>
>
--era Run2_2017,run2_nanoAOD_94XMiniAODv1: for 2017 94X MiniAODv1 samples
 
Changed:
<
<
cmsDriver.py test_data_80X -s NANO --data --eventcontent NANOAOD --datatier NANOAOD --filein /store/data/Run2016H/SingleMuon/MINIAOD/03Feb2017_ver2-v1/110000/F836EEF3-E8EA-E611-AB15-0CC47AA9943A.root --conditions auto:run2_data_relval -n 100 --era Run2_2016,run2_miniAOD_80XLegacy
>
>
--era Run2_2017,run2_nanoAOD_94XMiniAODv2: for 2017 94X MiniAODv2 samples (including 31Mar2018 data)
 
Added:
>
>
--era Run2_2018: for all 2018 10XY samples
 

Where to get the ntuples

Revision 392018-10-23 - MarcoPeruzzi

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

NanoAOD Documentation

Line: 71 to 71
 

A dump of the documentation of content for different releases is available here (NB: the actual content of a branch could dependent on the input sample for some generator quantities):

Changed:
<
<
>
>
 

NanoAOD -Tools Documentation

Line: 140 to 140
  You may want to make ntuples yourself for testing new features, for signal samples with special additional variables or for other particular workflows.
Changed:
<
<

Recipe for CMSSW_9_4_X and the current 94X HEAD of NanoAOD:

>
>

Recipe for the current HEAD of NanoAOD:

  From CMSSW_9_4_X a version of NanoAOD is included in CMSSW.
Changed:
<
<
>
>
For a basic test you can use that one, for the latest version and developments please merge the HEAD of the repository as shown below.
 
Changed:
<
<
cmsrel CMSSW_9_4_11_cand1 cd CMSSW_9_4_11_cand1/src cmsenv
>
>
If you want to integrate a new feature, please start with the master-102X branch. The new feature will then be rebased (if code functionality allows that) on the master-94X branch during the review of the pull request.

NanoAOD is supposed to run on all supported data (e.g. MINIAOD files created with CMSSW_9_4_X) from the same CMSSW_10_2_X area. You will just have to configure the correct era in the cmsDriver command.

 
Changed:
<
<
# the above is sufficient for a first version of NanoAOD # in order to get the latest developments
>
>
# run the commands below in the src/ directory of a # CMSSW_9_4_11_cand1 resp. CMSSW_10_2_6 release area
 
Changed:
<
<
git cms-merge-topic cms-nanoAOD:master
>
>
git cms-merge-topic cms-nanoAOD:master-94X resp. cms-nanoAOD:master-102X
 # create a branch directly on the nanoAOD branch, skipping the unnecessary merge commit from cms-merge-topic
Changed:
<
<
git checkout -b nanoAOD cms-nanoAOD/master
>
>
git checkout -b nanoAOD cms-nanoAOD:master-94X resp. cms-nanoAOD:master-102X
 
Changed:
<
<
# get also the support library
>
>
# get also the support library if you want to use it
 git clone https://github.com/cms-nanoAOD/nanoAOD-tools.git PhysicsTools/NanoAODTools

# compile it

Revision 382018-10-18 - MarcoPeruzzi

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

NanoAOD Documentation

Line: 103 to 103
 

Quickly make plots with NanoAODTools postprocessor

The postprocessor can be used also to simply make histograms. Quick recipe for impatient user:

Changed:
<
<
cmsrel CMSSW_9_4_9
cd CMSSW_9_4_9/src

>
>
cmsrel CMSSW_9_4_11_cand1
cd CMSSW_9_4_11_cand1/src

 cmsenv git cms-init #not really needed unless you later want to add some other cmssw stuff git clone https://github.com/cms-nanoAOD/nanoAOD-tools.git PhysicsTools/NanoAODTools
Line: 145 to 145
 From CMSSW_9_4_X a version of NanoAOD is included in CMSSW.
Changed:
<
<
cmsrel CMSSW_9_4_9 cd CMSSW_9_4_9/src
>
>
cmsrel CMSSW_9_4_11_cand1 cd CMSSW_9_4_11_cand1/src
 cmsenv

# the above is sufficient for a first version of NanoAOD

Line: 217 to 217
 

Quickly make plots with NanoAODTools postprocessor

The postprocessor can be used also to simply make histograms. Quick recipe for impatient user:

Changed:
<
<
cmsrel CMSSW_9_4_9
cd CMSSW_9_4_9/src

>
>
cmsrel CMSSW_9_4_11_cand1
cd CMSSW_9_4_11_cand1/src

 cmsenv git cms-init #not really needed unless you later want to add some other cmssw stuff git clone https://github.com/cms-nanoAOD/nanoAOD-tools.git PhysicsTools/NanoAODTools

Revision 372018-07-17 - GiovanniPetrucciani

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

NanoAOD Documentation

Line: 103 to 103
 

Quickly make plots with NanoAODTools postprocessor

The postprocessor can be used also to simply make histograms. Quick recipe for impatient user:

Changed:
<
<
cmsrel CMSSW_9_4_6
cd CMSSW_9_4_6/src

>
>
cmsrel CMSSW_9_4_9
cd CMSSW_9_4_9/src

 cmsenv git cms-init #not really needed unless you later want to add some other cmssw stuff git clone https://github.com/cms-nanoAOD/nanoAOD-tools.git PhysicsTools/NanoAODTools
Line: 145 to 145
 From CMSSW_9_4_X a version of NanoAOD is included in CMSSW.
Changed:
<
<
cmsrel CMSSW_9_4_6_patch1 cd CMSSW_9_4_6_patch1/src
>
>
cmsrel CMSSW_9_4_9 cd CMSSW_9_4_9/src
 cmsenv

# the above is sufficient for a first version of NanoAOD

Line: 162 to 162
 # compile it scram build
Changed:
<
<
You can then produce NanoAODs with nano_cfg.py. You can also generate a configuration using
cmsDriver.py nanoAOD -s NANO [--mc | --data] --eventcontent NANOAOD[SIM] --datatier NANOAOD[SIM] --conditions [global tag] --era Run2_2017 --no_exec
>
>
See below for how to generate configuration files with cmsDriver
 

Running on various datasets from past campaigns

Line: 217 to 217
 

Quickly make plots with NanoAODTools postprocessor

The postprocessor can be used also to simply make histograms. Quick recipe for impatient user:

Changed:
<
<
cmsrel CMSSW_9_4_6_patch1
cd CMSSW_9_4_6_patch1/src

>
>
cmsrel CMSSW_9_4_9
cd CMSSW_9_4_9/src

 cmsenv git cms-init #not really needed unless you later want to add some other cmssw stuff git clone https://github.com/cms-nanoAOD/nanoAOD-tools.git PhysicsTools/NanoAODTools

Revision 362018-05-23 - AndreaRizzi

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

NanoAOD Documentation

Line: 63 to 63
 

Content auto-documentation

Changed:
<
<
Documentation of content for different releases:
>
>
Each branch has built-in documentation distributed with every file, if you want to get the description of the content of a branch named e.g. SoftActivityJetNjets5, it is enough to do
root [1] Events->GetBranch("SoftActivityJetNjets5")->GetTitle()
(const char *) "number of soft activity jet pt, pt >5"
root [2] 

A dump of the documentation of content for different releases is available here (NB: the actual content of a branch could dependent on the input sample for some generator quantities):

 

Revision 352018-05-04 - AshrafKasem

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

NanoAOD Documentation

NanoAOD format

Changed:
<
<
NanoAOD format consists of an Ntuple like format, readable with bare root and containing the per-event information that is needed in most generic analyses. A NanoAOD is not a CMSSW EDM file but several EDM features are available in this simplified format too.
>
>
NanoAOD format consists of an Ntuple like format, readable with bare root and containing the per-event information that is needed in most generic analyses. A NanoAOD is not a CMSSW EDM file but several EDM features are available in this simplified format too.
  The size per event of NanoAOD is order of 1Kb, they can be centrally produced in ~1-2 week time, central production can be triggered often if many new features are made available in "hot periods". Users can easily extend NanoAOD for their specific study making a private production when needed (suggested for specific studies on samples that are not too large or for some special needs) or can ask for new features inclusion in the central format (suggested when need to run on multi billion events)
Changed:
<
<
A NanoAOD file contains a main TTree named Events and some additional TTrees for run, lumi and metadata.
>
>
A NanoAOD file contains a main TTree named Events and some additional TTrees for run, lumi and metadata.
 

The Events TTree

Added:
>
>
 The main TTree contains only scalar branches or simple array branches. No special dataformat is needed to read NanoAODs. Physics objects features are grouped via naming conventions and by sharing the same array dimensionality.
Changed:
<
<
As an example there is a branch named Muon_pt[nMuon] and a branch named Muon_eta[nMuon], both are arrays of length nMuon. Access as whole objects can be obtained with light frameworks (see e.g. the NanoAOD-Tools below)
>
>
As an example there is a branch named Muon_pt[nMuon] and a branch named Muon_eta[nMuon], both are arrays of length nMuon. Access as whole objects can be obtained with light frameworks (see e.g. the NanoAOD -Tools below)
 
Changed:
<
<
References across objects are implemented via branches with idx suffix that are just bare indices that contains information about the collection they point to, e.g. Electron_jetIdx contains information about the jet, if available that contains a given electron. A value of -1 means the ref is null, a value i >= 0 means that given Electron is associated to the i-th object in the Jet collection. Another example Jet_genJetIdx is the index to the GenJet collection. The pointer could also point outside the boundary of the collection if some elements in the tail of the collection are dropped. For example before accessing the GenJet associated to a Jet one should verify that 0<= Jet_genJetIdx < nGenJet.
>
>
References across objects are implemented via branches with idx suffix that are just bare indices that contains information about the collection they point to, e.g. Electron_jetIdx contains information about the jet, if available that contains a given electron. A value of -1 means the ref is null, a value i >= 0 means that given Electron is associated to the i-th object in the Jet collection. Another example Jet_genJetIdx is the index to the GenJet collection. The pointer could also point outside the boundary of the collection if some elements in the tail of the collection are dropped. For example before accessing the GenJet associated to a Jet one should verify that 0<= Jet_genJetIdx < nGenJet.
  HLT bits are automatically generated based on HLT bits available in the input file. Each time files are merged (in CMSSW or via the haddnano.py utility) if the list of bits is not matching the new/old bits are zero-filled so that the branches are properly aligned.
Line: 25 to 26
  No skimming is applied at production level.
Changed:
<
<
Some linking to gen and trigger objects is available. No cleaning is applied but objects are cross referenced (with indices, see above). The cross referencing is based on common PF constituents, not on DeltaR. DeltaR matching is less accurate and can be reproduced from NanoAOD if needed, while PF matching can only be done while reading MiniAOD. Predefined clean bitmasks can be used to store persistently a given cleaning choice (e.g. specific of a whole PAG), so that the user can clean by simply checking the proper bit.
>
>
Some linking to gen and trigger objects is available. No cleaning is applied but objects are cross referenced (with indices, see above). The cross referencing is based on common PF constituents, not on DeltaR. DeltaR matching is less accurate and can be reproduced from NanoAOD if needed, while PF matching can only be done while reading MiniAOD. Predefined clean bitmasks can be used to store persistently a given cleaning choice (e.g. specific of a whole PAG), so that the user can clean by simply checking the proper bit.
 
Specific information on some branches/objects

Jets
  • Jets are corrected to latest available JECs and provided with latest available JetID, PU ID, b-tag and QGL.
  • No systematic variations of JECs are stored, but the quantities (rho, area, pt, eta) needed to recompute JECs or JEC systematic variations are available. Dedicated nanoAOD-tools modules exist to compute those quantities.
Changed:
<
<
  • b-jet energy regression, developed for final states with one or more H->bb, is also available.
>
>
  • b-jet energy regression, developed for final states with one or more H->bb, is also available.
 
  • Jets of MC samples are not smeared (dedicated nanoAOD-tools modules exist to apply the smearing).

Muons
Line: 43 to 43
 ..e.g. explain the bitmask...
MET
Trigger
Deleted:
<
<
Trigger bits information is available for all trigger in the input files. If multiple files are given in input with different trigger bits, the bits not available in some events are filled with zeros. When merging two nanoaod file with haddnano.py the same algorithm to fill missing trigger bits is used (i.e. the output file will have zeros for events that had a given trigger bit not available)
 
Changed:
<
<
Trigger objects are stored in nanoaod for some filters as defined here, if you need some additional object p4 to be stored please open an issue.
>
>
Trigger bits information is available for all trigger in the input files. If multiple files are given in input with different trigger bits, the bits not available in some events are filled with zeros. When merging two nanoaod file with haddnano.py the same algorithm to fill missing trigger bits is used (i.e. the output file will have zeros for events that had a given trigger bit not available)
 
Added:
>
>
Trigger objects are stored in nanoaod for some filters as defined here, if you need some additional object p4 to be stored please open an issue.
 
Generator, dressed leptons, LHE
Added:
>
>
 Several collections are available for gen level information:
  • a subset of gen particle selected according to this pruner configuration
  • dressed lepton are also available as configured [[https://github.com/cms-sw/cmssw/blob/master/PhysicsTools/NanoAOD/python/particlelevel_cff.py][here]
Line: 53 to 53
 Several collections are available for gen level information:
  • a subset of gen particle selected according to this pruner configuration
  • dressed lepton are also available as configured [[https://github.com/cms-sw/cmssw/blob/master/PhysicsTools/NanoAOD/python/particlelevel_cff.py][here]
Changed:
<
<
  • gen jets with pt > 10 are stored. Parton and Hadron flavour as well as link to gen jet is available in Jets
>
>
  • gen jets with pt > 10 are stored. Parton and Hadron flavour as well as link to gen jet is available in Jets
 
  • Some LHE information is stored (see here
Weigths
Deleted:
<
<
..e.g. explain the PDF weights...
 
Added:
>
>
..e.g. explain the PDF weights...
 

The Other TTrees

Line: 86 to 87
 

How to run via crab on NanoAOD using NanoAOD-Tools

Changed:
<
<
Such step can also be run on crab using the example give in PhysicsTools/NanoAODTools/crab/
>
>
Such step can also be run on crab using the example give in PhysicsTools /NanoAODTools/crab/
 
Changed:
<
<
It is sufficient to edit crab_script.py
>
>
It is sufficient to edit crab_script.py
  replacing the settings of the PostProcessor with the ones you want

Quickly make plots with NanoAODTools postprocessor

Deleted:
<
<
The postprocessor can be used also to simply make histograms. Quick recipe for impatient user:
 
Changed:
<
<
cmsrel CMSSW_9_4_6

>
>
The postprocessor can be used also to simply make histograms. Quick recipe for impatient user:
cmsrel CMSSW_9_4_6

 cd CMSSW_9_4_6/src cmsenv git cms-init #not really needed unless you later want to add some other cmssw stuff
Line: 113 to 110
 

More information

Changed:
<
<

Presentations on NanoAOD format

>
>
now the post-processing engine allows you to recalculate and overwrite an already existing branch/variable, but be careful if you managed to recalculate an already existing variable in the Events tree, you have to keep the same type of the variable, for examples if you would like to overwrite the variable "Muon_mediumId" which is of type "Bool" you should do that by assigning the recomputed variable to the type Bool as well.
 
Changed:
<
<
see presentation in the PPD general meeting of 21/09/2017 and PPD workshop
>
>
This is well discussed in :
 
Added:
>
>
issue 81, and solved by PR 84

Presentations on NanoAOD format

 
Added:
>
>
see presentation in the PPD general meeting of 21/09/2017 and PPD workshop
 

Centrally produced samples

Changed:
<
<
NanoAOD are centrally produced. Available campaigns are
>
>
NanoAOD are centrally produced. Available campaigns are
 
  • 05Feb2018 - 80X samples (2016 run, data and MC)
  • 12Apr2018 - 94X miniAODV2 samples (MC for 2017 run)
Line: 129 to 129
 

How to check out the code and produce ntuples yoursel:

Changed:
<
<
In general you should not need to produce nanoAOD yourself. If there is a missing variable in standard centrally-produced nanoAOD we encourage to request the addition of such variable via a PR or Issue on nanoaod github.
>
>
In general you should not need to produce nanoAOD yourself. If there is a missing variable in standard centrally-produced nanoAOD we encourage to request the addition of such variable via a PR or Issue on nanoaod github.
  You may want to make ntuples yourself for testing new features, for signal samples with special additional variables or for other particular workflows.

Recipe for CMSSW_9_4_X and the current 94X HEAD of NanoAOD:

Deleted:
<
<
From CMSSW_9_4_X a version of NanoAOD is included in CMSSW.
 
Added:
>
>
From CMSSW_9_4_X a version of NanoAOD is included in CMSSW.
 

cmsrel CMSSW_9_4_6_patch1
Line: 154 to 153
 git clone https://github.com/cms-nanoAOD/nanoAOD-tools.git PhysicsTools/NanoAODTools

# compile it

Changed:
<
<
scram build
>
>
scram build
 
Changed:
<
<
You can then produce NanoAODs with nano_cfg.py. You can also generate a configuration using
cmsDriver.py nanoAOD -s NANO [--mc | --data] --eventcontent NANOAOD[SIM] --datatier NANOAOD[SIM] --conditions [global tag] --era Run2_2017 --no_exec
>
>
You can then produce NanoAODs with nano_cfg.py. You can also generate a configuration using
cmsDriver.py nanoAOD -s NANO [--mc | --data] --eventcontent NANOAOD[SIM] --datatier NANOAOD[SIM] --conditions [global tag] --era Run2_2017 --no_exec
 

Running on various datasets from past campaigns

Deleted:
<
<
The following example cmsDriver commands provide the correct config for different Eras (you need a valid GRID Proxy as data is served from remote locations)
 
Added:
>
>
The following example cmsDriver commands provide the correct config for different Eras (you need a valid GRID Proxy as data is served from remote locations)
 
cmsDriver.py test80X -s NANO --mc --eventcontent NANOAODSIM --datatier NANOAODSIM --filein /store/mc/RunIISummer16MiniAODv2/DYJetsToLL_M-50_TuneCUETP8M1_13TeV-madgraphMLM-pythia8/MINIAODSIM/PUMoriond17_80X_mcRun2_asymptotic_2016_TrancheIV_v6_ext1-v2/120000/02A210D6-F5C3-E611-B570-008CFA197BD4.root   --no_exec  --conditions auto:run2_mc -n 1000 --era Run2_2016,run2_miniAOD_80XLegacy
Line: 198 to 195
  It is important to modify the input configuration so that it runs a single NanoAODOutputModule and add the following parameter as configuraion of the outputmodule.
Changed:
<
<
fakeNameForCrab =cms.untracked.bool(True),
>
>
fakeNameForCrab =cms.untracked.bool(True),
 

How to run via crab on NanoAOD using NanoAOD-Tools

Added:
>
>
 NanoAOD-tools provide a post processing framework for simple and fast skimming and computation of derived quantities.
Changed:
<
<
Such step can also be run on crab using the example give in PhysicsTools/NanoAODTools/crab/
>
>
Such step can also be run on crab using the example give in PhysicsTools /NanoAODTools/crab/
 
Changed:
<
<
It is sufficient to edit crab_script.py
>
>
It is sufficient to edit crab_script.py
  replacing the settings of the PostProcessor with the ones you want

Quickly make plots with NanoAODTools postprocessor

Deleted:
<
<
The postprocessor can be used also to simply make histograms. Quick recipe for impatient user:
 
Changed:
<
<
cmsrel CMSSW_9_4_6_patch1

>
>
The postprocessor can be used also to simply make histograms. Quick recipe for impatient user:
cmsrel CMSSW_9_4_6_patch1

 cd CMSSW_9_4_6_patch1/src cmsenv git cms-init #not really needed unless you later want to add some other cmssw stuff
Line: 232 to 224
 

How to contribute and request changes

Changed:
<
<
You can contribute to NanoAOD the same way you contribute to the rest of CMSSW via github. The only difference is that NanoAOD related PR should be first submited to github.com/cms-nanoAOD/cmssw fork and then from there they will be collected, tested and pushed in main CMSSW via central Pull Requests.
>
>
You can contribute to NanoAOD the same way you contribute to the rest of CMSSW via github. The only difference is that NanoAOD related PR should be first submited to github.com/cms-nanoAOD/cmssw fork and then from there they will be collected, tested and pushed in main CMSSW via central Pull Requests.
  If you want to help but do not know what you can do, have a look at the issues for NanoAOD and NanoAOD-Tools and tell us there that you are going to implement some of the missing features or fix some of the open issues.

If you want to add something first create an issue under https://github.com/cms-nanoAOD/cmssw/issues (unless a similar issue doesn't exist already, in that case comment in the existing issue) clearly telling what you want to add/change and what is the use case.

So the workflow to contribute is something like:

Changed:
<
<
  • set up a recent CMSSW release and take the master branch of cms-nanoAOD/cmssw (as explained in the top part of this twiki)
>
>
  • set up a recent CMSSW release and take the master branch of cms-nanoAOD/cmssw (as explained in the top part of this twiki)
 
  • add/modify your content, mostly editing PhysicsTools/NanoAOD/python/ and PhysicsTools/NanoAOD/plugins/
  • if your code adds some variable, run the prepareDQM.py script on a file having the new variable to update the DQM configuration
Changed:
<
<
    • prepareDQM.py -d -u nano_file_mc.root
>
>
    • prepareDQM.py -d -u nano_file_mc.root
 
    • if necessary, edit the file (e.g. to fix the range or binning of the plot; do not reoder the lines), and then re-run the script again so that the file is formatted properly (e.g. spacing, decimals, etc.)
  • push the code in your cmssw fork on github (please push only the stuff you need, without for example the merge commits to more recent versions of CMSSW than the one used in cms-nanoAOD master branch)
  • make a PR against cms-nanoAOD/cmssw
Changed:
<
<
    • NB: most likely github offers you as default to make a PR against central CMSSW, please be sure you do not by changing the "base fork:" from cms-sw/cmssw to cms-nanoAOD/cmssw
>
>
    • NB: most likely github offers you as default to make a PR against central CMSSW, please be sure you do not by changing the "base fork:" from cms-sw/cmssw to cms-nanoAOD/cmssw
 
  • mention in the PR the issue such PR is addressing (you should have created an issue first unless the change is trivial)

-- AndreaRizzi - 2017-09-21 \ No newline at end of file

Revision 332018-04-19 - ClemensLange

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

NanoAOD Documentation

Line: 158 to 158
 

You can then produce NanoAODs with nano_cfg.py.

Changed:
<
<
You can also generate a configuration using
cmsDriver nanoAOD -s NANO [--mc | --data] --eventcontent NANOAOD[SIM] --datatier NANOAOD[SIM] --conditions [global tag] --era Run2_2017 --no_exec
>
>
You can also generate a configuration using
cmsDriver.py nanoAOD -s NANO [--mc | --data] --eventcontent NANOAOD[SIM] --datatier NANOAOD[SIM] --conditions [global tag] --era Run2_2017 --no_exec
 

Running on various datasets from past campaigns

The following example cmsDriver commands provide the correct config for different Eras

Revision 322018-04-17 - AndreaRizzi

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

NanoAOD Documentation

Line: 6 to 6
 NanoAOD format consists of an Ntuple like format, readable with bare root and containing the per-event information that is needed in most generic analyses. A NanoAOD is not a CMSSW EDM file but several EDM features are available in this simplified format too.
Changed:
<
<
The size per event of NanoAOD is order of 1Kb, they can be centrally produced in ~1-2 week time, central production can be triggered often if many new features are made available in "hot periods". Users can easily extend NanoAOD for their specific study making a private production when needed (suggested for specific studies on samples that are not too large or for some special needs) or can ask for new features inclusion in the central format (suggested when need to run on multi billion events)
>
>
The size per event of NanoAOD is order of 1Kb, they can be centrally produced in ~1-2 week time, central production can be triggered often if many new features are made available in "hot periods". Users can easily extend NanoAOD for their specific study making a private production when needed (suggested for specific studies on samples that are not too large or for some special needs) or can ask for new features inclusion in the central format (suggested when need to run on multi billion events)
  A NanoAOD file contains a main TTree named Events and some additional TTrees for run, lumi and metadata.
Line: 31 to 31
 
Specific information on some branches/objects

Jets
Added:
>
>
  • Jets are corrected to latest available JECs and provided with latest available JetID, PU ID, b-tag and QGL.
  • No systematic variations of JECs are stored, but the quantities (rho, area, pt, eta) needed to recompute JECs or JEC systematic variations are available. Dedicated nanoAOD-tools modules exist to compute those quantities.
  • b-jet energy regression, developed for final states with one or more H->bb, is also available.
  • Jets of MC samples are not smeared (dedicated nanoAOD-tools modules exist to apply the smearing).
 
Muons
Added:
>
>
 
Electrons
..e.g. explain the bitmask...
MET
Line: 35 to 42
 
Electrons
..e.g. explain the bitmask...
MET
Added:
>
>
Trigger
Trigger bits information is available for all trigger in the input files. If multiple files are given in input with different trigger bits, the bits not available in some events are filled with zeros. When merging two nanoaod file with haddnano.py the same algorithm to fill missing trigger bits is used (i.e. the output file will have zeros for events that had a given trigger bit not available)

Trigger objects are stored in nanoaod for some filters as defined here, if you need some additional object p4 to be stored please open an issue.

 
Generator, dressed leptons, LHE
Added:
>
>
Several collections are available for gen level information:
  • a subset of gen particle selected according to this pruner configuration
  • dressed lepton are also available as configured [[https://github.com/cms-sw/cmssw/blob/master/PhysicsTools/NanoAOD/python/particlelevel_cff.py][here]
  • gen jets with pt > 10 are stored. Parton and Hadron flavour as well as link to gen jet is available in Jets
  • Some LHE information is stored (see here
 
Weigths
..e.g. explain the PDF weights...
Line: 101 to 120
 

Centrally produced samples

Changed:
<
<
NanoAOD are going to be centrally produced. Test campaigns are already available:
>
>
NanoAOD are centrally produced. Available campaigns are
 
Changed:
<
<
>
>
 

How to check out the code and produce ntuples yoursel:

Revision 282018-04-08 - KorbinianGuntherSchweiger1

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

NanoAOD Documentation

Line: 149 to 149
  cmsDriver.py test92X -s NANO --mc --eventcontent NANOAODSIM --datatier NANOAODSIM --filein /store/relval/CMSSW_9_2_12/RelValTTbar_13/MINIAODSIM/PU25ns_92X_upgrade2017_realistic_v11-v1/00000/080E2624-F59D-E711-ACEE-0CC47A7C35A4.root --no_exec --conditions auto:phase1_2017_realistic -n 1000 --era Run2_2017,run2_nanoAOD_92X
Changed:
<
<
cmsDriver.py test94X -s NANO --mc --eventcontent NANOAODSIM --datatier NANOAODSIM --filein /store/mc/RunIIFall17MiniAOD/TTToSemiLeptonic_TuneCP5_PSweights_13TeV-powheg-pythia8/MINIAODSIM/94X_mc2017_realistic_v10-v1/60000/A0D71AEE-13E1-E711-B3C9-FA163E629498.root --no_exec --conditions auto:phase1_2017_realistic -n 1000 --era Run2_2017
>
>
cmsDriver.py test94X -s NANO --mc --eventcontent NANOAODSIM --datatier NANOAODSIM --filein /store/mc/RunIIFall17MiniAOD/TTToSemiLeptonic_TuneCP5_PSweights_13TeV-powheg-pythia8/MINIAODSIM/94X_mc2017_realistic_v10-v1/60000/A0D71AEE-13E1-E711-B3C9-FA163E629498.root --no_exec --conditions auto:phase1_2017_realistic -n 1000 --era Run2_2017,run2_nanoAOD_94XMiniAODv1
  ## FOR DATA: cmsDriver.py test_data_92X -s NANO --data --eventcontent NANOAOD --datatier NANOAOD --filein /store/data/Run2017C/JetHT/MINIAOD/PromptReco-v1/000/299/649/00000/1A4E4E38-E871-E711-A5E3-02163E01A2A6.root --conditions auto:run2_data_relval -n 100 --era Run2_2017,run2_nanoAOD_92X

Revision 272018-03-22 - AndreaRizzi

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"
Added:
>
>

NanoAOD Documentation

NanoAOD format

NanoAOD format consists of an Ntuple like format, readable with bare root and containing the per-event information that is needed in most generic analyses. A NanoAOD is not a CMSSW EDM file but several EDM features are available in this simplified format too.
 
Changed:
<
<

Introduction to NanoAOD format

>
>
The size per event of NanoAOD is order of 1Kb, they can be centrally produced in ~1-2 week time, central production can be triggered often if many new features are made available in "hot periods". Users can easily extend NanoAOD for their specific study making a private production when needed (suggested for specific studies on samples that are not too large or for some special needs) or can ask for new features inclusion in the central format (suggested when need to run on multi billion events)
 
Changed:
<
<
see presentation in the PPD general meeting of 21/09/2017 and PPD workshop
>
>
A NanoAOD file contains a main TTree named Events and some additional TTrees for run, lumi and metadata.

The Events TTree

The main TTree contains only scalar branches or simple array branches. No special dataformat is needed to read NanoAODs. Physics objects features are grouped via naming conventions and by sharing the same array dimensionality.

As an example there is a branch named Muon_pt[nMuon] and a branch named Muon_eta[nMuon], both are arrays of length nMuon. Access as whole objects can be obtained with light frameworks (see e.g. the NanoAOD-Tools below)

References across objects are implemented via branches with idx suffix that are just bare indices that contains information about the collection they point to, e.g. Electron_jetIdx contains information about the jet, if available that contains a given electron. A value of -1 means the ref is null, a value i >= 0 means that given Electron is associated to the i-th object in the Jet collection. Another example Jet_genJetIdx is the index to the GenJet collection. The pointer could also point outside the boundary of the collection if some elements in the tail of the collection are dropped. For example before accessing the GenJet associated to a Jet one should verify that 0<= Jet_genJetIdx < nGenJet.

HLT bits are automatically generated based on HLT bits available in the input file. Each time files are merged (in CMSSW or via the haddnano.py utility) if the list of bits is not matching the new/old bits are zero-filled so that the branches are properly aligned.

Object systematic uncertainties (e.g. all JES variations) are not stored persistently but, instead, the per-event information needed to compute those corrections is saved.

Many variables are stored with limited precision (i.e. less than 32bits float) by zeroing a given number of bits in the float mantissa (this results in a higher compression when stored on disk).

No skimming is applied at production level.

Some linking to gen and trigger objects is available. No cleaning is applied but objects are cross referenced (with indices, see above). The cross referencing is based on common PF constituents, not on DeltaR. DeltaR matching is less accurate and can be reproduced from NanoAOD if needed, while PF matching can only be done while reading MiniAOD. Predefined clean bitmasks can be used to store persistently a given cleaning choice (e.g. specific of a whole PAG), so that the user can clean by simply checking the proper bit.

Specific information on some branches/objects

Jets
Muons
Electrons
..e.g. explain the bitmask...
MET
Generator, dressed leptons, LHE
Weigths
..e.g. explain the PDF weights...

The Other TTrees

 
Added:
>
>

Content auto-documentation

 Documentation of content for different releases:
Added:
>
>

NanoAOD-Tools Documentation

NanoAOD-Tools provide a simple and lightweight python post-processing framework for fast skimming, analysis and computation of derived quantities.

NanoAOD-Tools should contain code implementation of NanoAOD ready "POG recipes" so that no one has to translate from twiki to code and the code is already available to access the data in NanoAOD format.

We anticipate that many analysis would run skimming and computation of derived quantities or systematic variations in a single step. Such step can be easily implemented with the NanoAOD-Tools postprocessors.

Existing modules in NanoAOD-Tools

Generic

  • PU weighting

POG specific

  • BTV: tool to compute scale factors
  • JME: JEC uncertainties, JER smearing with MET propagation
  • MUO: momentum calibration (to be implemented)

How to run via crab on NanoAOD using NanoAOD-Tools

Such step can also be run on crab using the example give in PhysicsTools/NanoAODTools/crab/

It is sufficient to edit crab_script.py

replacing the settings of the PostProcessor with the ones you want

Quickly make plots with NanoAODTools postprocessor

The postprocessor can be used also to simply make histograms. Quick recipe for impatient user:

cmsrel CMSSW_9_4_4
cd CMSSW_9_4_4/src
cmsenv
git cms-init   #not really needed unless you later want to add some other cmssw stuff
git clone https://github.com/cms-nanoAOD/nanoAOD-tools.git PhysicsTools/NanoAODTools
scram b
voms-proxy-init -voms cms

cd PhysicsTools/NanoAODTools/python/postprocessing/examples/
python exampleAnalysis.py

More information

Presentations on NanoAOD format

see presentation in the PPD general meeting of 21/09/2017 and PPD workshop

 

Centrally produced samples

NanoAOD are going to be centrally produced. Test campaigns are already available:

Revision 242018-02-21 - AndreaRizzi

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"
Line: 104 to 105
  replacing the settings of the PostProcessor with the ones you want
Added:
>
>

Quickly make plots with NanoAODTools postprocessor

The postprocessor can be used also to simply make histograms. Quick recipe for impatient user:

cmsrel CMSSW_9_4_4
cd CMSSW_9_4_4/src
cmsenv
git cms-init   #not really needed unless you later want to add some other cmssw stuff
git clone https://github.com/cms-nanoAOD/nanoAOD-tools.git PhysicsTools/NanoAODTools
scram b
voms-proxy-init -voms cms

cd PhysicsTools/NanoAODTools/python/postprocessing/examples/
python exampleAnalysis.py
 

How to contribute and request changes

You can contribute to NanoAOD the same way you contribute to the rest of CMSSW via github. The only difference is that NanoAOD related PR should be first submited to

Revision 212018-02-06 - AndreaRizzi

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

Introduction to NanoAOD format

Changed:
<
<
see presentation in the PPD general meeting of 21/09/2017: link
>
>
see presentation in the PPD general meeting of 21/09/2017 and PPD workshop
  Documentation of content for different releases:
Added:
>
>
 
Deleted:
<
<
 
Changed:
<
<

How to check out the code and produce ntuples:

>
>
--++ Centrally produced samples NanoAOD are going to be centrally produced. Test campaigns are already available:

How to check out the code and produce ntuples yoursel:

In general you should not need to produce nanoAOD yourself. If there is a missing variable in standard centrally-produced nanoAOD we encourage to request the addition of such variable via a PR or Issue on nanoaod github.

You may want to make ntuples yourself for testing new features, for signal samples with special additional variables or for other particular workflows.

 

Recipe for CMSSW_9_4_X and the current 94X HEAD of NanoAOD:

From CMSSW_9_4_X a version of NanoAOD is included in CMSSW.
Line: 39 to 51
 You can then produce NanoAODs with test_nano.cfg. You can also generate a configuration using
cmsDriver nanoAOD -s NANO [--mc | --data] --eventcontent NANOAOD[SIM] --datatier NANOAOD[SIM] --conditions [global tag] --era Run2_2017 --no_exec
Changed:
<
<

Running on older datasets (92X and 80X)

>
>

Running on various datasets from past campaigns

 The following example cmsDriver commands provide the correct config for different Eras (you need a valid GRID Proxy as data is served from remote locations)

Revision 182018-01-09 - GiovanniPetrucciani

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"
Line: 48 to 48
  cmsDriver.py test92X -s NANO --mc --eventcontent NANOAODSIM --datatier NANOAODSIM --filein /store/relval/CMSSW_9_2_12/RelValTTbar_13/MINIAODSIM/PU25ns_92X_upgrade2017_realistic_v11-v1/00000/080E2624-F59D-E711-ACEE-0CC47A7C35A4.root --no_exec --conditions auto:phase1_2017_realistic -n 1000 --era Run2_2017,run2_nanoAOD_92X
Changed:
<
<
cmsDriver.py test94X -s NANO --mc --eventcontent NANOAODSIM --datatier NANOAODSIM --filein /store/relval/CMSSW_9_4_0_pre1/RelValTTbar_13/MINIAODSIM/PU25ns_93X_mc2017_realistic_v3-v1/00000/92FD5642-509D-E711-ADAB-0025905B85C6.root --no_exec --conditions auto:phase1_2017_realistic -n 1000 --era Run2_2017
>
>
cmsDriver.py test94X -s NANO --mc --eventcontent NANOAODSIM --datatier NANOAODSIM --filein /store/mc/RunIIFall17MiniAOD/TTToSemiLeptonic_TuneCP5_PSweights_13TeV-powheg-pythia8/MINIAODSIM/94X_mc2017_realistic_v10-v1/60000/A0D71AEE-13E1-E711-B3C9-FA163E629498.root --no_exec --conditions auto:phase1_2017_realistic -n 1000 --era Run2_2017
  ## FOR DATA: cmsDriver.py test_data_92X -s NANO --data --eventcontent NANOAOD --datatier NANOAOD --filein /store/data/Run2017C/JetHT/MINIAOD/PromptReco-v1/000/299/649/00000/1A4E4E38-E871-E711-A5E3-02163E01A2A6.root --conditions auto:run2_data_relval -n 100 --era Run2_2017,run2_nanoAOD_92X

Revision 162017-12-20 - GiovanniPetrucciani

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"
Line: 18 to 18
 
Changed:
<
<
cmsrel CMSSW_9_4_1 cd CMSSW_9_4_1/src
>
>
cmsrel CMSSW_9_4_2 cd CMSSW_9_4_2/src
 cmsenv

# the above is sufficient for a first version of NanoAOD

Line: 104 to 104
 
  • add/modify your content, mostly editing PhysicsTools/NanoAOD/python/ and PhysicsTools/NanoAOD/plugins/
  • if your code adds some variable, run the prepareDQM.py script on a file having the new variable to update the DQM configuration
    • prepareDQM.py -d -u nano_file_mc.root
Added:
>
>
    • if necessary, edit the file (e.g. to fix the range or binning of the plot; do not reoder the lines), and then re-run the script again so that the file is formatted properly (e.g. spacing, decimals, etc.)
 
  • push the code in your cmssw fork on github (please push only the stuff you need, without for example the merge commits to more recent versions of CMSSW than the one used in cms-nanoAOD master branch)
  • make a PR against cms-nanoAOD/cmssw
    • NB: most likely github offers you as default to make a PR against central CMSSW, please be sure you do not by changing the "base fork:" from cms-sw/cmssw to cms-nanoAOD/cmssw

Revision 152017-11-30 - GiovanniPetrucciani

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"
Line: 8 to 8
  Documentation of content for different releases:
Changed:
<
<
>
>
 

How to check out the code and produce ntuples:

Changed:
<
<

Recipe for CMSSW_9_4_0_pre3 and the current 94X HEAD of NanoAOD:

From CMSSW_9_4_0_pre3 a version of NanoAOD is included in CMSSW.
>
>

Recipe for CMSSW_9_4_X and the current 94X HEAD of NanoAOD:

From CMSSW_9_4_X a version of NanoAOD is included in CMSSW.
 
Changed:
<
<
cmsrel CMSSW_9_4_0 cd CMSSW_9_4_0/src
>
>
cmsrel CMSSW_9_4_1 cd CMSSW_9_4_1/src
 cmsenv

#the above is sufficient for a first version of NanoAOD

#in order to get the latest developments git cms-merge-topic cms-nanoAOD:master

Added:
>
>
# create a branch directly on the nanoAOD branch, skipping the unnecessary merge commit from cms-merge-topic git checkout -b nanoAOD cms-nanoAOD/master
  # get also the support library git clone https://github.com/cms-nanoAOD/nanoAOD-tools.git PhysicsTools/NanoAODTools
Line: 35 to 37
 

You can then produce NanoAODs with test_nano.cfg.

Changed:
<
<
You can also generate a configuration using
cmsDriver nanoAOD -s NANO [--mc | --data] --eventcontent NANOAOD[SIM] --datatier NANO --conditions [global tag] --no_exec
>
>
You can also generate a configuration using
cmsDriver nanoAOD -s NANO [--mc | --data] --eventcontent NANOAOD[SIM] --datatier NANOAOD[SIM] --conditions [global tag] --era Run2_2017 --no_exec
 

Running on older datasets (92X and 80X)

The following example cmsDriver commands provide the correct config for different Eras

Revision 122017-11-03 - AndreaRizzi

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"
Line: 42 to 42
 cmsDriver.py test92X -s NANO --mc --eventcontent NANOAODSIM --datatier NANOAODSIM --filein /store/relval/CMSSW_9_2_12/RelValTTbar_13/MINIAODSIM/PU25ns_92X_upgrade2017_realistic_v11-v1/00000/080E2624-F59D-E711-ACEE-0CC47A7C35A4.root --no_exec --conditions auto:phase1_2017_realistic -n 1000 --era Run2_2017,run2_nanoAOD_92X

cmsDriver.py test94X -s NANO --mc --eventcontent NANOAODSIM --datatier NANOAODSIM --filein /store/relval/CMSSW_9_4_0_pre1/RelValTTbar_13/MINIAODSIM/PU25ns_93X_mc2017_realistic_v3-v1/00000/92FD5642-509D-E711-ADAB-0025905B85C6.root --no_exec --conditions auto:phase1_2017_realistic -n 1000 --era Run2_2017

Added:
>
>
## FOR DATA: cmsDriver.py test_data_92X -s NANO --data --eventcontent NANOAOD --datatier NANOAOD --filein /store/data/Run2017C/JetHT/MINIAOD/PromptReco-v1/000/299/649/00000/1A4E4E38-E871-E711-A5E3-02163E01A2A6.root --conditions auto:run2_data_relval -n 100 --era Run2_2017,run2_nanoAOD_92X

cmsDriver.py test_data_80X -s NANO --data --eventcontent NANOAOD --datatier NANOAOD --filein /store/data/Run2016H/SingleMuon/MINIAOD/03Feb2017_ver2-v1/110000/F836EEF3-E8EA-E611-AB15-0CC47AA9943A.root --conditions auto:run2_data_relval -n 100 --era Run2_2016,run2_miniAOD_80XLegacy

 
Added:
>
>
 

Where to get the ntuples

Some test ntuples are produced via crab (see below if you want to do it yourself) and you can have a look/play with them at the following links

Revision 112017-10-26 - AndreaRizzi

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"
Line: 37 to 37
 (you need a valid GRID Proxy as data is served from remote locations)
Changed:
<
<
cmsDriver.py test80X -s NANO --mc --eventcontent NANOAODSIM --datatier NANOAODSIM --filein /store/relval/CMSSW_8_0_0/RelValTTbar_13/MINIAODSIM/PU25ns_80X_mcRun2_asymptotic_v4-v1/10000/A65CD249-BFDA-E511-813A-0025905A6066.root --no_exec --conditions auto:run2_mc -n 1000 --era Run2_2016,run2_miniAOD_80XLegacy
>
>
cmsDriver.py test80X -s NANO --mc --eventcontent NANOAODSIM --datatier NANOAODSIM --filein /store/mc/RunIISummer16MiniAODv2/DYJetsToLL_M-50_TuneCUETP8M1_13TeV-madgraphMLM-pythia8/MINIAODSIM/PUMoriond17_80X_mcRun2_asymptotic_2016_TrancheIV_v6_ext1-v2/120000/02A210D6-F5C3-E611-B570-008CFA197BD4.root --no_exec --conditions auto:run2_mc -n 1000 --era Run2_2016,run2_miniAOD_80XLegacy
  cmsDriver.py test92X -s NANO --mc --eventcontent NANOAODSIM --datatier NANOAODSIM --filein /store/relval/CMSSW_9_2_12/RelValTTbar_13/MINIAODSIM/PU25ns_92X_upgrade2017_realistic_v11-v1/00000/080E2624-F59D-E711-ACEE-0CC47A7C35A4.root --no_exec --conditions auto:phase1_2017_realistic -n 1000 --era Run2_2017,run2_nanoAOD_92X

Revision 102017-10-25 - GiovanniPetrucciani

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"
Line: 37 to 37
 (you need a valid GRID Proxy as data is served from remote locations)
Changed:
<
<
cmsDriver.py test80X -s NANO --mc --eventcontent NANOAODSIM --datatier NANO --filein /store/relval/CMSSW_8_0_0/RelValTTbar_13/MINIAODSIM/PU25ns_80X_mcRun2_asymptotic_v4-v1/10000/A65CD249-BFDA-E511-813A-0025905A6066.root --no_exec --conditions auto:run2_mc -n 1000 --era Run2_2016,run2_miniAOD_80XLegacy
>
>
cmsDriver.py test80X -s NANO --mc --eventcontent NANOAODSIM --datatier NANOAODSIM --filein /store/relval/CMSSW_8_0_0/RelValTTbar_13/MINIAODSIM/PU25ns_80X_mcRun2_asymptotic_v4-v1/10000/A65CD249-BFDA-E511-813A-0025905A6066.root --no_exec --conditions auto:run2_mc -n 1000 --era Run2_2016,run2_miniAOD_80XLegacy
 
Changed:
<
<
cmsDriver.py test92X -s NANO --mc --eventcontent NANOAODSIM --datatier NANO --filein /store/relval/CMSSW_9_2_12/RelValTTbar_13/MINIAODSIM/PU25ns_92X_upgrade2017_realistic_v11-v1/00000/080E2624-F59D-E711-ACEE-0CC47A7C35A4.root --no_exec --conditions auto:phase1_2017_realistic -n 1000 --era Run2_2017,run2_nanoAOD_92X
>
>
cmsDriver.py test92X -s NANO --mc --eventcontent NANOAODSIM --datatier NANOAODSIM --filein /store/relval/CMSSW_9_2_12/RelValTTbar_13/MINIAODSIM/PU25ns_92X_upgrade2017_realistic_v11-v1/00000/080E2624-F59D-E711-ACEE-0CC47A7C35A4.root --no_exec --conditions auto:phase1_2017_realistic -n 1000 --era Run2_2017,run2_nanoAOD_92X
 
Changed:
<
<
cmsDriver.py test94X -s NANO --mc --eventcontent NANOAODSIM --datatier NANO --filein /store/relval/CMSSW_9_4_0_pre1/RelValTTbar_13/MINIAODSIM/PU25ns_93X_mc2017_realistic_v3-v1/00000/92FD5642-509D-E711-ADAB-0025905B85C6.root --no_exec --conditions auto:phase1_2017_realistic -n 1000 --era Run2_2017
>
>
cmsDriver.py test94X -s NANO --mc --eventcontent NANOAODSIM --datatier NANOAODSIM --filein /store/relval/CMSSW_9_4_0_pre1/RelValTTbar_13/MINIAODSIM/PU25ns_93X_mc2017_realistic_v3-v1/00000/92FD5642-509D-E711-ADAB-0025905B85C6.root --no_exec --conditions auto:phase1_2017_realistic -n 1000 --era Run2_2017
 

Where to get the ntuples

Revision 92017-10-25 - AndreaRizzi

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"
Line: 8 to 8
 

How to check out the code and produce ntuples:

Changed:
<
<

Recipe for CMSSW_9_4_0_pre2 and the current 94X HEAD of NanoAOD:

>
>

Recipe for CMSSW_9_4_0_pre3 and the current 94X HEAD of NanoAOD:

From CMSSW_9_4_0_pre3 a version of NanoAOD is included in CMSSW.
 
Changed:
<
<
cmsrel CMSSW_9_4_0_pre2 cd CMSSW_9_4_0_pre2/src
>
>
cmsrel CMSSW_9_4_0_pre3 cd CMSSW_9_4_0_pre3/src
 cmsenv
Added:
>
>
#the above is sufficient for a first version of NanoAOD

#in order to get the latest developments

 git cms-merge-topic cms-nanoAOD:master

# get also the support library

Revision 82017-10-18 - AndreaRizzi

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"
Line: 74 to 74
 You can contribute to NanoAOD the same way you contribute to the rest of CMSSW via github. The only difference is that NanoAOD related PR should be first submited to github.com/cms-nanoAOD/cmssw fork and then from there they will be collected, tested and pushed in main CMSSW via central Pull Requests.
Added:
>
>
If you want to help but do not know what you can do, have a look at the issues for NanoAOD and NanoAOD-Tools and tell us there that you are going to implement some of the missing features or fix some of the open issues.
 If you want to add something first create an issue under https://github.com/cms-nanoAOD/cmssw/issues (unless a similar issue doesn't exist already, in that case comment in the existing issue) clearly telling what you want to add/change and what is the use case.

So the workflow to contribute is something like:

  • set up a recent CMSSW release and take the master branch of cms-nanoAOD/cmssw (as explained in the top part of this twiki)
  • add/modify your content, mostly editing PhysicsTools/NanoAOD/python/ and PhysicsTools/NanoAOD/plugins/
Added:
>
>
  • if your code adds some variable, run the prepareDQM.py script on a file having the new variable to update the DQM configuration
    • prepareDQM.py -d -u nano_file_mc.root
 
  • push the code in your cmssw fork on github (please push only the stuff you need, without for example the merge commits to more recent versions of CMSSW than the one used in cms-nanoAOD master branch)
  • make a PR against cms-nanoAOD/cmssw
    • NB: most likely github offers you as default to make a PR against central CMSSW, please be sure you do not by changing the "base fork:" from cms-sw/cmssw to cms-nanoAOD/cmssw

Revision 72017-10-06 - AndreaRizzi

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"
Line: 40 to 40
 

Where to get the ntuples

Changed:
<
<
Some test ntuples are produced via crab (see below if you want to do it yourself) and you can have a look/play with them at DAS link
>
>
Some test ntuples are produced via crab (see below if you want to do it yourself) and you can have a look/play with them at the following links

NB: this is not the way NanoAOD are meant to be produced. Production will soon be handled centrally by computing

How to make NanoAOD via crab

While central production will take care soon of making nanoAOD for all CMS samples there are two use cases for NanoAOD made with crab:
  • test NanoAOD format before production is ready to handle them (expected late 2017)
  • add analysis specific information in the format for a few samples (e.g. signal samples), i.e. run your customized version of NanoAOD
 
Changed:
<
<

How to make NanoAOD via crab

While we wait for central integration of nanoAOD in production, individuals or groups can test NanoAOD production via crab using the provided config in PhysicsTools/NanoAOD/test/crab_mc.py
>
>
Example config is available here: PhysicsTools/NanoAOD/test/crab_mc.py
  It is important to modify the input configuration so that it runs a single NanoAODOutputModule and add the following parameter as configuraion of the outputmodule.

Revision 62017-10-05 - AndreaRizzi

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"
Line: 32 to 32
 (you need a valid GRID Proxy as data is served from remote locations)
Changed:
<
<
cmsDriver.py test80X -s NANO --mc --eventcontent NANOAODSIM --datatier NANO --filein /store/mc/RunIISummer16MiniAODv2/DYJetsToLL_M-50_TuneCUETP8M1_13TeV-madgraphMLM-pythia8/MINIAODSIM/PUMoriond17_80X_mcRun2_asymptotic_2016_TrancheIV_v6_ext1-v2/120000/02A210D6-F5C3-E611-B570-008CFA197BD4.root --no_exec --conditions auto:run2_mc -n 1000 --era Run2_2016,run2_miniAOD_80XLegacy
>
>
cmsDriver.py test80X -s NANO --mc --eventcontent NANOAODSIM --datatier NANO --filein /store/relval/CMSSW_8_0_0/RelValTTbar_13/MINIAODSIM/PU25ns_80X_mcRun2_asymptotic_v4-v1/10000/A65CD249-BFDA-E511-813A-0025905A6066.root --no_exec --conditions auto:run2_mc -n 1000 --era Run2_2016,run2_miniAOD_80XLegacy
 
Changed:
<
<
cmsDriver.py test92X -s NANO --mc --eventcontent NANOAODSIM --datatier NANO --filein /store/mc/RunIISummer17MiniAOD/TT_TuneCUETP8M2T4_13TeV-powheg-pythia8/MINIAODSIM/92X_upgrade2017_realistic_v10_ext1-v1/110000/187F7EDA-0986-E711-ABB3-02163E014C21.root --no_exec --conditions auto:phase1_2017_realistic -n 1000 --era Run2_2017,run2_nanoAOD_92X
>
>
cmsDriver.py test92X -s NANO --mc --eventcontent NANOAODSIM --datatier NANO --filein /store/relval/CMSSW_9_2_12/RelValTTbar_13/MINIAODSIM/PU25ns_92X_upgrade2017_realistic_v11-v1/00000/080E2624-F59D-E711-ACEE-0CC47A7C35A4.root --no_exec --conditions auto:phase1_2017_realistic -n 1000 --era Run2_2017,run2_nanoAOD_92X
  cmsDriver.py test94X -s NANO --mc --eventcontent NANOAODSIM --datatier NANO --filein /store/relval/CMSSW_9_4_0_pre1/RelValTTbar_13/MINIAODSIM/PU25ns_93X_mc2017_realistic_v3-v1/00000/92FD5642-509D-E711-ADAB-0025905B85C6.root --no_exec --conditions auto:phase1_2017_realistic -n 1000 --era Run2_2017

Revision 52017-10-04 - AndreaRizzi

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"
Line: 27 to 27
 You can then produce NanoAODs with test_nano.cfg. You can also generate a configuration using
cmsDriver nanoAOD -s NANO [--mc | --data] --eventcontent NANOAOD[SIM] --datatier NANO --conditions [global tag] --no_exec
Added:
>
>

Running on older datasets (92X and 80X)

The following example cmsDriver commands provide the correct config for different Eras (you need a valid GRID Proxy as data is served from remote locations)

cmsDriver.py test80X -s NANO --mc --eventcontent NANOAODSIM --datatier NANO --filein /store/mc/RunIISummer16MiniAODv2/DYJetsToLL_M-50_TuneCUETP8M1_13TeV-madgraphMLM-pythia8/MINIAODSIM/PUMoriond17_80X_mcRun2_asymptotic_2016_TrancheIV_v6_ext1-v2/120000/02A210D6-F5C3-E611-B570-008CFA197BD4.root  --no_exec  --conditions auto:run2_mc -n 1000 --era Run2_2016,run2_miniAOD_80XLegacy

cmsDriver.py test92X -s NANO --mc --eventcontent NANOAODSIM --datatier NANO --filein /store/mc/RunIISummer17MiniAOD/TT_TuneCUETP8M2T4_13TeV-powheg-pythia8/MINIAODSIM/92X_upgrade2017_realistic_v10_ext1-v1/110000/187F7EDA-0986-E711-ABB3-02163E014C21.root  --no_exec  --conditions auto:phase1_2017_realistic -n 1000 --era Run2_2017,run2_nanoAOD_92X

cmsDriver.py test94X -s NANO --mc --eventcontent NANOAODSIM --datatier NANO --filein /store/relval/CMSSW_9_4_0_pre1/RelValTTbar_13/MINIAODSIM/PU25ns_93X_mc2017_realistic_v3-v1/00000/92FD5642-509D-E711-ADAB-0025905B85C6.root  --no_exec  --conditions auto:phase1_2017_realistic -n 1000 --era Run2_2017
 

Where to get the ntuples

Added:
>
>
Some test ntuples are produced via crab (see below if you want to do it yourself) and you can have a look/play with them at DAS link
 
Added:
>
>

How to make NanoAOD via crab

While we wait for central integration of nanoAOD in production, individuals or groups can test NanoAOD production via crab using the provided config in PhysicsTools/NanoAOD/test/crab_mc.py

It is important to modify the input configuration so that it runs a single NanoAODOutputModule and add the following parameter as configuraion of the outputmodule.

    fakeNameForCrab =cms.untracked.bool(True),

How to run via crab on NanoAOD using NanoAOD-Tools

NanoAOD-tools provide a post processing framework for simple and fast skimming and computation of derived quantities.

Such step can also be run on crab using the example give in PhysicsTools/NanoAODTools/crab/

It is sufficient to edit crab_script.py

replacing the settings of the PostProcessor with the ones you want

 

How to contribute and request changes

Added:
>
>
You can contribute to NanoAOD the same way you contribute to the rest of CMSSW via github. The only difference is that NanoAOD related PR should be first submited to github.com/cms-nanoAOD/cmssw fork and then from there they will be collected, tested and pushed in main CMSSW via central Pull Requests.

If you want to add something first create an issue under https://github.com/cms-nanoAOD/cmssw/issues (unless a similar issue doesn't exist already, in that case comment in the existing issue) clearly telling what you want to add/change and what is the use case.

So the workflow to contribute is something like:

  • set up a recent CMSSW release and take the master branch of cms-nanoAOD/cmssw (as explained in the top part of this twiki)
  • add/modify your content, mostly editing PhysicsTools/NanoAOD/python/ and PhysicsTools/NanoAOD/plugins/
  • push the code in your cmssw fork on github (please push only the stuff you need, without for example the merge commits to more recent versions of CMSSW than the one used in cms-nanoAOD master branch)
  • make a PR against cms-nanoAOD/cmssw * NB: most likely github offers you as default to make a PR against central CMSSW, please be sure you do not by changing the "base fork:" from cms-sw/cmssw to cms-nanoAOD/cmssw
  • mention in the PR the issue such PR is addressing (you should have created an issue first unless the change is trivial)
  -- AndreaRizzi - 2017-09-21

Revision 42017-10-03 - GiovanniPetrucciani

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"
Line: 8 to 8
 

How to check out the code and produce ntuples:

Changed:
<
<

Recipe for CMSSW_9_4_0_pre1 and the current 94X HEAD of NanoAOD:

>
>

Recipe for CMSSW_9_4_0_pre2 and the current 94X HEAD of NanoAOD:

 
Changed:
<
<
cmsrel CMSSW_9_4_0_pre1 cd CMSSW_9_4_0_pre1/src
>
>
cmsrel CMSSW_9_4_0_pre2 cd CMSSW_9_4_0_pre2/src
 cmsenv
Changed:
<
<
# get the nanoAOD CMSSW branch git cms-merge-topic cms-nanoAOD:nano_94X

# get the data files (will not be needed in 9_4_0_pre2, or in a recent integration build) mkdir -p $CMSSW_BASE/external/$SCRAM_ARCH/data git clone https://github.com/cms-data/PhysicsTools-NanoAOD.git $CMSSW_BASE/external/$SCRAM_ARCH/data/PhysicsTools/NanoAOD/data

>
>
git cms-merge-topic cms-nanoAOD:master
  # get also the support library git clone https://github.com/cms-nanoAOD/nanoAOD-tools.git PhysicsTools/NanoAODTools
Line: 29 to 24
 scram build
Changed:
<
<
You can then produce NanoAODs with test_nano.cfg.
>
>
You can then produce NanoAODs with test_nano.cfg.
 You can also generate a configuration using
cmsDriver nanoAOD -s NANO [--mc | --data] --eventcontent NANOAOD[SIM] --datatier NANO --conditions [global tag] --no_exec

Where to get the ntuples

Revision 22017-09-22 - GiovanniPetrucciani

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

Introduction to NanoAOD format

Added:
>
>
see presentation in the PPD general meeting of 21/09/2017: link

How to check out the code and produce ntuples:

Recipe for CMSSW_9_4_0_pre1 and the current 94X HEAD of NanoAOD:

cmsrel CMSSW_9_4_0_pre1
cd CMSSW_9_4_0_pre1/src
cmsenv

# get the nanoAOD CMSSW branch
git cms-merge-topic cms-nanoAOD:nano_94X

# get the data files (will not be needed in 9_4_0_pre2, or in a recent integration build)
mkdir -p $CMSSW_BASE/external/$SCRAM_ARCH/data 
git clone https://github.com/cms-data/PhysicsTools-NanoAOD.git $CMSSW_BASE/external/$SCRAM_ARCH/data/PhysicsTools/NanoAOD/data 

# get also the support library
git clone https://github.com/cms-nanoAOD/nanoAOD-tools.git PhysicsTools/NanoAODTools

You can then produce NanoAODs with test_nano.cfg. You can also generate a configuration using
cmsDriver nanoAOD -s NANO [--mc | --data] --eventcontent NANOAOD[SIM] --datatier NANO --conditions [global tag] --no_exec

 

Where to get the ntuples

Added:
>
>
 

How to contribute and request changes

 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback