4.2.4.2 Exercise 02: How to create a PAT Tuple

Contents

• Objectives
• Setting up the environment
• Having a look at the PAT configuration file
• Having a deeper look at the PAT configuration
• Running pat::Candidate production on top of RECO/AOD
• Inspecting the output of pat::Candidate production
• Exercises
• Review status

Objectives

From this example you will learn how to create a PAT Tuple on your own. The PAT team provides standard configuration files to create PAT Tuples, which contain already sensible default values for many parameters. The default output of a standard PAT Tuple are the most important collections of high analysis objects like photons, electrons, muons, tau leptons, jets, MET. We will go through the different production steps and change a few basic parameters. This example will provide you with the following information:

• How to run PAT with a standard configuration.
• Basic understanding of the configuration.
• Basic understanding of the output.

Note:

This web course is part of the PAT Tutorial, which takes regularly place at cern and in other places. When following the PAT Tutorial the answers of questions marked in RED should be filled into the exercise form that has been introduced at the beginning of the tutorial. Also the solutions to the Exercises should be filled into the form. The exercises are marked in three colours, indicating whether this exercise is basic (obligatory), continuative (recommended) or optional (free). The colour coding is summarized in the table below:

Color Code	Explanation
	Basic exercise, which is obligatory for the PAT Tutorial.
	Continuative exercise, which is recommended for the PAT Tutorial to deepen what has been learned.
	Optional exercise, which shows interesting applications of what has been learned.

Setting up of the environment

First of all connect to lxplus and go to some work directory. You can choose any directory, provided that you have enough space. You need ~500 MB of free disc space for this exercise. We recommend you to use your ~/scratch0 space. In case you don't have this (or do not even know what it is) check your quota typing fs lq and follow this link or change your quota here (click manage near "Linux and AFS" -> Increase quota to 2 GB). If you don't have enough space, you may instead use the temporary space ( /tmp/your_user_name), but be aware that this is lost once you log out of lxplus (or within something like a day). We will expect in the following that you have such a ~/scratch0 directory.

ssh your_lxplus_Name@lxplus6.cern.ch 
[ ... enter password ... ]
cd scratch0/

Create a directory for this exercise (to avoid interference with code from the other exercises).

mkdir exercise02
cd exercise02

Create a local CMSSW release area and enter it.

cmsrel CMSSW_7_4_1_patch4
cd CMSSW_7_4_1_patch4/src

The first command creates all directories needed in a local CMSSW release area. Setting up the CMSSW environment is done by invoking the following script:

cmsenv

Having a look at the PAT configuration file

To check out the recommended modules and configuration files we want to use do the following:

git cms-addpkg PhysicsTools/PatAlgos
git cms-addpkg FWCore/GuiBrowsers
git cms-merge-topic -u CMS-PAT-Tutorial:CMSSW_7_4_1_patTutorial
scram b -j 4

This will checkout and compile the standard PAT, its workflow, and the necessary classes. The standard PAT sequence is run on standard AOD/RECO input files, specified in PhysicsTools/PatAlgos/test/patTuple_standard_cfg.py. Have a look at this file with your favourite editor, for example xemacs:

xemacs PhysicsTools/PatAlgos/test/patTuple_standard_cfg.py

You will see that this configuration file does not contain too much code:

## import skeleton process from PhysicsTools.PatAlgos.patTemplate_cfg import *

## let it run process.p = cms.Path( process.patDefaultSequence )

## from PhysicsTools.PatAlgos.patInputFiles_cff import filesRelValProdTTbarAODSIM process.source.fileNames = filesRelValProdTTbarAODSIM # process.maxEvents.input = 100

## output file process.out.fileName = 'patTuple_standard.root'

We left out the commented regions on the screen. This is due to the fact that most information you might have expected to see from a typical cmsRun cfg file (like the cms.OutputModule to write output to an edm output file) are imported from another file: the patTemplate_cfg.py. We import it in the very first line. You may want to have a closer look into the patTemplate_cfg.py file (which is indeed imported for all example files in the test directory of the PatAlgos package). The patTemplate_cfg.py file provides defaults so that any python script (in the test directory) in which it is included runs fine. But of course you can override these defaults by parameter replacements at any time. We will do this later during this exercise. It is indeed the recommended way to change parameters by replacements instead of editing the files the parameters were defined in. The reason is obvious: editing the source file would change the parameter for any further cfg file which imports the source file - no chance to control what you did.

Try the following:

cmsRun PhysicsTools/PatAlgos/test/patTuple_standard_cfg.py

In case cmsRun gives a message that the file patTemplate_cfg.py does not exist check the following: Did you setup your environment properly? Did you forget to type cmsenv? You can test this checking the variable environment variable CMSSW_RELEASE_BASE ( echo $CMSSW_RELEASE_BASE) and see whether it points into the release you are expecting to work with (which is /afs/cern.ch/cms/slc5_amd64_gcc462/cms/cmssw/CMSSW_7_4_1_patch4 in our case).

To have a look into the patTemplate_cfg.py file open it with your favourite editor:

xemacs PhysicsTools/PatAlgos/python/patTemplate_cfg.py &

We now come to the content of the patTemplate_cfg.py file. At the top level the process is defined:

import FWCore.ParameterSet.Config as cms

process = cms.Process("PAT")

Followed by the configuration of the SWGuideMessageLogger and the output report.

## MessageLogger process.load("FWCore.MessageLogger.MessageLogger_cfi")

## Options and Output Report process.options = cms.untracked.PSet( wantSummary = cms.untracked.bool(True) )

This will just print a timing and trigger summary at the end of the job.

For this example we will only loop over 100 events from this test input file. The tool pickRelValInputFiles will just pick up an up to date RelVal file and return a string corresponding to the I/o protocol and the logical filename (LFN). You could replace it for the code fragment to look like this:

## Source

process.source = cms.Source("PoolSource", fileNames = cms.untracked.vstring( '/store/relval/CMSSW_7_4_0_pre8/RelValProdTTbar_13/AODSIM/MCRUN2_74_V7-v1/00000/44E1E4BA-50BD-E411-A57A-002618943949.root' ) )

## Maximal Number of Events

process.maxEvents = cms.untracked.PSet( input = cms.untracked.int32(100) )

For more information on how to configure the input source, have a look at the relevant section of SWGuidePoolInputSources. If you wish to only change the input files, you can start from the top level configuration (patTuple_standard_cfg.py),by adding the lines:

process.source.fileNames = ['/store/relval/CMSSW_7_4_0_pre8/RelValProdTTbar_13/AODSIM/MCRUN2_74_V7-v1/00000/44E1E4BA-50BD-E411-A57A-002618943949.root']<span data-mce-mark="1"><span data-mce-mark="1"><span data-mce-mark="1">

and replace the example file by the file(s) of your choice. Note that the RECO or AOD content may have changed from release to release. So there is no guarantee that you will still be able to use a input file which has been produced with an older release that the one you are using.

In the following section of the patTemplate_cfg.py configuration file additional information is loaded to the process, which is needed by some components during the pat::Candidate production steps as explained in WorkBookPATWorkflow:

## Geometry and Detector Conditions (needed for a few patTuple production steps) process.load("Configuration.Geometry.GeometryIdeal_cff") process.load("Configuration.StandardSequences.FrontierConditions_GlobalTag_cff") from [[Configuration.AlCa.GlobalTag]] import GlobalTag process.GlobalTag = GlobalTag (process.GlobalTag, 'auto:startup') process.load("Configuration.StandardSequences.MagneticField_cff")

Note: In this section the geometry information, calibration and alignment information (note the global tag) and magnetic field information is loaded into the process, which is needed for the calculation of Jet/MET corrections for instance. For the simulation, the global tag should always correspond to the global tag used for the reconstruction of these events. This can be inferred from the dataset name, or by looking at the "Configuration files" linked in DBS. For data, the latest set of calibration constants and frontier conditions can be found at SWGuideFrontierConditions, which is maintained by the calibration and alignment group.

Typically the line that defines the global tag to be used would look like this:

process.GlobalTag.globaltag = cms.string('auto:run2_mc')

We are using the so called autoCond struct to do this automatically for us here. In the following part of the configuration file the sequence for the production of pat::Candidates is loaded and made known to the process:

## Standard PAT Configuration File process.load("PhysicsTools.PatAlgos.patSequences_cff")

The last part of the configuration file is concerned with the configuration of the output file:

process.out = cms.OutputModule("PoolOutputModule", fileName = cms.untracked.string('patTuple.root'),

## save only events passing the full path

#SelectEvents = cms.untracked.PSet( SelectEvents = cms.vstring('p') ),

## save PAT output; you need a '*' to unpack the list of commands

##'patEventContent'

outputCommands = cms.untracked.vstring('drop *', *patEventContentNoCleaning ) )

process.outpath = cms.EndPath(process.out)

We want to save the produced layer of pat::Candidates in the output file. We will only save those events, which passed the whole path, and by default we drop everything else. Finally, we add the output module to the end path of our process.

Having a deeper look at the PAT configuration

The edmConfigEditor. will allow allow you a detailed and interactive look into the Configuration. You can start it and load your config file like this:

edmConfigEditor PhysicsTools/PatAlgos/test/patTuple_standard_cfg.py

You can take a look at the modules scheduled to run, by choosing paths and then scheduled in the left tab.

Show result... Hide result...

You can choose the single modules, by clicking on them in the middle tab. Choose selectedPatElectrons.

Show result... Hide result...

This is one of the last steps of pat::Candidate production. The final acceptance cuts. It is fed by the result of patElectrons:

Show result... Hide result...

Have a look at WorkBookPATConfiguration to learn more how to configure pat::Candidates during their production. You can also have a look at the WorkBookPATDocNavigationExercise.

Question 2 a):
The selectedPatElectrons has the patElectrons collection as input. How would you verify this? As the parameter embedGenMatch for the creation of pat::Electrons in the patElectrons collection module is switched to True, will the generator match also be embedded in the pat::Electrons of the selectedPatElectrons collection?

Note: you can exit the interactive python mode hitting CTRL-d. For now, let us just make a small modification to the acceptance cuts above. We'll restrict ourselves to electrons with pt>10 GeV/c. Open the file patTuple_standard_cfg.py in your favourite editor:

[edit] PhysicsTools/PatAlgos/test/patTuple_standard_cfg.py
[... go to the end ...]

and add the following line at the end:

process.selectedPatElectrons.cut = 'pt > 10. && abs(eta) < 12.' 

Note: The syntax is not exactly the same as in cut = cms.string('') above, where the type is specified with cms.string(...). That's because we are replacing an existing attribute of selectedPatElectrons, so python already knows the type.

Question 2 b):
As you applied a selection on the transverse momentum of the electrons in the selectedPatElectrons collection. Will it also be applied for the electrons in the cleanPatElectrons collection?

Running pat::Candidate production on top of RECO/AOD

To start the production of pat::Candidates do the following:

cmsRun PhysicsTools/PatAlgos/test/patTuple_standard_cfg.py >& output.log

Note: In this examples the log output is redirected to a file output.log. You may want to have a look into this file after the processing is done. Open it in your favourite text editor, you will see that a message is set out for the input file opening and for each event. To learn how to configure this log messages further have a look at SWGuideMessageLogger:

globaltag = PRE_STA71_V4::All
==> using COND/Services/RelationalAuthenticationService for auth, sys 1
==> using COND/Services/RelationalAuthenticationService for auth, sys 1
12-Jun-2014 15:32:18 CEST  Initiating request to open file root://eoscms//eos/cms/store/relval/CMSSW_7_1_0_pre4_AK4/RelValProdTTbar/AODSIM/START71_V1-v2/00000/7A3637AA-28B5-E311-BC25-003048678B94.root?svcClass=default
12-Jun-2014 15:32:20 CEST  Successfully opened file root://eoscms//eos/cms/store/relval/CMSSW_7_1_0_pre4_AK4/RelValProdTTbar/AODSIM/START71_V1-v2/00000/7A3637AA-28B5-E311-BC25-003048678B94.root?svcClass=default
Begin processing the 1st record. Run 1, Event 1, LumiSection 1 at 12-Jun-2014 15:32:29.202 CEST
Begin processing the 2nd record. Run 1, Event 2, LumiSection 1 at 12-Jun-2014 15:32:54.961 CEST
Begin processing the 3rd record. Run 1, Event 3, LumiSection 1 at 12-Jun-2014 15:32:55.008 CEST
Begin processing the 4th record. Run 1, Event 4, LumiSection 1 at 12-Jun-2014 15:32:55.038 CEST
Begin processing the 5th record. Run 1, Event 5, LumiSection 1 at 12-Jun-2014 15:32:55.109 CEST
[...]

Jump to the end of the processing, where you can find the timing and trigger report summary:

TrigReport ---------- Event  Summary ------------
TrigReport Events total = 100 passed = 100 failed = 0

TrigReport ---------- Path   Summary ------------
TrigReport  Trig Bit#        Run     Passed     Failed      Error Name

TrigReport -------End-Path   Summary ------------
TrigReport  Trig Bit#        Run     Passed     Failed      Error Name
TrigReport     0    0        100        100          0          0 outpath

TrigReport ------ Modules in End-Path: outpath ------------
TrigReport  Trig Bit#    Visited     Passed     Failed      Error Name
TrigReport     0    0        100        100          0          0 out

TrigReport ---------- Module Summary ------------
TrigReport    Visited        Run     Passed     Failed      Error Name
TrigReport        100        100        100          0          0 out
TrigReport          0          0          0          0          0 ak4PFJetsPtrs
TrigReport          0          0          0          0          0 caloJetMETcorr
TrigReport          0          0          0          0          0 caloType1CorrectedMet
TrigReport          0          0          0          0          0 caloType1p2CorrectedMet
TrigReport        100        100        100          0          0 elPFIsoDepositCharged
TrigReport        100        100        100          0          0 elPFIsoDepositChargedAll
TrigReport        100        100        100          0          0 elPFIsoDepositGamma
TrigReport        100        100        100          0          0 elPFIsoDepositNeutral
TrigReport        100        100        100          0          0 elPFIsoDepositPU
TrigReport          0          0          0          0          0 elPFIsoValueCharged03NoPFId
TrigReport          0          0          0          0          0 elPFIsoValueCharged03PFId
TrigReport        100        100        100          0          0 elPFIsoValueCharged04NoPFId
TrigReport        100        100        100          0          0 elPFIsoValueCharged04PFId
TrigReport          0          0          0          0          0 elPFIsoValueChargedAll03NoPFId
TrigReport          0          0          0          0          0 elPFIsoValueChargedAll03PFId
TrigReport        100        100        100          0          0 elPFIsoValueChargedAll04NoPFId
TrigReport        100        100        100          0          0 elPFIsoValueChargedAll04PFId
TrigReport          0          0          0          0          0 elPFIsoValueGamma03NoPFId
TrigReport          0          0          0          0          0 elPFIsoValueGamma03PFId
TrigReport        100        100        100          0          0 elPFIsoValueGamma04NoPFId
TrigReport        100        100        100          0          0 elPFIsoValueGamma04PFId
[...]

In our case, it doesn't convey much information because none of the modules actually rejects events. Nevertheless it is confirmed that 100 events were processed.

Note: In the context of analyses the name trigger report does not have anything to do with the L1 Trigger or HLT for physics events. It just tells you which modules were visited and whether events passed or failed when visiting the module. The fact that some electrons are rejected due to the object selection cuts that we defined above is not reflected here since in this step only objects (i.e. electrons) as part of an object collection are rejected and not whole events. An object count filter as also defined in the PatAlgos/python/selectionLayer1 directory would have had an effect on whole events and hence be reflected in the summary report.

Finally, a timing report for each module is printed at the end of the job:

TimeReport ---------- Event  Summary ---[sec]----
TimeReport CPU/event = 0.000000 Real/event = 0.000000

TimeReport ---------- Path   Summary ---[sec]----
TimeReport             per event          per path-run 
TimeReport        CPU       Real        CPU       Real Name
TimeReport        CPU       Real        CPU       Real Name
TimeReport             per event          per path-run 

TimeReport -------End-Path   Summary ---[sec]----
TimeReport             per event       per endpath-run 
TimeReport        CPU       Real        CPU       Real Name
TimeReport   0.296235   0.304723   0.296235   0.304723 outpath
TimeReport        CPU       Real        CPU       Real Name
TimeReport             per event       per endpath-run 
TimeReport        CPU       Real        CPU       Real Name
TimeReport             per event      per module-visit 

TimeReport ------ Modules in End-Path: outpath ---[sec]----
TimeReport             per event      per module-visit 
TimeReport        CPU       Real        CPU       Real Name
TimeReport   0.296235   0.000000   0.296235   0.000000 out
TimeReport        CPU       Real        CPU       Real Name
TimeReport             per event      per module-visit 

TimeReport ---------- Module Summary ---[sec]----
TimeReport             per event        per module-run      per module-visit 
TimeReport        CPU       Real        CPU       Real        CPU       Real Name
TimeReport   0.296235   0.304719   0.296235   0.304719   0.296235   0.304719 out
TimeReport   0.000000   0.000000   0.000000   0.000000   0.000000   0.000000 ak4PFJetsPtrs
TimeReport   0.000000   0.000000   0.000000   0.000000   0.000000   0.000000 caloJetMETcorr
TimeReport   0.000000   0.000000   0.000000   0.000000   0.000000   0.000000 caloType1CorrectedMet
TimeReport   0.000000   0.000000   0.000000   0.000000   0.000000   0.000000 caloType1p2CorrectedMet
TimeReport   0.000790   0.000765   0.000790   0.000765   0.000790   0.000765 elPFIsoDepositCharged
TimeReport   0.000380   0.000325   0.000380   0.000325   0.000380   0.000325 elPFIsoDepositChargedAll
TimeReport   0.000470   0.000473   0.000470   0.000473   0.000470   0.000473 elPFIsoDepositGamma
TimeReport   0.000050   0.000095   0.000050   0.000095   0.000050   0.000095 elPFIsoDepositNeutral
TimeReport   0.000050   0.000051   0.000050   0.000051   0.000050   0.000051 elPFIsoDepositPU
TimeReport   0.000000   0.000000   0.000000   0.000000   0.000000   0.000000 elPFIsoValueCharged03NoPFId
TimeReport   0.000000   0.000000   0.000000   0.000000   0.000000   0.000000 elPFIsoValueCharged03PFId
TimeReport   0.000000   0.000022   0.000000   0.000022   0.000000   0.000022 elPFIsoValueCharged04NoPFId
TimeReport   0.000260   0.000266   0.000260   0.000266   0.000260   0.000266 elPFIsoValueCharged04PFId
TimeReport   0.000000   0.000000   0.000000   0.000000   0.000000   0.000000 elPFIsoValueChargedAll03NoPFId
TimeReport   0.000000   0.000000   0.000000   0.000000   0.000000   0.000000 elPFIsoValueChargedAll03PFId
TimeReport   0.000040   0.000026   0.000040   0.000026   0.000040   0.000026 elPFIsoValueChargedAll04NoPFId
TimeReport   0.000010   0.000027   0.000010   0.000027   0.000010   0.000027 elPFIsoValueChargedAll04PFId
TimeReport   0.000000   0.000000   0.000000   0.000000   0.000000   0.000000 elPFIsoValueGamma03NoPFId
[...]

Question 2 c):
Can you tell from the timing report of your output what the most time consuming part is during the PAT tuple production?

Inspecting the output of pat::Candidate production

You might want to have a quick look at the event content of the original AOD/RECO input file before inspecting the event content of the newly created PAT tuple. You can do this by invoking the edm tool edmDumpEventContent or by using the TBrowser in an interactive ROOT session. Note that the AOD/RECO file is located on castor. A way to access it is given below:

Note: Due to the large file size it might take a short while to open the file.

Note: If you would like to view the event content of the copied AOD and later of the PAT tuple with the root Browser make sure that you have loaded the FWLite libraries in advance, which will allow us to easily plot physical quantities in ROOT. Have a look at SWGuideFWLite to learn more about the meaning of FWLite.

root -l root://eoscms//eos/cms/store/relval/CMSSW_7_1_0_pre4_AK4/RelValProdTTbar_13/AODSIM/POSTLS171_V1-v2/00000/0AD19371-15B6-E311-9DB6-0025905A6110.root
gSystem->Load("libFWCoreFWLite.so");
AutoLibraryLoader::enable();
TBrowser b

You might have added these lines to your .rootrc or rootLogon.C file to have these libraries loaded automatically when starting root.

Then do the following:

• double-click on ROOT files on the left sidebar;
• double-click on the ROOT file name ( root://eoscms//eos/cms/store/relval/CMSSW_5_3_6/RelValTTbar/GEN-SIM-RECO/PU_START53_V14-v1/0003/3E3EDF4A-E92C-E211-A1BF-003048D2BD66.root) in the in the left sidebar;
• double-click on the Events;1 directory in the left side bar.

Show image... Hide image...

Those collections ending with _HLT have been produced (or processed) by the High Level Trigger, which performs some fast reconstruction; those ending with _RECO have been produced by standard reconstruction algorithms.

You can now exit root and go back to your working directory to inspect the PAT tuple that you have created before. Open root again in your working directory and proceed as above:

• double-click on ROOT files on the left sidebar;
• double-click on the ROOT file name in the main window (on the right side);
• double-click on the Events;1 directory in the main window.

Show image... Hide image...

Those collections ending with _PAT are the pat::Candidate collections created by the process above. Other collections (if any) are copied over from the input file. Have a look at SWGuidePATEventSize to learn more about event size management and how to configure the event content with PAT.

To do a raw plot of the transverse momentum distribution of the pat::Electrons do the following:

• double-click on the directory patElectrons_selectedPatElectrons__PAT (this opens the PAT electron collection)
• double-click on patElectrons_selectedPatElectrons__PAT.obj
• scroll down to pt() and double-click on it (look at the image below to locate it)

Show image... Hide image...

If everything went smoothly, you should obtain the electron pt spectrum, with a cut at 10 GeV, as expected. You can exit root typing .q in the prompt.

Exercises

Now that you have run your first PAT job and before leaving this page try to do the following exercises:

Exercise 2 a):
How can you increase the number of events from 100 to 1000? (You might also want to change the default dataset to your favourite dataset. But note that you will get slightly different results then).

You will find the solution here:

Show Hide

process.maxEvents.input = 1000

Exercise 2 b): Add a clone of the selectedPatJets collection (the default pat::Jet collection without object disambiguation) to the event content, with the collection label goodPatJets. You can achieve this by cloning the selectedPatJet module in your configuration file. Have a look to WorkBookConfigFileIntro if you don't know how to clone a module. Change the selection string for your goodPatJets collection to "pt>30 & abs(eta)<3". What is the mean jet multiplicity in your event sample for this new jet collection? Note: Make sure that your new collection will be written to the event output. You can use the feature of cloning modules together with the string selection feature to apply object and event selections in your analysis in a very elegant and human readable way.

You will find the solution here:

Show Hide

## jet selector
from PhysicsTools.PatAlgos.selectionLayer1.jetSelector_cfi import *
<!--verbatim14-->

## make sure to write it to the event content of your pat tuple, this is needed to run the process (unscheduled mode)
from PhysicsTools.PatAlgos.patEventContent_cff import *
process.out.outputCommands += ["keep *_goodPatJets*_*_*"]

Exercise 2 c): add the ak4PFJets collection to your event content (have a look to patTuple_addJets_cfg.py to find out how to do that) and inspect the event content of the produced file with edmDumpEventContent (type edmDumpEventContent --help in your shell to find out how to use it).
Can you see what has changed in the event content with respect to the standard output you saw before? (Copy and paste the additional line into the form.) Note: You may observe more than one additional collection due to what is called embedding in PAT. You will learn about this in Exercise 3 .

You will find the solution here:

Show Hide

## uncomment the following line to add different jet collections
## to the event content
from PhysicsTools.PatAlgos.tools.jetTools import addJetCollection
labelAK4PF = 'AK4PF'
addJetCollection(
   process,
   labelName = labelAK4PF,
   jetSource = cms.InputTag('ak4PFJets'),
   jetCorrections = ('AK5PF', cms.vstring(['L1FastJet', 'L2Relative', 'L3Absolute']), 'Type-1'),
   )
process.out.outputCommands.append( 'drop *_selectedPatJets%s_caloTowers_*'%( labelAK4PF ) )

Exercise 2 d): In the PAT section of the SWGuide you find a section that explains how to add customized, user defined data to a pat::Candidate. Have a look to SWGuidePATUserData to learn more about this feature. Follow the instruction given there to add the relative isolation variable ( relIso) to the pat::Muon in your pat::Tuple.

Note:

In case of problems don't hesitate to contact the SWGuidePAT#Support. Having successfully finished Exercise 2 you might want to proceed to the other exercises of the WorkBookPATTutorial to learn more about PAT.

Review status

Show Hide