This page describes how to skim/slim NTUP (D3PD) in athena with a view to run jobs in the central production system of ATLAS.

See also this tutorial :SoftwareTutorialAnalyzingD3PDsInAthena for the details of reading/writing D3PDs in athena.

Athena setup

Please follow this instruction.

In this page, we use release 17.2.1. Make test area like:

mkdir -p $HOME/testarea/17.2.1
cd $HOME/testarea/17.2.1
export AtlasSetup=/afs/cern.ch/atlas/software/dist/AtlasSetup
alias asetup='source $AtlasSetup/scripts/asetup.sh'
asetup 17.2.1, here

In addition, check out PyUtils-00-12-06 for scripts to make D3PDObjects below.

cmt co -r PyUtils-00-12-06 Tools/PyUtils 
cd Tools/PyUtils/cmt

Checking out main package and running examples

Checking out main pacakge: NTUPtoNTUPExample

cd $TestArea
cmt co -r NTUPtoNTUPExample-00-00-03 -o svn+ssh://svn.cern.ch/reps/atlasusr/mkaneda/atlasoff/ PhysicsAnalysis/NTUPtoNTUP/NTUPtoNTUPExample

Running example 1: using jobOptions file

mkdir -p $TestArea/run/test1
cd $TestArea/run/test1
get_files NTUPtoMyNTUP.py
athena NTUPtoMyNTUP.py >log 2>&1

Check: myNtup.root is new file

Running example 2: using job transform script

mkdir -p $TestArea/run/test2
cd $TestArea/run/test2
NTUPtoNTUP_trf.py maxEvents=-1 inputNTUP_SMWZFile=root://castoratlas//castor/cern.ch/grid/atlas/atlt3/scratch/mkaneda/data/valid1.106046.PythiaZee_no_filter.merge.NTUP_SMWZ.e815_s1272_s1274_r3397_p948_tid752536_00/NTUP_SMWZ.752536._000068.root.1   outputNTUP_MYNTUPFile=myNtup.root outputNTUP_MYNTUP2File=myNtup2.root >log 2>&1

Check: There should be two files: myNtup.root and myNtup2.root

In both cases, "NTUPtoNTUPExample/MyNTUP_prodJobOFragment.py" is the key file to setup output file (myNtup.root).

Adding new output definition

Add your output definition at the bottom of NTUPtoNTUPExample/python/NTUPtoNTUPProdFlags.py:

class WriteMyTestNTUP (JobProperty):
"""test NTUP""" 
    statusOn = True
    allowedTypes = ['bool']
    StoredValue = False
    StreamName = 'StreamNTUP_MYTESTNTUP'
    FileName = '' NTUPScript = "MyTestNTUP/MyTestNTUP_prodJobOFragment.py"
    TreeNames = ['physics']
    SubSteps = ['n2n']
prodFlags.add_JobProperty (WriteMyTestNTUP)
listAllKnownNTUPtoNTUP.append (prodFlags.WriteMyTestNTUP) 

  • MyTestNTUP/MyTestNTUP_prodJobOFragment.py: package name and job fragment files which will be made in the next section
  • TreeNames: tree name of ntuple
  • StreamName: Stream name. It is also used to define the argument name for NTUPtoNTUP_trf.py. argument name is "outputNTUP_MYTESTNTUPFile" in this case.

Preparing new package and job fragment file

Make new package to construct your new ntuple:

cd $TestArea
acmd.py cmt new-pkg Tutorial/MyTestNTUP

Make jobFragment file: Tutorial/MyTestNTUP/share/MyNTUPTest_prodJobFragment.py

cd Tutorial/MyTestNTUP/share/

  • MyNTUPTest_prodJobFragment.py (simple example to copy a few branches from original ntuple)
## This jobO should not be included more than once:
include.block( "MyNTUPTest/MyNTUPTest_prodJobFragment.py" )
## Common import(s):
from AthenaCommon.AlgSequence import AlgSequence
topSequence = AlgSequence()
from AthenaCommon.JobProperties import jobproperties
prodFlags = jobproperties.NTUPtoNTUPProdFlags
from PrimaryDPDMaker.PrimaryDPDHelpers import buildFileName


## Set up a logger:
from AthenaCommon.Logging import logging
MyNTUPTestStream_msg = logging.getLogger( 'MyNTUPTest_prodJobOFragment' )

## Construct the stream and file names for the SUSY NTUP:
streamName = mytestntup.StreamName
fileName   = buildFileName( mytestntup )
MyNTUPTestStream_msg.info( "Configuring MyNTUPTest with streamName '%s' and fileName '%s'" % \
                            ( streamName, fileName ) )

## set input tree:
from AthenaCommon.JobProperties import jobproperties

## Create the NTUP streams:
from NTUPtoNTUPExample.MultipleNTUPStreamManager import MNSMgr
MyNTUPTestStream = MNSMgr.NewNTUPStream( streamName, fileName, tree_name)
    "el_n", # add el_n
    "el_pt", # add el_pt
    "el_Etcone*", # add any branches matching el_Etcone*
    "-el_Etcone15", # remove el_Etcone15

The branches in new ntuple is defined by the line "MyNTUPTestStream.AddItem([...])".

Compile the package:

cd ../cmt/
sed -i "s/apply_pattern component_library/#apply_pattern component_library/g" requirements # comment out library requirement for the moment

And test new output:

mkdir -p $TestArea/run/test3
cd $TestArea/run/test3
NTUPtoNTUP_trf.py maxEvents=-1 inputNTUP_SMWZFile=root://castoratlas//castor/cern.ch/grid/atlas/atlt3/scratch/mkaneda/data/valid1.106046.PythiaZee_no_filter.merge.NTUP_SMWZ.e815_s1272_s1274_r3397_p948_tid752536_00/NTUP_SMWZ.752536._000068.root.1   outputNTUP_MYTTESTNTUPFile=myTestNtup.root >log 2>&1

Check output file:

 acmd.py dump-root filtered.myTestNtup.root -t physics

Skimming/Adding new branches

This section shows example of Z->ee selection.

  • Select only electrons pt > 20GeV
  • Select events with more than one electrons and 60GeV<m_Z(ee)<120GeV

Making new algorithm to skim events and add new branches

cd $TestArea/Tutorial/MyTestNTUP/src
#Make skeleton files (MyTestNTUPAlg.h/cxx)
acmd.py gen-klass  --klass MyTestZeeAlg --pkg MyTestNTUP --type alg -o MyTestZeeAlg
#make load file
mkdir -p components
cat >| components/MyTestNTUP_load.cxx << EOF
#include "GaudiKernel/LoadFactoryEntries.h"
#make entries file
cat >| components/MyTestNTUP_entries.cxx << EOF
#include "GaudiKernel/DeclareFactoryEntries.h"
#include "../MyTestZeeAlg.h"

Preparing D3PDObject

Prepare D3PDObject code from D3PD:

cd $TestArea
mkdir -p run/d3pd
cd run/d3pd
atl-gen-athena-d3pd-reader root://castoratlas//castor/cern.ch/grid/atlas/atlt3/scratch/mkaneda/data/valid1.105200.T1_McAtNlo_Jimmy.merge.NTUP_SMWZ.e835_s1310_s1300_r3397_p948_tid752537_00/NTUP_SMWZ.752537._000074.root.1


There will be codes: code/*D3PDObject.cxx/h.

Copy ElectronD3PDObject to MyTestNTUP package

cd $TestArea/Tutorial/MyTestNTUP/src
cp $TestArea/run/d3pd/code/ElectronD3PDObject* .

Writing Z->ee algorithm

cd $TestArea/Tutorial/MyTestNTUP/src

Edit MyTestZeeAlg.* like: MyTestZeeAlg.cxx, MyTestZeeAlg.h

  • RVar/WVar is used to read/write single variable from/to StoreGate
  • el_* variables are retrieved as ElectronD3PDObject: const ElectronD3PDObject el("el_"), and can be used like: el.pt(i)
  • New electrons are stored as "myEl_", by using ElectronD3PDObject
  • In the function execute(), setFilterPassed function is used to decide whether the event should be passed or not.
Go to share directory and edit MyTestNTUP _prodJobOFragment.py.
cd $TestArea/Tutorial/MyTestNTUP/share

Add "myEl_*" and "my_mZ" to AddItem list and add filter algorithm:

    "el_n", # add el_n
    "el_pt", # add el_pt
    "el_Etcone*", # add any branches matching el_Etcone*
    "-el_Etcone15", # remove el_Etcone15
    "myEl_*", # new electron
    "my_mZ", # Z mass
## Algorithm for filter and new variables
from MyTestNTUP.MyTestNTUPConf import MyTestZeeAlg
                            MinNumberPassed      = 2,
                            PtMin                = 20.*GeV,
                            MZMin                = 60.*GeV,
                            MZMax                = 160.*GeV,

Go to cmt directory, and udpate requirements file: requirements, and make.

cd $TestArea/Tutorial/MyTestNTUP/cmt

And test new output:

mkdir -p $TestArea/run/test4
cd $TestArea/run/test4
NTUPtoNTUP_trf.py maxEvents=-1 inputNTUP_SMWZFile=%inputNTUP_SMWZFile%  outputNTUP_MYTTESTNTUPFile=myTestNtup.root >log 2>&1

In the log file, you can find a result of the filter

MyTestZeeAlg         INFO Finalizing MyTestZeeAlg...
MyTestZeeAlg         INFO ******Summary******
MyTestZeeAlg         INFO Number of processed events:          10000
MyTestZeeAlg         INFO Number of    passed events:          7287
MyTestZeeAlg         INFO Average number of electrons (all) :  4.5943
MyTestZeeAlg         INFO Average number of electrons (rem):  2.04556
MyTestZeeAlg         INFO *******************

And check the contents if it has correctly values "myEl_" and "my_mZ"

Another method of D3PD->D3PD

NTUPtoNTUPExample has another method to skim/slim D3PD by using simple script.

Run example:

mkdir -p $TestArea/run/test5
cd $TestArea/run/test5
SkimNTUP_trf.py maxEvents=-1 inputNTUP_SMWZFile=root://castoratlas//castor/cern.ch/grid/atlas/atlt3/scratch/mkaneda/data/valid1.106046.PythiaZee_no_filter.merge.NTUP_SMWZ.e815_s1272_s1274_r3397_p948_tid752536_00/NTUP_SMWZ.752536._000068.root.1   outputNTUP_MYSKIMNTUPFile=myTestNtup.root >log 2>&1

The script file for skimming/slimming is PhysicsAnalysis/NTUPtoNTUP/NTUPtoNTUPExample/python/slim.py, which has PyROOT function, doSkim().

JobOFragment file is PhysicsAnalysis/NTUPtoNTUP/NTUPtoNTUPExample/python/MySkimNTUP_prodJobOFragment.py, which calls doSkim().

To add new type, add new definition to PhysicsAnalysis/NTUPtoNTUP/NTUPtoNTUPExample/python/SkimNTUP_ProdFlags.py as described above for NTUPtoNTUP.

Major updates:
-- MichiruKaneda - 05-Apr-2012

%RESPONSIBLE% MichiruKaneda
%REVIEW% Never reviewed

Edit | Attach | Watch | Print version | History: r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r1 - 2012-07-11 - MichiruKaneda
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Main All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback