EPAMTutorial < Main

Main Web>TWikiUsers>MohamedGouighri>EPAMTutorial (2020-06-18, MaximKonyushikhin) (raw view)
%CERTIFY%
---+!! <nop>%TOPIC%

%TOC% <!--optional--> 
%STARTINCLUDE%

<noautolink>

---+ Scope of the Tutorial:

The goal of this tutorial is to make sure that by the end of the school, the participants will have a general idea about the data ta<script type="text/javascript" src="https://twiki.cern.ch/twiki/pub/TWiki/TinyMCEPlugin/tinymce/jscripts/tiny_mce/themes/advanced/langs/en.js"></script><script type="text/javascript" src="https://twiki.cern.ch/twiki/pub/TWiki/TinyMCEPlugin/tinymce/jscripts/tiny_mce/plugins/twikibuttons/langs/en.js"></script><script type="text/javascript" src="https://twiki.cern.ch/twiki/pub/TWiki/TinyMCEPlugin/tinymce/jscripts/tiny_mce/plugins/twikiimage/langs/en.js"></script><script type="text/javascript" src="https://twiki.cern.ch/twiki/pub/TWiki/TinyMCEPlugin/tinymce/jscripts/tiny_mce/themes/advanced/langs/en.js"></script><script type="text/javascript" src="https://twiki.cern.ch/twiki/pub/TWiki/TinyMCEPlugin/tinymce/jscripts/tiny_mce/plugins/twikibuttons/langs/en.js"></script><script type="text/javascript" src="https://twiki.cern.ch/twiki/pub/TWiki/TinyMCEPlugin/tinymce/jscripts/tiny_mce/plugins/twikiimage/langs/en.js"></script><script type="text/javascript" src="https://twiki.cern.ch/twiki/pub/TWiki/TinyMCEPlugin/tinymce/jscripts/tiny_mce/themes/advanced/langs/en.js"></script><script type="text/javascript" src="https://twiki.cern.ch/twiki/pub/TWiki/TinyMCEPlugin/tinymce/jscripts/tiny_mce/plugins/twikibuttons/langs/en.js"></script><script type="text/javascript" src="https://twiki.cern.ch/twiki/pub/TWiki/TinyMCEPlugin/tinymce/jscripts/tiny_mce/plugins/twikiimage/langs/en.js"></script>king at LHC, the whole steps to follow to have at the end the souhitable data for the analysis and be able to run ATLAS analy<script type="text/javascript" src="https://twiki.cern.ch/twiki/pub/TWiki/TinyMCEPlugin/tinymce/jscripts/tiny_mce/themes/advanced/langs/en.js"></script><script type="text/javascript" src="https://twiki.cern.ch/twiki/pub/TWiki/TinyMCEPlugin/tinymce/jscripts/tiny_mce/plugins/twikibuttons/langs/en.js"></script><script type="text/javascript" src="https://twiki.cern.ch/twiki/pub/TWiki/TinyMCEPlugin/tinymce/jscripts/tiny_mce/plugins/twikiimage/langs/en.js"></script>sis on their own. There are two important requirements that need to be satisfied before the start of the tutorial, namely:
   * Have an account on Lxplus.
   * Have a valid grid certificate to be able to run analysis using the GRID (second session given tomorrow by David, Eric, Farida et Ghita).

We will cover the following main areas:
   * Setup the directory structure.
   * Setup a particular version of the ATLAS software releases.
   * Understand various physics analysis objects (how to access more information about each object).
   * How to select good Good Runs Lists, based on selecting only good LBs
   * Calculate the Luminosity 
   * Use the physics objects in an AOD analysis.
   * Create a new package (algorithm) 
   * Use the GRID (PAnda, Ganga) to analyze data, etc... (in second session)

---+ Session 1:
---++ Setup the lxplus account s

1. For the practical session it is needed to have an account on LXPLUS, and have about 100MB of space quota. You can check this by typing =fs lq=; the result is given in kilobytes. For people who do not have an lxplus account they can use my secondary account for this tutorial, to log on use:
<verbatim>
Loging: mgouigh 
password: 
</verbatim>

Open a terminal and log in to the LXPLUS by: =ssh -l mgouigh@lxplus.cern.ch -X -Y=

It's recommended to use bash or zsh as your shell. To check this type
<verbatim>
ps -p $$ 
</verbatim>

To switch type
<verbatim>
bash -l 
</verbatim>

The main thing is that you shouldn't use tcsh. You could change your shell on LXPLUS using the self-service tool available on a Web Portal
<verbatim>
http://www.cern.ch/account
</verbatim>

Click on the link ="Applications and Resources"= and from there, Click on =[Manage]= corresponding to the ="Linux and AFS"= line within ="Common Services"=.

Under the ="Linux settings"= section, you can select your Unix Shell between: =/bin/tcsh, /bin/bash, /bin/csh, /bin/ksh, /bin/sh and /bin/zsh=.

Press on the ="Save Selection"= blue button to activate your change. Please wait for about an hour to make sure the change is propagated to the Linux environment.

---+ Setting up your account for ATLAS software

There are currently two methods for setting up your lxplus account to enable running ATLAS software. The old method required setting up CMT (described below), which is now superseded by the [AtlasSetup] environment setup package. From Athena 16.1.0 both methods are supported, although the old method will be phased out, so it is essential to start to use the new (and simpler) [AtlasSetup].

---+ Setting up AtlasSetup on Lxplus

Prior to using Atlas.AtlasSetup to obtain the ATLAS software you must make the following modifications to you lxplus (or, with appropriate changes, your local institute) account. Full details and can be found in the Atlas.AtlasSetup page.

   * Identify the type of shell you are using; bash / ash
   * The reco<span style="color: black;" class="WYSIWYG_COLOR">mmende</span>d places on LXPLUS for these lines are: sh/zsh/bash shells (e.g. ${HOME}/.bash_login); and, csh/tcsh shells (e.g. ${HOME}/.tcshrc):
<verbatim>
export AtlasSetup=/afs/cern.ch/atlas/software/dist/AtlasSetup
alias asetup='source $AtlasSetup/scripts/asetup.sh'
</verbatim>

Now you are ready to setup Athena, type:
<verbatim>
asetup 16.6.0,here.
</verbatim>

Instead of typing these commands each time, one can create a very short *.sh (Exp: =test.sh=) file in your work area (the place you prefer to setup athena) and put these lines:
<verbatim>
export AtlasSetup=/afs/cern.ch/atlas/software/dist/AtlasSetup
alias asetup='source $AtlasSetup/scripts/asetup.sh'
asetup 16.6.0,here
</verbatim>

This will setup the release 16.6.0 just by doing:
<verbatim>
source test.sh
</verbatim>

---+ Setting up Athena using the old method (CMT):

This method can be used only for releases up to 16.0.1, since ATLAS start to change the management of the soft by using the setup. CMT home is the that allow the management of the packages, to use this method:

   * Create a directory named cmthome.
<verbatim>
mkdir cmthome   
cd cmthome
</verbatim>
   * Create a file using your favorite editor (nano, emacs ...) named requirements ( =nano requirements=) and paste into it these lines:
<verbatim>
set   CMTSITE  CERN 
set   SITEROOT /afs/cern.ch 
macro ATLAS_DIST_AREA ${SITEROOT}/atlas/software/dist 
macro ATLAS_TEST_AREA ${HOME}/
use AtlasLogin AtlasLogin-* $(ATLAS_DIST_AREA)
</verbatim>

In case you prefer to setup athena in an other area rather than your HOME directory (exp scratch0), you just have to include it's path in your requirement (fourth line):
<verbatim>
macro ATLAS_TEST_AREA ${HOME}/scratch0
</verbatim>

   * Set up CMT by:
<verbatim>
source /afs/cern.ch/sw/contrib/CMT/v1r21/mgr/setup.sh
cmt config
</verbatim>
   * Now you are ready to setup athena, just type:
<verbatim>
source cmthome/setup.sh -tag=16.0.1,setup,32
</verbatim>

You force the 32-bit mode to be used.

   * Create a work directory by:
<verbatim>
echo $TestArea
mkdir $TestArea
</verbatim>
   * Go to the TestArea and get ready ready to run the ATLAS software!
<verbatim>
cd $TestArea
</verbatim>

For participants who do not have an LXPLUS account and like to do the work on mgouigh@lxplus.cern.ch, it's preferable to make a directory in a temporary area (/tmp) of the LXPLUS machine you are logged on, like that we will not have problem of space in $HOME directory.
<verbatim>
cd /tmp
</verbatim>

Create a directory under your name
<verbatim>
mkdir Your_name
cd Your_name
</verbatim>

then do:
<verbatim>
cp /afs/cern.ch/user/m/mgouigh/Control/AthenaExamples/AthExRegTutorial/run/*.py .
</verbatim>

---+ Selection of Good Luminosity Blocks

The selection of events in good luminosity blocks can be done using the [[http://alxr.usatlas.bnl.gov/lxr-stb4/source/atlas/DataQuality/GoodRunsLists/GoodRunsLists/][GoodRunListSelectorTool]], to do so, please:

1. Setup Athena 2. Then checkout the packages needed (for both GRLs production and Luminosity calculation):
<verbatim>
cmt co -r GoodRunsListsUser-00-00-12 DataQuality/GoodRunsListsUser
cmt co -r GoodRunsLists-00-00-89 DataQuality/GoodRunsLists
cmt co -r LumiBlockComps-00-02-11 LumiBlock/LumiBlockComps
cmt co -r CoolRunQuery-00-03-35 Database/CoolRunQuery
</verbatim>

To install, do:
<verbatim>
cd $TestArea/Database/CoolRunQuery/cmt/
cmt make; source setup.sh
cd $TestArea/DataQuality/GoodRunsLists/cmt/
cmt make; source setup.sh
cd $TestArea/LumiBlock/LumiBlockComps/cmt/
cmt make; source setup.sh
cd $TestArea/DataQuality/GoodRunsListsUser/cmt/
cmt make; source setup.sh
</verbatim>

It is useful to make the script init.sh . cd $TestArea Open your favorite editor to make init.sh, and insert the lines:
<verbatim>
source $TestArea/Database/CoolRunQuery/cmt/setup.sh
source $TestArea/DataQuality/GoodRunsLists/cmt/setup.sh
source $TestArea/LumiBlock/LumiBlockComps/cmt/setup.sh
source $TestArea/DataQuality/GoodRunsListsUser/cmt/setup.sh
</verbatim>

alternatively you can:
<verbatim> 
cd $TestArea 
setupWorkArea.py 
cd WorkArea/cmt 
cmt bro cmt config 
source setup.sh 
</verbatim>

To look for the exact tags needed for a giving package, do:
<verbatim>
get_tag DataQuality/GoodRunsLists
</verbatim>

If you are interested of the head version of a given package, do:
<verbatim>
cmt co DataQuality/GoodRunsLists
</verbatim>

To use this tools you need first to setup athena as explained before

A possible selection rule can be the following:

<verbatim>
AtlRunQuery.py 'find run 90270-90350 and dq em y+ and dq pixb y+ / show dq pix,sct,em,till'
AtlRunQuery.py 'find run 140541+ and partition ATLAS and db DATA and st physics_MinBias and ptag data09_900GeV and duration 60s+ and events 100+ and mag s and dq til y+ and dq lar y+ and dq atlsol y+ and lhc beamenergy 449-451 and lhc stablebeams true/ show dq til and dq lar and dq pix'
</verbatim>

Example
<verbatim>
AtlRunQuery.py 'find run 167575-167844 and partition ATLAS and db DATA and lhc stablebeams T and st physics_MinBias and ready 1 and ptag data10_7TeV and dq ATLGL LBSUMM#DetStatus-v03-repro05-00 g and dq atltor LBSUMM#DetStatus-v03-repro05-00 g and dq atlsol LBSUMM#DetStatus-v03-repro05-00 g and dq pix LBSUMM#DetStatus-v03-repro05-00 g and dq sct LBSUMM#DetStatus-v03-repro05-00 g and dq trtb,trte LBSUMM#DetStatus-v03-repro05-00 y+ and dq mdt LBSUMM#DetStatus-v03-repro05-00 g and dq rpc LBSUMM#DetStatus-v03-repro05-00 g and dq tgc LBSUMM#DetStatus-v03-repro05-00 g and dq csc LBSUMM#DetStatus-v03-repro05-00 g'



[mgouigh@lxplus417]~/Tutorial/Control/AthenaExamples/AthExRegTutorial/run% AtlRunQuery.py 'find run 90270-90350 and dq em y+ and dq pixb y+ / show dq pix,sct,em,till'
Parser: 'find run 90270-90350 and dq em y+ and dq pixb y+ / show dq pix,sct,em,till'
  find argument: "run 90270-90350 and dq em y+ and dq pixb y+"
  show argument: "dq pix,sct,em,till"
  extra argument: ""
AtlRunQuery.py --run "90270-90350" --dq "EMBA yellow+ SHIFTOFL" --dq "EMBC yellow+ SHIFTOFL" --dq "EMECA yellow+ SHIFTOFL" --dq "EMECC yellow+ SHIFTOFL"  --dq "PIXB yellow+ SHIFTOFL"  --show run --show events --show time --show "dq PIXB SHIFTOFL" --show "dq PIX0 SHIFTOFL" --show "dq PIXEA SHIFTOFL" --show "dq PIXEC SHIFTOFL" --show "dq SCTB SHIFTOFL" --show "dq SCTEA SHIFTOFL" --show "dq SCTEC SHIFTOFL" --show "dq EMBA SHIFTOFL" --show "dq EMBC SHIFTOFL" --show "dq EMECA SHIFTOFL" --show "dq EMECC SHIFTOFL"  --projecttag "data08*,data09*,data10*" --partition "ATLAS" --verbose
__Warning: use COOL_DISABLE_CORALCONNECTIONPOOLCLEANUP__
Data source lookup using /afs/cern.ch/atlas/software/builds/AtlasCore/16.6.0/InstallArea/XML/AtlasAuthentication/dblookup.xml file
SELOUT Checking for runs in run range [[90270, 90350]]  ==> 8 runs found (0.69 sec)
SELOUT Checking in the DQ folder SHIFTOFL
SELOUT (SHIFTOFL:EMBA) and (SHIFTOFL:EMBC) and (SHIFTOFL:EMECA) and (SHIFTOFL:EMECC) and (SHIFTOFL:PIXB) is 'yellow+'  ==> 3 runs found (0.26 sec)
SELOUT Checking if the filename tag matches "data08*,data09*,data10*"  ==> 3 runs found (0.10 sec)
SELOUT Checking if partition name matches "ATLAS"  ==> 3 runs found (0.10 sec)
Retrieving event numbers  ==> Done (0.0282331 sec)
---------------------------------------------------------------------
Run             #LB                             Start and endtime (CEST)     #Events  SHIFTOFL:EMBA  SHIFTOFL:EMBC  SHIFTOFL:EMECA  SHIFTOFL:EMECC  SHIFTOFL:PIXB  SHIFTOFL:PIX0  SHIFTOFL:PIXEA  SHIFTOFL:PIXEC  SHIFTOFL:SCTB  SHIFTOFL:SCTEA  SHIFTOFL:SCTEC  
                                                                                                                                                                                                                          
run    90270     10      Sat Sep 27 2008 23:55:14 - Sun Sep 28, 03:57:42        n.a.           G           Y           G           G           G           G           G           G           G           G           G  
run    90272     58                  Sun Sep 28 2008 04:27:42 - 20:19:46        n.a.           G           G           G           G           G           G           G           G           G           G           G  
run    90275     47      Sun Sep 28 2008 22:19:31 - Mon Sep 29, 10:49:45        n.a.           G           Y           G           G           G           G           G           G           G           G           G  
Summary:
                 Run : 3
Total execution time : 4.2376 sec
-------------------------------------------------------------------- 
</verbatim>

A folder =data= is created by the AtlRunQuery.py in your work area, it contains more details concerning the DataQuality that follow the criteria you selected, a Good Run List file "=MyLBCollection.xml=" is created too we interest to use it later for our analysis

In case you get a problem like:
<verbatim>
__Warning: use COOL_DISABLE_CORALCONNECTIONPOOLCLEANUP__
Data source lookup using /afs/cern.ch/atlas/software/releases/16.6.0/DBRelease/current/XMLConfig/dblookup.xml file
(file "", line 0) The database does not exist (C++ exception)
</verbatim>

You have to include a very short file =.asetup= in your $HOME were you put these lines:

<verbatim>
[defaults]
standalone = False

# your test area directory
testarea = $PWD

# each release has its own directory
multi=False

# the release dir name has the project prepended
projtest = False
</verbatim>

The xml file is used as input in the athena analysis, it allows to run on the data conditions you select to produce it, make a not on the name which include =<name>= near the top of the file, we will use it on the jobOption. If you are not sure simply run the command:
<verbatim>
less <inputGRL.xml> | grep /Name
</verbatim>

---+ Integrated luminosity calculation in physics analysis

As we saw, we could calculate the luminosity using the iLumiCalc.exe which uses the Good Run List as input, the iLumiCalc.exe is loaded from the package =LumiBlock/LumiBlockComps=
<verbatim>
iLumiCalc.exe -t L1_MBTS_1 --lumitag=OflLumi-7TeV-002 -x data10_7TeV.periodI.167575-167844_LBSUMM_DetStatus-v03-repro05-00_muon_7TeV.xml

iLumiCalc.exe --lumitag=OflLumi-7TeV-002 --livetrigger=L1_MBTS_2 --trigger=L1_MU10 --xml=/afs/cern.ch/user/g/gouighri/scratch0/MyXtalk/Control/AthenaExamples/AthExRegTutorial/run/data10_7TeV.periodI.167575-167844_LBSUMM_DetStatus-v03-repro05-00_muon_7TeV.xml
</verbatim>

we got something like:
<verbatim>
[gouighri@lxplus250]~/scratch0/MyXtalk/Control/AthenaExamples/AthExRegTutorial/run/LUMINO% iLumiCalc.exe --lumitag=OflLumi-7TeV-002 --livetrigger=L1_MBTS_2 --trigger=L1_MU10 --xml=/afs/cern.ch/user/g/gouighri/scratch0/MyXtalk/Control/AthenaExamples/AthExRegTutorial/run/data10_7TeV.periodI.167575-167844_LBSUMM_DetStatus-v03-repro05-00_muon_7TeV.xml
--- iLumiCalc           : Trigger(s): L1_MU10, 
--- iLumiCalc           : Trigger used for livetime: L1_MBTS_2
--- iLumiCalc           : Lumitag: OflLumi-7TeV-002
--- iLumiCalc           : No lumimethod or lumichannel is given, defaulting to Lumimethod: ATLAS_PREFERRED
--- iLumiCalc           : Being in XML file mode...
--- iLumiCalc           : Processing file: </afs/cern.ch/user/g/gouighri/scratch0/MyXtalk/Control/AthenaExamples/AthExRegTutorial/run/data10_7TeV.periodI.167575-167844_LBSUMM_DetStatus-v03-repro05-00_muon_7TeV.xml>
--- iLumiCalc           : --------------------------------------------
--- LumiCalculator      : Luminosity database: COOLOFL_TRIGGER/COMP200
--- LumiCalculator      : Trigger database: COOLONL_TRIGGER/COMP200
--- CoolQuery           : Trying to connect to database COOLONL_TRIGGER/COMP200...
Using machine hostname lxplus250.cern.ch for DB replica resolution
Frontier server at (proxyurl=http://atlast0fsquid.cern.ch:3128)(proxyurl=http://atlassquid1.cern.ch:3128)(proxyurl=http://atlassquid2.cern.ch:3128)(serverurl=http://atlasfrontier.cern.ch:8000/atlr)(serverurl=http://atlasfrontier1.cern.ch:8000/atlr)(serverurl=http://atlasfrontier2.cern.ch:8000/atlr)(serverurl=http://ccsqfatlasli01.in2p3.fr:23128/ccin2p3-AtlasFrontier) will be considered
Total of 6 servers found for host lxplus250.cern.ch
__Warning: use COOL_DISABLE_CORALCONNECTIONPOOLCLEANUP__
Data source lookup using /afs/cern.ch/atlas/software/builds/AtlasCore/16.6.0/InstallArea/XML/AtlasAuthentication/dblookup.xml file
Allowed replica to try (priority -700) : frontier://ATLF/()/ATLAS_COOLONL_TRIGGER
Allowed replica to try (priority -699) : oracle://ATLAS_COOLPROD/ATLAS_COOLONL_TRIGGER
Allowed replica to try (priority -200) : frontier://ATLF/()/ATLAS_COOLONL_TRIGGER
Allowed replica to try (priority -199) : oracle://FZK/ATLAS_COOLONL_TRIGGER
Allowed replica to try (priority -198) : oracle://LYON/ATLAS_COOLONL_TRIGGER
--- CoolQuery           : Trying to connect to database COOLOFL_TRIGGER/COMP200...
Using machine hostname lxplus250.cern.ch for DB replica resolution
Frontier server at (proxyurl=http://atlast0fsquid.cern.ch:3128)(proxyurl=http://atlassquid1.cern.ch:3128)(proxyurl=http://atlassquid2.cern.ch:3128)(serverurl=http://atlasfrontier.cern.ch:8000/atlr)(serverurl=http://atlasfrontier1.cern.ch:8000/atlr)(serverurl=http://atlasfrontier2.cern.ch:8000/atlr)(serverurl=http://ccsqfatlasli01.in2p3.fr:23128/ccin2p3-AtlasFrontier) will be considered
Total of 6 servers found for host lxplus250.cern.ch
Allowed replica to try (priority -700) : frontier://ATLF/()/ATLAS_COOLOFL_TRIGGER
Allowed replica to try (priority -699) : oracle://ATLAS_COOLPROD/ATLAS_COOLOFL_TRIGGER
Allowed replica to try (priority -200) : frontier://ATLF/()/ATLAS_COOLOFL_TRIGGER
Allowed replica to try (priority -199) : oracle://FZK/ATLAS_COOLOFL_TRIGGER
Allowed replica to try (priority -198) : oracle://LYON/ATLAS_COOLOFL_TRIGGER
--- LumiCalculator      : -----------------------------------
--- LumiCalculator      : Beginning calculation for Trigger L1_MU10, Run 167575 LB [125-138]
--- LumiCalculator      : -----------------------------------
--- LumiCalculator      : >== Trigger  : L1_MU10
--- LumiCalculator      :        Run    L1-Acc    L2-Acc    L3-Acc  LiveTime  IntL rec.(ub^-1)  IntL del.(ub^-1)
--- LumiCalculator      :     167575   1665584         0         0   821.188            121471            158385
--- LumiCalculator      : L1/2/3 accept: 1665584   0         0         
--- LumiCalculator      : L1BeforePresc: 3363760   
--- LumiCalculator      : Livetime     : 821.188
--- LumiCalculator      : Prescale Weighted Livetime: 821.188
--- LumiCalculator      : Lumi w/o prescale (ub^-1):  121471
--- LumiCalculator      : Good LBs     : 14
--- LumiCalculator      : Bad LBs      : 0
--- LumiCalculator      : IntL delivered (ub^-1) : 158385
--- LumiCalculator      : IntL recorded (ub^-1) : 121471
--- LumiCalculator      : L1 Prescales: [167575,125]:1, [167575,129]:1, [167575,130]:1, [167575,134]:1, 
--- LumiCalculator      : -----------------------------------
--- LumiCalculator      : Beginning calculation for Trigger L1_MU10, Run 167576 LB [3-293]
--- LumiCalculator      : -----------------------------------
--- LumiCalculator      : >== Trigger  : L1_MU10
--- LumiCalculator      :        Run    L1-Acc    L2-Acc    L3-Acc  LiveTime  IntL rec.(ub^-1)  IntL del.(ub^-1)
--- LumiCalculator      :     167576  52012950         0         0   33949.3       3.75406e+06       3.80898e+06
--- LumiCalculator      : L1/2/3 accept: 52012950  0         0         
--- LumiCalculator      : L1BeforePresc: 52749220  
--- LumiCalculator      : Livetime     : 33949.3
--- LumiCalculator      : Prescale Weighted Livetime: 33833.7
--- LumiCalculator      : Lumi w/o prescale (ub^-1):  3.77062e+06
--- LumiCalculator      : Good LBs     : 291
--- LumiCalculator      : Bad LBs      : 0
--- LumiCalculator      : IntL delivered (ub^-1) : 3.80898e+06
--- LumiCalculator      : IntL recorded (ub^-1) : 3.75406e+06
--- LumiCalculator      : L1 Prescales: [167576,3]:1, [167576,4]:30, [167576,5]:1, [167576,6]:1, [167576,7]:1, [167576,27]:1, [167576,31]:1, [167576,37]:1, [167576,59]:1, [167576,61]:1, [167576,86]:1, [167576,119]:1, [167576,122]:1, [167576,176]:1, 
--- LumiCalculator      : -----------------------------------
--- LumiCalculator      : Beginning calculation for Trigger L1_MU10, Run 167607 LB [86-430]
--- LumiCalculator      : -----------------------------------
--- LumiCalculator      : >== Trigger  : L1_MU10
--- LumiCalculator      :        Run    L1-Acc    L2-Acc    L3-Acc  LiveTime  IntL rec.(ub^-1)  IntL del.(ub^-1)
--- LumiCalculator      :     167607  76179160         0         0   37868.1        5.4771e+06       5.76007e+06
--- LumiCalculator      : L1/2/3 accept: 76179160  0         0         
--- LumiCalculator      : L1BeforePresc: 80310842  
--- LumiCalculator      : Livetime     : 37868.1
--- LumiCalculator      : Prescale Weighted Livetime: 37868.1
--- LumiCalculator      : Lumi w/o prescale (ub^-1):  5.4771e+06
--- LumiCalculator      : Good LBs     : 345
--- LumiCalculator      : Bad LBs      : 0
--- LumiCalculator      : IntL delivered (ub^-1) : 5.76007e+06
--- LumiCalculator      : IntL recorded (ub^-1) : 5.4771e+06
--- LumiCalculator      : L1 Prescales: [167607,86]:1, [167607,87]:1, [167607,90]:1, [167607,93]:1, [167607,96]:1, [167607,99]:1, [167607,102]:1, [167607,107]:1, [167607,115]:1, [167607,126]:1, [167607,127]:1, [167607,138]:1, [167607,140]:1, [167607,143]:1, [167607,188]:1, [167607,189]:1, [167607,232]:1, [167607,239]:1, [167607,241]:1, [167607,274]:1, [167607,284]:1, [167607,287]:1, [167607,291]:1, [167607,330]:1, [167607,362]:1, [167607,369]:1, [167607,382]:1, [167607,425]:1, 
--- LumiCalculator      : -----------------------------------
--- LumiCalculator      : Beginning calculation for Trigger L1_MU10, Run 167661 LB [372-439]
--- LumiCalculator      : -----------------------------------
--- LumiCalculator      : >== Trigger  : L1_MU10
--- LumiCalculator      :        Run    L1-Acc    L2-Acc    L3-Acc  LiveTime  IntL rec.(ub^-1)  IntL del.(ub^-1)
--- LumiCalculator      :     167661  17728830         0         0   7361.96       1.33562e+06       1.38041e+06
--- LumiCalculator      : L1/2/3 accept: 17728830  0         0         
--- LumiCalculator      : L1BeforePresc: 18333164  
--- LumiCalculator      : Livetime     : 7361.96
--- LumiCalculator      : Prescale Weighted Livetime: 7361.96
--- LumiCalculator      : Lumi w/o prescale (ub^-1):  1.33562e+06
--- LumiCalculator      : Good LBs     : 68
--- LumiCalculator      : Bad LBs      : 0
--- LumiCalculator      : IntL delivered (ub^-1) : 1.38041e+06
--- LumiCalculator      : IntL recorded (ub^-1) : 1.33562e+06
--- LumiCalculator      : L1 Prescales: [167661,372]:1, [167661,375]:1, [167661,377]:1, [167661,379]:1, [167661,384]:1, [167661,385]:1, [167661,386]:1, [167661,393]:1, [167661,395]:1, [167661,413]:1, [167661,414]:1, [167661,421]:1, 
--- LumiCalculator      : -----------------------------------
--- LumiCalculator      : Beginning calculation for Trigger L1_MU10, Run 167680 LB [33-266]
--- LumiCalculator      : -----------------------------------
--- LumiCalculator      : >== Trigger  : L1_MU10
--- LumiCalculator      :        Run    L1-Acc    L2-Acc    L3-Acc  LiveTime  IntL rec.(ub^-1)  IntL del.(ub^-1)
--- LumiCalculator      :     167680  47860798         0         0   23839.6       3.68099e+06       3.84451e+06
--- LumiCalculator      : L1/2/3 accept: 47860798  0         0         
--- LumiCalculator      : L1BeforePresc: 49888768  
--- LumiCalculator      : Livetime     : 23839.6
--- LumiCalculator      : Prescale Weighted Livetime: 23839.6
--- LumiCalculator      : Lumi w/o prescale (ub^-1):  3.68099e+06
--- LumiCalculator      : Good LBs     : 234
--- LumiCalculator      : Bad LBs      : 0
--- LumiCalculator      : IntL delivered (ub^-1) : 3.84451e+06
--- LumiCalculator      : IntL recorded (ub^-1) : 3.68099e+06
--- LumiCalculator      : L1 Prescales: [167680,33]:1, [167680,40]:1, [167680,45]:1, [167680,54]:1, [167680,58]:1, [167680,60]:1, [167680,66]:1, [167680,77]:1, [167680,83]:1, [167680,86]:1, [167680,89]:1, [167680,96]:1, [167680,99]:1, [167680,101]:1, [167680,104]:1, [167680,107]:1, [167680,121]:1, [167680,124]:1, [167680,128]:1, [167680,141]:1, [167680,146]:1, [167680,151]:1, [167680,164]:1, [167680,172]:1, [167680,234]:1, [167680,258]:1, 
--- LumiCalculator      : -----------------------------------
--- LumiCalculator      : Beginning calculation for Trigger L1_MU10, Run 167776 LB [124-546]
--- LumiCalculator      : -----------------------------------
--- LumiCalculator      : >== Trigger  : L1_MU10
--- LumiCalculator      :        Run    L1-Acc    L2-Acc    L3-Acc  LiveTime  IntL rec.(ub^-1)  IntL del.(ub^-1)
--- LumiCalculator      :     167776  78320321         0         0   49707.8       6.05068e+06       6.11436e+06
--- LumiCalculator      : L1/2/3 accept: 78320321  0         0         
--- LumiCalculator      : L1BeforePresc: 79133722  
--- LumiCalculator      : Livetime     : 49707.8
--- LumiCalculator      : Prescale Weighted Livetime: 49707.8
--- LumiCalculator      : Lumi w/o prescale (ub^-1):  6.05068e+06
--- LumiCalculator      : Good LBs     : 423
--- LumiCalculator      : Bad LBs      : 0
--- LumiCalculator      : IntL delivered (ub^-1) : 6.11436e+06
--- LumiCalculator      : IntL recorded (ub^-1) : 6.05068e+06
--- LumiCalculator      : L1 Prescales: [167776,124]:1, [167776,130]:1, [167776,132]:1, [167776,191]:1, [167776,208]:1, [167776,210]:1, [167776,212]:1, [167776,214]:1, [167776,283]:1, [167776,347]:1, [167776,372]:1, [167776,378]:1, [167776,384]:1, [167776,390]:1, [167776,401]:1, [167776,432]:1, 
--- LumiCalculator      : -----------------------------------
--- LumiCalculator      : Beginning calculation for Trigger L1_MU10, Run 167844 LB [79-270]
--- LumiCalculator      : -----------------------------------
--- LumiCalculator      : >== Trigger  : L1_MU10
--- LumiCalculator      :        Run    L1-Acc    L2-Acc    L3-Acc  LiveTime  IntL rec.(ub^-1)  IntL del.(ub^-1)
--- LumiCalculator      :     167844  33607002         0         0   21950.5       2.53309e+06       2.55123e+06
--- LumiCalculator      : L1/2/3 accept: 33607002  0         0         
--- LumiCalculator      : L1BeforePresc: 33782287  
--- LumiCalculator      : Livetime     : 21950.5
--- LumiCalculator      : Prescale Weighted Livetime: 21950.5
--- LumiCalculator      : Lumi w/o prescale (ub^-1):  2.53309e+06
--- LumiCalculator      : Good LBs     : 192
--- LumiCalculator      : Bad LBs      : 0
--- LumiCalculator      : IntL delivered (ub^-1) : 2.55123e+06
--- LumiCalculator      : IntL recorded (ub^-1) : 2.53309e+06
--- LumiCalculator      : L1 Prescales: [167844,79]:1, [167844,81]:1, [167844,86]:1, [167844,90]:1, [167844,91]:1, [167844,95]:1, [167844,112]:1, [167844,116]:1, [167844,121]:1, [167844,135]:1, [167844,142]:1, [167844,189]:1, [167844,224]:1, 
--- LumiCalculator      : -----------------------------------
--- LumiCalculator      :      Total    L1-Acc    L2-Acc    L3-Acc  LiveTime  IntL rec.(ub^-1)  IntL del.(ub^-1)
--- LumiCalculator      :            307374645         0         0    175498        2.2953e+07        2.3618e+07
--- LumiCalculator      : Total L1/2/3 accept: 307374645 0         0         
--- LumiCalculator      : Total L1 livetime trigger accept: 408026357 
--- LumiCalculator      : Total L1BeforePresc: 317561763 
--- LumiCalculator      : Total Livetime     : 175498
--- LumiCalculator      : Total prescale weighted Livetime: 175383
--- LumiCalculator      : Total lumi w/o prescale (ub^-1):  2.29696e+07
--- LumiCalculator      : Total Good LBs     : 1567
--- LumiCalculator      : Total Bad LBs     : 0
--- LumiCalculator      : Total IntL delivered (ub^-1) : 2.3618e+07
--- LumiCalculator      : Total IntL recorded (ub^-1) : 2.2953e+07
--- iLumiCalc           : --------------------------------------------
--- TGoodRunsListWriter : GoodRunsList stored as        : ilumicalc_merged_grl_167575.125-167844.270.xml
--- iLumiCalc           : Real time: 8.5147 s
--- iLumiCalc           : CPU time:   4.41 s
</verbatim>

Other info can be added to the command line, like =-r= specify a specific run(s), =--lbstart=xx --lbend=yy= to select a LB range[xx,yy], =-t= specify the trigger item and =--lumitag= the Cool database folders are tagged so one needs to specify the tag.

For more details about the Luminosity calculation, have a look on this tutorial [[Atlas.CoolLumiCalcTutorial#Exercise_1_Using_i_LumiCalc_exe][LuminosityCalculation]]

---+ Running the Full simulation chain

To run the full simulation chain (Generation to Reconstruction) you need an ATHENA production cache release like =16.6.0.X=, normally, it is recommended to use directly the official files ( =RDO, ESD, AOD...=) produced by your physic group, some stages especially at the reconstruction level are complex and difficult to configure correctly. The production is generally done on the Grid using the Production System (ProdSys).

---+ Running an athena analysis code

After setting up athena using asetup:
<verbatim>
cd TestArea
asetup 16.6.0,here
</verbatim>

Now we checkout a package using the =cmt co=
<verbatim>
cmt co -r AthExRegTutorial-00-00-03 Control/AthenaExamples/AthExRegTutorial
</verbatim>

Once you checkout the package, let us have a look at the contents of different parts and understand how it works. =cd Control/AthenaExamples/AthExRegTutorial=, =ls=, you will see something like:
<verbatim>
AthExRegTutorial  cmt  genConf	i686-slc5-gcc43-opt  python  run  scripts  share  src
</verbatim>

   * =AthExRegTutorial= contains the C++ header files for the algorithm(s) and tool(s) in the package - the convention is that this has the same name as the package itself
   * =src= contains the C++ source code for the algorithm(s) and tool(s) in the package
   * =share= contains the job option files for the algorithm(s) and tool(s) in the package - have a look in there for HelloWorldOptions.py which you used earlier
   * =genConf= is a text file containing a record of modifications to the package
   * =cmt= is the cmt configuration directory containing the requirements file for the package
   * =python= (is probably not here in this particular example but in general) is used for the declaration of configurables for the job options, and should not normally be touched
   * =scripts= is optional in case you like to put your script (root) into your directory.

Let's have a look at different folders and try to understand its contents.

1. Now let us start running a first short example, for this please, copy the jobOptions from the =share= directory to your run directory.
<verbatim>
</verbatim>

2. For people who working on mgouigh, since you already create =/tmp/mgouigh/your_name= please create inside a =run= directory and copy the python files into it.
<verbatim>
cd /tmp/gouighri/your_name
mkdir run
cp /afs/cern.ch/user/m/mgouigh/Tutorial/Control/AthenaExamples/AthExRegTutorial/share/*.py .
</verbatim>

3. You can start now by: =athena master.py=
<verbatim>
</verbatim>

4. Print the output in a log file: =athena master.py |tee output_name.log=.
<verbatim>
</verbatim>

5. Have a look at the files created.
<verbatim>
</verbatim>

6. Now let us play a little with the jobOptions to have a clear idea on how things work.

Once we have now a clear idea, the next steps is how you could create a new package on Athena.

---+ Create a new package on Athena.

We will try to create a new package based on our existing package =AthExRegTutorial= where we will try to include the trigger. it's not so easy, please be patient :)

You can find more information on how to create a new package on the [[Atlas.WorkBookCreatingNewPackage][WorkBook]]

1. Go into
cd $Tutorial/Control/AthenaExamples/AthExRegTutorial/src=

=cp SimpleAnalysisSkeleton.cxx SimpleAnalysisSkeletonTrig.cxx= <BR>
=cp SimpleAnalysisSkeleton.h SimpleAnalysisSkeletonTrig.h=

I already created these files and include the trigger informations, all we need is to have a look and see what's are the differences.

2. Declare your new algorithm in the folder =components= under =src=. You will see two files =AthExRegTutorial_entries.cxx= and  =AthExRegTutorial_load.cxx=.

Since we make a new algorithm inside the same package, the changes should be only on the AthExRegTutorial_entries.cxx, should be something like:

<verbatim>
#include "AthExRegTutorial/SimpleAnalysisSkeleton.h"
#include "AthExRegTutorial/SimpleAnalysisSkeletonTrig.h"

#include "GaudiKernel/DeclareFactoryEntries.h"

DECLARE_ALGORITHM_FACTORY( SimpleAnalysisSkeleton )
DECLARE_ALGORITHM_FACTORY( SimpleAnalysisSkeletonTrig )

DECLARE_FACTORY_ENTRIES( AthExRegTutorial ) {
  DECLARE_ALGORITHM( SimpleAnalysisSkeleton )
  DECLARE_ALGORITHM( SimpleAnalysisSkeletonTrig )
} 
</verbatim>  
 
3. We need now to include the new algorithm on the job Option =run/jobOptions.py=, since we are interested to compile our code with the new informations. This is what you need to add:

Replace:
<verbatim>
from AthExRegTutorial.AthExRegTutorialConf import SimpleAnalysisSkeleton
</verbatim>
with
<verbatim>
from AthExRegTutorial.AthExRegTutorialConf import SimpleAnalysisSkeletonTrig
</verbatim>
and 
<verbatim>
job.ModSequence1 += SimpleAnalysisSkeletonTrig( "SimpleAnalysisSkeleton" )
</verbatim>
With:
<verbatim>
job.ModSequence1 += SimpleAnalysisSkeletonTrig( "SimpleAnalysisSkeletonTrig" )
</verbatim>

You have to include also this line.
<verbatim>
include("trigger.py")
</verbatim>
and to setup the trigger include:
<verbatim>
job.ModSequence1.SimpleAnalysisSkeletonTrig.useTrigger = True
</verbatim>

To run now on the new algorithm, copy the new jobOption into your Workarea:
<verbatim>
cp /afs/cern.ch/user/m/mgouigh/Tutorial/Control/AthenaExamples/AthExRegTutorial/run/TRIGGER/*.py .
</verbatim>

Run on the new algorithm and store the output file to see what's the new (differences).
<verbatim>
athena master.py |tee output_trigger.log
</verbatim>

The main interesting changes we will see from the output will be related to the trigger settings:

The loading of all items =L1= and chains for =HLT=, you see few lines in your output like (we list here only L1 items):
<verbatim>
SimpleAnalysisSkeletonTrig                           INFO L1 Items: 
[L1_EM3, L1_EM5, L1_EM2, L1_EM10, L1_EM10I, L1_EM14, L1_EM14_XE10, L1_BGRP1, L1_NIM_S8C2B21, L1_2EM2, L1_2EM3, L1_2EM5, L1_NIM_S8C2B22, L1_NIM_S8C2B23, L1_NIM_S8C2B24, L1_MBTS_4_4, L1_4J30, L1_RD0_FILLED, L1_RD0_FIRSTEMPTY, L1_RD0_UNPAIRED_ISO, L1_MBTS_4_4_UNPAIRED_ISO, L1_ZDC_UNPAIRED, L1_MU0_J55, L1_J5_UNPAIRED_ISO, L1_FJ10_UNPAIRED_ISO, L1_TAU5_UNPAIRED_ISO, L1_NIM_S8C2B25, L1_NIM_S8C2B26, L1_NIM_S8C2B27, L1_NIM_S9C0B21, L1_MU0_MV, L1_EM3_MV, L1_TAU5_MV, L1_TAU5_XE10, L1_4J10, L1_TAU5_MU6, L1_2TAU5_EM5, L1_2TAU6_EM5, L1_2TAU6_EM10, L1_TAU5_3J5_2J15, L1_3J5_J30, L1_4J5_J15, L1_4J5_J30, L1_2J10_J30, L1_2J10_J55, L1_3J10_2J30, L1_4J10_3J15, L1_TAU6_MU10, L1_TAU6_MU15, L1_TAU5_4J5_3J15, L1_MU0_COMM_UNPAIRED_ISO, L1_MU0_J30, L1_MU0_UNPAIRED_ISO, L1_EM2_UNPAIRED_ISO, L1_J10_FJ10, L1_2J30, L1_2J55, L1_2J95, L1_EM3_FIRSTEMPTY, L1_TAU5_FIRSTEMPTY, L1_J5_FIRSTEMPTY, L1_FJ10_FIRSTEMPTY, L1_RD0_EMPTY, L1_RD1_FILLED, L1_TAU5, L1_TAU6, L1_TAU6I, L1_TAU50, L1_TAU11, L1_TAU11I, L1_2MU6, L1_2MU10, L1_TAU20, L1_TAU30, L1_NIM_S9C0B22, L1_2MU0_MU6, L1_TRT, L1_L1A_Mon, L1_NIM_S9C0A20, L1_NIM_S9C1C19, L1_NIM_S9C1C20, L1_3J30, L1_EM85, L1_6J10, L1_2J15_J75, L1_MU15, L1_MU0, L1_MU6, L1_MU10, L1_MU20, L1_MU0_COMM, L1_2MU0, L1_2TAU11, L1_J5, L1_J10, L1_J15, L1_J30, L1_J75, L1_J15_XE15_MU15, L1_J55, L1_J95, L1_2J5, L1_2J10, L1_4J10_MU15, L1_MU0_J10, L1_2J15, L1_2TAU5, L1_2TAU6, L1_J115, L1_J15_MV, L1_J15_LV, L1_J15_NL, L1_XE25, L1_XE35, L1_XE40, L1_XE50, L1_FJ10, L1_FJ30, L1_NIM_S9C0B23, L1_NIM_S9C0B24, L1_2FJ55, L1_2FJ10, L1_2FJ30, L1_FJ55, L1_3J10, L1_FJ95, L1_XE10, L1_4J5, L1_XE15, L1_2J15_XE25, L1_XE20, L1_2J15_XE10, L1_XE30, L1_2J15_XE15, L1_JE60, L1_TE20, L1_TE50, L1_TE100, L1_TE180, L1_MU6_J5, L1_MU0_EM3, L1_J15_XE15_EM10, L1_JE100, L1_JE140, L1_JE200, L1_MBTS_1_EMPTY, L1_MBTS_1_1_EMPTY, L1_MBTS_2_EMPTY, L1_TAU5_EMPTY, L1_TAU6_4J5_3J15, L1_2MU6_EM5, L1_MU10_J10, L1_J10_FIRSTEMPTY, L1_J30_FIRSTEMPTY, L1_J10_UNPAIRED, L1_J30_UNPAIRED, L1_MU0_FIRSTEMPTY, L1_MU0_EMPTY, L1_2MU0_FIRSTEMPTY, L1_MU10_FIRSTEMPTY, L1_MU6_FIRSTEMPTY, L1_MU0_COMM_EMPTY, L1_MU0_COMM_FIRSTEMPTY, L1_J5_EMPTY, L1_J30_XE10, L1_J30_XE15, L1_J10_EMPTY, L1_J30_EMPTY, L1_TAU6_3J5_2J15, L1_J30_XE25, L1_FJ10_EMPTY, L1_4J10_EM10, L1_EM3_EMPTY, L1_2EM14, L1_EM14I, L1_2MU20, L1_ZDC_AND_EMPTY, L1_EM5_MU6, L1_EM14_XE15, L1_J10_MV, L1_2EM10, L1_3J15, L1_4J15, L1_5J10, L1_ZDC_AND, L1_ZDC_A, L1_ZDC_C, L1_ZDC_A_C, L1_ZDC, L1_ZDC_A_C_EMPTY, L1_ZDC_EMPTY, L1_TAU6_XE10, L1_RD1_EMPTY, L1_LUCID_A_C_UNPAIRED_ISO, L1_LUCID_A_C_UNPAIRED_NONISO, L1_MBTS_2_UNPAIRED_NONISO, L1_MBTS_1_UNPAIRED_ISO, L1_MBTS_1_1_UNPAIRED_ISO, L1_MBTS_2_UNPAIRED_ISO, L1_LUCID_A_UNPAIRED, L1_LUCID_C_UNPAIRED, L1_LUCID_A_C_UNPAIRED, L1_LUCID_A, L1_LUCID_C, L1_LUCID_A_C, L1_LUCID, L1_BCM_AC_CA_BGRP0, L1_BCM_Wide_BGRP0, L1_BCM_HT_BGRP0, L1_MU0_UNPAIRED_NONISO, L1_MU0_COMM_UNPAIRED_NONISO, L1_EM2_UNPAIRED_NONISO, L1_TAU5_UNPAIRED_NONISO, L1_J5_UNPAIRED_NONISO, L1_MBTS_1, L1_MBTS_2, L1_MBTS_1_1, L1_LUCID_A_EMPTY, L1_LUCID_C_EMPTY, L1_LUCID_A_C_EMPTY, L1_ZB, L1_BPTX0_BGRP0, L1_BPTX1_BGRP0, L1_EM5_MU10, L1_MU0_J15, L1_LUCID_EMPTY, L1_LUCID_COMM, L1_MU6_EM3, L1_LUCID_COMM_EMPTY, L1_MU0_J5, L1_CALREQ0, L1_CALREQ1, L1_CALREQ2]
</verbatim>

Also you will see many lines like: 
<verbatim>
SimpleAnalysisSkeletonTrig                           INFO L1_MU20 passed - True triggerCounter is 27
</verbatim>
That means the counting for each event that pass the trigger we were interesting (trigger we include into the =src/SimpleAnalysisSkeletonTrig=)

At the printing level you will see the number of events that pass each chains, look in your log file for lines start with:

<verbatim>
Level-1 Triggers defined: 233
-------------------------
Events passing L1_MU15			31
Events passing L1_MU0			28
Events passing L1_MU6			0
Events passing L1_MU10			67
Events passing L1_MU20			29
Events passing L1_MU0_COMM			35
Events passing L1_2MU0			29
Events passing L1_2TAU11			1
Events passing L1_J5			0
...

</verbatim>

And the finalize steps in our analysis will print the percentage of each trigger chain:

<verbatim> 
TRIGGER SUMMARY out of 100 events 
===============================
Number of events passing Level 1:      100
Number of events passing Level 2:      99
Number of events passing Event Filter: 98

Triggered events for defined trigger L1_MU0 		28	(28.0000%)
Triggered events for defined trigger L1_MU6 		0	(0.0000%)
Triggered events for defined trigger L1_MU10 		67	(67.0000%)
Triggered events for defined trigger L1_MU11 		0	(0.0000%)
Triggered events for defined trigger L1_MU20 		29	(29.0000%)
Triggered events for defined trigger L1_MU40 		0	(0.0000%)
Triggered events for defined trigger L1_2MU4 		0	(0.0000%)
Triggered events for defined trigger L1_2MU6 		17	(17.0000%)
Triggered events for defined trigger L1_2MU10 		8	(8.0000%)
Triggered events for defined trigger L1_2MU11 		0	(0.0000%)
Triggered events for defined trigger L1_2MU20 		0	(0.0000%)
</verbatim>

From this percentage one could have a clear idea on the acceptance of the chain you are interested for you analysis (=Cross-section measurements=) for example.



-- Main.MohamedGouighri - 11-Mar-2011
Topic revision: r13 - 2020-06-18 - MaximKonyushikhin
Main
Webs
Public webs
Welcome Guest
- Cern Search
- TWiki Search
- Google Search
Main All webs
Copyright &© 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback