VKaushikSandbox < Sandbox

<!---------------------------------------------------------  snip snip ----------------------------------------------------------------->
%TOC%     <!--optional-->

%STARTINCLUDE%

---+!! <nop>ARATopAnalysis
---++ Introduction
<nop>ARATopAnalysis is a physics analysis package which uses <nop>AthenaROOTAccess (ARA)
to read <nop>ESD/<nop>AOD input in the form of a transient ROOT tree and performs a
basic %$t \bar{t}$% candidate selection. The offline objects are selected
using different selectors. The selector classes access an object (<nop>EventData) which holds the
required event information to reconstruct top pairs from the transient tree. The transient tree
itself is packaged in the <nop>ARATree object. The selector classes are unaware of how
the branches of the transient tree are accessed. They are independent of
this step. The interface between the <nop>ARATree (data) and their <nop>accessors (selectors)
is the <nop>EventData class.

---++ Intended Users
This package is intended for users starting out on top physics analysis using ARA, although it should
be relatively easy to tailor this package to suit any other analysis. The users are required to write their own
steering class which is able to access all the information from the transient tree. Once this is done
the user has a choice of using/modifying the selector classes already provided or writing their
own object selection routines.

#TagsAnchor

---++ List of Tags 
Newest tag always contains latest updates. Please use those. The SVN location for [[https://svnweb.cern.ch/trac/atlasgrp/browser/Institutes/Arizona/ARATopAnalysis][<nop>ARATopAnalysis]].
A list of tags for specific <nop>Athena releases is given in the table below:

%TABLE{ sort="on" tableborder="0" cellpadding="4" cellspacing="3" cellborder="0" headerbg="#D5CCB1" headercolor="#666666" databg="#FAF0D4, #F3DFA8" headerrows="2" footerrows="1" }%
| *Athena* | *<nop>AthenaROOTAccess* | *<nop>ARATopAnalysis* | *<nop>AZAnalysis* | *Comment* |
| 15.5.2   | <nop>AthenaROOTAccess-00-05-57 |<nop>ARATopAnalysis-00-03-02  | <nop>AZAnalysis-00-00-05 | Added <nop>TopoVariables, <nop>TopoTreeMaker |
%BR%
%BR%
%TABLE{ sort="on" tableborder="0" cellpadding="4" cellspacing="3" cellborder="0" headerbg="#D5CCB1" headercolor="#666666" databg="#FAF0D4, #F3DFA8" headerrows="2" footerrows="1" }%
| *Athena* | *<nop>AthenaROOTAccess* | *<nop>ARATopAnalysis* | *Comment* |
| 14.4.0 | <nop>AthenaROOTAccess-00-05-31 | <nop>ARATopAnalysis-00-00-00 | First import |
| 14.5.0 | <nop>AthenaROOTAccess-00-05-33 | <nop>ARATopAnalysis-00-00-01 | No updates |
| ^          | ^                                                         | <nop>ARATopAnalysis-00-00-02 | added dummy version.cmt |
| ^          | ^                                                         | <nop>ARATopAnalysis-00-00-03 | added <nop>TrigStudy class       |
| ^          | ^                                                         | <nop>ARATopAnalysis-00-00-04 | added <nop>TopFitter class        |
| 15.0.0   | <nop>AthenaROOTAccess-00-05-43       | <nop>ARATopAnalysis-00-01-03 | added <nop>TruthSelector class         |
| ^          | ^                                                         | ^ | modifications to compile with gcc4.3        |
| 15.1.0   | <nop>AthenaROOTAccess-00-05-46-01       | <nop>ARATopAnalysis-00-01-04 | added <nop>TagAndProbe class         |
| 15.2.0   | <nop>AthenaROOTAccess-00-05-49            |<nop>ARATopAnalysis-00-01-05  | verified cutflow for PUB note, merged e,mu |
| ^          | ^                                                              |<nop>ARATopAnalysis-00-01-06  | added an example showing the use of EventData | 
| 15.2.0   | <nop>AthenaROOTAccess-00-05-49 |<nop>ARATopAnalysis-00-02-00  | Using <nop>TriggerDecisionTool, added Top/W containers, first SVN version|
| ^          | ^                                                              |<nop>ARATopAnalysis-00-02-02  | fixed crash, added myMain.cxx |
| ^          | ^                                                              |<nop>ARATopAnalysis-00-02-04  | added runARA.py module, <nop>NTupleHandler|
| ^          | ^                                                              |<nop>ARATopAnalysis-00-02-05  | added <nop>JESSystematic|
| 15.4.0   | <nop>AthenaROOTAccess-00-05-55 |<nop>ARATopAnalysis-00-02-06  | fixes to use P4Helpers |


---++ Analysis Steering
Steering the analysis involves the following steps: <br />
   * Access all the required branches from <nop>ARATree <br />
   * Store this information in <nop>EventData <br />
   * Loop over all events and perform the selection <br />
   * Save the resulting subset of events, histograms etc. <br />

An analysis base-class (<nop>myAnalysis) is created which has methods to perform these steps.
It has pure virtual methods and all the analysis steering classes (one of them is <nop>TopAnalysis)
inherit from this base-class and implement these steps. The analysis base class is made a friend
of <nop>ARATree, to enable the steering classes to implement the above steps.

---++ Configuration
User analysis usually involves selection of physics objects based on cuts. To facilitate easy
modification of the selection criteria without the additional burden of re-compile / building of the
package upon making these modifications, a configuration file (an ASCII text file)
is used by the selectors. A <nop>Config class reads the user configuration during run-time and passes
them to the selectors. The user configuration could be acceptance regions in %$ \eta-\phi $%,
%$ p_T $% cuts, names of the branches of the transient tree to be read etc. to name a few.

---++ Extras
Lastly, it is possible to store/retrieve a subset of selected objects or new objects. This is done
by the <nop>ObjCollection class. Message service is provided by <nop>SLogger (which is originally from the
<nop>SFrame analysis package). Message service enables user to output messages to standard output / error
streams with different levels of detail.

---++ Class Overview
---+++!! Main Classes

   * *ARATree*: Has the ARA transient tree and methods to access the branches <br />
   * *EventData*: Contains only relevant transient tree branches to reconstruct %$ t \bar{t} $% pairs from their semi-leptonic decay <br />
   * *myAnalysis*: Analysis base class. All steering classes (analysers) must inherit from this class <br />
   * *TopAnalysis*: Analyzer for top selection based on top cross section CSC note (T6) <br />
   * *ObjCollection*: Stores collection of user selected objects. The objects can be stored / retrieved by their unique names <br />
   * *myAnalysisTools*: Its a helper class used to sort collection of objects based on some of their properties (descending order by %$ p_T $% for example) <br />
   * *Config*: Reads the user configuration (cuts) from an ASCII file <br />
   * *SLogger*: Holds the message stream objects <br />
   * *SLogWriter*: Writes the formatted messages to standard output with various levels of detail <br />

---+++!! Helper Classes
   * *Histogram*: Plot kinematic properties of objects <br />
   * *TopoKineVars*: Calculate some event shape / topological quantities specific to top decay and store them in a rootuple <br />
   * *JESSystematic*: Scales the 4-momenta of all the jets in an event and re-computes the missing ET<br />

---+++!! Container Classes
   * *TopContainer*:  To hold leptonic %$ t \rightarrow (W \rightarrow \ell \nu) b$% and hadronic top %$ t \rightarrow (W \rightarrow qq) b$% <br />


---+++!! Selector Classes

   * *ElectronSelector*: Select reconstructed electrons <br />
   * *JetSelector*: Select reconstructed jets <br />
   * *METSelector*: Select missing %$ E_T $% <br />
   * *MuonSelector*: Selected reconstructed muons <br />
   * *TopSelector*: Reconstuct a top pair from their semi-leptonic decay <br />
   * *TriggerSelector*: Select events based on certain triggers <br />
   * *TruthSelector*: Select MC truth information for truth-matching  <br />

---++ Example 1. Running <nop>TopAnalysis
For running the top analysis out of the box, one has to check out the package (either from head or a specific tag)
and compile it first. The default analysis can be run out of the box with tag [[https://svnweb.cern.ch/trac/atlasgrp/browser/Institutes/Arizona/ARATopAnalysis/tags/ARATopAnalysis-00-02-00][ARATopAnalysis-00-02-00]] or later.
A python module (=runARA.py= aliased to =runARA=) is defined to facilitate running the analysis.

      * compile and create a link to config file%BR%
         =cd $TestArea/<nop>ARATopAnalysis/cmt/ && gmake && cd $TestArea= %BR%
         =source $TestArea/<nop>ARATopAnalysis/cmt/setup.sh= %BR%
         =ln -s <nop>ARATopAnalysis/test/config/analysis.ttbar.config analysis.config=

      * show =runARA= options:%BR%
         =runARA -h= or =runARA --help= or =runARA.py -h=
      <verbatim>
      Options:
        -h, --help            show this help message and exit
        -f FILENAME, --file=FILENAME
                              read a set of AOD/ESD/DPD's from inputfile
        -n EVENTS,   --nevts=EVENTS
                              number of events to analyze
      </verbatim>
      * run over 100 events, with input files listed in test.ttbar.list:%BR%
         =runARA -n 100 -f /afs/cern.ch/user/v/venkat/public/test.ttbar.list=
      * run over all events%BR%
         =runARA -f /afs/cern.ch/user/v/venkat/public/test.ttbar.list=
      * run over all events from two input files: Notice the comma after each input file in the first line and no commas in the second line %BR%
         =echo /raid01/venkat/dataset/check/AOD.026357._00511.pool.root.1, /raid01/venkat/dataset/check/AOD.026357._00540.pool.root.1 | runARA=
         =runARA /raid01/venkat/dataset/check/AOD.026357._00511.pool.root.1 /raid01/venkat/dataset/check/AOD.026357._00540.pool.root.1= 
      * Using runARA.exe directly. Use with caution or be prepared for consequences!%BR%
         =runARA.exe 100 /afs/cern.ch/user/v/venkat/public/test.ttbar.list=

---++ Example 2. Standalone test
---+++!!! Setup Athena Release
Refer [[https://twiki.cern.ch/twiki/bin/view/Atlas/WorkBookSetAccount][WorkBookSetAccount]] on how to setup your Athena account for a particular release.
This will setup your release and workarea pointed to by $TestArea environment variable. You should also have $CMTROOT setup properly and $CMTPATH set to
include your working area. You have to check to make sure these are set properly.

---+++!!! Checkout the package from <nop>CVS
Assuming you have setup your release and are using bash shell, execute the following commands. 
Refer to a list [[#TagsAnchor][tags]] for a given Athena release. You will also need the usual 
[[https://twiki.cern.ch/twiki/bin/viewauth/AtlasProtected/AthenaROOTAccess][AthenaROOTAccess]] tags for the 
same release. In the commands below, my top-level working directory is called *work* <br />

<verbatim>
cd $TestArea
work> export CVSROOT=isscvs.cern.ch:/local/reps/atlas
work> export CVS_RSH=ssh
work> cvs co -d ARATopAnalysis -r ARATopAnalysis-xx-yy-zz atlas/groups/Arizona/ARATopAnalysis
</verbatim>
Alternatively, if you are using cmt you can do the following
<verbatim>
cd $TestArea
work> export CMTCVSOFFSET=groups/Arizona
work> cmt co ARATopAnalysis      #from head OR
work> cmt co -r ARATopAnalysis-xx-yy-zz ARATopAnalysis     #specific tag
work> export CMTCVSOFFSET=offline
</verbatim>


---+++!!! Checkout the package from <nop>SVN
<verbatim>
cd $TestArea
export SVNGRP=svn+ssh://svn.cern.ch/reps/atlasgrp/Institutes/Arizona
work> svn co -d ARATopAnalysis -r ARATopAnalysis-xx-yy-zz $SVNGRP/ARATopAnalysis (specific tag)
work> svn co $SVNGRP/ARATopAnalysis/trunk ARATopAnalysis  (from trunk [HEAD])
</verbatim>

Alternatively, if you are using cmt you need to use the following version when setting up cmt
<verbatim>
cmthome> source /afs/cern.ch/sw/contrib/CMT/v1r20p20090520/mgr/setup.sh
cmthome> cmt config
</verbatim> 

Now you can do the usual setup of Athena and then:
<verbatim>
cd $TestArea
work> export SVNROOT=svn+ssh://svn.cern.ch/reps/atlasgrp/Institutes/Arizona
work> cmt co ARATopAnalysis      #from head OR
work> cmt co -r ARATopAnalysis-xx-yy-zz ARATopAnalysis     #specific tag
work> export SVNROOT=svn+ssh://svn.cern.ch/reps/atlasoff
</verbatim>

---+++!!! Some useful SVN commands
---+++!!! Updating, adding/removing files to a package
<verbatim>
work> cd $TestArea/ARATopAnalysis
ARATopAnalysis> export SVNGRP=svn+ssh://svn.cern.ch/reps/atlasgrp
ARATopAnalysis> svn add ARATopAnalysis/TopContainer.h  (add)
ARATopAnalysis> svn rm ARATopAnalysis/TruthContainer.h  (remove) 
ARATopAnalysis> svn update  (update) 
ARATopAnalysis> svn ci -m "See ChangeLog"  (commit) 
ARATopAnalysis> svn cp . $SVNGRP/Institutes/Arizona/ARATopAnalysis/tags/ARATopAnalysis-00-02-00  (tag)
</verbatim>

---+++!!! Creating and importing a package (first time)
<verbatim>
MyDir> ls 
HLTAnalysis

MyDir> svn ls $SVNGRP/Institutes/Arizona/AZAnalysis
MyDir> svn mkdir $SVNGRP/Institutes/Arizona/HLTAnalysis  -m "creating new package"
MyDir> svn mkdir $SVNGRP/Institutes/Arizona/HLTAnalysis/trunk -m "adding trunk"
MyDir> svn mkdir $SVNGRP/Institutes/Arizona/HLTAnalysis/tags -m "adding tags"
MyDir> svn mkdir $SVNGRP/Institutes/Arizona/HLTAnalysis/branches -m "adding branches"
MyDir> cd HLTAnalysis/
HLTAnalysis> ls 
ChangeLog  HLTAnalysis  cmt  config  exec  src

HLTAnalysis> svn import -m "HLTAnalysis first import" . $SVNGRP/Institutes/Arizona/HLTAnalysis/trunk  (importing first time)

</verbatim>

---+++!!! Prepare a configuration file
The input data can be in the form of a text file. An example configuration file: ARATopAnalysis/test/config/testARA.config
is provided. It reads a set of input parameters from the user. In the example provided, the reconstructed muon branch
"StacoMuonCollection" branch is accessed and for all events with one or more muons, the %$p_T$% spectrum and multiplicity
are plotted. Here, you can either create a soft-link to that config file or set an environment variable that points to the file. The
only requirement is that the configuration file that the program picks up is named *analysis.config*. If there are more than one
analysis.config files in the current search path (starting from your work area) the last modified one is picked up by the program.
 
<verbatim>
work> cat ARATopAnalysis/test/config/testARA.config
##############  myTestARA Configuration ################

# This is a comment line -- will be ignored

#set message level
myTestARA.DebugLevel: INFO

#select muons from this branch
myTestARA.MuonsFromBranch: StacoMuonCollection
myTestARA.ElectronsFromBranch: ElectronAODCollection

## End of configuration
</verbatim>
There are two options: either you can create a symbolic link to the configuration file
or set an environment variable ANSLYSIS_CONFIG pointing to it.
<verbatim>
work> ln -s ARATopAnalysis/test/config/testARA.config analysis.config
                           OR equivalently
work> export ANALYSIS_CONFIG=ARATopAnalysis/test/config/testARA.config
</verbatim>

---+++!!! Compile and run a test
Almost all of the requirements for this package are similar to <nop>_UserAnalysis_ package. 
In order to compile, do the following.

<verbatim>
work> cd ARATopAnalysis/cmt
cmt> cmt config
cmt> source setup.sh
cmt> cmt bro gmake
cmt> cd $TestArea
</verbatim>
Alternatively you can use the script provided which will compile all packages in $TestArea
especially if you have a large number of packages checked out and need to compile all
of them. This script uses *cmt broadcast gmake* and checks to make sure all packages compile
correctly.
<verbatim>
cd $TestArea
work> ln -s ARATopAnalysis/share/compileARA.sh
work> ./compileARA.sh -f         # without prompting the user OR
work> ./compileARA.sh      # user is prompted before compiling each package
</verbatim>


This will create a directory $TestArea/InstallArea with links to the executables, libraries etc. required to run the example.
If everything goes well during compilation, you should be able to run the test program called _runTest.exe_
In order to test, you will need a set of input files (AOD's or ESD's) in some location. The test program reads a list of
input files (listed in a text file, one per line) and creates a transient tree (or chain of trees). It then loops through (N > 0)
events specified at the command line (or alternatively over all events if you specify -1 from command line).
You can get a sample AOD/ESD file and specify its location in a text file. 

<verbatim>
work> ls ./ARATopAnalysis/$CMTCONFIG/runTest.exe
work> runTest.exe
--> Usage: runTest.exe <nEvents> <sampleName>
work> ls /path/to/some.AOD.pool.root.1 > myList.ext
work> cat myList.txt
/path/to/some/AOD.pool.root.1
work> runTest.exe 10 myList.txt
</verbatim>
---+++!!! Program Output
The user should see the following output: %BR%
%GREEN%
 INFO  runTest.exe        : Initializing ARA transient tree. Could take a while, please wait...
%ENDCOLOR%
%BLACK%
---------------------------------------------------------------------------------------------------
--> reading 1 files from /path/to%BR%
--> some.AOD.pool.root.1%BR%
type/key :  HLT / <nop>TrigRoiDescriptorCollection  to be aliased to:  HLT_Roi%BR%
type/key :  HLT_TrigT2CaloEgamma / <nop>TrigRoiDescriptorCollection  to be aliased to:  HLT_TrigT2CaloEgamma_Roi%BR%
type/key :  HLT_TrigT2CaloJet / <nop>TrigRoiDescriptorCollection  to be aliased to:  HLT_TrigT2CaloJet_Roi%BR%
type/key :  HLT_TrigT2CaloTau / <nop>TrigRoiDescriptorCollection  to be aliased to:  HLT_TrigT2CaloTau_Roi%BR%
type/key :  HLT / <nop>MuonFeatureContainer  to be aliased to:  HLT_Mu%BR%
type/key :  HLT / <nop>IsoMuonFeatureContainer  to be aliased to:  HLT_IsoMu%BR%
type/key :  HLT / <nop>CombinedMuonFeatureContainer  to be aliased to:  HLT_CombMu%BR%
type/key :  HLT_TrigT2CaloJet / TrigT2JetContainer  to be aliased to:  HLT_TrigT2CaloJet_T2Jet%BR%
type/key :  HLT / <nop>TrigTauContainer  to be aliased to:  HLT_tau%BR%
type/key :  HLT / <nop>TileMuFeatureContainer  to be aliased to:  HLT_TileMu%BR%
type/key :  HLT / <nop>TileTrackMuFeatureContainer  to be aliased to:  HLT_TileTrackMu%BR%
type/key :  HLT / <nop>TrigEMClusterContainer  to be aliased to:  HLT_EMCl%BR%
type/key :  HLT_TrigT2CaloEgamma / <nop>TrigEMClusterContainer  to be aliased to:  HLT_TrigT2CaloEgamma_EMCl%BR%
type/key :  HLT_TrigT2CaloTau / <nop>TrigTauClusterContainer  to be aliased to:  HLT_TrigT2CaloTau_TauCl%BR%
type/key :  HLT / <nop>TrigTauTracksInfoCollection  to be aliased to:  HLT_TauTrInfo%BR%
type/key :  HLT / <nop>TrigL2BphysContainer  to be aliased to:  HLT_L2Bphys%BR%
type/key :  HLT / <nop>TrigInDetTrackCollection  to be aliased to:  HLT_InDetTrk%BR%
type/key :  HLT / <nop>CaloClusterContainer  to be aliased to:  HLT_CaloCl%BR%
type/key :  HLT / <nop>CaloShowerContainer  to be aliased to:  HLT_CaloSh%BR%
type/key :  HLT_egamma / <nop>egammaContainer  to be aliased to:  HLT_egamma_eg%BR%
type/key :  HLT / <nop>TrigTrackCountsCollection  to be aliased to:  HLT_TrkCount%BR%
type/key :  HLT / <nop>JetCollection  to be aliased to:  HLT_JetColl%BR%
type/key :  HLT_egamma / <nop>egDetailContainer  to be aliased to:  HLT_egamma_egDet%BR%
<nop>JobIDSvc.Servic...VERBOSE <nop>ServiceLocatorHelper::<nop>createService: found service <nop>JobIDSvc%BR%
<nop>AthenaHitsVector    DEBUG  initialized.%BR%
---------------------------------------------------------------------------------------------------
%ENDCOLOR%
%GREEN%
 INFO  runTest.exe        : ARA Initialization: <nop>RealTime: 21.3409 s      <nop>CpuTime: 21.04 s%BR%
 INFO  runTest.exe        : Looping over events%BR%
 INFO  Config             : Reading configuration file: ./analysis.config%BR%
 INFO  myTestARA          : ----------------- Configuration ------------------------------%BR%
 INFO  myTestARA          : Electrons From Branch : <nop>ElectronAODCollection%BR%
 INFO  myTestARA          : Muons From Branch     : <nop>StacoMuonCollection%BR%
 INFO  myTestARA          : <nop>DebugLevel            : INFO%BR%
 INFO  myTestARA          : --------------------------------------------------------------%BR%
 INFO  myTestARA          : Event #: 0          6 kB%BR%
 INFO  myTestARA          : Event #: 1          2 kB%BR%
 INFO  myTestARA          : Event #: 2          4 kB%BR%
 INFO  myTestARA          : Event #: 3          10 kB%BR%
 INFO  myTestARA          : Event #: 4          7 kB%BR%
 INFO  myTestARA          : Event #: 5          4 kB%BR%
 INFO  myTestARA          : Event #: 6          2 kB%BR%
 INFO  myTestARA          : Event #: 7          10 kB%BR%
 INFO  myTestARA          : Event #: 8          2 kB%BR%
 INFO  myTestARA          : Event #: 9          3 kB%BR%
 INFO  runTest.exe        : <nop>Analysed events: 10%BR%
 INFO  runTest.exe        : <nop>RealTime: 0.405415 seconds.          <nop>CpuTime: 0.09 seconds%BR%
 INFO  runTest.exe        : Execution time: 24.6661 events / <nop>RealTime second.   111.111 events / <nop>CpuTime second.%BR%
%ENDCOLOR%
---------------------------------------------------------------------------------------------------
%BR%
It will also create a ROOT file called *myTestARA.hists.root* in the working area, which contains two histograms - the muon multiplicity
and muon %$p_T$% histograms. The histograms obtained by running this example are shown below. %BR%
<br />
| *m_muon_all_n* | *m_muon_pt* | 
|<img alt="m_muon_all_n.gif" src="%ATTACHURLPATH%/m_muon_all_n.gif" style="width: 363px; height: 244px;" />|<img alt="m_muon_pt.gif" src="%ATTACHURLPATH%/m_muon_pt.gif" style="width: 363px; height: 244px;" /> |
%BR%
---------------------------------------------------------------------------------------------------%BR%

%BEGINLATEX{ engine="ps"}%
\begin{table}
\begin{tabular}{|c|c|}
a & b \\
1 & 2 \\
\hline
\end{tabular}
\end{table}
%ENDLATEX%


---++ Example 3. Running on the grid
---+++!! Submitting a grid job using prun
[[https://twiki.cern.ch/twiki/bin/view/Atlas/PandaRun][PandaRun]] twiki describes how to submit general
jobs on the grid. The user has options of submitting ROOT (CINT, C++, pyROOT), ARA, Python, user executables
or even shell scripts. Using the flexibility it offers, we can choose to submit a general job running a user executable.
In our case, the executable is called gridrunARA.exe. Its a "grid-aware" version of the test program which was explained
in example 1. The submission command is shown below. Before you issue *prun* you must make sure you have a 
valid membership of a VO (Virtual Organization ATLAS) and obtain and install the <nop>PandaRun package. When you
specify output dataset (which will hold all your output) its required of you to follow some convention : userNN.FirstNameLastName.XYZ
where NN is the last two digits of the current year, First-, Last- names of the user followed by a description of your output data.
For more options refer the [[https://twiki.cern.ch/twiki/bin/view/Atlas/DAonPanda#example_1_evgen][examples]].
<verbatim>
work> prun --exec "source gridcompileARA.sh 14.4.0 AtlasOffline; echo %IN | gridrunARA.exe" \
--athenaTag 14.4.0 --inDS mc08.105200.T1_McAtNlo_Jimmy.recon.AOD.e357_s462_r579 \
--outDS user09.VenkateshKaushik.try25 --nFiles 5 --cloud FR --outputs "my*.root"
</verbatim>
%BR%

---+++!!! Checking Job Status
---++++!!! Using the command line interface
Once you issue the above command, you will see the following output: %BR%
%GREEN%
query files in dataset:mc08.105200.T1_McAtNlo_Jimmy.recon.AOD.e357_s462_r579%BR%
gathering files under /raid01/venkat/testarea/TopAnalysis/14.5.0/TrigARA%BR%
upload sources%BR%
submit%BR%
===================%BR%
 <nop>JobID  : 31%BR%
 Status : 0%BR%
  > build%BR%
    <nop>PandaID=23480737%BR%
  > run%BR%
    <nop>PandaID=23480738%BR%
%ENDCOLOR%
-----------------------------------------------------------------------------------
In order to check the status of your job, you can either use web interface or the command line
interface. Using the command line interface involves issuing the *pbook* which is a book-keeping
tool along with *show* command with the jobID. 
<verbatim>
work> pbook
</verbatim>
Any general job is split into two parts. The "build" stage and the "run" stage that follows it.
If the build-job fails, the run-job is killed. Once submitted, you will see the status of the
build-job shows "running", whereas the run-job shows "defined". You might see a job status
set to *%YELLOW%holding%ENDCOLOR%* between different stages. Once the job is finished the status is changed to *%GREEN%finished%ENDCOLOR%*.
%BR%
>>> show(31)%BR%
      ...%BR%
      ...%BR%
      *jobStatus : running%BR%*
            *defined : 1%BR%*
            *holding : 1*
-----------------------------------------------------------------------------
>>> show(31) %BR%
      ...%BR%
      ...%BR%
      *jobStatus : running%BR%*
           *finished : 1%BR%*
            *running : 1*
-----------------------------------------------------------------------------
>>> show(31)%BR%
INFO : Getting status for <nop>JobID=31 ...%BR%
INFO : Updated <nop>JobID=31%BR%
======================================%BR%
          <nop>JobID : 31%BR%
           type : prun%BR%
        <nop>PandaID : 23480737-23480738%BR%
          nJobs : 1 + 1(build)%BR%
           site : ANALY_TOKYO%BR%
          cloud : FR%BR%
           inDS : mc08.105200.T1_McAtNlo_Jimmy.recon.AOD.e357_s462_r579%BR%
          outDS : user09.VenkateshKaushik.try25%BR%
          libDS : user09.VenkateshKaushik.lib._1232292662.66.lib.tgz%BR%
        retryID : 0%BR%
   provenanceID : 0%BR%
   creationTime : 2009-01-18 15:31:05%BR%
     lastUpdate : 2009-01-18 16:28:00%BR%
         params : --exec "source gridcompileARA.sh 14.4.0 AtlasOffline; echo %IN gridrunARA.exe" --athenaTag 14.4.0 --inDS mc08.105200.T1_McAtNlo_Jimmy.recon.AOD.e357_s462_r579 --outDS user09.VenkateshKaushik.try25 --nFiles 5 --cloud FR --outputs "my*.root"%BR%
      *jobStatus : frozen%BR%*
           *finished : 2%BR%*
--------------------------------------------------------------------------------
---++++!!! Using the web interface
The [[http://panda.cern.ch/][panda monitor]] web interface can be used to find out the status of jobs. 
The [[https://twiki.cern.ch/twiki/bin/view/Atlas/DAonPanda#5_Bookkeeping_Retry][book-keeping]] 
and the [[https://twiki.cern.ch/twiki/bin/view/Atlas/DAonPanda#6_How_to_debug_jobs_when_they_fa][debug-jobs]]
sections are especially useful. Given below are the screen-shots of the different stages of job execution for the user
analysis job above. Once the job finishes (or fails) an e-mail message will be sent to the user notifying the completion
of the job.
%BR%

--------------------------------------------------------------------------------
*Build Stage:* %BR%
<img alt="runningbuild.jpg" src="%ATTACHURLPATH%/runningbuild.jpg" style="width: 676px; height: 279px;" />
%BR%
--------------------------------------------------------------------------------

*Run Stage:* %BR%
<img alt="runningjob.jpg" src="%ATTACHURLPATH%/runningjob.jpg" style="width: 676px; height: 279px;" />
%BR%
--------------------------------------------------------------------------------

*Finished stage:* %BR%
<img alt="runfinished.jpg" src="%ATTACHURLPATH%/runfinished.jpg" style="width: 676px; height: 279px;" />
%BR%
--------------------------------------------------------------------------------

---++!! Bugs/Known Issues
   * TriggerDecisionTool cannot find valid IOV for a given trigger configuration - (mostly harmless)
<verbatim>
Did not find valid IOV for _TRIGGER_LVL1_Menu
Did not find valid IOV for _TRIGGER_HLT_Menu
Did not find valid IOV for _TRIGGER_LVL1_Lvl1ConfigKey
Did not find valid IOV for _TRIGGER_HLT_HltConfigKeys
Did not find valid IOV for _TRIGGER_LVL1_Prescales
TrigDecisionToo...  FATAL ERROR: Loading of new trigger configuration for run/lb = 0/0 failed
</verbatim>

---++!! <nop>CMSsandbox.ToDo List

%GREEN%   
   * Use [[http://www-d0.fnal.gov/results/publications_talks/thesis/snyder/html/thesis.html][HitFit]] for constrained fit and compute    %Y%
   * Single muon trigger efficiency from Z sample  %Y%
   * Use trigger decision tool instead of trigger bit decoder %Y%
   * Add provision for storing JES correction (nominal, plus minus) jets %Y%
   * Add Ntuple making capability -- Thanks Caleb!
   * Cleanup access to <nop>ObjCollection -- Thanks Caleb!
      * Removed <nop>ObjCollection and <nop>EventData classes. 
      * Replaced with <nop>DataStore to store user/<nop>ARATree objects

%ENDCOLOR%
%RED%
   * Add muon isolation study    
   * Muon+Jet trigger efficiency
   * Estimate QCD from template / matrix method
   * Add TauContainer to user objects
%ENDCOLOR%


%BR%
<!--***********************************************************--> <!--Do NOT remove the remaining lines, but add requested info as appropriate--> <!--***********************************************************-->
Responsible: [[mailto:Venkat.Kaushik@cernSPAMNOT.ch][Venkat Kaushik]] %BR%

Major updates:
--Main.VenkateshKaushik 16 Jan 2009

<!--%REVINFO{"Main.VenkateshKaushik" rev="1.1"}% -->
%BR% <!--Once this page has been reviewed, please add the name and the date e.g. Main.VenkateshKaushik - 16 Jan 2009 -->
Review: *Never reviewed*

%STOPINCLUDE%
Topic revision: r31 - 2010-12-08 - VenkateshKaushik
Account
- Log In
- Cern Search
- TWiki Search
- Google Search
Sandbox All webs
Edit
Attach
Copyright &© 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback