Ryan Reece's Sandbox

Athena Notes

Here you will find notes from my experiences computing in ATLAS. You can find some more notes on ROOT, python, pyROOT, and makefiles at my website: http://www.hep.upenn.edu/~rreece/computing.html.

Other places to find athena help are:

Using Athena at BNL

You will need a computing account at BNL and you will have to create a register a public key. Following the advice of the USATLAS TWiki, my cmthome/requirements file at BNL is
macro PROJ_RELEASE   "latest" \
         11.3.0         "11.3.0" \
         12.0.6         "12.0.6" \
         12.0.7         "12.0.7" \
         13.0.10        "13.0.10" \
         13.0.20        "13.0.20" \
         13.0.25        "13.0.25" \
         13.0.26        "13.0.26" \
         13.0.28        "13.0.28" \
         13.0.30        "13.0.30" \
         13.1.0         "13.1.0" \
         13.2.0         "13.2.0" \
         rel_0          "rel_0" \
         rel_1          "rel_1" \
         rel_2          "rel_2" \
         rel_3          "rel_3" \
         rel_4          "rel_4" \
         rel_5          "rel_5" \
         rel_6          "rel_6"

macro PROJ_BASE_RELEASE   "$(PROJ_RELEASE)" \      "13.0.25" \      "13.0.25" \      "13.0.25" \      "13.0.25" \      "13.0.25" \      "13.0.25" \      "13.0.25" \      "13.0.25"

         bugfix      "bugfix/$(PROJ_BASE_RELEASE)" \
         dev         "dev/$(PROJ_BASE_RELEASE)"

set SITEROOT /opt/usatlas/kit_rel/${PROJ_SUBDIR}
macro ATLAS_GROUP_AREA "/afs/cern.ch/atlas/groups/PAT/Tutorial/EventViewGroupArea/EVTags-"
apply_tag oneTest
apply_tag setupCMT
apply_tag setup
apply_tag noCVSROOT
apply_tag 32
use AtlasLogin AtlasLogin-* $(ATLAS_DIST_AREA)
set PATHENA_GRID_SETUP_SH /afs/usatlas.bnl.gov/lcg/current/etc/profile.d/grid_env.sh
macro setup_slc3compat "" \
      gcc323  "/opt/usatlas/kit_rel/SLC3/setup_slc3compat"
setup_script $(setup_slc3compat)
I have added the ATLAS_GROUP_AREA line in order to use EventView. You should also set the CVSROOT in your .bashrc file so that you can check out packages.
export CVSROOT=/afs/usatlas.bnl.gov/software/cvs
echo "CVSROOT set for BNL to " $CVSROOT

I find it convenient when setting up athena and running jobs, to define the following functions in my .bashrc file:

function setup {
    source ${HOME}/cmthome/setup.sh -tag=$*
function run {
    nohup athena.py $* &> $1.log &
    tail -f $1.log
Using the above to set up athena, all I have to do is run setup follow by the comma separated tags, e.g.
>> setup 13.0.30,groupArea

And to run an athena job, all I do is

>> run example_jobOptions.py
This creates a log file of the athena ouput, example_jobOptions.py.log and prints it to the screen real time using tail -f.

In general, refer to the Computing Workbook for learning how to setup and run athena.

Back to top

Checking Out a Package

After setting up athena, go to the appropriate place to check out packages, your testarea.
>> cd $TestArea
>> pwd

First, you must find which package tag is appropriate for your version of Athena.

>> cmt show versions PhysicsAnalysis/AnalysisCommon/UserAnalysis
PhysicsAnalysis/AnalysisCommon/UserAnalysis UserAnalysis-00-10-12 /opt/usatlas/kit_rel/13.0.30/AtlasAnalysis/13.0.30
Alternatively, you can see a list of all the possible tags for the package by running:
>> get_tag PhysicsAnalysis/AnalysisCommon/UserAnalysis

To checkout the package with this tag do:

>> cmt co -r UserAnalysis-00-10-12 PhysicsAnalysis/AnalysisCommon/UserAnalysis

After checkout, go inside the package and see what's there.

>> cd PhysicsAnalysis/AnalysisCommon/UserAnalysis/
>> ls
cmt/  CVS/  doc/  python/  Root/  run/  share/  src/  UserAnalysis/
Every pacakge has the following directories:
  • cmt - where you configure and build the package
  • share - contains python job options
  • src - contains the C++ sources
  • PackageName - contains the C++ headers
  • run - directory for you to run jobs. a place to hold your job options and data.

To configure and make the package do

>> cd cmt
>> cmt config
>> source setup.sh
>> gmake

Back to top

Using DQ2 to Get Datasets

You can browse for datasets on the web at the DQ2 Browser.

The following describes how to setup DQ2 at BNL. This requires that you have a grid certificate. Create the file dq2_setup.sh containing:

source /afs/usatlas.bnl.gov/lcg/current/etc/profile.d/grid_env.sh
source /afs/usatlas.bnl.gov/Grid/Don-Quijote/dq2_user_client/setup.sh.BNL
Now source this script:
>> source dq2_setup.sh
This will prompt you for the password for your grid certificate. Now you can use the DQ2 commmands. For example:
>> dq2_ls trig1_misal1_mc12.005144.PythiaZee.recon.AOD.*
To copy locally all the files in one of these data sets, do:
>> dq2_get trig1_misal1_mc12.005144.PythiaZee.recon.AOD.v12000604_tid008230

Back to top

Grid Submission

In order to submit athena jobs to the computing grid, you need to checkout pathena:

>> cd $TestArea
>> cmt co PhysicsAnalysis/DistributedAnalysis/PandaTools
>> cd PhysicsAnalysis/DistributedAnalysis/PandaTools/cmt
>> cmt config
>> source setup.sh
>> gmake
To submit a job, do:
>> pathena ZeeZmmOnAODExample_jobOptions.py
    --outDS user.RyanDReece.005144.AnalysisExampleZee.01
    --inDS trig1_misal1_mc12.005144.PythiaZee.recon.AOD.v12000604_tid008230
  • ZeeZmmOnAODExample_jobOptions.py is some job option file.
  • --outDS precedes your specified output dataset that your job will create. Its name must have the format of user...
  • --inDS is optional. It specifies the input dataset to use for your job. If given, it overides any EventSelector.InputCollections in your job option file.

Following this command, you will be prompted for the password for your grid certificate. Then your job's IDs will be printed.

There are two ways to monitor the status of your jobs. First, you can run pathen_util:

>> pathena_util
>>> show()
will print a list of the jobs you have submitted. You can specify a specific job ID to get more information:
>>> show(3)
JobID   : 3
time    : 2007-11-19 16:14:23
inDS    : trig1_misal1_mc12.005144.PythiaZee.recon.AOD.v12000604_tid008230
outDS   : user.RyanDReece.005144.AnalysisExampleZee.01
libDS   : user.RyanDReece.acas0003_5.lib._000003
build   : 4668038
run     : 4668039
jobO    : ZeeZmmOnAODExample_jobOptions.py
site    : ANALY_BNL_ATLAS_1
buildJob   : ----
runAthena  :
          total : 1
      succeeded : 0
         failed : 0
        running : 1
        unknown : 0

To quit pathena_util, type CTRL-D.

Second, you can visit your user page on the Panda Monitor website. You will receive an email you notifying you when your job has finished. Then you can use DQ2 to retrieve the dataset your job created:

>> dq2_get -rv user.RyanDReece.005144.AnalysisExampleZee.01

Back to top

Writing Your Own Athena Algorithms

In writing your own athena algorithm, you must write a C++ class that inherits from the algorithm class. Like all C++ classes, it will need a header file with a declaration and a source file with implementation of the algorithm. They should be named MyAlg.h and MyAlg.cxx, where MyAlg is whatever you want to name your algorithm. The header file goes in the PackageName/PackageName/ directory and the source file goes in the PackageName/src/ directory.

The Header File

The following is a template for the header file:

#ifndef MYALG_H
#define MYALG_H

#include "GaudiKernel/ToolHandle.h"
#include "GaudiKernel/Algorithm.h"
#include "GaudiKernel/ObjectVector.h"
#include "GaudiKernel/ITHistSvc.h"
#include "CLHEP/Units/SystemOfUnits.h"
#include "StoreGate/StoreGateSvc.h"
#include "AnalysisTools/AnalysisTools.h"

#include "egammaEvent/ElectronContainer.h"
#include "egammaEvent/Electron.h"

#include <string>
#include "TTree.h"

class MyAlg : public Algorithm
        MyAlg(const std::string& name, ISvcLocator* pSvcLocator);

        StatusCode initialize();
        StatusCode finalize();
        StatusCode execute();


//      methods
        bool selectElectron(const Analysis::Electron& elect) const;

//      data members
        ToolHandle<AnalysisTools> m_analysisTools;

        StoreGateSvc* m_storeGate;
        ITHistSvc * m_histSvc;
        TTree* m_tree;
        int m_numEvents;
        float m_electron_Et;

//      configurable data members
        float m_electron_Et_min_cut;
        float m_electron_abs_eta_cut;
#endif // MYALG_H

The first group of includes should be in every algorithm. The egammaEvent includes are examples of what you might need to read the data in an AOD. Every athena algorithm has to have a constructor of the form MyAlg(const std::string& name, ISvcLocator* pSvcLocator) and initialize(), finalize(), and execute() methods. selectElectron is an example of a custom method you can write yourself. The following data members are just examples of things an algorithm might use, but are not necessary. We will see what makes m_electron_Et_min_cut and m_electron_abs_eta_cut "configurable" in the source file.

Back to top

The Source File

The following is a template for the source file:

#include "GaudiKernel/MsgStream.h"
#include "GaudiKernel/AlgFactory.h"
#include "GaudiKernel/IToolSvc.h"
#include "StoreGate/StoreGateSvc.h"
#include "StoreGate/DataHandle.h"
#include "egammaEvent/Electron.h" // the Electron
#include "Navigation/NavigationToken.h" // Constituent navigation
#include "ParticleEvent/ParticleBaseContainer.h" // common implementation of all particles
// #include "CompositeParticleEvent/CompositeParticle.h" // the composite particle
#include "AnalysisUtils/AnalysisCombination.h" // analysis tools

#include "PackageName/MyAlg.h"// the header file

#include <stdint.h>
#include <algorithm>
#include <math.h>

using namespace Analysis;

/// Constructor
MyAlg::MyAlg(const std::string& name,
  ISvcLocator* pSvcLocator) : Algorithm(name, pSvcLocator),
  m_analysisTools( "AnalysisTools", this )
    m_storeGate = NULL;
    m_histSvc = NULL;
    m_tree = NULL;
    m_numEvents = 0;
    m_numEventsPassed = 0;
    m_electron_Et = 0.0;

    declareProperty("electron_Et_min_cut", m_electron_Et_min_cut = 20*GeV);
    declareProperty("electron_abs_eta_cut", m_electron_abs_eta_cut = 2.0);

/// Destructor
MyAlg::~MyAlg() {}

/// Initialize
StatusCode MyAlg::initialize()
    MsgStream mLog( messageService(), name() );
    mLog << MSG::DEBUG << "Initializing MyAlg" << endreq;

    // Get handle StoreGate
    StatusCode sc = service("StoreGateSvc", m_storeGate);
        mLog << MSG::ERROR
             << "Unable to retrieve pointer to StoreGate service."
             << endreq;

    // Get a handle on the NTuple and histogramming service
    sc = service("THistSvc", m_histSvc);
        mLog << MSG::ERROR
             << "Unable to retrieve pointer to THistSvc"
             << endreq;
        return sc;

    // Create TTree and register it to THistSvc
    m_tree = new TTree("MyTree" , "MyTree");
    std::string fullTreeName =  "/AANT/MyTree" ;
    sc = m_histSvc->regTree(fullTreeName, m_tree);
        mLog << MSG::ERROR << "Unable to register TTree : " << fullTreeName << endreq;
        return sc;

    // Create TTree branches.
    m_tree->Branch("electron_Et", &m_electron_Et, "electron_Et/F");

    return StatusCode::SUCCESS;

/// Finalize
StatusCode MyAlg::finalize()
{ }

/// Execute - called by the event loop on event by event
StatusCode MyAlg::execute()
    [Code called event by event here]
    return StatusCode::SUCCESS;

Notice the calls of the declareProperty("electron_Et_min_cut", m_electron_Et_min_cut = 20*GeV) in the constructor. This makes the C++ m_electron_Et_min_cut variable configurable from the Python job options. The first argument is a string containing the Python name of the variable. The second is the C++ variable with an optional equals-sign followed by a default value. Configuration from Python will be explained more later when we get to the job options.

This also shows how to use the Message Service to print to the screen and how to create TTrees in a root file. All that is missing is the assignment of variables in branches and calling Fill() in the execute() function.

Back to top


In order to make your algorithm configurable in Python job options, there is one last thing to do. Every package has a PackageName/src/components/PackageName_entries.cxx file. In it, there is some code that tells cmt to create the appropriate Python modules for the algorithms in the package when you make the package.

#include "PackageName/MyAlg.h"
#include "PackageName/AnotherAlg.h"



For each algorithm in your package, you need to include the header file, add a DECLARE_ALGORITHM_FACTORY line, and add a DECLARE_ALGORITHM line like above. After editing this file, you can now compile your code. Go to the PackageName/cmt/ directory and run

>> cmt config
>> source setup.sh
>> gmake

Back to top

The Job Options

Here are some example Python job options needed to run a job with your new algorithm.

from AthenaCommon.Constants import *
from AthenaCommon.AppMgr import theApp
from AthenaCommon.AppMgr import ServiceMgr
import AthenaPoolCnvSvc.ReadAthenaPool

# Input Dataset
ServiceMgr.EventSelector.InputCollections = [
theApp.EvtMax = -1 # -1 means all events

# Message Service
# Set output level threshold (2=DEBUG, 3=INFO, 4=WARNING, 5=ERROR, 6=FATAL )
ServiceMgr.MessageSvc.OutputLevel = WARNING

# Algorithms
from AthenaCommon.AlgSequence import AlgSequence
theJob = AlgSequence()
from PackageName.PackageNameConf import MyAlg
theJob += MyAlg(OutputLevel = INFO)

# Histogram and Tree Service
THistSvc = Service ( "THistSvc" )
THistSvc.Output = ["AANT DATAFILE='MyAlg.AAN.root' OPT='RECREATE'"]
THistSvc.OutputLevel = INFO

print theJob

Notice the following lines. One gets a handle on the top algorithm sequence by

from AthenaCommon.AlgSequence import AlgSequence
theJob = AlgSequence()

Then one adds algorithms to the sequence by

from PackageName.PackageNameConf import MyAlg
theJob += MyAlg(OutputLevel = INFO)

Now, using the bash function I defined above, you can run your athena job by

>> run example_jobOptions.py

Back to top


In my notes above on writing your own athena algorithms, the example code shows the use of the Message Service to print to the terminal and the THistSvc to make a TTree saved in a ROOT file. In the next two sections, I explain how to use of these services in more detail.

Message Service

You can use the Message Service to print text in the terminal. In the C++ code for your algorithm, you must #include "GaudiKernel/MsgStream.h". Then define an instance of the MsgStream by
MsgStream mLog( msgSvc(), name() );
Now you can use it to print to the terminal by
mLog << MSG::INFO << "Hello World!" << endreq;
where INFO specifies the Output Level of the message. It can be one of the following 2=DEBUG, 3=INFO, 4=WARNING, 5=ERROR, or 6=FATAL. If the message's Output Level is not above the Output Level specified in the job options, then the message will be suppressed.

Now in your job options, you must have the line

from AthenaCommon.AppMgr import ServiceMgr
To specify the global Output Level in job options, do
ServiceMgr.MessageSvc.OutputLevel = WARNING
This will suppress output of all message with output level INFO or lower. (Note the difference in syntax between msgSvc() in the C++ code and MessageSvc in the Python job options.) One can specify that a specific algorithm have an Output Level differing from the global Output Level by setting the OutputLevel member variable of that algorithm.
someAlg.OutputLevel = INFO
Alternatively, one can set the Output Level when declaring an instance of the algorithm, as is done in the job options above.
theJob += MyAlg(OutputLevel = INFO)

Back to top



Back to top



Back to top


Here I test out using LaTeX in a TWiki.

 We next turn our attention studying the solutions of the Klein-Gordon equation. Consider the following. \begin{equation}     \label{eq:fk}     f_{k}(x) \equiv \frac{1}{\sqrt{(2 \pi)^3 \: 2 \omega_k}} \: e^{- i \: k \cdot x} \end{equation} where  \begin{equation}     k^\mu \equiv \left(\omega_k, \vec{k}\right)^\mu \end{equation} and \begin{equation}     \label{eq:omega-k}     \omega_k \equiv + \: \sqrt{\left.\vec{k}\right.^2 + m^2} \end{equation} Plugging in the \textbf{plane-waves} $f_{k}(x)$ in for $\phi(x)$ shows that they are solutions. The functions $f_{k}(x)$ form a complete basis for a complex function space. The appropriate inner product in this space involves the following operation. \begin{equation}     \label{eq:arrow-derivative} a \stackrel{\leftrightarrow}{\partial_0} b \equiv a \: \partial_0 b - \left(\partial_0 a\right) b \end{equation} The completeness relation is derived by pluggin equations (\ref{eq:fk}) and (\ref{eq:arrow-derivative}) into the following. \begin{eqnarray*}     \int  d^3x \: f^*_{k'}(x) \: i \stackrel{\leftrightarrow}{\partial_0} f_k(x)     &amp;=&amp; \frac{i}{2 \: \sqrt{\omega_k \: \omega_{k'}}} \: \frac{1}{(2\pi)^3}         \int d^3x \left[ e^{i \: k' \cdot x} \: \partial_0 e^{- i \: k \cdot x}         - \left(\partial_0 e^{i \: k' \cdot x}\right) e^{- i \: k \cdot x}\right]\\     &amp;=&amp; \frac{i}{2 \: \sqrt{\omega_k \: \omega_{k'}}} \: \frac{1}{(2 \pi)^3}         \int d^3x \: (-i) \: \left(\omega_k + \omega_{k'}\right) e^{i (k' - k) \cdot x}\\     &amp;=&amp; \frac{\omega_k + \omega_{k'}}{2 \: \sqrt{\omega_k \: \omega_{k'}}} \:         e^{i(\omega_{k'} - \omega_k) t} \:         \underbrace{\frac{1}{(2 \pi)^3} \int d^3x \: e^{- i \left(\vec{k'} - \vec{k}\right) \cdot \vec{x}}}_{             \delta^3 \left(\vec{k} - \vec{k'}\right)}\\     &amp;=&amp; \frac{2 \: \omega_k}{2 \: \omega_k} \: e^{0} \: \delta^3\big(\vec{k} - \vec{k'}\big) \end{eqnarray*} \begin{equation} \therefore \qquad \int d^3x \: f^*_{k'}(x) \: i \stackrel{\leftrightarrow}{\partial_0} f_k(x)     = \delta^3\big(\vec{k} - \vec{k'}\big) \end{equation} Similary, the orthoginality relation is derived by \begin{eqnarray*}     \int  d^3x \: f_{k'}(x) \: i \stackrel{\leftrightarrow}{\partial_0} f_k(x)     &amp;=&amp; \frac{i}{2 \: \sqrt{\omega_k \: \omega_{k'}}} \: \frac{1}{(2\pi)^3}         \int d^3x \left[ e^{- i \: k' \cdot x} \: \partial_0 e^{- i \: k \cdot x}         - \left(\partial_0 e^{- i \: k' \cdot x}\right) e^{- i \: k \cdot x}\right]\\     &amp;=&amp; \frac{i}{2 \: \sqrt{\omega_k \: \omega_{k'}}} \: \frac{1}{(2 \pi)^3}         \int d^3x \: (-i) \: \left(\omega_k - \omega_{k'}\right) e^{- i (k' + k) \cdot x}\\     &amp;=&amp; \frac{\omega_k - \omega_{k'}}{2 \: \sqrt{\omega_k \: \omega_{k'}}} \:         e^{- i (\omega_{k'} + \omega_k) t} \:         \underbrace{\frac{1}{(2 \pi)^3} \int d^3x \: e^{i \left(\vec{k'} + \vec{k}\right) \cdot \vec{x}}}_{             \delta^3 \left(\vec{k} + \vec{k'}\right)}\\     &amp;=&amp; \cancelto{0}{\frac{\omega_k - \omega_k}{2 \: \omega_k}} \: e^{-i \:2\:\omega_k} \: \delta^3\big(\vec{k} + \vec{k'}\big) \end{eqnarray*} \begin{equation} \therefore \qquad \int d^3x \: f_{k'}(x) \: i \stackrel{\leftrightarrow}{\partial_0} f_k(x) = 0 \end{equation}

Back to top

Major updates:
-- RyanReece - 21 Dec 2007
-- RyanReece - 05 Jan 2008
Latex rendering error!! dvi file was not created.
Edit | Attach | Watch | Print version | History: r56 | r15 < r14 < r13 < r12 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r13 - 2008-01-06 - RyanReece
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Sandbox All webs login

  • Edit
  • Attach
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback