-- SebastianOlivares - 25-May-2010


The Basics

Setting up your account

It is assumed that you have an account on LXPLUS already, and have about 100MB of spare quota. You can check this by typing;
fs lq
the result is given in kilobytes.

The first thing we have to do is entry to lxplus, in this case we will specify at the end of the command control keys to access to graphical modes (for root framework and emacs). In this tutorial we need computers with SL4, we don't have to specify a SL4 machine because lxplus connect to this kind of O.S by default.

ssh yourlogin@lxplus.cern.ch -X -Y

Setting up CMT

This is intended for people who doesn't have a cmthome and want to know the whole process of setting up his account. If you already have done this before and want a quicker way to do this, please go to the express point.

1. Create a cmthome directory in your home directory:

cd $HOME
mkdir cmthome
cd cmthome

2. Make your login requirements file - use your favorite editor to create a file called requirements and paste into it these lines:

set   SITEROOT /afs/cern.ch
macro ATLAS_DIST_AREA ${SITEROOT}/atlas/software/dist
macro ATLAS_TEST_AREA ${HOME}/scratch0/
use AtlasLogin AtlasLogin-* $(ATLAS_DIST_AREA)
Important notes:

* Line 4: change the path after ${HOME} so that it points to wherever you want to work. If you have scratch space it is recommended that you work in there. Of course if you want to make a new directory inside this space, that is fine.

3. Save and exit this file

4. Now set up CMT as follows:

source /afs/cern.ch/sw/contrib/CMT/v1r20p20090520/mgr/setup.sh
cmt config

The first line sets the CMT version. This changes from time to time. When it changes you need to re-do these two lines. Check from time to time that your CMT is compatible with the software release you're using, with the ATLAS release status page. It is particularly important that you use this version now, due to the change to SVN

Note: Steps (1)-(4) above only have to be done once. Step (5) has to be done every time you log into a new session.

5. Now set up the ATLAS software. For this part we'll use release 15.6.9.

cd $HOME
source cmthome/setup.sh -tag=15.6.9,setup,32
(Note that you'll get a complaint about directories not existing - don't worry about this - it will only happen once)
echo $TestArea
mkdir $TestArea

6. Go into your $TestArea, and get ready to run the ATLAS software!

cd $TestArea 

Setting up DQ2

1. Set up the LXPLUS UI:

source /afs/cern.ch/project/gd/LCG-share/current/etc/profile.d/grid_env.sh

(please note that later you won't need to do this step anymore, once you get introduced to the DQ2 framework)

2. Make your proxy certificate (you'll be asked for a password):

voms-proxy-init --voms atlas  

3. To load the environment on lxplus do:

source /afs/cern.ch/atlas/offline/external/GRID/ddm/DQ2Clients/setup.sh
source /afs/cern.ch/atlas/offline/external/GRID/ddm/DQ2Clients/setup.zsh
depending on your shell.

Express Setup

A quicker way to do this is making two requirements files for setting up athena-cmt and DQ2.

1. Create a directory called myscripts in your home directory:

cd $HOME
mkdir myscripts
cd myscripts

2. Make your athena-cmt requirements file - use your favorite editor to create a file called g15.6.9.sh and paste into it these lines:

export ATHENA_VERSION=15.6.9
cd ~/cmthome
cp rg${ATHENA_VERSION} requirements
source /afs/cern.ch/sw/contrib/CMT/v1r20p20090520/mgr/setup.sh
cmt config
source ~/cmthome/setup.sh -tag=${ATHENA_VERSION},setup,32
cmt show path
cd $HOME/myscripts

3. Make your DQ2 requirements file - use your favorite editor to create a file called setdq2.sh and paste into it these lines:

source /afs/cern.ch/project/gd/LCG-share/current/etc/profile.d/grid_env.sh
source /afs/cern.ch/atlas/offline/external/GRID/ddm/DQ2Clients/setup.sh 
voms-proxy-init --voms atlas

4. Run both files:

source g15.6.9.sh
source setdq2.sh
Note: We only have to run the dq2 file when we want to download something from the grid.

TTP packages Installation

Downloading Packages

First we have to look for the packages tags compatibles with our athena release (15.6.9) in this twiki page (on the latest tags section): https://twiki.cern.ch/twiki/bin/view/Atlas/TrigTauPerformNtuple.

1. Go into your $TestArea.

cd $TestArea 

2. For Validation we will download three packages: TrigTauPerformNtuple, TrigTauPerformAnalysis, TrigTauPerformValidation.

cmt co -r TrigTauPerformNtuple-XX-YY-ZZ Trigger/TrigAnalysis/TrigTauAnalysis/TrigTauPerformNtuple
cmt co -r TrigTauPerformAnalysis-XX-YY-ZZ Trigger/TrigAnalysis/TrigTauAnalysis/TrigTauPerformAnalysis
cmt co -r TrigTauPerformValidation-XX-YY-ZZ Trigger/TrigAnalysis/TrigTauAnalysis/TrigTauPerformValidation
* -XX-YY-ZZ: packages tags.

Compiling the packages

1. Compile each package in this order: Ntuple->Analysis->Validation.

cd Trigger/TrigAnalysis/TrigTauAnalysis/TrigTauPerformNtuple/cmt
gmake -f Makefile.standalone
Note: You have to do the same with the other two packages.

2. Create a run directory in the Trigger area (we will use this folder later for the python code).

cd Trigger/TrigAnalysis/TrigTauAnalysis/
mkdir run

Searching Datasets

Datasets are produced with diferent kind of variables and athena releases. When the datasets are done they will have a specific tag, in this tutorial we will use a dataset for athena release 15.6.9.


AMI (ATLAS Metadata Interface) is a generic cataloging framework used in ATLAS for dataset discovery (also a Tag Collector + one or two other things).

1. The first thing we have to do is entry the AMI web page (you need a grid certificate correctly loaded into your browser): http://ami.in2p3.fr:8080/AMI/servlet/net.hep.atlas.Database.Bookkeeping.AMI.Servlet.Command?linkId=512

2. In the first rectangle we can write any option we want to look for, in this case we will search a medium pT signal coming from a Z boson (write: ztautau).

3. On the top of the page (just under Home) you will find a "clickeable" rectangle with the kinds of dataset we can find for search, in this case we will use the mc09-production.

4. On the third column (dataType) click on the Grupby button (left icon) and select Browse only AOD's.

5. We will use a sample with ~10.000 events and with a transformation package higher that, so let's find it.

6. When we already find a good dataset, then you have to click on the details hyperlink on the left.

7. In this web page you can see the details of the dataset we are looking for. We can check the athena version, the number of AOD's in it (40 is a good statistic) and something we have to write for a future step: the Geometry Version (in our case ATLAS-GEO-10-00-00).

8. Click on the DQ2 hyperlink, just down the dataset name.

9. On the right we have three rows. The first one, Existing Replicas, tell us in which sites our dataset is (there have to be at least one for download it). In the third row, Contained datasets, there is a hyperlink to the dataset location, click it.

10. Now we have almost the same page, but the difference is that now it appears a new button: dq2-get, click it. This will show you the command you have to write on lxplus to download the dataset from the grid, copy it.

Downloading the dataset with DQ2

1. Go to your temporally folder.

cd /tmp/username

2. Paste the dq2-get command you just copy from AMI and wait untill the download is done.

dq2-get valid1.105188.A3_Ztautau_filter.recon.AOD.e380_s764_r1266_tid131286_00

Creating an Ntuple

We can make an Ntuple from the AOD's that are inside de dataset, we can also do the same with RDO's files.

Editing the Ntuple

2. Create a directory in the Trigger area where we will put the ntuples (the ntuples can be quite heavier ~150M).

cd Trigger/TrigAnalysis/TrigTauAnalysis/
mkdir ntuples

3. Download the python file that create Ntuples from the dataset.

cd ntuples
get_files TTPNtuple.py

4. Edit the file - use your favorite editor to edit the following lines:

* Uncomment the first line and replace the DetDescrVersion with the Geometry Version we wrote before (ATLAS-GEO-10-00-00). This is the detector geometry used by the dataset creation.

* In line 66, change EvtMax from 1000 to -1, this will runs over all events that are available.

* In line 70, change: include("TrigTauPerformAthena/tauPerform_RDO_topOptions.py"), to: include("TrigTauPerformAthena/tauPerform_AOD_topOptions.py").

* Copy this two lines on line 64. This way is easier to access to a large quantity of files:

  from glob import glob
  PoolAODInput= glob('/tmp/yourlogin/data/datasetfolder/AOD*')

* In line 75 change the root file name to: trigtau.15691.AOD.root

Compiling the Ntuple

Just run it typing:

athena TTPNtuple.py
Note: This will last ~30min, so we will leave the first part of the tutorial up here.

I suggest you to save que ntuple in your own PC or Server, because the file weight ~150Mb and most of the lxplus accounts have 300Mb.

Major updates:
-- Main.Sebastian Olivares - 27 May 2010

RESPONSIBLE SebastianOlivares
REVIEW Never reviewed

Edit | Attach | Watch | Print version | History: r17 | r15 < r14 < r13 < r12 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r13 - 2010-06-10 - SebastianOlivares
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Main All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback