TWiki> Main Web>WISComputingGuide (revision 1)EditAttachPDF

Table Of Content

Running on lxplus


You have to get an account from Cern. Instructions to do it are at:

After getting an account you have to connect to (the public computer of Cern) using SSH program. A very convenient program is PuTTY which can be downloaded freely from: (There is no installation, just double click on the file after downloading it).

Really not necessary: To make the unix environment more friendly I use the files REMEMBER TO ADD THE FILES LATER

Just put them in your home directory, and they will be available on your next login. If you have your own aliases, shortcuts or configuration files that make your life easy feel free to update these files

Setting the environment

This section follows:

Setting your environment for the 1st time

The environment of lxplus is based on CMT, which is a Configuration Management Environment. The CMT is responsible (the way I understand it) to manage the different packages in Athena (For more details have a look at: Anyway, no matter what the CMT is, you have to configure it. For this task you have to create the CMT home directory, and prepare a working directory.
cd scratch0/
mkdir cmthome
mkdir testarea

Now you have to decide the version of Athena you are interested to run, for example 12.0.4 or 11.0.5. In what follows I describe how to run version 12.0.4, but feel free (and cross your fingers while doing so) to change it to your desired version. Here we create two work directories, one for version 12.0.4 and one for 11.0.5.

mkdir testarea/12.0.4
mkdir testarea/11.0.5

To setup the CMT:

cd cmthome
source /afs/

The last line tells the shell to execute the script From time to time this file is updated, so choose the updated one. Be careful this will not work properly if you have a directory ~/cmt. Now you have to put a requirements file. You can write it in a text editor or just put: REMEMBER TO ADD THE FILE LATER

The macro ATLAS_TEST_AREA points to your working area. If you want to change your working area you have to change this macro to point to the right directory.

To configure the CMT according to the requirements file just run:

cmt config

Configure the environment of your shell must do every login

Every time you login (in every shell) you have to choose the version of Athena. Here we choose to work with 12.0.4.
cd ~/scratch0/cmthome
source ~/scratch0/cmthome/ -tag=12.0.4
If you choose to work with version 11.0.5 you have to run:
cd ~/scratch0/cmthome
source ~/scratch0/cmthome/ -tag=11.0.5
source /afs/*/cmt/
To check that it works, type:
cmt show path
You should see all the paths of the CMT packages, among them you should see the path to your working area.

Job Options in Athena

Athena is a framework for all the packages in Atlas. The way to choose what to execute is by writing what you want in a Python script, this is the "" file. The easiest way to run something on Athena is to locate the appropriate job option file and edit the parameters to match your needs. if you are interested to know more have a look at: (I didn't bother to look at it). The way to run something is using the command:
This will execute Athena according to the job option file. While executing Athena sends a log to the standard output. It is recommended to save the log, because sometimes it has errors and you can view them after the execution. You can save it using: :
athena |& tee logfile.log
I like to paint the lines according to their status using the file REMEMBER TO ADD THE FILE LATER that I placed in my home directory, and I run:
athena |& tee logfile.log | color

Running Hello World

It is always fun to run a Hello World program because it gives you the feeling that you are on the right track. So we follow here the instruction of the Atlas WokBook:


cd ~/scratch0/testarea/12.0.4
cmt co -r UserAnalysis-00-08-43 PhysicsAnalysis/AnalysisCommon/UserAnalysis
This will check-out the relevant packages. Checking out is to take a package for a local use and edit (I think…). For version 11.0.5 replace UserAnalysis-00-08-43 with UserAnalysis-00-05-11.

Go to the relevant directory, setup and compile the package:

cd PhysicsAnalysis/AnalysisCommon/UserAnalysis/cmt/

Finally, go to the run directory, get the job option file and run Athena:

cd ../run
athena |& tee helloWorld.log | color
The get_files command looks for a job options file with the name indicated and copy it to your current directory. If you see the following lines (disregard the FATAL error lines) it means that you ran it successfully.

HelloWorld           INFO initialize()
HelloWorld           INFO   MyInt =    42
HelloWorld           INFO   MyBool =   1
HelloWorld           INFO   MyDouble = 3.14159
HelloWorld           INFO   MyStringVec[0] = Welcome
HelloWorld           INFO   MyStringVec[1] = to
HelloWorld           INFO   MyStringVec[2] = Athena
HelloWorld           INFO   MyStringVec[3] = Framework
HelloWorld           INFO   MyStringVec[4] = Tutorial
HistogramPersis...   INFO "CnvServices": ["HbookHistSvc", "RootHistSvc"]

Running Batch Jobs

Every process that takes more than an hour (more or less) is killed by lxplus. Instead there are queues for jobs where you can place your own job. First you have to write a script to run your job. Here we show how to run HelloWorld. Create a script file, for example '' with a text editor and put inside (but only after you checked out the package built it and got the job option file):
cd ~/scratch0/cmthome
source ~/scratch0/cmthome/ -tag=12.0.4
cd ~/scratch0/testarea/12.0.4/PhysicsAnalysis/AnalysisCommon/UserAnalysis/cmt
cd ~/scratch0/testarea/12.0.4/PhysicsAnalysis/AnalysisCommon/UserAnalysis/run
The first line opens a new shell for the job. The 2nd-3rd lines configure Athena for version 12.0.4. The 4th-last lines run Athena. You have to change the permissions of your script to be executable file:
chmod +x

And then place the job in the queue:

bsub -q 8nm
This will place your job in a queue for processes that run under 8 minutes. For processes that take up to 1 hour you have to replace 8nm with 1nh, for 8 hours 8nh, one day 1nd, one week 1nw.

There are a lot more options for this. You can check the status of your job by:

bjobs -l jobID
and you can kill a job:
bkill jobID

You will be notified through email when the job is over, and where to find the log file.


Following: Useful information can be found in:

If you haven't run Hello World before then you must(???) execute:

cd ~/scratch0/testarea/12.0.4
cmt co -r UserAnalysis-00-08-43 PhysicsAnalysis/AnalysisCommon/UserAnalysis
cd PhysicsAnalysis/AnalysisCommon/UserAnalysis/cmt/

From this section is for everyone. Go to the run directory, get the PDG table file and the job options file.

cd ~/scratch0/testarea/12.0.4/PhysicsAnalysis/AnalysisCommon/UserAnalysis/run

You have to add to the job options file: after all the AODFlags the line:

AODFlags.McEventKey = "GEN_EVENT"
Also, change INPUT to your input file (pythia.pool.root), OUTPUT to the output file (Fast_AOD.pool.root) and EVTMAX to the max number of events to process (-1 for all events)

Run Athena:

athena |& tee atlfast.log | color

You should get an AOD file as output.

-- Main.osilbert - 28 Feb 2007

Edit | Attach | Watch | Print version | History: r25 | r4 < r3 < r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r1 - 2007-02-28 - OhadSilbert
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Main All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback