Using Ganga to send FLUKA jobs on the Grid

Before you start

You need to have a valid Grid Certificate installed at the Linux machine, and be a member of the LHCb Virtual Organization. A compact but sufficient guide how to do this can be found in the LHCb Certificate FAQ.

If you are using Linux at your own PC or laptop you need to have a local installation of Ganga, the LHCb frontend for job definition and management in Grid - the instruction how to do this can be found at the Ganga Install and Configure pages.

Ganga general and specific questions can be solved with the documentation at the Ganga homepage.

Hint: To check that everything is in place, at the Linux prompt try to call lhcb-proxy-init (you will need to enter your Grid Certificate password) and after it lhcb-proxy-info.

An example how it should look like (at IHEP in Protvino using Russian RDIG certificate) is below:

[talanov][ui0004]:~/ lhcb-proxy-init
Enter Certificate password:
[talanov][ui0004]:~/ lhcb-proxy-info
subject      : /C=RU/O=RDIG/OU=users/OU=ihep.su/CN=Vadim Talanov/CN=proxy/CN=proxy
issuer       : /C=RU/O=RDIG/OU=users/OU=ihep.su/CN=Vadim Talanov/CN=proxy
identity     : /C=RU/O=RDIG/OU=users/OU=ihep.su/CN=Vadim Talanov
timeleft     : 23:53:37
DIRAC group  : lhcb_user
path         : /tmp/x509up_u24867
username     : vtalanov
VOMS         : True
VOMS fqan    : ['/lhcb/Role=user']

You also have to be a registered FLUKA user, and have a local installation of FLUKA (at present). How to do this and other FLUKA general and specific questions can be solved using comprehensive FLUKA manual at the FLUKA site.

Quick run

You will need to prepare separately a tar-ball with a FLUKA executable, and a FLUKA input file splitted into two parts.

In addition you will need a script to send FLUKA job in Ganga, a wrapper script to start FLUKA execution at the CE, and another script to retrieve the FLUKA output from the SEs.

You can copy fully working templates of these scripts: myFlkJob.py, rrr_ganga and myFlkCat.py from ~/vtalanov/public/myFlkGanga/v1r3/myFlkGanga.tar.gz.

Finally, some knowledge of FLUKA, Ganga and Linux is certainly required.

Note: Most of the file names used below can be customized at user choice by editing the scripts.

FLUKA executable

Compile and link FLUKA executable with (any) your user routines.

Ensure that the executable name is the default flukahp.

Put flukahp together with any additional input files (magnetic field maps etc.) EXCEPT the FLUKA input file in the gzipped tar-ball with the name fluka_exe.tar.gz.

Note: Check that the size of your fluka_exe.tar.gz DOES NOT EXCEED 10 Mb.

Place the fluka_exe.tar.gz tar-ball in your working directory from where you will send the jobs.

FLUKA input files

Cut your FLUKA input file into two parts.

Put into flk.head file everything above the RANDOMIZe card and into flk.tail everything below. DO NOT put the RANDOMIZe card itself in any of two input files.

Hint: Put all your FLUKA input into flk.head and only START and STOP cards into flk.tail file.

FLUKA output files

Present algorithm of job submission uses the following FLUKA convention:

  • If you write some output file filename from the MGDRAW routine it will have the final name flk001_filename after completion of FLUKA run

  • If you ask FLUKA for an estimator (USRBIN etc.) on unit XX the output file name will be flk001_fort.XX

Depending on your case set the value of the myFlkOutputFile in the myFlkJob.py either to filename or to fort (without the FLUKA unit number).

Hint: Your working directory at this point should look like as below (note that rrr_ganga is an executable script):

[talanov][ui0004]:~/Desktop/2010/G_Genoa/11_30/ ls -alF
total 6025
drwx------ 2 talanov talanov    2048 Nov 30 14:44 ./
drwx------ 8 talanov talanov    2048 Nov 30 13:51 ../
-rw------- 1 talanov talanov    1163 Nov 30 14:37 flk.head
-rw------- 1 talanov talanov      26 Nov 30 14:37 flk.tail
-rw------- 1 talanov talanov 6158283 Nov 30 13:52 fluka_exe.tar.gz
-rw------- 1 talanov talanov     535 Nov 30 13:56 myFlkCat.py
-rw------- 1 talanov talanov    1879 Nov 30 14:06 myFlkJob.py
-rwx------ 1 talanov talanov     515 Nov 30 14:11 rrr_ganga*

Sending the Ganga jobs

Set the value of the myGangaJobName in the myFlkJob.py and change the myNumofJobs to the number of the jobs to be sent .

Do SetupProject Ganga and start Ganga by typing ganga at the Linux prompt from your working directory (you may be asked to renew your Grid proxy at this point).

From Ganga prompt do execfile('myFlkJob.py').

Cross your fingers.

Hint: If something happens remember that at Ganga prompt you can always get particular job by the job id with j = jobs(id) and then with j.peek('stdout') try to see what has happened with the job in the GRID.

Retrieving the job output files

Set the value of the myGangaJobName in the myFlkCat.py to the same as in the myFlkJob.py.

Start Ganga as in the previous section.

From Ganga prompt do execfile('myFlkCat.py').

The gzipped tar archives with the output files from all completed jobs with the given job name myGangaJobName will be downloaded into the current working directory.

Each archive will have the name myGangaJobName_myFlkOutputFile_JobNumber.tgz with JobNumber in a range ( 1000, 1000 + myNumofJobs ).

Hint: You can monitor the status of your jobs by typing jobs at the Ganga prompt.

GRID job monitoring in DIRAC (from the DIRAC pages)

You can check the status of your jobs in the GRID here.

You need a valid GRID certificate in your web browser to use these pages and your certificate must be loaded into your web browser.

All information about CERN certificates and how to load them into your browser you can find here.

After loading certificate into browser you can use it for authentication in your browser and work with DIRAC GUI to monitor your jobs here.

There you can manage your jobs: kill, reschedule, delete etc. An interface should look like in a picture below:

Ganga26.jpg

Combining jobs output

To be completed.

User Guide

To be completed.

-- VadimTalanov - 11-Feb-2011

Edit | Attach | Watch | Print version | History: r7 < r6 < r5 < r4 < r3 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r7 - 2011-02-11 - VadimTalanov
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LHCb All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback