Running Ganga as a Service

Introduction

As of Ganga 5.8.7, it is possible to run Ganga not only daemonised but as a light weight socket-style service. This Twiki will go through the basics of how to use this functionality as well as some examples.

Any comments, questions, bugs, etc. please email Mark Slater (mws@hepNOSPAMPLEASE.ph.bham.ac.uk)

Basic use: Running Ganga as a daemon

The very basic way of running ganga as a daemon is to specify it at the command line:

source /afs/cern.ch/sw/ganga/install/etc/setup-atlas.sh
ganga --daemon <script_name>

This will start ganga as a separate daemon process with the stdout/stdin redirected to:

<gangadir>/server/server-<hostname>.stdout
<gangadir>/server/server-<hostname>.stderr

Note that for obvious reasons, you can only run Ganga as a daemon in batch mode, i.e. when specifying a script to run.

Advanced Use: Active Server Mode

In addition to the basic daemonising of Ganga, there are scripts that utilise this to produce a full Ganga Service that can have commands sent to it through socket commands. This can be accessed through the GangaService API python functions. To get access to these, do the following after setting up Ganga:

export PYTHONPATH=$PYTHONPATH:/afs/cern.ch/sw/ganga/install/5.8.7/python

Note that this directory should point to the latest install of Ganga.

This now allows the following python script to be used:

from GangaService.Lib.ServiceAPI.ServiceAPI import GangaService

# create the Ganga Service ojecct
gs = GangaService()

# Add any commands to set up Ganga before running (e.g. GANGA_CONFIG_PATH)
#gs.prerun = "export GANGA_CONFIG_PATH='GangaAtlas/Atlas.ini'"

# Specify the Ganga command line
gs.gangacmd = "/afs/cern.ch/sw/ganga/install/5.8.7/bin/ganga"

# specify how long the server should stay up for (in minutes)
gs.timeout = 24*60

# send a basic command
print gs.sendCmd("""
j = Job()
j.submit()
""")

This will setup a server on the default port of 43434 that will hang around for 24 hours. Note that you need to give the command line to start Ganga (the above works for lxplus). Other options available through the GangaService object are:

gs.killServer()   # kill any active server
gs.port     # the port number to start the server on
gs.gangadir    # The ganga directory to use - defaults to ~/gangadir-server

The server keeps running until the 'killServer' command is issued (from any machine that has access to the same gangadir) or it hits the timeout parameter without any commands being received. The server output is as above:

<gangadir>/server/server-<hostname>.stdout
<gangadir>/server/server-<hostname>.stderr

but there is an additional watchdog file that gives the server info:

<gangadir>/server/server.info

Example: Running an Atlas Task

The server is most useful in conjunction with the Tasks framework as you can set a Task going that will take a number of hours without having to stay logged in or (actively) running Ganga for the jobs to be submitted, resubmitted, etc. A typical example is:

from GangaService.Lib.ServiceAPI.ServiceAPI import GangaService

gs = GangaService()
gs.prerun = "export GANGA_CONFIG_PATH='GangaAtlas/Atlas.ini'"
gs.gangacmd = "/home/mws/Ganga/install/5.8.7-pre/bin/ganga"
gs.timeout = 24*60

print gs.sendCmd("""                                                                                                                                                    
t = AtlasTask()                                                                                                                                                         
t.float = 1000                                                                                                                                                          
                                                                                                                                                                        
trf = AtlasTransform()                                                                                                                                                  
bk = Panda()                                                                                                                                                            
app = Athena()                                                                                                                                                          
app.option_file=['AnalysisSkeleton_topOptions.py' ]                                                                                                                     
app.prepare()                                                                                                                                                           
                                                                                                                                                                        
data11_7TeV_Datasets = {                                                                                                                                                
        'PeriodD' : 'data11_7TeV.periodD.physics_Egamma.PhysCont.AOD.pro10_v01/',                                                                                       
        'PeriodE' : 'data11_7TeV.periodE.physics_Egamma.PhysCont.AOD.pro10_v01/',                                                                                       
        'PeriodF' : 'data11_7TeV.periodF.physics_Egamma.PhysCont.AOD.pro10_v01/',                                                                                       
        'PeriodG' : 'data11_7TeV.periodG.physics_Egamma.PhysCont.AOD.pro10_v01/',                                                                                       
        'PeriodH' : 'data11_7TeV.periodH.physics_Egamma.PhysCont.AOD.pro10_v01/',                                                                                       
        'PeriodI' : 'data11_7TeV.periodI.physics_Egamma.PhysCont.AOD.pro10_v01/',                                                                                       
        'PeriodJ' : 'data11_7TeV.periodJ.physics_Egamma.PhysCont.AOD.pro10_v01/',                                                                                       
        'PeriodK' : 'data11_7TeV.periodK.physics_Egamma.PhysCont.AOD.pro10_v01/',                                                                                       
        'PeriodL' : 'data11_7TeV.periodL.physics_Egamma.PhysCont.AOD.pro10_v01/',                                                                                       
        'PeriodM' : 'data11_7TeV.periodM.physics_Egamma.PhysCont.AOD.pro10_v01/'                                                                                        
    }                                                                                                                                                                   
                                                                                                                                                                        
t.initializeFromDictionary(data11_7TeV_Datasets, application = app, backend = bk)                                                                                       
t.run()                                                                                                                                                                 
                                                                                                                                                                        
""")

This will process ~5000 jobs and will take ~24 hours but you can execute this python script, forget about it and Ganga will continue to process the Task until completion.

-- MarkWSlater - 20-Jul-2012