Using Ganga as a Job Submission Interface for the DIANE applications

Using the Ganga arhitecture we developed a backend handler that may be dynamically loaded in the Ganga interface (GUI/CLI) and used to submit/monitor the evolution of DIANE applications.

How to load GangaDIANE plugins

Edit your .gangarc file and add the path to the DIANE handlers to the RUNTIME_PATH:
RUNTIME_PATH = GangaLHCb:/afs/cern.ch/project/asddat/lhcxx/kuba/DIANE/specific/slc3_gcc323/HEAD/DIANE/dev/GangaDIANE
Next time when you start Ganga the DIANE plugins will be automatically loaded.

Creating a Ganga Job for a DIANE Application:

j=Job()
j.backend=DIANEBackend()
j.application=DIANEApplication(name='crashTest',data=[100,1.2,-1,15,3])
In this example we created a Ganga Job, we set the DIANEBackend() as the backend for the Ganga Job and initialized the 'crashTest' DIANE application For the complete documentation use the help function from the Ganga session: help DIANEApplication help DIANEBackend

How to setup DIANE workers agents

You can control the number and the destination of DIANE workers using the j.backend.numberOfWorkers and j.backend.destination fields of the DIANEBackend. In the following example we setup 7 DIANE workers to be ran on LCG Grid
j.backend.numberOfWorkers=7
j.backend.destination=LCG()

Start processing of DIANE Application

In order to start the DIANE Master locally on your UI machine and submit the configured number of DIANE workers using the destination backend you need to submit the Ganga job (your DIANE application).

You can choose to start the master in interactive mode (by default it starts as a background job in your UI machine)

j.backend.interactiveMaster=True

The Ganga Job is splitted in multiple subjobs:

  • the subjob at index 0 is the DIANE Master
  • the subjobs with index>0 are DIANE Workers

Submit:

j.submit()

Monitoring application

  • Ping/Trace/Kill DIANE Master:
j.backend.pingMaster()
j.backend.traceMaster()
j.backend.killMaster()

  • Pretty-printing of DIANE Master/Worker:
You can get the current status of the DIANE master/workers by calling status method of DIANEBackend()

j.backend.status()

  • Access Master log file from Ganga:
 !tail -f $j.subjobs[0].backend.workdir/stdout

Add more workers on-the-fly

DIANE allows on-demand creation of new workers to increase the task completion throughput :
j.backend.addWorkers(2, LSF())
j.backend.addWorkers(4, Local())

Getting the Master output

!ls $j.subjobs[0].outputdir

-- Main.adim - 06 Nov 2006

Edit | Attach | Watch | Print version | History: r6 < r5 < r4 < r3 < r2 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r6 - 2007-02-05 - AdrianMuraru
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    ArdaGrid All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback