DIANE Integration with Ganga

Proposal 1

Example 1

GPI code which corresponds to DIANE/dev/workspace/testErrorRecovery.job file:

# master job
j = Job()
j.backend = DIANEMaster()
j.backend.failRecovery = 'failRecovery_Ignore'
j.backend.workerNumber = 20
j.backend.workerTarget = LCG()
j.backend.masterTarget = Local() # optional, default is Local
j.application = DIANEApplication()
j.application.name = 'crashTest'
j.application.data = [10,1.2,0.3,0,5,"error"] # JobData is a tuple (taskNum, seed, problem_prob, avg_wait, std_dev, pref_fail<opt>, init_fail_prob<opt>)
j.submit()

#additional worker job j2 would be created and submitted automatically (after the MasterOID is aviable), in such a way that:
j2.application == Executable(...) # exe = wrapper script prepared by DIANEMaster
j2.backend == j.backend.targetBackend 
j2.splitter = NSplitter(n=j.backend.workerNumber)
j2.name = 'Worker for %d'%j.id

Notes:

  • the MasterOID is available in $j.outputdir/MasterOID
  • all files generated by master are copied to $j.outputdir at the end (so effectvely j.outputbox is ['*'])

Example 2

GPI code which corresponds to diane/HEAD/DIANE/dev/workspace/Athena/athena.job file:

j = Job()
j.backend = DIANEMaster()
j.backend.failRecovery ---> currently impossible to pickle functions
j.backend.workerTarget = LCG()
j.backend.workerNumber = 30
j.application = DIANEApplication()
j.application.name = 'ATLAS_Athena1'
j.application.data = {
                'pool_catalog' : File('/afs/cern.ch/sw/arda/install/ATLAS/diane_demo2/InputSandbox/rome.004100.recov10.T1_McAtNLO_top-cern-castor.xml.gz'),
                'algorithm_pkg'     : File('/afs/cern.ch/sw/arda/install/ATLAS/diane_demo2/InputSandbox/AnalysisExamples-001.tar.gz'),
                'algorithm_set'     : 'File(/afs/cern.ch/sw/arda/install/ATLAS/diane_demo2/InputSandbox/athenajob-setup.sh'),
                'output_data_files' : ['ttbar.hist.root'],
                'file_grouping'     :1,
                'async_merge'       :1
              }
j.submit()

File IS NOT currently implemented

-- JakubMoscicki - 15 Aug 2006

Edit | Attach | Watch | Print version | History: r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r2 - 2006-11-30 - JakubMoscicki
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    ArdaGrid All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback