Creating the Ganga application plugin.

  1. Make a copy of Ganga/Lib/Executable and save it as myAppDir/myApp (for example). Source code for CVS Ganga/Lib/Executable
  2. Modify myApp as required. Remember to change the _name attribute!
  3. Do not forget that runtime handlers in Ganga connect applications to the backends. Without a runtime handler the application is not usable (j.submit() will fail)

Some principles

  • An application has to be derived from IApplication
  • An Application has a schema. This is necessary to support persistency and other python magic. Currently we support SimpleItem (any python simple type without type checking), FileItem (a file) and CompositeItem (another GANGA object)
  • An application has to implement the configure method. This method is called during submit to do some common preparations. Optionally it can support a master_configure routine. This can be useful for jobsplitting
  • The main work is done in the runtime handler. It is another object that is derived from IRuntimeHandler. It connects an application with an backend. Its main purpose is to define a JobConfig object, that might be specific for a backend object (as LCG, gLite, PBS etc). Typically the run time handler would construct the sandbox and fill the required information into the JobConfig object. This allows then the backend handler to do its work during submission.

Running the myApp plugin

Add myAppDir to the RUNTIME_PATH in your ~/.gangarc file. myApp will then be loaded at startup automatically.

Snippet from the ~/.gangarc file.

      [Configuration]
      RUNTIME_PATH = GangaAtlas:myAppDir

To be fully compliant with the Ganga initialization model, myApp/__init__.py must contain the loadPlugins() function. See GangaLHCb/__init__.py for an example.

Note on the $GANGA_CONFIG_PATH variable

If you specify config file in GANGA_CONFIG_PATH as a relative path (e.g. GangaLHCb/LHCb.ini) then this config file will be taken from the release area and NOT from your private copy of GangaLHCb as specified by RUNTIME_PATH.

Exporting any objects and functions to GPI

The Ganga.GPI package is a namespace which holds public objects, which are available automatically in Ganga. Plugin classes are automatically visible in the public namespace. You may additionally export other objects and functions in the loadPlugins() function at the top level of the runtime package:

myPackage/__init_.py_

from Ganga.Runtime.GPIexport import exportToGPI

def loadPlugins(c):
       .....
       exportToGPI(name,obj,...)

Plugin package structure

Importing yourPackage should not have side effects. Instead, you should define the function loadPlugins(config) at the top package level, which will be called at startup. It should then import modules and subpackages with plugin classes.

Pitfalls to avoid

Definition of the schema

The schema of the plugin should be defined correctly.

  1. SimpleItem objects should be given sensible defvalue values and None values should be avoided. ComponentItem objects, on the other hand, can have a defvalue of None.
  2. If an attribute is defined as a sequence, the defvalue should be [].

Handlers should not modify the schema properties which users may directly modify

The properties which are declared protected in the schema are meant to be modified by the system. The other properties are under user's control.

Your handler should not modify the properties which are under user's control. This will confuse the users (they will not understand why the properties they have set changed).

Handler should not modify or rely on shared environment

If possible you should not modify environment of the Ganga process because it may affect other handlers. So avoid doing things like os.environ['PATH'] = mypath. If you do it, you may break other handlers. The better way is to use private, cached environment and source it on-the-fly when invoking a subprocess. For example consider the LCG handler. It uses the lcg-job-submit shell command to submit jobs. Each time Ganga is about to execute this command, the LCG environment which has previously been cached, is set.

Sometimes you must rely on the common environment of the Ganga process. For example, you use some extension modules (compiled libraries dynamically loaded by python) then at least you must have the LD_LIBRARY_PATH set correctly. Use the getEnvironment() function in the runtime package. Ganga will know that you need to modify common environment and at least you will get better diagnostics when there is a clash with other package.

Be careful with storing filenames automatically in the schema properties

If during job submission you create additional files (which are stored in the job's input directory) you should be careful with storing their filenames in the persistent schema. You may silently create a dependency on the files which are specific to a certain job. When such job is copied the references will be copied as well (check copyable declaration in the schema) and the job defintion will not be self-contained. This means that a user may encounter wierd effects on the copy if the original is removed.

Other useful references

Refer to the documentation included in the following code:

  1. Ganga/GPIDev/Adapters/IApplication.py
  2. Ganga/GPIDev/Adapters/IRuntimeHandler.py

-- AlvinTan - 10 Jul 2006

Edit | Attach | Watch | Print version | History: r6 < r5 < r4 < r3 < r2 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r6 - 2011-02-18 - JakubMoscicki
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    ArdaGrid All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback