Instructions for running jobs on the Manchester Linux 8 cluster

A small python module (written by Jiri & Mark) rempyexec is available to use to submit jobs to the linux8 cluster. It can deal with any executable. Before use, add the location of the module to the python path:

export PYTHONPATH=/afs/hep.man.ac.uk/g/atlas/JobRunning/:$PYTHONPATH

Generic Executables

A script sending a simple test job is available (don't forget the above path), first login to linux8, go to some temporary directory in your manchester afs space and then do:

python /afs/hep.man.ac.uk/g/atlas/JobRunning/testjob.py

The results (just a log file) will appear in a sub directory called output.

An example script for multiple jobs is at:

/afs/hep.man.ac.uk/g/atlas/JobRunning/multi_testjob.py

It's run in the same way as the previous example. For this test the results (just a log file) will appear in a sub directory called output_jobid, where job id = {0,1,2}.

It should be straight forward to modify these to deal with your favorite executable.

SFrame Jobs

There is also a specific class provided for running sframe jobs. Example python scripts for submitting sframe jobs are available: /afs/hep.man.ac.uk/g/atlas/JobRunning/example_sframe.py /afs/hep.man.ac.uk/g/atlas/JobRunning/example_multi_sframe.py

The first demonstrates a single sframe job, the second shows how to submit two. The scripts should be self explanatory. Note for sframe you must submit from your SFrame directory (where you run setup.sh) since it copies the full contents of this directory to the worker node. Make sure any files / libraries you need are specified with an absolute & not relative path. Copy one of the testscripts, edit it to run one of your config files, set up the path as above & do:

python example_sframe.py

-- MarkOwen - 05 Mar 2009