PandaPilotJobState < PanDA

%CERTIFY%
<!-- This is the default ATLAS template. 
Please modify it in the sections indicated to create your topic!
If you have any comments/complaints about this template, then please email me:
edward.moyse at cern.ch
--> 


<!-- By default the title is the WikiWord used to create this topic !-->
<!-- if you want to modify it to something more meaningful, just replace %TOPIC% below with i.e "My Topic"!-->
---+!!<nop>Job State
%TOC%
%STARTINCLUDE%

---+Introduction
<!-- Add an introduction here, describing the purpose of this topic. !--> 
The purpose of the job state algorithm is to provide a job recovery mechanism for the Panda pilot.
Whenever the pilot updates the Panda server with the progress of a job, it also creates/updates
a job state file with all information necessary for later job recovery. If there is a crash and the
job is lost, the job state file will remain in the job work directory for later recovery by a different pilot.
This new pilot will find the lost job state file and will attempt to recover the lost job. If the payload of the
lost job finished (but e.g. failed to transfer the data and log files), the job recovery algorithm will move
the log to the local SE and will register the data file.

---+Algorithm
<!--Add the main topic here. i.e. create some new headings as follows:!-->
<img src="%ATTACHURLPATH%/updateJobState.png" alt="diagram" />

Activity diagram for the job state algorithm. [Correction: a job state file will in fact be re-created
on every Panda server update. There is no check if the file already exists.]

---+Additional details
The job state object knows how to store itself and how to be read back, as well as delete itself and the old work directory. If there are
no more job state files in the site work directory, this directory will also be deleted. When a job finishes (successfully) and all output files and log are copied to the SE, the job state file is removed from the site work directory.

---+File format
*File name*: _<verbatim>jobState-<jobId>.pickle</verbatim>_

---++File object information
The job state file is a container for all information necessary for job recovery. The file contains the following objects stored in pickle format:
   * [[http://atlas-sw.cern.ch/cgi-bin/viewcvs-atlas.cgi/offline/Production/panda/pilot2/Job.py][job]]
   * [[http://atlas-sw.cern.ch/cgi-bin/viewcvs-atlas.cgi/offline/Production/panda/pilot2/Site.py][site]]
   * [[http://atlas-sw.cern.ch/cgi-bin/viewcvs-atlas.cgi/offline/Production/panda/pilot2/Node.py][node]]
There are a few overlaps with these objects but they are insignificant. A typical job state file size is about 4kB.

<!-- MAJOR UPDATES
For significant updates to the topic, consider adding your 'signature' (beneath this editing box) !-->
-----
*Major updates*:%BR%
<!--Future editors should add their signatures beneath yours!-->
-- Main.PaulNilsson - 06 Oct 2006


%STOPINCLUDE%


<!--Please add the name of someone who is responsible for this page (probably you!) so that they can be contacted if the content needs updating!-->
%RESPONSIBLE% Main.PaulNilsson
%BR%
<!--Once this page has been reviewed, please add the name and the date e.g. Main.StephenHaywood - 31 Oct 2006 -->
%REVIEW%

Attachments

Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
jpeg	updateJobState.jpg.jpeg	r1	manage	23.7 K	2006-10-06 - 17:56	UnknownUser	Activity diagram for job state algorithm
png	updateJobState.png	r1	manage	21.0 K	2006-10-06 - 18:02	UnknownUser	job state algorithm

This topic: PanDA > AtlasDistributedComputing > PanDA > PandaPilot > PandaPilotJobState
Topic revision: r5 - 2006-12-22 - PaulNilssonSecondary

Copyright &© 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback