Job Report for production jobs

Required job report fields

The payload / transform in a production job is expected to create a job report (json dictionary) containing several fields that are needed by the pilot and by Harvester (on HPCs). In ATLAS, it contains many additional fields that are not used by the pilot or Harvester, but is used by other components so the pilot sends the entire file along with the final server update ('metaData' field). The default file name is "jobReport.json" but can be defined in the pilot configuration file (pilot/util/default.cfg, "jobreport"). The pilot expects to find the following fields:

  • exitCode - the payload exit code
  • exitMsg - the payload exit message

(the fields expected by Harvester may be documented elsewhere).

ATLAS specific job report fields (expected)

For ATLAS (i.e. used in the ATLAS plugin only) there are several other fields used, including

  • nevents - the number of processed events
  • Avg - average memory info
  • Max - max memory info
  • dbData
  • dbTime
  • cpuTime - consumed CPU time

Furthermore, the dictionary format (relevant for the above fields) is:

{
   ..
   "exitCode": [integer],
   "exitMsg": "[string]",
   "resource": {"executor": {"nevents": <int>, "memory": {"Avg": .., "Max": ..}, "cpuTime": <int>, "dbData": <int>, "dbTime": <float>}
 }

Note: on (at least) Titan, the "logfileReport" is reset before jobReport is uploaded.

-- PaulNilsson - 2019-03-05

Edit | Attach | Watch | Print version | History: r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r1 - 2019-03-05 - PaulNilsson
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    PanDA All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback