Job Report for production jobs

Required job report fields

The payload / transform in a production job is expected to create a job report (json dictionary) containing several fields that are needed by the pilot and by Harvester (on HPCs). In ATLAS, it contains many additional fields that are not used by the pilot or Harvester, but is used by other components so the pilot sends the entire file along with the final server update ('metaData' field). The default file name is "jobReport.json" but can be defined in the pilot configuration file (pilot/util/default.cfg, "jobreport"). The pilot expects to find the following fields:

• exitCode - the payload exit code
• exitMsg - the payload exit message

(the fields expected by Harvester may be documented elsewhere).

ATLAS specific job report fields (expected)

For ATLAS (i.e. used in the ATLAS plugin only) there are several other fields used, including

• nevents - the number of processed events
• Avg - average memory info
• Max - max memory info
• dbData
• dbTime
• cpuTime - consumed CPU time

Furthermore, the dictionary format (relevant for the above fields) is:

{
   ..
   "exitCode": [integer],
   "exitMsg": "[string]",
   "resource": {"executor": {"nevents": <int>, "memory": {"Avg": .., "Max": ..}, "cpuTime": <int>, "dbData": <int>, "dbTime": <float>}
 }

Note: on (at least) Titan, the "logfileReport" is reset before jobReport is uploaded.

-- PaulNilsson - 2019-03-05