Event Records

Introduction

In order to do accounting, we need to retrieve equvalent information from the log files of different batch systems. It can be difficult to ensure that this information is directly comparable.

The batch systems we support are LSF, PBS/Torque and SGE. We plan to support SLURM. In the past we have supported Condor.

Notes

  • WallDuration should logically be EndTime - StartTime. In LSF it is calculated from these values.
  • SGE appears to present MPI jobs as different lines in the log file.
  • SubmitHost only appears in LSF log files, and is not the same as SubmitHost in the JobRecord

Field comparison

  • Numbers represent the item in arrays, where the lines have been appropriately split
  • Strings represent the keys where the log file is presented in key-value form
  • 'o' in the LSF fields represents an offset due to variable-length arrays

Field LSF PBS SGE [Condor] Notes
SiteName        
JobName 3 2 5    
LocalUserID 11 'user' 3    
LocalUserGroup not available 'group' 2    
EventTime 2 'end' 10    
WallDuration 2 - 10 (!) 'resources_used.walltime' 13    
CpuDuration (28+o) + (29+o) 'resources_used.cput' 36    
StartTime 10 'start' 9    
StopTime 2 'end' 10    
SubmitHost 16 not available not available    
MemoryReal 54 + o 'resources_used.mem' 37    
MemoryVirtual 55 + o 'resources_used.vmem' 42    
Processors numExec parse from 'exec_host' 34    
NodeCount unique(numExec) parse from 'exec_host' ?    

-- WillRogers - 13-Aug-2012

Edit | Attach | Watch | Print version | History: r9 < r8 < r7 < r6 < r5 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r9 - 2012-10-30 - unknown
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    EMI All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback