Autopyfactory condor logs contain useful information about the state of Grid computing fabric which are otherwise invisible to PanDA WMS. We will store these logs and analyze using the lxhadoop cluster.

Usecases and specific questions

  • how much wallclock time is wasted, aggregated and per panda queue
  • identify broken sites which otherwise missed by existing tools

Data volume and format

These logs are condor history logs, about 200MB each and rotated by condor. A single factory has a few of these per day so for 7 factories this is a total of 5GB per day. The content are condor classads per job with records separated by the '*** Offset' line. A sample is here.

Major updates:
-- PeterLove - 2014-12-17

Responsible: PeterLove
Last reviewed by: Never reviewed

Edit | Attach | Watch | Print version | History: r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r2 - 2016-05-16 - IlijaVukotic
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Atlas All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback