The raw Panda service logs contain information not available elsewhere. We will store these logs on HDFS as needed in the usecases below.

Usecases and specific questions

  • count number of requests to the panda jobDispatcher via getJob, grouping on each panda queue
  • extract job identifier and use this to join on pilot factory records to identify pilots receiving payload verses not receiving a payload

Data volume and format

JobDispatcher logs (/data/atlpan/srv/var/log/panda/panda-JobDispatcher.log) are rotated daily and gzipped with filesize 2-300MB. There are 8 machines producing this file (aipanda030-7) so say 2.5GB per day. The format on HDFS is the raw log output by flume DataStream. Currently the export of data to hdfs is manual pending setup of crons to handle the kerberos authentication.

Sample lines although the format varies a lot:

2015-02-26 12:12:47,444 panda.log.JobDispatcher: DEBUG    getJob : CONNECT_CLOUD useGLEXEC=False ret -> StatusCode=20
2015-02-26 12:12:47,447 panda.log.JobDispatcher: DEBUG    getJob(UKI-SCOTGRID-DURHAM_MCORE)
2015-02-26 12:12:47,447 panda.log.JobDispatcher: DEBUG    getJob(UKI-SCOTGRID-DURHAM_MCORE,2668.0,32000,20000,test,,UKI-SCOTGRID-DURHAM_MCORE,None,None,False,None,,None,DN:/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=atlpilo1/CN=614260/CN=Robot: ATLAS Pilot1/CN=548551862/CN=3084574165/CN=2008730830,role:True,token:None,val:True,FQAN:['/atlas/lcg1/Role=NULL/Capability=NULL', '/atlas/Role=NULL/Capability=NULL', '/atlas/usatlas/Role=NULL/Capability=NULL', '/atlas/Role=production/Capability=NULL'])
2015-02-26 12:12:47,500 panda.log.JobDispatcher: DEBUG    updateJob : 2400219050 ret -> command=NULL&StatusCode=0
2015-02-26 12:12:47,509 panda.log.JobDispatcher: DEBUG    updateJob : 2400072969 ret -> command=NULL&StatusCode=0

Major updates:
-- PeterLove - 2015-02-12

Responsible: PeterLove
Last reviewed by: Never reviewed

Edit | Attach | Watch | Print version | History: r4 < r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r4 - 2016-05-16 - IlijaVukotic
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Atlas All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback