Purpose

The level of detail presented when printing a Ganga object (especially a Ganga job object) can be overwhelming. There needs to be some control mechanism to determine the amount of detail returned.

Requirements

  • Objects defined in the schema that possess the sequence=1 parameter should only display n entries in the sequence or simply display its length.
    • could only show diffs between entries in a sequence

Notes: [3 subjob example - 430 lines reduced to 98 lines]
       1) Sequences of more than n number will be reduced to '...n entries...'
       2) Default values omitted. (Should we indicate config changes?)
       3) First entry shows all, taking (1) into account. Subsequent entries shows changes (across all entries) only.

Job (
    status = 'completed' ,
    name = 'split-mer-job,DQ2_LOCAL, 5200' ,
    inputdir = '/afs/cern.ch/user/c/ctan/gangadir/workspace/Local/418/input/' ,
    outputdir = '/afs/cern.ch/user/c/ctan/gangadir/workspace/Local/418/output/' ,
    id = 418 ,
    splitter = AthenaSplitterJob (
       numsubjobs = 3 
       ) ,
    merger = AthenaOutputMerger (
       ) ,
    application = Athena (
       max_events = '100' ,
       atlas_release = '11.0.5' ,
       option_file = File (
          name = '/afs/cern.ch/user/c/ctan/athena/testarea/11.0.5/InstallArea/jobOptions/UserAnalysis/AnalysisSkeleton_jobOptions.py' ,
          ) ,
       user_area = File (
          name = '/afs/cern.ch/user/c/ctan/gangadir/workspace/Local/UserAnalysis-00015.tar.gz' ,
          ) 
       ) ,
    outputdata = ATLASOutputDataset (
       output = [['guid:5ca26bf4-f259-4151-ab64-edc4ea34f4c2'], ['guid:06e693fd-cf98-49ba-aa98-66cb9ff5f4d6'], ['guid:ce122022-31b1-4082-b5a7-fe8ff908d85a']] ,
       outputdata = ['AnalysisSkeleton.aan.root'] ,
       ) ,
    inputdata = DQ2Dataset (
       dataset = 'csc11.005200.T1_McAtNlo_Jimmy.recon.AOD.v11004204' 
       ) ,
    backend = LCG (
       CE = 'lcg2ce.ific.uv.es:2119/jobmanager-pbs-short' ,
       ) ,
    subjobs = [ Job (
       status = 'completed' ,
       inputdir = '/afs/cern.ch/user/c/ctan/gangadir/workspace/Local/418/0/input/' ,
       outputdir = '/afs/cern.ch/user/c/ctan/gangadir/workspace/Local/418/0/output/' ,
       id = 0 ,
       application = Athena (
          max_events = '100' ,
          atlas_release = '11.0.5' ,
          option_file = File (
             name = '/afs/cern.ch/user/c/ctan/athena/testarea/11.0.5/InstallArea/jobOptions/UserAnalysis/AnalysisSkeleton_jobOptions.py' ,
             subdir = '.' 
             ) ,
          user_area = File (
             name = '/afs/cern.ch/user/c/ctan/gangadir/workspace/Local/UserAnalysis-00015.tar.gz' ,
             subdir = '.' 
             ) 
          ) ,
       outputdata = ATLASOutputDataset (
          output = ['guid:5ca26bf4-f259-4151-ab64-edc4ea34f4c2'] ,
          outputdata = ['AnalysisSkeleton.aan.root'] ,
          location = 'srm://srm.cern.ch/castor/cern.ch/grid/atlas/scratch/ctan/ganga/418.0' 
          ) ,
       inputdata = DQ2Dataset (
          tagdataset = '' ,
          type = 'DQ2_LOCAL' ,
          names = ...60 entries... , 
          match_ce = 0 ,
          dataset = 'csc11.005200.T1_McAtNlo_Jimmy.recon.AOD.v11004204' 
          ) ,
       backend = LCG (
          status = 'Done (Success)' ,
          reason = 'Job terminated successfully' ,
          CE = 'lcg2ce.ific.uv.es:2119/jobmanager-pbs-short' ,
          actualCE = 'lcg2ce.ific.uv.es:2119/jobmanager-pbs-short' ,
          id = 'https://gdrb01.cern.ch:9000/p1DwtKGwKPFQodf5hpL4iQ' ,
          exitcode = '0' ,
          ) 
       ) , 
       Job (
       inputdir = '/afs/cern.ch/user/c/ctan/gangadir/workspace/Local/418/1/input/' ,
       outputdir = '/afs/cern.ch/user/c/ctan/gangadir/workspace/Local/418/1/output/' ,
       id = 1 ,
       outputdata = ATLASOutputDataset (
          output = ['guid:06e693fd-cf98-49ba-aa98-66cb9ff5f4d6'] ,
          location = 'srm://srm.cern.ch/castor/cern.ch/grid/atlas/scratch/ctan/ganga/418.1' 
          ) ,
       inputdata = DQ2Dataset (
          names = ...60 entries... , 
          ) ,
       backend = LCG (
          id = 'https://rb105.cern.ch:9000/Or5qfvu2nPTKmaBEGg_LeQ' ,
       ) , 
       Job (
       inputdir = '/afs/cern.ch/user/c/ctan/gangadir/workspace/Local/418/2/input/' ,
       outputdir = '/afs/cern.ch/user/c/ctan/gangadir/workspace/Local/418/2/output/' ,
       id = 2 ,
       outputdata = ATLASOutputDataset (
          output = ['guid:ce122022-31b1-4082-b5a7-fe8ff908d85a'] ,
          location = 'srm://srm.cern.ch/castor/cern.ch/grid/atlas/scratch/ctan/ganga/418.2' 
          ) ,
       inputdata = DQ2Dataset (
          names = ...60 entries... ,
          ) ,
       backend = LCG (
          id = 'https://rb105.cern.ch:9000/eH-4GUAnmDk6gv1EBi9Kkw' ,
          ) 
       ) ]
   )

  • Link to config level control mechanism.?
  • Introduction of simple method to effect print detail level e.g. setdetail(NEWLEVEL)
  • Enable display depth restriction
  • Remind user of current detail level

Implications for other components

Kuba: export function depends on printing?

None.

Proposal for controlling verbosity when printing Jobs

The Current Implementation:

Any object that inherits from GangaObject can be printed by calling the printTree method on GangaObject. This uses the Visitor pattern to traverse the GangaObject, iterating over each attribute in a depth first manner. The visitor used can be found in Ganga.GPIDev.Base.VPrinter. This VPrinter does a full dump of all copyable schema attributes.

Proposal for Summary Printing:

This proposal is based on the assertion that doing a full print is fine of most GangaObjects, and needs to be controlled only on a case by case basis. This aim of this proposal is then to give a mechanism for this control.

Default Verbosity

The default verbosity will be changed from full to summary, so that when a user does print j or >> j they will only see the summary. Users who wish to see a full print may call GangaObject.printTree() directly. We may introduce some convenience command into the GPI such as print_full in order to make this more usable. We encourage developers to produce output that is valid python syntax.

A New Visitor - VSummaryPrinter

The summary print will be initiated by calling a new printSummaryTree method on GangaObject. This will be very similar to printTree, but will use a modified visitor.

The VSummaryPrinter should inherit from VPrinter and overide a few of the methods to add in extra functionality. It is proposed that there will be two ways of providing summary information:

  • Via a new function pointer in the schema
  • By overriding the printSummaryTree method of GangaObject.

Function Pointers in the schema

As is currently done for the checkset property of a schema attribute, a plug-in author can write provide a function with a signature:

_summary_print(object, verbosity_level, interactive)

where verbosity_level will be related to the indent provided by the VPrinter (This is currently used to correctly format output) and perhaps some preference set in .gangarc. If a schema item has defined a _summary_print method then it will be called during the visitor traversal. This will be done both for simple attributes and component attributes. The method should return a formatted string that will then be printed at the appropriate indent level. The string should only contain the value of the attribute. It will later be printed with name = %(value)s. interactive is a boolean which specifies whether the print was triggered from iPython or by __str__ being called (print jobs vs jobs). Prints may want to use colour for interactive prints.

For the initial implementation of summary printing, verbosity_level will always be DEFAULT, to be defined in the VSummaryPrinter object. Further levels can be introduced as required. The VSummaryPrinter will provide a method to access the current verbosity level.

Overriding the printSummaryTree method

In order to support more complex summary printing objects and to provide a more natural implementation for objects that subclass GangaObject, if a component attribute has no _summary_print function pointer defined then the printSummaryTree will be called. A GangaObject is free to do what it likes in this function and is responsible for its own formatting. The current indentation level will be passed in to the method by its parent so that the formatting can be retained if required. A string containing a white space marker will also be passed. This is the text that should be inserted in front of the any output (on a line by line basis) in order to appear uniform with other entries. An example white space marker would be a string indent*tab_string. The method is free to ignore either the indent or the white space marker. The first line of output must not have a white space marker, so that the format will be correct for name = %(value)s type printing.

The proposed method signature for printSummaryTree is:

printSummaryTree(indent, verbosity_level, whitespace_marker)

The return type will be a formatted string which will be treated as for the function pointer above.


-- AlvinTan - 02 Dec 2006
Edit | Attach | Watch | Print version | History: r7 < r6 < r5 < r4 < r3 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r7 - 2007-11-20 - WillReece
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    ArdaGrid All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback