XMLSummary F.A.Q.

This page will be populated with Frequently Asked Questions, and discussion concerning the XMLSummary

Add your own Question

Did you not find the answer you were looking for? If not please submit your question here.

Current user-submitted FAQ's can be found here

What is the XMLSummary?

The XML summary is a shorter human and computer readable version of the stdout.

It stores key summary information about the job. Which files and events were processed, what counters were produced, what stage the job reached.

How can I turn on the XMLSummary?

In any Gaudi LHCb job:

from Configurables import LHCbApp

#write out a summary file called summary.xml
LHCbApp().XMLSummary="summary.xml"

#run the service, but do not write a file
LHCbApp().XMLSummary=""

#default, do not create the service.
LHCbApp().XMLSummary=None

The default options can just be included, as in DaVinciTutorial0

$XMLSUMMARYKERNELROOT/options/add-XMLSummary.py

Where is the service documented?

This FAQ.

• The talk at software week here.
• The XMLSummarySvc doxygen svn.
• XMLSummary configurable svn,
• LHCbApp Configurable svn.
• summary.py python modulesvn
• The internal help in the summary.py module

from XMLSummaryBase import summary

#make a summary object
sum=summary.Summary()
help(sum)

How can I configure the service?

It will work and be good for most things without you configuring anything. If you want to change the default behaviour you need to get a handle on the underlying service itself.

In any Gaudi LHCb job:

from Configurables import XMLSummarySvc

xmlsum=XMLSummarySvc("CounterSummarySvc")
#change the outputlevel
xmlsum.OutputLevel=1
#change which counters are saved with a list of regexp
xmlsum.CounterList=[ "MyAlg.*", ".*rlambert.*" ]
#change how often the xmlfile is written
#write/print out at least once each X changes.
xmlsum.UpdateFreq=500

How can I create/manipulate a summary outside Gaudi?

In the future you will be able to do many things in Ganga.

The XMLSummary is written/parsed with Python code. Helper modules have been written to help you with this. They are in the Kernel/XMLSummaryBase backage.

You should have access to this package by doing SetupProject . The modules schema and summary will then be on your pythonpath.

from XMLSummaryBase import summary

#make a summary object
sum=summary.Summary()
#load an existing summary into a tree-structure
sum.parse("pathtojob/summary.xml")

#merge a list files into a new file
sumTotal=summary.Merge(["pathtojob/summary.xml", "pathtojob2/summary.xml"])
#print to the screen
sumTotal.dump()

For example xml files see $XMLSUMMARYROOT/xml

For more example manipulations see $XMLSUMMARYROOT/test/test_parse_schema.py, or lookat the documentation of the code with python's help functions.

I currently parse stdout, is there a better way?

The XMLSummary is guaranteed to confirm to the schema, therefore it is much more stable than stdout and much smaller. It is therefore better to parse. Earlier in this FAQ we showed how to manipulate the summary directly in python, which gives you an easy-to-navigate tree structure.

How can I keep track of which files I've run over?

the <input> entry in the summary contains the list of input files which were opened, or attempted to be opened in your job.

How can I keep track of which events I've run over?

You can also sum up all the events processed in <input>.

Also, if you add an event counter, it will be recorded in the summary. A simple event/efficiency counter can be created at any stage using the algorithm EventCountAlg:

from Configurables import EventCountAlg

countalg=EventCountAlg("testcounter")
#don't forget to add it to your sequence, or to DaVinci user algs

How can I tell if my job was sucessful?

If the finalize step was reached, and the Success flag in the summary reads true.

Probably you should also check the number of input events is what you expected.

My job had an error, how can I tell?

You should know which files/events you expected to run over. If the output in the XMLSummary does not agree, you probably had an error. If the finalize step was not reached, you definately had an error.

My job had an error, where did it happen?

The error probably happened on the last event that was read. This will be equal to, or after, the last event documented in the XMLSummary file.

To know exactly, you can re-run your job, writing out the xmlfile at every event, by changing the UpdateFreq to 1, as discussed above.

My job had an error, but the XMLSummary still says successful!

When the error is a Gaudi exception, which is smoothly handled, or a StatusCode::FAILURE, algorithms and services are still finalised correctly. As long as your job reached the finalise step, the status will be recorded successful.

If the error is a segfault or if the job is killed for some reason, which is not smoothly handled, finalise will not be called. The XMLSummary will be left as it was in the last update, and record success as False.

How can I merge XMLSummaries in Ganga?

This will be implimented in the future. For the time being you can use the Merge function of the python summary module as discussed above.

I have a different CMT project, can I use/look at the summary?

The XMLSummaryBase package contains all that is required to read/manipulate the xml in python and nothing more. It will not pollute your environment.

Add a project dependency on LHCb v28r1. Add a use statement for XMLSummaryBase v* Kernel.

re-releasing your project should then be enough for you to use/manipulate the xml as given above.

What's the difference between a simple Counter and a StatEntity?

• A simple counter is the simplest way to store an integer, which is incremented when some situation occurs. A simple counter should be used when you have no reason to store anything other than a cumulative integer, because it gives you all required information.
• A stat entity is a powerful generalisation of a counter, which stores not only the total of the counter, but how many times it was changed, what the maximum and minimum additions were, and what the sum of squares of all entries were. A stat entity should only be used when you need more information than just one integer. For example, to calculate an efficiency, or an RMS.

How can I store my counter as a StatEntity?

For each algorithm and tool, you can change what gets recorded as a StatEntity, and what gets recorded as a Counter, through the properties StatEntityList and CounterList.

#StatEntityList is a list of regular expressions
MyAlg.StatEntityList=[ ".*fficiency.*" ]

By default, all counters are recorded as simple counters.

-- RobLambert - 2009-09-16