Building the statistics tables for Monte-Carlo simulations (Sim08)

Foreword

The user of a Monte Carlo sample is interested in a number of efficiency values to make use of his sample. This information is available in xml files produced by the simulation jobs and associated to each production. The statistics tables gather this information in a well-formatted way, and are published so that the information is easily accessible to users. It is the task of the MC contact to generate the statistics tables for the production of his WG. There are several steps:

Get the productions ID number for your request(s).

Use the Dirac webpage (alternate webpage) to retrieve the Dirac Production ID associated to your request. Follow 'Production' -> 'Request manager', and then use the left hand panel to filter the displayed request and pin down the request you are interested in. Click on the request, and select 'Production monitor'. This will bring you to another Dirac webpage, which you could use directly if you know the request ID number. Each request have several steps, each with a Dirac Production ID. You are interested only in the step of type 'MCSimulation'. Write down its ProdID, shown in the first column.

Do it for all the requests you want to process as the rest of the workflow can be performed on several prodIDs in one go.

Another way to get this information from the a bookkeeping path is to use the LHCbDirac command dirac-bookkeeping-prod4path. Mind that if there are spaces in the BK path, you should enclose it in quotes.

$ SetupProject LHCbDirac
$ dirac-bookkeeping-prod4path --BK '/MC/2012/Beam4000GeV-2012-MagDown-Nu2.5-Pythia8/Sim08c/Digi13/Trig0x409f0045/Reco14a/Stripping20NoPrescalingFlagged/41900006 ( ttbar_gg_1l17GeV ) /ALLSTREAMS.DST'
For BK path /MC/2012/Beam4000GeV-2012-MagDown-Nu2.5-Pythia8/Sim08c/Digi13/Trig0x409f0045/Reco14a/Stripping20NoPrescalingFlagged/41900006 ( ttbar_gg_1l17GeV ) /ALLSTREAMS.DST: 
Productions found (Merge): 32263 
Parent productions (MCSimulation): 32262 

You are interested only in the prodID for the 'MCSimulation'.

Produce the tables.

This step consists into retrieving the xml files of a set of ProdIDs and constructing statistics tables for them. The latter operation of done by a script called GaussStat.py. The full operation is steered by DownloadAndBuild.py, which runs GaussStat.py internally.

To set up the correct environment:

SetupProject LHCbDirac --runtime DBASE --use MCStatTools

Then cd to a directory with a lot of free space. Copy the most recent statistics tables of your WG in this directory:

cp $LHCBDOC/STATISTICS/SIM08STAT/<your WG>-WG/*.html .

Launch the script DownloadAndBuildStat.py giving a list of ProdID, comma-separated, and the number of xml files to use to build the tables (500 is a good value). E.g. for the ProdIDs 28381, 28383 and 500 jobs:

python $MCSTATTOOLSSCRIPTS/DownloadAndBuildStat.py -n500 28381,28383
This will try to get the logs first from the web, then from castor. It will also filter the logs for obviously malformed and incomplete log files. Then it calls the script that builds the tables up. It will then update the existing tables that you copied earlier with the information corresponding to the ProdIDs passed.

Publish the tables.

Create a JIRA task at https://its.cern.ch/jira/browse/LHCBGAUSS (you have to login with your CERN SSO account). Choose as Component Generators Statistics and either upload the statistics pages or give a pointer to a folder containing the updated tables in a public-readable area. The tables will then be added to the Gauss web site.

Edit | Attach | Watch | Print version | History: r7 < r6 < r5 < r4 < r3 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r5 - 2015-10-12 - AlexPearce
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LHCb All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback