Difference: DownloadAndBuildSim08 (6 vs. 7)

Revision 72017-02-20 - MarcOlivierBettler

Line: 1 to 1
 
META TOPICPARENT name="LHCbSimulation"

Building the statistics tables for Monte-Carlo simulations (Sim08)

Changed:
<
<
>
>
Please refer to DownloadAndBuild
 
Deleted:
<
<

Foreword

The user of a Monte Carlo sample is interested in a number of efficiency values to make use of his sample. This information is available in xml files produced by the simulation jobs and associated to each production. The statistics tables gather this information in a well-formatted way, and are published so that the information is easily accessible to users. It is the task of the MC contact to generate the statistics tables for the production of his WG. There are several steps:

Get the productions ID number for your request(s).

Use the Dirac webpage (alternate webpage) to retrieve the Dirac Production ID associated to your request. Follow 'Production' -> 'Request manager', and then use the left hand panel to filter the displayed request and pin down the request you are interested in. Click on the request, and select 'Production monitor'. This will bring you to another Dirac webpage, which you could use directly if you know the request ID number. Each request have several steps, each with a Dirac Production ID. You are interested only in the step of type 'MCSimulation'. Write down its ProdID, shown in the first column.

Do it for all the requests you want to process as the rest of the workflow can be performed on several prodIDs in one go.

Another way to get this information from the a bookkeeping path is to use the LHCbDirac command dirac-bookkeeping-prod4path. Mind that if there are spaces in the BK path, you should enclose it in quotes.

<!-- SyntaxHighlightingPlugin -->
$ SetupProject LHCbDirac
$ dirac-bookkeeping-prod4path --BK '/MC/2012/Beam4000GeV-2012-MagDown-Nu2.5-Pythia8/Sim08c/Digi13/Trig0x409f0045/Reco14a/Stripping20NoPrescalingFlagged/41900006 ( ttbar_gg_1l17GeV ) /ALLSTREAMS.DST'
For BK path /MC/2012/Beam4000GeV-2012-MagDown-Nu2.5-Pythia8/Sim08c/Digi13/Trig0x409f0045/Reco14a/Stripping20NoPrescalingFlagged/41900006 ( ttbar_gg_1l17GeV ) /ALLSTREAMS.DST: 
Productions found (Merge): 32263 
Parent productions (MCSimulation): 32262 
<!-- end SyntaxHighlightingPlugin -->

You are interested only in the prodID for the 'MCSimulation'.

Produce the tables.

This step consists into retrieving the xml files of a set of ProdIDs and constructing statistics tables for them. The latter operation of done by a script called GaussStat.py. The full operation is steered by DownloadAndBuild.py, which runs GaussStat.py internally.

To set up the correct environment:

<!-- SyntaxHighlightingPlugin -->
SetupProject LHCbDirac --runtime DBASE --use MCStatTools
<!-- end SyntaxHighlightingPlugin -->

As of MCStatTools v3r* please, use the new way of setting up the environment:

<!-- SyntaxHighlightingPlugin -->
lb-run --use MCStatTools LHCbDirac bash
<!-- end SyntaxHighlightingPlugin -->

Then cd to a directory with a lot of free space. Copy the most recent statistics tables of your WG in this directory:

<!-- SyntaxHighlightingPlugin -->
cp $LHCBDOC/STATISTICS/SIM08STAT/<your WG>-WG/*.html .
<!-- end SyntaxHighlightingPlugin -->

Launch the script DownloadAndBuildStat.py giving a list of ProdID, comma-separated, and the number of xml files to use to build the tables (500 is a good value). E.g. for the ProdIDs 28381, 28383 and 500 jobs:

<!-- SyntaxHighlightingPlugin -->
python $MCSTATTOOLSSCRIPTS/DownloadAndBuildStat.py -n500 28381,28383
<!-- end SyntaxHighlightingPlugin -->
This will try to get the logs first from the web, then from castor. It will also filter the logs for obviously malformed and incomplete log files. Then it calls the script that builds the tables up. It will then update the existing tables that you copied earlier with the information corresponding to the ProdIDs passed.

Publish the tables.

Create a JIRA task at https://its.cern.ch/jira/browse/LHCBGAUSS (you have to login with your CERN SSO account). Choose as Component Generators Statistics and either upload the statistics pages or give a pointer to a folder containing the updated tables in a public-readable area. The tables will then be added to the Gauss web site.

  |*META PREFERENCE*|name="SYNTAXHIGHLIGHTINGPLUGIN_FORMAT_PREPEND" title="SYNTAXHIGHLIGHTINGPLUGIN_FORMAT_PREPEND" type="Set" value="
"|

 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback