Difference: DC06StrippingNewDST (1 vs. 3)

Revision 32008-04-15 - LesyaShchutska

Line: 1 to 1
 
META TOPICPARENT name="DC06StrippingHowTo"

How to run on re-stripped DC06 data with the new format DSTs

Line: 225 to 225
 
  • setc_newdst_v31.xml: File catalogue for SETC for new DSTs made from v31r11 reconstructed DSTs
Deleted:
<
<
META FILEATTACHMENT attachment="setc_v30.opts" attr="" comment="SETC for v30r17 reconstructed DSTs" date="1207861893" name="setc_v30.opts" path="setc_v30.opts" size="28615" stream="setc_v30.opts" user="Main.LesyaShchutska" version="2"
META FILEATTACHMENT attachment="setc_v30_lfn.opts" attr="" comment="List of LFNs for SETC for v30r17 reconstructed DSTs" date="1207861964" name="setc_v30_lfn.opts" path="setc_v30_lfn.opts" size="24841" stream="setc_v30_lfn.opts" user="Main.LesyaShchutska" version="2"
META FILEATTACHMENT attachment="setc_v30.xml" attr="" comment="File catalogue for SETC for v30r17 reconstructed DSTs" date="1207861936" name="setc_v30.xml" path="setc_v30.xml" size="64992" stream="setc_v30.xml" user="Main.LesyaShchutska" version="2"
META FILEATTACHMENT attachment="dst_v30.xml" attr="" comment="File catalogue for v30r17 reconstructed DSTs" date="1207861588" name="dst_v30.xml" path="dst_v30.xml" size="142853" stream="dst_v30.xml" user="Main.LesyaShchutska" version="2"
META FILEATTACHMENT attachment="newDST_v30.opts" attr="" comment="New DSTs made from v30r17 reconstructed DSTs" date="1207861716" name="newDST_v30.opts" path="newDST_v30.opts" size="26394" stream="newDST_v30.opts" user="Main.LesyaShchutska" version="2"
META FILEATTACHMENT attachment="newDST_v30_lfn.opts" attr="" comment="List of LFNs for new DSTs made from v30r17 reconstructed DSTs" date="1207861783" name="newDST_v30_lfn.opts" path="newDST_v30_lfn.opts" size="22617" stream="newDST_v30_lfn.opts" user="Main.LesyaShchutska" version="2"
META FILEATTACHMENT attachment="newDST_v30.xml" attr="" comment="File catalogue for new DSTs made from v30r17 reconstructed DSTs" date="1207861761" name="newDST_v30.xml" path="newDST_v30.xml" size="65436" stream="newDST_v30.xml" user="Main.LesyaShchutska" version="2"
META FILEATTACHMENT attachment="setc_newdst_v30.opts" attr="" comment="SETC for new DSTs made from v30r17 reconstructed DSTs" date="1207861806" name="setc_newdst_v30.opts" path="setc_newdst_v30.opts" size="2145" stream="setc_newdst_v30.opts" user="Main.LesyaShchutska" version="1"
META FILEATTACHMENT attachment="setc_newdst_v30_lfn.opts" attr="" comment="List of LFNs for SETC for new DSTs made from v30r17 reconstructed DSTs" date="1207861864" name="setc_newdst_v30_lfn.opts" path="setc_newdst_v30_lfn.opts" size="1942" stream="setc_newdst_v30_lfn.opts" user="Main.LesyaShchutska" version="1"
META FILEATTACHMENT attachment="setc_newdst_v30.xml" attr="" comment="File catalogue for SETC for new DSTs made from v30r17 reconstructed DSTs" date="1207861835" name="setc_newdst_v30.xml" path="setc_newdst_v30.xml" size="3804" stream="setc_newdst_v30.xml" user="Main.LesyaShchutska" version="2"
META FILEATTACHMENT attachment="setc_v31.opts" attr="" comment="SETC for v31r11 reconstructed DSTs" date="1207854712" name="setc_v31.opts" path="setc_v31.opts" size="28864" stream="setc_v31.opts" user="Main.LesyaShchutska" version="1"
META FILEATTACHMENT attachment="setc_v31_lfn.opts" attr="" comment="List of LFNs for SETC for v31r11 reconstructed DSTs" date="1207854744" name="setc_v31_lfn.opts" path="setc_v31_lfn.opts" size="25046" stream="setc_v31_lfn.opts" user="Main.LesyaShchutska" version="1"
META FILEATTACHMENT attachment="setc_v31.xml" attr="" comment="File catalogue for SETC for v31r11 reconstructed DSTs" date="1207854778" name="setc_v31.xml" path="setc_v31.xml" size="65578" stream="setc_v31.xml" user="Main.LesyaShchutska" version="1"
META FILEATTACHMENT attachment="dst_v31.xml" attr="" comment="File catalogue for v31r11 reconstructed DSTs" date="1207854809" name="dst_v31.xml" path="dst_v31.xml" size="143819" stream="dst_v31.xml" user="Main.LesyaShchutska" version="1"
META FILEATTACHMENT attachment="newDST_v31.opts" attr="" comment="New DSTs made from v31r11 reconstructed DSTs" date="1207854841" name="newDST_v31.opts" path="newDST_v31.opts" size="26637" stream="newDST_v31.opts" user="Main.LesyaShchutska" version="1"
META FILEATTACHMENT attachment="newDST_v31_lfn.opts" attr="" comment="List of LFNs for new DSTs made from v31r11 reconstructed DSTs" date="1207854866" name="newDST_v31_lfn.opts" path="newDST_v31_lfn.opts" size="22822" stream="newDST_v31_lfn.opts" user="Main.LesyaShchutska" version="1"
META FILEATTACHMENT attachment="newDST_v31.xml" attr="" comment="File catalogue for new DSTs made from v31r11 reconstructed DSTs" date="1207854890" name="newDST_v31.xml" path="newDST_v31.xml" size="66026" stream="newDST_v31.xml" user="Main.LesyaShchutska" version="1"
META FILEATTACHMENT attachment="setc_newdst_v31.opts" attr="" comment="SETC for new DSTs made from v31r11 reconstructed DSTs" date="1207854927" name="setc_newdst_v31.opts" path="setc_newdst_v31.opts" size="2146" stream="setc_newdst_v31.opts" user="Main.LesyaShchutska" version="1"
META FILEATTACHMENT attachment="setc_newdst_v31_lfn.opts" attr="" comment="List of LFNs for SETC for new DSTs made from v31r11 reconstructed DSTs" date="1207854956" name="setc_newdst_v31_lfn.opts" path="setc_newdst_v31_lfn.opts" size="1942" stream="setc_newdst_v31_lfn.opts" user="Main.LesyaShchutska" version="1"
META FILEATTACHMENT attachment="setc_newdst_v31.xml" attr="" comment="File catalogue for SETC for new DSTs made from v31r11 reconstructed DSTs" date="1207855006" name="setc_newdst_v31.xml" path="setc_newdst_v31.xml" size="3804" stream="setc_newdst_v31.xml" user="Main.LesyaShchutska" version="1"
 
META TOPICMOVED by="LesyaShchutska" date="1207831508" from="GeneratorServices.DC06StrippingNewDST" to="LHCb.DC06StrippingNewDST"

Revision 22008-04-10 - LesyaShchutska

Line: 1 to 1
 
META TOPICPARENT name="DC06StrippingHowTo"

How to run on re-stripped DC06 data with the new format DSTs

Line: 19 to 19
 
DST, the full stripped DST
This is the main output of the stripping: the complete DST with all what you need in it. The detailed set of instructions how to use it is available here.
SETC, the stripped event-tag collection
This event-tag collection allows you to navigate directly to the stripped DSTs containing events selected by your preselections. Examples are given below.
Changed:
<
<
newDST, the new stripped DST
This is the output data of the re-stripping. It contains MC truth information, /Event/Phys containers with the all intermediate particles and B-candidates selected by preselections, protoparticles, selection result and links to the DSTs in case other information is needed. Examples of using are given below.
>
>
newDST, the new stripped DST
This is the output data of the re-stripping. It contains MC truth information, /Event/Phys containers with the all intermediate particles and B-candidates selected by preselections, protoparticles, selection result and links to the DSTs in case the other information is needed. Examples of using are given below.
 
newSETC, the stripped event-tag collection for newDST
This event-tag collection allows you to navigate directly to the new DSTs containing events selected by your preselections. Examples are given below.

Why were the new DSTs produced.

Line: 29 to 29
 events and for the result of preselections. It was checked that e.g. selection algorithm for ${\mathrm{B^0_s}} \to \phi  \gamma$ runs four times faster on newSETC than on SETC so that processing of the stripped events takes 1.5 hour.
Changed:
<
<
Timing example for using different approaches using ${\mathrm{B^0_s}} \to \phi  \gamma$ selection:
  • DST:
  • SETC: 1630 s on ~3.2k events selected by PreselXb2GammaX <=> ~72k stripped events => 5.1 hours for all data
  • newDST: 1120 s on ~72k ev. => 3.5 hours for all data
  • newSETC: 460 s on ~3.2k events selected by PreselXb2GammaX <=> ~72k stripped events => 1.4 hours for all data
>
>
Timing example for using different approaches with ${\mathrm{B^0_s}} \to \phi  \gamma$ selection:
  • DST: 3310 s on ~36k events $\Rightarrow$ 21 hours for all data (note that here both PreselXb2GammaX and SelBs2PhiGamma ran in sequence )
  • SETC: 1630 s on ~3.2k events selected by PreselXb2GammaX $\Leftrightarrow$ ~72k stripped events $\Rightarrow$ 5.1 hours for all data
  • newDST: 1120 s on ~72k events $\Rightarrow$ 3.5 hours for all data
  • newSETC: 460 s on ~3.2k events selected by PreselXb2GammaX $\Leftrightarrow$ ~72k stripped events $\Rightarrow$ 1.4 hours for all data
 

How to use stripped data with the new files

There are three ways to get access to the result of your preselection. The third one is the fastest.
Line: 42 to 42
 Here's the recipe to get the result of your preselection and navigate to interesting events form DSTs.

Get the files with SETC

Changed:
<
<
You can find the file to be included to your options here: ~lshchuts/public/DST_v30/setc_v30.opts or ~lshchuts/public/DST_v31/setc_v31.opts. Add the name of your preselection as a SEL='PreselXXX>0' requirement at the end of each line. You can add as many selections as you like combined with && or ||. It's basically a root TCut. In our example that would be like
>
>
You can find the file to be included to your options here: setc_v30.opts or setc_v31.opts. Add the name of your preselection as a SEL='PreselXXX>0' requirement at the end of each line. You can add as many selections as you like combined with && or ||. It's basically a root TCut. For example that would be like
 %SYNTAX{ syntax="cpp"}% EventSelector.Input = { "COLLECTION='TagCreator/1'
Line: 57 to 57
 

Include a file catalogue for DSTs to your options

Next you need to tell where to find the files the ETC points to. This is done declaring a file catalogue which you can copy from the corresponding directory:
Changed:
<
<
~lshchuts/public/DST_v30/dst_v30.xml or ~lshchuts/public/DST_v31/dst_v31.xml
>
>
dst_v30.xml or dst_v31.xml
 an then add this line to your options:
<!-- SyntaxHighlightingPlugin -->
FileCatalog.Catalogs  = { "xmlcatalog_file:dst_v30.xml" } ;
<!-- end SyntaxHighlightingPlugin -->
Changed:
<
<
The file catalogue contains blocks like these
>
>
The file catalogue contains blocks like this one:
 %SYNTAX{ syntax="cpp"}%
Line: 75 to 75
  %ENDSYNTAX%
Changed:
<
<
which describe the location of the file. The File ID is what is stored in the ETC. Hence if you forget to declare the file catalogue you will get an error like
>
>
These blocks describe the location of the file. The File ID is what is stored in the ETC. Hence if you forget to declare the file catalogue you will get an error like
 
<!-- SyntaxHighlightingPlugin -->
PoolRootTreeEvtCnvSvc  ERROR Error: connectDatabase> Cannot connect to database: 
   PFN=gfal:guid:DCC1AB97-079D-DC11-9DBC-0030488A1192
<!-- end SyntaxHighlightingPlugin -->
Deleted:
<
<

Run on SETC with ganga

In order to run it with ganga you need logical file names (LFN) as the input and file catalogue that contain IDs for the SETC files. The files with LFNs can be found here: ~lshchuts/public/DST_v30/setc_v30_lfn.opts or ~lshchuts/public/DST_v31/setc_v31_lfn.opts File catalogues: ~lshchuts/public/DST_v30/setc_v30.xml or ~lshchuts/public/DST_v31/setc_v31.xml

<!-- SyntaxHighlightingPlugin -->
ApplicationMgr.ExtSvc += { "TagCollectionSvc/EvtTupleSvc" };
FileCatalog.Catalogs  = { "xmlcatalog_file:dst_v30.xml" } ;
FileCatalog.Catalogs += { "xmlcatalog_file:setc_v30.xml" } ;
#include "setc_v30_lfn.opts"
<!-- end SyntaxHighlightingPlugin -->

It surely works for lxbatch i.e. LSF() backend but won't work for Dirac().

 

Run on new re-stripped DST

Use the new re-stripped DST as input but look only at events that have passed your preselection. To achieve this you have to read the SelResult tag of each event to find the ones relevant for you. This way is faster than running on SETC that navigate you to the full stripped DSTs.

Get the files with new DSTs

Changed:
<
<
You can find the file to be included to your options here: ~lshchuts/public/DST_v30/newDST_v30.opts or ~lshchuts/public/DST_v31/newDST_v31.opts.
>
>
You can find the file to be included to your options here: newDST_v30.opts or newDST_v31.opts.
 

Using CheckSelResult algorithm

You should put all your analysis code in a sequencer starting with a CheckSelResult algorithm.
Line: 124 to 108
 %ENDSYNTAX%

Here for instance you look for events having passed the PreselXb2GammaX preselection and run the selection only for those.

Changed:
<
<
Be aware of absence any + signs in the sequence declarations. You don't have to make all the standard particles again since they are already stored in the corresponding containers. Also you can select the B-candidate from the "Phys/PreselXXX" container. So there is no need to re-run your preselection you can directly apply the selection algorithm.
>
>
Be aware of absence any + signs in the sequence declarations. You don't have to make all the standard particles again since they are already stored in the corresponding containers. Also you can select the B-candidate from the "Phys/PreselXXX" container. So there is no need to re-run your preselection you can directly apply the selection algorithm.
 

Include a file catalogue to your options

Changed:
<
<
These new DSTs also have links to the full stripped DSTs. In order to tell where these DSTs can be find you should add the same file catalogue as when using SETC:
>
>
These new DSTs also have links to the full stripped DSTs. In order to tell where these DSTs can be find you should add the same file catalogue as when using SETC (dst_v30.xml or dst_v31.xml):
 %SYNTAX{ syntax="cpp"}% FileCatalog.Catalogs = { "xmlcatalog_file:dst_v30.xml" } ; #include "newDST_v30.opts"
Line: 135 to 119
 

Run on new DSTs with ganga

In order to run it with ganga you need logical file names (LFN) as the input:
Changed:
<
<
~lshchuts/public/DST_v30/newDST_v30_lfn.opts or ~lshchuts/public/DST_v31/newDST_v31_lfn.opts
>
>
newDST_v30_lfn.opts or newDST_v31_lfn.opts
 and file catalogue that contain IDs for the newDST files:
Changed:
<
<
~lshchuts/public/DST_v30/newDST_v30.xml or ~lshchuts/public/DST_v31/newDST_v31.xml
>
>
newDST_v30.xml or newDST_v31.xml
  %SYNTAX{ syntax="cpp"}% FileCatalog.Catalogs = { "xmlcatalog_file:dst_v30.xml" } ;
Line: 145 to 129
 #include "newDST_v30_lfn.opts" %ENDSYNTAX%
Added:
>
>
It surely works for lxbatch i.e. LSF() backend but won't work for Dirac().
 

Run on stripped ETC for new stripped DSTs.

Changed:
<
<
This way is the fastest one and depending on the selection algorithm one even can run through the all stripped bbbar data interactively. Here's the recipe to get the result of your preselection and navigate to interesting events form the new stripped DSTs. Again one doesn't need to make the standard particles and can put only the selection algorithm in the
>
>
This way is the fastest one and depending on the selection algorithm one even can run through the all stripped $b\bar{b}$-data interactively. Here's the recipe to get the result of your preselection and navigate to interesting events form the new stripped DSTs. Again one doesn't need to make the standard particles and can put only the selection algorithm in the
 sequence:

%SYNTAX{ syntax="cpp"}%

Line: 164 to 150
 %ENDSYNTAX%

Get the files with new SETC

Changed:
<
<
You can find the file to be included to your options here: ~lshchuts/public/DST_v30/setc_newdst_v30.opts or ~lshchuts/public/DST_v31/setc_newdst_v31.opts.
>
>
You can find the file to be included to your options here: setc_newdst_v30.opts or setc_newdst_v31.opts.
 Add the name of your preselection as a SEL='PreselXXX>0' requirement at the end of each line. You can add as many selections as you like combined with && or ||. In our example that would be like %SYNTAX{ syntax="cpp"}% EventSelector.Input = {
Line: 178 to 164
 

Include a file catalogues for new DSTs and DSTs to your options

Next you need to tell where to find the files the SETC points to and where to find files where newDSTs point to. This is done declaring a file catalogue which you can take from the corresponding directory:
Changed:
<
<
~lshchuts/public/DST_v30/dst_v30.xml or ~lshchuts/public/DST_v31/dst_v31.xml
>
>
dst_v30.xml or dst_v31.xml and newDST_v30.xml or newDST_v31.xml
 an then add this line to your options: %SYNTAX{ syntax="cpp"}% FileCatalog.Catalogs = { "xmlcatalog_file:dst_v30.xml" } ;
Line: 186 to 174
 #include "setc_newdst_v30.opts" %ENDSYNTAX%
Deleted:
<
<

Run on new SETC with ganga

In order to run it with ganga you need logical file names (LFN) as the input: ~lshchuts/public/DST_v30/setc_newdst_v30_lfn.opts or ~lshchuts/public/DST_v31/setc_newdst_v31_lfn.opts and file catalogue that contain IDs for the SETC for newDSTs files: ~lshchuts/public/DST_v30/setc_newdst_v30.xml or ~lshchuts/public/DST_v31/setc_newdst_v31.xml

<!-- SyntaxHighlightingPlugin -->
FileCatalog.Catalogs  = { "xmlcatalog_file:dst_v30.xml" } ;
FileCatalog.Catalogs += { "xmlcatalog_file:newDST_v30.xml" } ;
FileCatalog.Catalogs += { "xmlcatalog_file:setc_newdst_v30.xml" } ;
#include "setc_newdst_v30_lfn.opts"
<!-- end SyntaxHighlightingPlugin -->
 

How much BB is that?

All the data sets were produced from the DC06 - Stripping-v30-lumi2 and the DC06 - Stripping-v31-lumi2 stripped DST data available
Changed:
<
<
in the bookkeping. This means that new files correspond to 804584 stripped events and 22085009 bbbar inclusive events. So this number should be used for the final evaluation. However I would like to emphasize once more that the safe way is to use v30r17 configuration
>
>
in the bookkeping. This means that new files correspond to 804584 stripped events and 22085009 $b\bar{b}$ inclusive events. So this number should be used for the final evaluation. However it's worth to emphasize once more that the safe way is to use v30r17 configuration
 since it reproduces the original stripping in a correct way. But with the gain in time which the new DSTs provide it is not a problem to run on both sets of events and check the consistency of results.

-- LesyaShchutska - 10 Apr 2008

Added:
>
>

  • setc_v30.xml: File catalogue for SETC for v30r17 reconstructed DSTs

  • dst_v30.xml: File catalogue for v30r17 reconstructed DSTs

  • newDST_v30.xml: File catalogue for new DSTs made from v30r17 reconstructed DSTs

  • setc_newdst_v30.xml: File catalogue for SETC for new DSTs made from v30r17 reconstructed DSTs

  • setc_v31.xml: File catalogue for SETC for v31r11 reconstructed DSTs

  • dst_v31.xml: File catalogue for v31r11 reconstructed DSTs

  • newDST_v31.xml: File catalogue for new DSTs made from v31r11 reconstructed DSTs

  • setc_newdst_v31.xml: File catalogue for SETC for new DSTs made from v31r11 reconstructed DSTs

META FILEATTACHMENT attachment="setc_v30.opts" attr="" comment="SETC for v30r17 reconstructed DSTs" date="1207861893" name="setc_v30.opts" path="setc_v30.opts" size="28615" stream="setc_v30.opts" user="Main.LesyaShchutska" version="2"
META FILEATTACHMENT attachment="setc_v30_lfn.opts" attr="" comment="List of LFNs for SETC for v30r17 reconstructed DSTs" date="1207861964" name="setc_v30_lfn.opts" path="setc_v30_lfn.opts" size="24841" stream="setc_v30_lfn.opts" user="Main.LesyaShchutska" version="2"
META FILEATTACHMENT attachment="setc_v30.xml" attr="" comment="File catalogue for SETC for v30r17 reconstructed DSTs" date="1207861936" name="setc_v30.xml" path="setc_v30.xml" size="64992" stream="setc_v30.xml" user="Main.LesyaShchutska" version="2"
META FILEATTACHMENT attachment="dst_v30.xml" attr="" comment="File catalogue for v30r17 reconstructed DSTs" date="1207861588" name="dst_v30.xml" path="dst_v30.xml" size="142853" stream="dst_v30.xml" user="Main.LesyaShchutska" version="2"
META FILEATTACHMENT attachment="newDST_v30.opts" attr="" comment="New DSTs made from v30r17 reconstructed DSTs" date="1207861716" name="newDST_v30.opts" path="newDST_v30.opts" size="26394" stream="newDST_v30.opts" user="Main.LesyaShchutska" version="2"
META FILEATTACHMENT attachment="newDST_v30_lfn.opts" attr="" comment="List of LFNs for new DSTs made from v30r17 reconstructed DSTs" date="1207861783" name="newDST_v30_lfn.opts" path="newDST_v30_lfn.opts" size="22617" stream="newDST_v30_lfn.opts" user="Main.LesyaShchutska" version="2"
META FILEATTACHMENT attachment="newDST_v30.xml" attr="" comment="File catalogue for new DSTs made from v30r17 reconstructed DSTs" date="1207861761" name="newDST_v30.xml" path="newDST_v30.xml" size="65436" stream="newDST_v30.xml" user="Main.LesyaShchutska" version="2"
META FILEATTACHMENT attachment="setc_newdst_v30.opts" attr="" comment="SETC for new DSTs made from v30r17 reconstructed DSTs" date="1207861806" name="setc_newdst_v30.opts" path="setc_newdst_v30.opts" size="2145" stream="setc_newdst_v30.opts" user="Main.LesyaShchutska" version="1"
META FILEATTACHMENT attachment="setc_newdst_v30_lfn.opts" attr="" comment="List of LFNs for SETC for new DSTs made from v30r17 reconstructed DSTs" date="1207861864" name="setc_newdst_v30_lfn.opts" path="setc_newdst_v30_lfn.opts" size="1942" stream="setc_newdst_v30_lfn.opts" user="Main.LesyaShchutska" version="1"
META FILEATTACHMENT attachment="setc_newdst_v30.xml" attr="" comment="File catalogue for SETC for new DSTs made from v30r17 reconstructed DSTs" date="1207861835" name="setc_newdst_v30.xml" path="setc_newdst_v30.xml" size="3804" stream="setc_newdst_v30.xml" user="Main.LesyaShchutska" version="2"
META FILEATTACHMENT attachment="setc_v31.opts" attr="" comment="SETC for v31r11 reconstructed DSTs" date="1207854712" name="setc_v31.opts" path="setc_v31.opts" size="28864" stream="setc_v31.opts" user="Main.LesyaShchutska" version="1"
META FILEATTACHMENT attachment="setc_v31_lfn.opts" attr="" comment="List of LFNs for SETC for v31r11 reconstructed DSTs" date="1207854744" name="setc_v31_lfn.opts" path="setc_v31_lfn.opts" size="25046" stream="setc_v31_lfn.opts" user="Main.LesyaShchutska" version="1"
META FILEATTACHMENT attachment="setc_v31.xml" attr="" comment="File catalogue for SETC for v31r11 reconstructed DSTs" date="1207854778" name="setc_v31.xml" path="setc_v31.xml" size="65578" stream="setc_v31.xml" user="Main.LesyaShchutska" version="1"
META FILEATTACHMENT attachment="dst_v31.xml" attr="" comment="File catalogue for v31r11 reconstructed DSTs" date="1207854809" name="dst_v31.xml" path="dst_v31.xml" size="143819" stream="dst_v31.xml" user="Main.LesyaShchutska" version="1"
META FILEATTACHMENT attachment="newDST_v31.opts" attr="" comment="New DSTs made from v31r11 reconstructed DSTs" date="1207854841" name="newDST_v31.opts" path="newDST_v31.opts" size="26637" stream="newDST_v31.opts" user="Main.LesyaShchutska" version="1"
META FILEATTACHMENT attachment="newDST_v31_lfn.opts" attr="" comment="List of LFNs for new DSTs made from v31r11 reconstructed DSTs" date="1207854866" name="newDST_v31_lfn.opts" path="newDST_v31_lfn.opts" size="22822" stream="newDST_v31_lfn.opts" user="Main.LesyaShchutska" version="1"
META FILEATTACHMENT attachment="newDST_v31.xml" attr="" comment="File catalogue for new DSTs made from v31r11 reconstructed DSTs" date="1207854890" name="newDST_v31.xml" path="newDST_v31.xml" size="66026" stream="newDST_v31.xml" user="Main.LesyaShchutska" version="1"
META FILEATTACHMENT attachment="setc_newdst_v31.opts" attr="" comment="SETC for new DSTs made from v31r11 reconstructed DSTs" date="1207854927" name="setc_newdst_v31.opts" path="setc_newdst_v31.opts" size="2146" stream="setc_newdst_v31.opts" user="Main.LesyaShchutska" version="1"
META FILEATTACHMENT attachment="setc_newdst_v31_lfn.opts" attr="" comment="List of LFNs for SETC for new DSTs made from v31r11 reconstructed DSTs" date="1207854956" name="setc_newdst_v31_lfn.opts" path="setc_newdst_v31_lfn.opts" size="1942" stream="setc_newdst_v31_lfn.opts" user="Main.LesyaShchutska" version="1"
META FILEATTACHMENT attachment="setc_newdst_v31.xml" attr="" comment="File catalogue for SETC for new DSTs made from v31r11 reconstructed DSTs" date="1207855006" name="setc_newdst_v31.xml" path="setc_newdst_v31.xml" size="3804" stream="setc_newdst_v31.xml" user="Main.LesyaShchutska" version="1"
 
META TOPICMOVED by="LesyaShchutska" date="1207831508" from="GeneratorServices.DC06StrippingNewDST" to="LHCb.DC06StrippingNewDST"

Revision 12008-04-10 - LesyaShchutska

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="DC06StrippingHowTo"

How to run on re-stripped DC06 data with the new format DSTs

Introduction

This page is the follow up from the official DC06 Stripping page by Patrick Koppenburg and it introduces the new convenient way to run on DC06 stripped data.

The problems with official stripping results

  • The first problem with stripped DSTs is that the SelResult container is not saved on the DST for the latest stripping. So in order to get the result of specific preselection one should always rerun it.
  • The second one is that one cannot use the recipe for SETC now since the SETC in bookkeping do not contain the results of preselections.

Two sets of new data.

The stripping was rerun with DaVinci v19r11 on the stripped DSTs. Since there are two sets of DSTs corresponding to reconstruction with Brunel v30r17 or Brunel v31r11 ( the corresponding configurations are DC06 - stripping-v30-lumi2 and DC06 - stripping-v31-lumi2) the two sets of SETC, newDSTs and newSETCs were produced. The difference between the reconstructions leads to the fact that efficiency of all preselections on v30r17 reconstructed stripped data is ~94% and on v31r11 only ~60%. These numbers can vary for the individual preselections. So the safe way is to use v30r17 reconstructed data and than make a check of your final selection on v31r11 data.

Output data

Now it is possible to use the following data formats:
DST, the full stripped DST
This is the main output of the stripping: the complete DST with all what you need in it. The detailed set of instructions how to use it is available here.
SETC, the stripped event-tag collection
This event-tag collection allows you to navigate directly to the stripped DSTs containing events selected by your preselections. Examples are given below.
newDST, the new stripped DST
This is the output data of the re-stripping. It contains MC truth information, /Event/Phys containers with the all intermediate particles and B-candidates selected by preselections, protoparticles, selection result and links to the DSTs in case other information is needed. Examples of using are given below.
newSETC, the stripped event-tag collection for newDST
This event-tag collection allows you to navigate directly to the new DSTs containing events selected by your preselections. Examples are given below.

Why were the new DSTs produced.

When analyzing stripped events it often appears that one should rerun the preselection/selection algorithm in order to include some new features that were absent in the previous version. So waiting the output makes the analysis really time consuming. The new concept of stripped DSTs provides a faster access to the stripped events and for the result of preselections. It was checked that e.g. selection algorithm for ${\mathrm{B^0_s}} \to \phi  \gamma$ runs four times faster on newSETC than on SETC so that processing of the stripped events takes 1.5 hour.

Timing example for using different approaches using ${\mathrm{B^0_s}} \to \phi  \gamma$ selection:

  • DST:
  • SETC: 1630 s on ~3.2k events selected by PreselXb2GammaX <=> ~72k stripped events => 5.1 hours for all data
  • newDST: 1120 s on ~72k ev. => 3.5 hours for all data
  • newSETC: 460 s on ~3.2k events selected by PreselXb2GammaX <=> ~72k stripped events => 1.4 hours for all data

How to use stripped data with the new files

There are three ways to get access to the result of your preselection. The third one is the fastest.

Run on stripped ETC

Here's the recipe to get the result of your preselection and navigate to interesting events form DSTs.

Get the files with SETC

You can find the file to be included to your options here: ~lshchuts/public/DST_v30/setc_v30.opts or ~lshchuts/public/DST_v31/setc_v31.opts. Add the name of your preselection as a SEL='PreselXXX>0' requirement at the end of each line. You can add as many selections as you like combined with && or ||. It's basically a root TCut. In our example that would be like
<!-- SyntaxHighlightingPlugin -->
EventSelector.Input   = {
"COLLECTION='TagCreator/1' 
DATAFILE='PFN:castor:/castor/cern.ch/user/l/lshchuts/stripping/SETC_v30/1.root' TYP='POOL_ROOT' 
SEL='(PreselXb2GammaX>0)'",
[...]
};
ApplicationMgr.ExtSvc  += { "TagCollectionSvc/EvtTupleSvc" };
<!-- end SyntaxHighlightingPlugin -->
This last line tells DaVinci what to use to understand the ETC.

Include a file catalogue for DSTs to your options

Next you need to tell where to find the files the ETC points to. This is done declaring a file catalogue which you can copy from the corresponding directory: ~lshchuts/public/DST_v30/dst_v30.xml or ~lshchuts/public/DST_v31/dst_v31.xml an then add this line to your options:
<!-- SyntaxHighlightingPlugin -->
FileCatalog.Catalogs  = { "xmlcatalog_file:dst_v30.xml" } ;
<!-- end SyntaxHighlightingPlugin -->

The file catalogue contains blocks like these

<!-- SyntaxHighlightingPlugin -->
<File ID="DCC1AB97-079D-DC11-9DBC-0030488A1192">
 <physical>
  <pfn name="castor:/castor/cern.ch/grid/lhcb/production/DC06/v1r0/00002000/DST/0000/00002000_00000107_2.dst" filetype="ROOT_All"/>
 </physical>
 <logical>
  <lfn name="/lhcb/production/DC06/v1r0/00002000/DST/0000/00002000_00000107_2.dst"/>
 </logical>
</File>
<!-- end SyntaxHighlightingPlugin -->
which describe the location of the file. The File ID is what is stored in the ETC. Hence if you forget to declare the file catalogue you will get an error like
<!-- SyntaxHighlightingPlugin -->
PoolRootTreeEvtCnvSvc  ERROR Error: connectDatabase> Cannot connect to database: 
   PFN=gfal:guid:DCC1AB97-079D-DC11-9DBC-0030488A1192
<!-- end SyntaxHighlightingPlugin -->

Run on SETC with ganga

In order to run it with ganga you need logical file names (LFN) as the input and file catalogue that contain IDs for the SETC files. The files with LFNs can be found here: ~lshchuts/public/DST_v30/setc_v30_lfn.opts or ~lshchuts/public/DST_v31/setc_v31_lfn.opts File catalogues: ~lshchuts/public/DST_v30/setc_v30.xml or ~lshchuts/public/DST_v31/setc_v31.xml

<!-- SyntaxHighlightingPlugin -->
ApplicationMgr.ExtSvc += { "TagCollectionSvc/EvtTupleSvc" };
FileCatalog.Catalogs  = { "xmlcatalog_file:dst_v30.xml" } ;
FileCatalog.Catalogs += { "xmlcatalog_file:setc_v30.xml" } ;
#include "setc_v30_lfn.opts"
<!-- end SyntaxHighlightingPlugin -->

It surely works for lxbatch i.e. LSF() backend but won't work for Dirac().

Run on new re-stripped DST

Use the new re-stripped DST as input but look only at events that have passed your preselection. To achieve this you have to read the SelResult tag of each event to find the ones relevant for you. This way is faster than running on SETC that navigate you to the full stripped DSTs.

Get the files with new DSTs

You can find the file to be included to your options here: ~lshchuts/public/DST_v30/newDST_v30.opts or ~lshchuts/public/DST_v31/newDST_v31.opts.

Using CheckSelResult algorithm

You should put all your analysis code in a sequencer starting with a CheckSelResult algorithm.
<!-- SyntaxHighlightingPlugin -->
#include "$DAVINCIROOT/options/DaVinciCommon.opts"
ApplicationMgr.TopAlg = { "GaudiSequencer/MainSeq" };
MainSeq.Members = { "CheckSelResult/CheckPreselXb2GammaX"
                        , "PrintHeader/PrintOfficialPreselXb2GammaX"
                        , "GaudiSequencer/SeqSelBs2PhiGamma" };
SeqSelBs2PhiGamma.Algorithms = { "SelBs2PhiGamma" };
// Run Bs to phi gamma selection
#include "$RADIATIVEOPTS/options/DVSelBs2PhiGamma.opts"
SelBs2PhiGamma.PhysDesktop.InputLocations ={
  "Phys/StdLooseAllPhotons", 
  "Phys/StdLooseKaons" ,
  "Phys/StdLoosePhi2KK",
  "Phys/PreselXb2GammaX"    
};
<!-- end SyntaxHighlightingPlugin -->

Here for instance you look for events having passed the PreselXb2GammaX preselection and run the selection only for those. Be aware of absence any + signs in the sequence declarations. You don't have to make all the standard particles again since they are already stored in the corresponding containers. Also you can select the B-candidate from the "Phys/PreselXXX" container. So there is no need to re-run your preselection you can directly apply the selection algorithm.

Include a file catalogue to your options

These new DSTs also have links to the full stripped DSTs. In order to tell where these DSTs can be find you should add the same file catalogue as when using SETC:
<!-- SyntaxHighlightingPlugin -->
FileCatalog.Catalogs  = { "xmlcatalog_file:dst_v30.xml" } ;
#include "newDST_v30.opts"
<!-- end SyntaxHighlightingPlugin -->

Run on new DSTs with ganga

In order to run it with ganga you need logical file names (LFN) as the input: ~lshchuts/public/DST_v30/newDST_v30_lfn.opts or ~lshchuts/public/DST_v31/newDST_v31_lfn.opts and file catalogue that contain IDs for the newDST files: ~lshchuts/public/DST_v30/newDST_v30.xml or ~lshchuts/public/DST_v31/newDST_v31.xml

<!-- SyntaxHighlightingPlugin -->
FileCatalog.Catalogs  = { "xmlcatalog_file:dst_v30.xml" } ;
FileCatalog.Catalogs += { "xmlcatalog_file:newDST_v30.xml" } ;
#include "newDST_v30_lfn.opts"
<!-- end SyntaxHighlightingPlugin -->

Run on stripped ETC for new stripped DSTs.

This way is the fastest one and depending on the selection algorithm one even can run through the all stripped bbbar data interactively. Here's the recipe to get the result of your preselection and navigate to interesting events form the new stripped DSTs. Again one doesn't need to make the standard particles and can put only the selection algorithm in the sequence:

<!-- SyntaxHighlightingPlugin -->
ApplicationMgr.TopAlg = { "GaudiSequencer/SeqBs2PhiGamma" } ;
/// the selection sequence itself
SeqBs2PhiGamma.Members = {
  "SelBs2PhiGamma"    // the selection algo
} ;
SelBs2PhiGamma.PhysDesktop.InputLocations ={
  "Phys/StdLooseAllPhotons", 
  "Phys/StdLooseKaons" ,
  "Phys/StdLoosePhi2KK",     
  "Phys/PreselXb2GammaX"
};
<!-- end SyntaxHighlightingPlugin -->

Get the files with new SETC

You can find the file to be included to your options here: ~lshchuts/public/DST_v30/setc_newdst_v30.opts or ~lshchuts/public/DST_v31/setc_newdst_v31.opts. Add the name of your preselection as a SEL='PreselXXX>0' requirement at the end of each line. You can add as many selections as you like combined with && or ||. In our example that would be like
<!-- SyntaxHighlightingPlugin -->
EventSelector.Input   = {
"COLLECTION='TagCreator/1' 
DATAFILE='PFN:castor:/castor/cern.ch/user/l/lshchuts/stripping/SETC_newDST_v30/1.root' TYP='POOL_ROOT' SEL='(PreselXb2GammaX>0)'",
[...]
};
ApplicationMgr.ExtSvc  += { "TagCollectionSvc/EvtTupleSvc" };
<!-- end SyntaxHighlightingPlugin -->
This last line tells DaVinci what to use to understand the ETC.

Include a file catalogues for new DSTs and DSTs to your options

Next you need to tell where to find the files the SETC points to and where to find files where newDSTs point to. This is done declaring a file catalogue which you can take from the corresponding directory: ~lshchuts/public/DST_v30/dst_v30.xml or ~lshchuts/public/DST_v31/dst_v31.xml an then add this line to your options:
<!-- SyntaxHighlightingPlugin -->
FileCatalog.Catalogs  = { "xmlcatalog_file:dst_v30.xml" } ;
FileCatalog.Catalogs += { "xmlcatalog_file:newDST_v30.xml" } ;
#include "setc_newdst_v30.opts"
<!-- end SyntaxHighlightingPlugin -->

Run on new SETC with ganga

In order to run it with ganga you need logical file names (LFN) as the input: ~lshchuts/public/DST_v30/setc_newdst_v30_lfn.opts or ~lshchuts/public/DST_v31/setc_newdst_v31_lfn.opts and file catalogue that contain IDs for the SETC for newDSTs files: ~lshchuts/public/DST_v30/setc_newdst_v30.xml or ~lshchuts/public/DST_v31/setc_newdst_v31.xml

<!-- SyntaxHighlightingPlugin -->
FileCatalog.Catalogs  = { "xmlcatalog_file:dst_v30.xml" } ;
FileCatalog.Catalogs += { "xmlcatalog_file:newDST_v30.xml" } ;
FileCatalog.Catalogs += { "xmlcatalog_file:setc_newdst_v30.xml" } ;
#include "setc_newdst_v30_lfn.opts"
<!-- end SyntaxHighlightingPlugin -->

How much BB is that?

All the data sets were produced from the DC06 - Stripping-v30-lumi2 and the DC06 - Stripping-v31-lumi2 stripped DST data available in the bookkeping. This means that new files correspond to 804584 stripped events and 22085009 bbbar inclusive events. So this number should be used for the final evaluation. However I would like to emphasize once more that the safe way is to use v30r17 configuration since it reproduces the original stripping in a correct way. But with the gain in time which the new DSTs provide it is not a problem to run on both sets of events and check the consistency of results.

-- LesyaShchutska - 10 Apr 2008

META TOPICMOVED by="LesyaShchutska" date="1207831508" from="GeneratorServices.DC06StrippingNewDST" to="LHCb.DC06StrippingNewDST"
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback