How to run on re-stripped DC06 data with the new format DSTs

Introduction

This page is the follow up from the official DC06 Stripping page by Patrick Koppenburg and it introduces the new convenient way to run on DC06 stripped data.

The problems with official stripping results

• The first problem with stripped DSTs is that the SelResult container is not saved on the DST for the latest stripping. So in order to get the result of specific preselection one should always rerun it.
• The second one is that one cannot use the recipe for SETC now since the SETC in bookkeping do not contain the results of preselections.

Two sets of new data.

The stripping was rerun with DaVinci v19r11 on the stripped DSTs. Since there are two sets of DSTs corresponding to reconstruction with Brunel v30r17 or Brunel v31r11 ( the corresponding configurations are DC06 - stripping-v30-lumi2 and DC06 - stripping-v31-lumi2) the two sets of SETC, newDSTs and newSETCs were produced. The difference between the reconstructions leads to the fact that efficiency of all preselections on v30r17 reconstructed stripped data is ~94% and on v31r11 only ~60%. These numbers can vary for the individual preselections. So the safe way is to use v30r17 reconstructed data and than make a check of your final selection on v31r11 data.

Output data

Now it is possible to use the following data formats:

DST, the full stripped DST

This is the main output of the stripping: the complete DST with all what you need in it. The detailed set of instructions how to use it is available here.

SETC, the stripped event-tag collection

This event-tag collection allows you to navigate directly to the stripped DSTs containing events selected by your preselections. Examples are given below.

newDST, the new stripped DST

This is the output data of the re-stripping. It contains MC truth information, /Event/Phys containers with the all intermediate particles and B-candidates selected by preselections, protoparticles, selection result and links to the DSTs in case the other information is needed. Examples of using are given below.

newSETC, the stripped event-tag collection for newDST

This event-tag collection allows you to navigate directly to the new DSTs containing events selected by your preselections. Examples are given below.

Why were the new DSTs produced.

When analyzing stripped events it often appears that one should rerun the preselection/selection algorithm in order to include some new features that were absent in the previous version. So waiting the output makes the analysis really time consuming. The new concept of stripped DSTs provides a faster access to the stripped events and for the result of preselections. It was checked that e.g. selection algorithm for runs four times faster on newSETC than on SETC so that processing of the stripped events takes 1.5 hour.

Timing example for using different approaches with selection:

• DST: 3310 s on ~36k events 21 hours for all data (note that here both PreselXb2GammaX and SelBs2PhiGamma ran in sequence )
• SETC: 1630 s on ~3.2k events selected by PreselXb2GammaX ~72k stripped events 5.1 hours for all data
• newDST: 1120 s on ~72k events 3.5 hours for all data
• newSETC: 460 s on ~3.2k events selected by PreselXb2GammaX ~72k stripped events 1.4 hours for all data

How to use stripped data with the new files

There are three ways to get access to the result of your preselection. The third one is the fastest.

Run on stripped ETC

Here's the recipe to get the result of your preselection and navigate to interesting events form DSTs.

Get the files with SETC

You can find the file to be included to your options here: setc_v30.opts or setc_v31.opts. Add the name of your preselection as a SEL='PreselXXX>0' requirement at the end of each line. You can add as many selections as you like combined with && or ||. It's basically a root TCut. For example that would be like

EventSelector.Input   = {
"COLLECTION='TagCreator/1' 
DATAFILE='PFN:castor:/castor/cern.ch/user/l/lshchuts/stripping/SETC_v30/1.root' TYP='POOL_ROOT' 
SEL='(PreselXb2GammaX>0)'",
[...]
};
ApplicationMgr.ExtSvc  += { "TagCollectionSvc/EvtTupleSvc" };

This last line tells DaVinci what to use to understand the ETC.

Include a file catalogue for DSTs to your options

Next you need to tell where to find the files the ETC points to. This is done declaring a file catalogue which you can copy from the corresponding directory: dst_v30.xml or dst_v31.xml an then add this line to your options:

FileCatalog.Catalogs  = { "xmlcatalog_file:dst_v30.xml" } ;

The file catalogue contains blocks like this one:

<File ID="DCC1AB97-079D-DC11-9DBC-0030488A1192">
 <physical>
  <pfn name="castor:/castor/cern.ch/grid/lhcb/production/DC06/v1r0/00002000/DST/0000/00002000_00000107_2.dst" filetype="ROOT_All"/>
 </physical>
 <logical>
  <lfn name="/lhcb/production/DC06/v1r0/00002000/DST/0000/00002000_00000107_2.dst"/>
 </logical>
</File>

These blocks describe the location of the file. The File ID is what is stored in the ETC. Hence if you forget to declare the file catalogue you will get an error like

PoolRootTreeEvtCnvSvc  ERROR Error: connectDatabase> Cannot connect to database: 
   PFN=gfal:guid:DCC1AB97-079D-DC11-9DBC-0030488A1192

Run on new re-stripped DST

Use the new re-stripped DST as input but look only at events that have passed your preselection. To achieve this you have to read the SelResult tag of each event to find the ones relevant for you. This way is faster than running on SETC that navigate you to the full stripped DSTs.

Get the files with new DSTs

You can find the file to be included to your options here: newDST_v30.opts or newDST_v31.opts.

Using CheckSelResult algorithm

You should put all your analysis code in a sequencer starting with a CheckSelResult algorithm.

#include "$DAVINCIROOT/options/DaVinciCommon.opts"
ApplicationMgr.TopAlg = { "GaudiSequencer/MainSeq" };
MainSeq.Members = { "CheckSelResult/CheckPreselXb2GammaX"
                        , "PrintHeader/PrintOfficialPreselXb2GammaX"
                        , "GaudiSequencer/SeqSelBs2PhiGamma" };
SeqSelBs2PhiGamma.Algorithms = { "SelBs2PhiGamma" };
// Run Bs to phi gamma selection
#include "$RADIATIVEOPTS/options/DVSelBs2PhiGamma.opts"
SelBs2PhiGamma.PhysDesktop.InputLocations ={
  "Phys/StdLooseAllPhotons", 
  "Phys/StdLooseKaons" ,
  "Phys/StdLoosePhi2KK",
  "Phys/PreselXb2GammaX"    
};

Here for instance you look for events having passed the PreselXb2GammaX preselection and run the selection only for those. Be aware of absence any + signs in the sequence declarations. You don't have to make all the standard particles again since they are already stored in the corresponding containers. Also you can select the B-candidate from the "Phys/PreselXXX" container. So there is no need to re-run your preselection you can directly apply the selection algorithm.

Include a file catalogue to your options

These new DSTs also have links to the full stripped DSTs. In order to tell where these DSTs can be find you should add the same file catalogue as when using SETC (dst_v30.xml or dst_v31.xml):

FileCatalog.Catalogs  = { "xmlcatalog_file:dst_v30.xml" } ;
#include "newDST_v30.opts"

Run on new DSTs with ganga

In order to run it with ganga you need logical file names (LFN) as the input: newDST_v30_lfn.opts or newDST_v31_lfn.opts and file catalogue that contain IDs for the newDST files: newDST_v30.xml or newDST_v31.xml

FileCatalog.Catalogs  = { "xmlcatalog_file:dst_v30.xml" } ;
FileCatalog.Catalogs += { "xmlcatalog_file:newDST_v30.xml" } ;
#include "newDST_v30_lfn.opts"

It surely works for lxbatch i.e. LSF() backend but won't work for Dirac().

Run on stripped ETC for new stripped DSTs.

This way is the fastest one and depending on the selection algorithm one even can run through the all stripped -data interactively. Here's the recipe to get the result of your preselection and navigate to interesting events form the new stripped DSTs. Again one doesn't need to make the standard particles and can put only the selection algorithm in the sequence:

ApplicationMgr.TopAlg = { "GaudiSequencer/SeqBs2PhiGamma" } ;
/// the selection sequence itself
SeqBs2PhiGamma.Members = {
  "SelBs2PhiGamma"    // the selection algo
} ;
SelBs2PhiGamma.PhysDesktop.InputLocations ={
  "Phys/StdLooseAllPhotons", 
  "Phys/StdLooseKaons" ,
  "Phys/StdLoosePhi2KK",     
  "Phys/PreselXb2GammaX"
};

Get the files with new SETC

You can find the file to be included to your options here: setc_newdst_v30.opts or setc_newdst_v31.opts. Add the name of your preselection as a SEL='PreselXXX>0' requirement at the end of each line. You can add as many selections as you like combined with && or ||. In our example that would be like

EventSelector.Input   = {
"COLLECTION='TagCreator/1' 
DATAFILE='PFN:castor:/castor/cern.ch/user/l/lshchuts/stripping/SETC_newDST_v30/1.root' TYP='POOL_ROOT' SEL='(PreselXb2GammaX>0)'",
[...]
};
ApplicationMgr.ExtSvc  += { "TagCollectionSvc/EvtTupleSvc" };

This last line tells DaVinci what to use to understand the ETC.

Include a file catalogues for new DSTs and DSTs to your options

Next you need to tell where to find the files the SETC points to and where to find files where newDSTs point to. This is done declaring a file catalogue which you can take from the corresponding directory: dst_v30.xml or dst_v31.xml and newDST_v30.xml or newDST_v31.xml an then add this line to your options:

FileCatalog.Catalogs  = { "xmlcatalog_file:dst_v30.xml" } ;
FileCatalog.Catalogs += { "xmlcatalog_file:newDST_v30.xml" } ;
#include "setc_newdst_v30.opts"

How much BB is that?

All the data sets were produced from the DC06 - Stripping-v30-lumi2 and the DC06 - Stripping-v31-lumi2 stripped DST data available in the bookkeping. This means that new files correspond to 804584 stripped events and 22085009 inclusive events. So this number should be used for the final evaluation. However it's worth to emphasize once more that the safe way is to use v30r17 configuration since it reproduces the original stripping in a correct way. But with the gain in time which the new DSTs provide it is not a problem to run on both sets of events and check the consistency of results.

-- LesyaShchutska - 10 Apr 2008

• setc_v30.opts: SETC for v30r17 reconstructed DSTs

• setc_v30_lfn.opts: List of LFNs for SETC for v30r17 reconstructed DSTs

• setc_v30.xml: File catalogue for SETC for v30r17 reconstructed DSTs

• dst_v30.xml: File catalogue for v30r17 reconstructed DSTs

• newDST_v30.opts: New DSTs made from v30r17 reconstructed DSTs

• newDST_v30_lfn.opts: List of LFNs for new DSTs made from v30r17 reconstructed DSTs

• newDST_v30.xml: File catalogue for new DSTs made from v30r17 reconstructed DSTs

• setc_newdst_v30.opts: SETC for new DSTs made from v30r17 reconstructed DSTs

• setc_newdst_v30_lfn.opts: List of LFNs for SETC for new DSTs made from v30r17 reconstructed DSTs

• setc_newdst_v30.xml: File catalogue for SETC for new DSTs made from v30r17 reconstructed DSTs

• setc_v31.opts: SETC for v31r11 reconstructed DSTs

• setc_v31_lfn.opts: List of LFNs for SETC for v31r11 reconstructed DSTs

• setc_v31.xml: File catalogue for SETC for v31r11 reconstructed DSTs

• dst_v31.xml: File catalogue for v31r11 reconstructed DSTs

• newDST_v31.opts: New DSTs made from v31r11 reconstructed DSTs

• newDST_v31_lfn.opts: List of LFNs for new DSTs made from v31r11 reconstructed DSTs

• newDST_v31.xml: File catalogue for new DSTs made from v31r11 reconstructed DSTs

• setc_newdst_v31.opts: SETC for new DSTs made from v31r11 reconstructed DSTs

• setc_newdst_v31_lfn.opts: List of LFNs for SETC for new DSTs made from v31r11 reconstructed DSTs

• setc_newdst_v31.xml: File catalogue for SETC for new DSTs made from v31r11 reconstructed DSTs