Difference: MicroDST (1 vs. 67)

Revision 672013-03-28 - MarcoCattaneo

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Line: 408 to 408
 
  • Storage of Particle -> primary vertex relations in case of PV re-fitting needs to be re-designed.
  • Storage of neutral proto-particles (and associated MC truth) is not yet tested
  • ProcStatus information
Added:
>
>
  • Task to implement MicroDST formats for MC data
 

Presentations

Changed:
<
<
>
>
 

Revision 662011-08-26 - RobLambert

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Line: 246 to 246
 full DST available but instead of reading in the complete event, only the relevant information (e.g. the B candidate) is read from the microDST and further information (say, track hits) are then obtained via the SmartRefs.

Running TisTos on a MicroDST produced in the stripping

Changed:
<
<
Certain stripping lines produce MicroDSTs as output. These contain raw banks for the HltSelReports and HltDecReports in /Event//. It is necessary to unpack these, and put the resulting objects also in /Event//, so that they are found by algorithms and tools for which RootInTES has been set as suggested above. This can be achieved with the following configuration lines:
>
>
Certain stripping lines produce MicroDSTs as output. These contain raw banks for the HltSelReports and HltDecReports. It is necessary to unpack these, and put the resulting objects also in /Event/<stream name>/, so that they are found by algorithms and tools for which RootInTES has been set as suggested above. This can be achieved with the following configuration lines:

The location where these are stored has been optimized for the reprocessing:

  • Stripping16 and under in /Event/<stream name>/.
  • Stripping17 and over in /Event/Trigger/RawEvent/
 %SYNTAX{syntax="python"}%

locationRoot = '/Event//'

Revision 652011-08-02 - VanyaPolyakov

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Line: 288 to 288
  %ENDSYNTAX%
Added:
>
>
Note , that you also need to set up L0 decoding. This is done by DaVinci automaticaly for DSTs , but not for microDSTs . Read https://twiki.cern.ch/twiki/bin/view/LHCb/MicroDSTTisTos

 Finally, it is necessary to access the LHCb::LHCbIDs that the particle under investigation has been built from. In a full DST, this information is obtained by navigating back, via the particle's ProtoParticle, to the LHCb::Track, LHCb::CaloHypos, LHCb::RichPID, and LHCb::MuonPID. A map linking each particle to an std::vector<LHCb::LHCbID> is stored in /Particle2LHCbIDMap.

%SYNTAX{syntax="cpp"}%

Revision 642011-03-24 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Line: 130 to 130
 

Creating a MicroDST

There are two ways to create a MicroDST:
Changed:
<
<

1. Use the new MicroDSTWriter configurable.

>
>

1. Use the MicroDSTWriter configurable.

 The MicroDSTWriter configurable is a light-weight configurable designed for a "typical" physics use-case, but has limited flexibility and hides away many details of what is going on. There is an example script in Ex/MicroDSTExample/options/TestMicroDSTMakeNewConf.py, where one can clearly see how it is standalone and requires some interplay with another LHCb application configurable, in this case, DaVinci(). The MicroDSTWriter requires that the selections that are to be used to write the MicroDST be in the form of SelectionSequences. For more details on how to write selections like that, see the Particle Selections page and DaVinci Tutorial 4.

%SYNTAX{syntax="python"}%

Revision 632011-02-10 - RobLambert

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Line: 327 to 327
 

Accessing MC truth information from MicroDST in TupleTools

Provided MC truth information has been stored in a standard way, there should be a relations table linking particles in a given location to their associated MCParticles. From DaVinci v26r3p2 onwards, it is also possible to read back stored background catagory values for candidates. First, let's start with an example on how to configure a job to configure TupleToolMCTruth to find the associates MCParticles. Here we assume the MicroDST contains candidates in location "/Event/MicroDST/MyCandidates/Particles", and that the DecayTreeTuples get added to a sequence with RootInTES='/Event/MicroDST'.
Added:
>
>
Recently for the latest DecayTreeTuple >=v3r11p1 we added helper decorators to make configuation a little easier...
 %SYNTAX{syntax="python"}%
Changed:
<
<
from Configurables import DecayTreeTuple
>
>
from DecayTreeTuple.Configuration import *
 from Configurables import MCMatchObjP2MCRelator decayTreeTuple = DecayTreeTuple(.....)
Deleted:
<
<
decayTreeTuple.ToolList += [..., "TupleToolMCTruth", ...]
 # configure the TupleToolMCTruth
Changed:
<
<
decayTreeTuple.addTool(TupleToolMCTruth)
>
>
MCTruth=decayTreeTuple.addTupleTool("TupleToolMCTruth")
 # change the associator type to something that can read the stored associaitons from the MicroDST
Changed:
<
<
decayTreeTuple.TupleToolMCTruth.IP2MCPAssociatorType = 'MCMatchObjP2MCRelator'
>
>
IP2MCPAssociatorType = 'MCMatchObjP2MCRelator'
 # configure the associator
Changed:
<
<
TupleToolMCTruth.addTool(MCMatchObjP2MCRelator,
>
>
MCTruth.addTool(MCMatchObjP2MCRelator,
  name="IP2MCPAssociatorType") # tell it where to find the Particle->MCParticle relations
Changed:
<
<
RelTableLocations = ['Phys/MyCandidates/P2MCPRelations']
>
>
RelTableLocations = ['Phys/MyCandidates/P2MCPRelations']
  %ENDSYNTAX%

Next, we can see how to obtain the background categiries for each particle via TupleToolMCBackgroundInfo:

%SYNTAX{syntax="python"}%

Changed:
<
<
from Configurables import BackgroundCategoryViaRelations, TupleToolMCBackgroundInfo decayTreeTuple.addTool(TupleToolMCBackgroundInfo) decayTreeTuple.TupleToolMCBackgroundInfo.IBackgroundCategoryType = 'BackgroundCategoryViaRelations' decayTreeTuple.TupleToolMCBackgroundInfo.addTool(BackgroundCategoryViaRelations,
>
>
from Configurables import BackgroundCategoryViaRelations BackCat=decayTreeTuple.addTupleTool("TupleToolMCBackgroundInfo") IBackgroundCategoryType = 'BackgroundCategoryViaRelations' BackCat.addTool(BackgroundCategoryViaRelations,
  name="IBackgroundCategoryType")
Changed:
<
<
decayTreeTuple.TupleToolMCBackgroundInfo.IBackgroundCategoryType.inputTable = ['Phys/MyCandidates/P2BCRelations']
>
>
IBackgroundCategoryType.inputTable = ['Phys/MyCandidates/P2BCRelations']
 %ENDSYNTAX%

-- JuanPalacios - 20-Jan-2011

Revision 622011-01-20 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Line: 324 to 324
  -- JuanPalacios - 23-Oct-2010
Added:
>
>

Accessing MC truth information from MicroDST in TupleTools

Provided MC truth information has been stored in a standard way, there should be a relations table linking particles in a given location to their associated MCParticles. From DaVinci v26r3p2 onwards, it is also possible to read back stored background catagory values for candidates. First, let's start with an example on how to configure a job to configure TupleToolMCTruth to find the associates MCParticles. Here we assume the MicroDST contains candidates in location "/Event/MicroDST/MyCandidates/Particles", and that the DecayTreeTuples get added to a sequence with RootInTES='/Event/MicroDST'.

<!-- SyntaxHighlightingPlugin -->
from Configurables import DecayTreeTuple
from Configurables import MCMatchObjP2MCRelator
decayTreeTuple = DecayTreeTuple(.....)
decayTreeTuple.ToolList += [..., "TupleToolMCTruth", ...]
# configure the TupleToolMCTruth
decayTreeTuple.addTool(TupleToolMCTruth)
# change the associator type to something that can read the stored associaitons from the MicroDST
decayTreeTuple.TupleToolMCTruth.IP2MCPAssociatorType = 'MCMatchObjP2MCRelator'
# configure the associator
DecayTreeB2PiMuMu.TupleToolMCTruth.addTool(MCMatchObjP2MCRelator,
                                           name="IP2MCPAssociatorType")
# tell it where to find the Particle->MCParticle relations
DecayTreeB2PiMuMu.TupleToolMCTruth.IP2MCPAssociatorType.RelTableLocations = ['Phys/MyCandidates/P2MCPRelations']
<!-- end SyntaxHighlightingPlugin -->

Next, we can see how to obtain the background categiries for each particle via TupleToolMCBackgroundInfo:

<!-- SyntaxHighlightingPlugin -->
from Configurables import BackgroundCategoryViaRelations, TupleToolMCBackgroundInfo
decayTreeTuple.addTool(TupleToolMCBackgroundInfo)
decayTreeTuple.TupleToolMCBackgroundInfo.IBackgroundCategoryType = 'BackgroundCategoryViaRelations'
decayTreeTuple.TupleToolMCBackgroundInfo.addTool(BackgroundCategoryViaRelations,
                                                    name="IBackgroundCategoryType")
decayTreeTuple.TupleToolMCBackgroundInfo.IBackgroundCategoryType.inputTable = ['Phys/MyCandidates/P2BCRelations']
<!-- end SyntaxHighlightingPlugin -->

-- JuanPalacios - 20-Jan-2011

 

Examples of reading microDST in C++

Examples of reading MicroDST in C++ within the DaVinci framework can be found from MultiVariateSelections page and in Ex/MicroDSTExample/options/TestMicroDSTRead.py, which forms the basis for the example below. Note that here we create a SelectionSequence on the fly, and we use the SelectionSequence used to make the MicroDST just to get the particles' location.

Revision 612010-11-26 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Line: 283 to 283
 l0DecReportsMaker = L0DecReportsMaker(OutputHltDecReportsLocation = l0DecReportsLoc) l0SelReportsMaker = L0SelReportsMaker(OutputHltSelReportsLocation = l0SelReportsLoc)
Changed:
<
<
DataOnDemandSvc().AlgMap[l0DecReportsLoc] = l0DecReportsLoc DataOnDemandSvc().AlgMap[l0SelReportsLoc] = l0SelReportsLoc
>
>
DataOnDemandSvc().AlgMap[l0DecReportsLoc] = l0DecReportsMaker DataOnDemandSvc().AlgMap[l0SelReportsLoc] = l0SelReportsMaker
  %ENDSYNTAX%

Revision 602010-10-26 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Line: 284 to 284
 l0SelReportsMaker = L0SelReportsMaker(OutputHltSelReportsLocation = l0SelReportsLoc)

DataOnDemandSvc().AlgMap[l0DecReportsLoc] = l0DecReportsLoc

Changed:
<
<
DataOnDemandSvc().AlgMap[l0DecReportsLoc] = l0SelReportsLoc
>
>
DataOnDemandSvc().AlgMap[l0SelReportsLoc] = l0SelReportsLoc
  %ENDSYNTAX%

Revision 592010-10-25 - PhilipHunt

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Line: 277 to 277
 %SYNTAX{syntax="python"}% from Configurables import L0DecReportsMaker, L0SelReportsMaker
Changed:
<
<
l0DecReportsLoc = locationRoot + 'HltLikeL0/DecReports') l0SelReportsLoc = locationRoot + 'HltLikeL0/SelReports')
>
>
l0DecReportsLoc = locationRoot + 'HltLikeL0/DecReports' l0SelReportsLoc = locationRoot + 'HltLikeL0/SelReports'
 
Changed:
<
<
l0DecReportsMaker = L0DecReportsMaker(OutputHltDecReportsLocation = l0DecReportsLoc l0SelReportsMaker = L0SelReportsMaker(OutputHltSelReportsLocation = l0SelReportsLoc
>
>
l0DecReportsMaker = L0DecReportsMaker(OutputHltDecReportsLocation = l0DecReportsLoc) l0SelReportsMaker = L0SelReportsMaker(OutputHltSelReportsLocation = l0SelReportsLoc)
  DataOnDemandSvc().AlgMap[l0DecReportsLoc] = l0DecReportsLoc DataOnDemandSvc().AlgMap[l0DecReportsLoc] = l0SelReportsLoc

Revision 582010-10-23 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Line: 277 to 277
 %SYNTAX{syntax="python"}% from Configurables import L0DecReportsMaker, L0SelReportsMaker
Changed:
<
<
l0decrm = L0DecReportsMaker(OutputHltDecReportsLocation = locationRoot + 'HltLikeL0/DecReports') l0selrm = L0SelReportsMaker(OutputHltSelReportsLocation = locationRoot + 'HltLikeL0/SelReports') l DataOnDemandSvc().AlgMap["SeqJpsi/HltLikeL0/DecReports"] = l0decrm DataOnDemandSvc().AlgMap["SeqJpsi/HltLikeL0/SelReports"] = l0selrm
>
>
l0DecReportsLoc = locationRoot + 'HltLikeL0/DecReports') l0SelReportsLoc = locationRoot + 'HltLikeL0/SelReports')

l0DecReportsMaker = L0DecReportsMaker(OutputHltDecReportsLocation = l0DecReportsLoc l0SelReportsMaker = L0SelReportsMaker(OutputHltSelReportsLocation = l0SelReportsLoc

DataOnDemandSvc().AlgMap[l0DecReportsLoc] = l0DecReportsLoc DataOnDemandSvc().AlgMap[l0DecReportsLoc] = l0SelReportsLoc

  %ENDSYNTAX%
Line: 319 to 322
  If you use DecayTreeTuple with TupleToolTISTOS, bear in mind that this TupleTool used two private instances if an ITriggerTisTos, namely TriggerTisTos and L0TriggerTisTos. To be able to TisTos with tuple tools you would have to configure your DecayTreeTuple's TupleToolTisTos suitably. Configuring a tool of a tool of an algorithm is a daunting task, but this link should make it easier.
Added:
>
>
-- JuanPalacios - 23-Oct-2010
 

Examples of reading microDST in C++

Examples of reading MicroDST in C++ within the DaVinci framework can be found from MultiVariateSelections page and in Ex/MicroDSTExample/options/TestMicroDSTRead.py, which forms the basis for the example below. Note that here we create a SelectionSequence on the fly, and we use the SelectionSequence used to make the MicroDST just to get the particles' location.

Revision 572010-10-22 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Line: 296 to 296
  get<DaVinci::Particle2LHCbIDs>("/Event/StreamX/Phys/SelY/Particle2LHCbIDMap");

// get one of the candidates

Changed:
<
<
const LHCb::Particle* signal = ...; const std::vector<LHCb::LHCbID>& signalIDs = p2LHCbIDs(signal); // will have to add the necessary TisTos c++ lines.
>
>
const LHCb::Particle* cand = ...; // get its LHCbIDs const std::vector<LHCb::LHCbID>& signalIDs = (*p2LHCbIDs)(cand);

// get your favourite TisTosTool ITriggerTisTos* tisTosTool = ...;

tisTosTool->setOfflineInput(); // erase the signal definition tisTosTool->addToOfflineInput(signalIDs); // set signal to particle cand's IDs

%ENDSYNTAX%

In the future, with the release of DaVinci v26r3, it will be possible to achieve skip the C++ steps and configure the ITriggerTisTos directly. This is currently available in Phys/TisTosTobbing revision 90050, which is compatible with DaVinci v26r2. The configuration works as follows:

%SYNTAX{syntax="python"}%

tisTosTool = ... # Configurable of something that inherits from ITriggerTisTos tisTosTool.UseParticle2LHCbIDsMap = 2

 %ENDSYNTAX%
Added:
>
>
If you use DecayTreeTuple with TupleToolTISTOS, bear in mind that this TupleTool used two private instances if an ITriggerTisTos, namely TriggerTisTos and L0TriggerTisTos. To be able to TisTos with tuple tools you would have to configure your DecayTreeTuple's TupleToolTisTos suitably. Configuring a tool of a tool of an algorithm is a daunting task, but this link should make it easier.
 

Examples of reading microDST in C++

Examples of reading MicroDST in C++ within the DaVinci framework can be found from MultiVariateSelections page and in Ex/MicroDSTExample/options/TestMicroDSTRead.py, which forms the basis for the example below. Note that here we create a SelectionSequence on the fly, and we use the SelectionSequence used to make the MicroDST just to get the particles' location.

Revision 562010-10-22 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Line: 218 to 218
 
  • Beware of tools that require more information than is stored on the MicroDST. Some DaVinci tools might require more information than appears at first sight.
  • You can specify the RootInTES of a GaudiSequencer. This will propagate to all the members.
Changed:
<
<
To specify the input location for the DVALgorithm, use
>
>
To specify the input location for the DVAlgorithm, use
 %SYNTAX{syntax="python"}% RootInTES = '/Event/' InputLocations = ["Path/to/Your/orig/Alg"]
Line: 272 to 272
 ANNDispatchSvc().RawEventLocation = rawEventLoc %ENDSYNTAX%
Added:
>
>
The stripping MicroDST contains L0 trigger information in standard locations, '/Event/Trig/L0/L0DUReport', '/Event/Trig/L0/MuonBCSU', and '/Event/Trig/L0/FullCalo' , but tools configured as suggested above expect them on the RootInTES. Therefore we need a different approach to ensure that the L0 decision reports end up where they are expected:

<!-- SyntaxHighlightingPlugin -->
from Configurables import L0DecReportsMaker, L0SelReportsMaker

l0decrm = L0DecReportsMaker(OutputHltDecReportsLocation = locationRoot + 'HltLikeL0/DecReports')
l0selrm = L0SelReportsMaker(OutputHltSelReportsLocation = locationRoot + 'HltLikeL0/SelReports')
l
DataOnDemandSvc().AlgMap["SeqJpsi/HltLikeL0/DecReports"] = l0decrm
DataOnDemandSvc().AlgMap["SeqJpsi/HltLikeL0/SelReports"] = l0selrm
<!-- end SyntaxHighlightingPlugin -->

Finally, it is necessary to access the LHCb::LHCbIDs that the particle under investigation has been built from. In a full DST, this information is obtained by navigating back, via the particle's ProtoParticle, to the LHCb::Track, LHCb::CaloHypos, LHCb::RichPID, and LHCb::MuonPID. A map linking each particle to an std::vector<LHCb::LHCbID> is stored in /Particle2LHCbIDMap.

<!-- SyntaxHighlightingPlugin -->
#include "Kernel/Particle2LHCbIDs.h"

...
   // get the Particle->vector<LHCbID> map
   const DaVinci::Particle2LHCbIDs* p2LHCbIDs =
                 get<DaVinci::Particle2LHCbIDs>("/Event/StreamX/Phys/SelY/Particle2LHCbIDMap");

   // get one of the candidates
   const LHCb::Particle* signal = ...;
   const std::vector<LHCb::LHCbID>& signalIDs = p2LHCbIDs(signal);
   // will have to add the necessary TisTos c++ lines.
<!-- end SyntaxHighlightingPlugin -->
 

Examples of reading microDST in C++

Examples of reading MicroDST in C++ within the DaVinci framework can be found from MultiVariateSelections page and in Ex/MicroDSTExample/options/TestMicroDSTRead.py, which forms the basis for the example below. Note that here we create a SelectionSequence on the fly, and we use the SelectionSequence used to make the MicroDST just to get the particles' location.

Revision 552010-10-21 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Line: 214 to 214
 

C++

When creating the MicroDST, the algorithms replicate the structure of the transient event store (TES) in a new branch and store the information about the relevant particles, etc. there. By default, this new branch has the prefix 'MicroDST', so everything is stored under '/Event/MicroDST'. Since DValgorithm and PhysDesktop use partial TES locations for various things, it is better to use the RootInTES property instead of setting the input locations with full paths.
Deleted:
<
<
Besides this, the EventClockSvc needs to know where to get the ODIN bank for in order to decode the time. Since it is not part of a sequence, it is necessary to set this separately (it is actually the OdinTimeDecoder tool of the EventClockSvc that needs this information):

<!-- SyntaxHighlightingPlugin -->
from Configurables import OdinTimeDecoder, EventClockSvc
ecs = EventClockSvc()
ecs.addTool(OdinTimeDecoder)
ecs.OdinTimeDecoder.RootInTES = "/Event/MicroDST/"
<!-- end SyntaxHighlightingPlugin -->
 There are a few important points to note:
  • Beware of tools that require more information than is stored on the MicroDST. Some DaVinci tools might require more information than appears at first sight.
  • You can specify the RootInTES of a GaudiSequencer. This will propagate to all the members.

To specify the input location for the DVALgorithm, use %SYNTAX{syntax="python"}%

Changed:
<
<
RootInTES = '/Event/MicroDST/'
>
>
RootInTES = '/Event/'
 InputLocations = ["Path/to/Your/orig/Alg"] %ENDSYNTAX%
Changed:
<
<
However, if the algorithm is in a sequencer,
>
>
However, if the algorithm is in a sequencer, you can use relative paths for your algorithms, and set RootInTES only for the sequencer:
 %SYNTAX{syntax="python"}% YourAlg0 = ... InputLocations = ["Path/to/Your/orig/Alg0"]
Line: 253 to 244
 microDST to access the information stored on the full DST (even if it is not available on the MicroDST). A possible use-case for this could be that you want to update the file or some plot with further information or that you have the full DST available but instead of reading in the complete event, only the relevant information (e.g. the B candidate) is read from the microDST and further information (say, track hits) are then obtained via the SmartRefs.
Added:
>
>

Running TisTos on a MicroDST produced in the stripping

Certain stripping lines produce MicroDSTs as output. These contain raw banks for the HltSelReports and HltDecReports in /Event//. It is necessary to unpack these, and put the resulting objects also in /Event//, so that they are found by algorithms and tools for which RootInTES has been set as suggested above. This can be achieved with the following configuration lines:
<!-- SyntaxHighlightingPlugin -->
locationRoot = '/Event/<sequence name>/'

# Set up on-demand unpacking of HLT report banks for TisTos                                                                                                                              
from Configurables import HltSelReportsDecoder, HltDecReportsDecoder, DataOnDemandSvc, ANNDispatchSvc

rawEventLoc = locationRoot + 'DAQ/RawEvent'
decReportLoc = locationRoot +"Hlt/DecReports"
selReportLoc = locationRoot + "Hlt/SelReports"

# get HltSelReports raw bank from rawEventLoc and put HltSelReports in selReportLoc                                                                                                      
selReportsDecoder = HltSelReportsDecoder( InputRawEventLocation = rawEventLoc,
                                          OutputHltSelReportsLocation = selReportLoc)

# get HltDecReports raw bank from rawEventLoc and put HltSelReports in decReportLoc                                                                                                      
decReportsDecoder = HltDecReportsDecoder( InputRawEventLocation = rawEventLoc,
                                          OutputHltDecReportsLocation = decReportLoc)

DataOnDemandSvc().AlgMap[selReportLoc] = selReportsDecoder
DataOnDemandSvc().AlgMap[decReportLoc] = decReportsDecoder

ANNDispatchSvc().RawEventLocation = rawEventLoc
<!-- end SyntaxHighlightingPlugin -->
 

Examples of reading microDST in C++

Examples of reading MicroDST in C++ within the DaVinci framework can be found from MultiVariateSelections page and in Ex/MicroDSTExample/options/TestMicroDSTRead.py, which forms the basis for the example below. Note that here we create a SelectionSequence on the fly, and we use the SelectionSequence used to make the MicroDST just to get the particles' location.
Line: 277 to 296
  BsSeq = SelectionSequence('Bs', TopSelection = BsFilterSel) seq = BsSeq.sequence() # this is a GaudiSequencer
Changed:
<
<
seq.RootInTES = "/Event/MicroDST/" # All members of the sequence get their RootInTES set seq.OutputLevel=4
>
>
seq.RootInTES = '/Event/' + selSequence.name() + '/' # All members of the sequence get their RootInTES set
  dv=DaVinci() dv.EvtMax=-1
Line: 286 to 304
 dv.UserAlgorithms = [seq] dv.InputType='MDST'
Deleted:
<
<
# some necessary framework stuff ecs = EventClockSvc() ecs.addTool(OdinTimeDecoder) ecs.OdinTimeDecoder.RootInTES = "/Event/MicroDST/"
 %ENDSYNTAX%

Note that this requires that DaVinci() recognise InputType='MDST', which will only be possible with DaVinci v24r4 or higher.

Revision 542010-09-15 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Line: 265 to 265
  BsSel = AutomaticData(Location = particleLocation)
Added:
>
>
filterCuts = "......."
 from Configurables import FilterDesktop _bsFilter = FilterDesktop('_bsFilter',
Changed:
<
<
Code = 'ALL')
>
>
Code = filterCuts)
  BsFilterSel = Selection('HelloWorld', Algorithm = _bsFilter,

Revision 532010-08-09 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Line: 261 to 261
 selSequence = SeqBs2Jpsi2MuMuPhi2KK particleLocation = selSequence.outputLocation()
Changed:
<
<
from PhysSelPython.Wrappers import DataOnDemand, Selection, SelectionSequence
>
>
from PhysSelPython.Wrappers import AutomaticData, Selection, SelectionSequence
 
Changed:
<
<
BsSel = DataOnDemand('DOD_Bs', Location = particleLocation)
>
>
BsSel = AutomaticData(Location = particleLocation)
  from Configurables import FilterDesktop _bsFilter = FilterDesktop('_bsFilter',

Revision 522010-04-20 - ChristopherRJones

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Revision 512010-03-12 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Line: 295 to 295
 Note that this requires that DaVinci() recognise InputType='MDST', which will only be possible with DaVinci v24r4 or higher.

Work in progress and known issues

Added:
>
>
  • Storage of Particle -> primary vertex relations in case of PV re-fitting needs to be re-designed.
 
  • Storage of neutral proto-particles (and associated MC truth) is not yet tested
  • ProcStatus information

Revision 502009-11-20 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Line: 311 to 311
 
META FILEATTACHMENT attachment="AnalysisMicroDST.py.txt" attr="" comment="Simple analysis script" date="1207298418" name="AnalysisMicroDST.py.txt" path="AnalysisMicroDST.py.txt" size="2116" stream="AnalysisMicroDST.py.txt" user="Main.UlrichKerzel" version="1"
META FILEATTACHMENT attachment="JpsiPhiData.opts" attr="" comment="Data card containing Bs -> J/Psi Phi MC signal. Used by TestMicroDSTMake.opts" date="1222098039" name="JpsiPhiData.opts" path="JpsiPhiData.opts" size="31309" stream="JpsiPhiData.opts" user="Main.JuanPalacios" version="1"
META FILEATTACHMENT attachment="MicroDSTReadingExample.py.txt" attr="" comment="Simple (and messy) python script to read a %22typical%22 MicroDST and make some plots." date="1222098220" name="MicroDSTReadingExample.py.txt" path="MicroDSTReadingExample.py" size="6081" stream="MicroDSTReadingExample.py" user="Main.JuanPalacios" version="1"
Added:
>
>
|*META PREFERENCE*|name="SYNTAXHIGHLIGHTINGPLUGIN_FORMAT_PREPEND" title="SYNTAXHIGHLIGHTINGPLUGIN_FORMAT_PREPEND" type="Set" value="
"|

Revision 492009-11-20 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Line: 22 to 22
 
LHCb::RecHeader CopyRecHeader  

This is how one of these is now defined in C++ (doxygen comments removed for clarity):

Changed:
<
<
>
>
%SYNTAX{syntax="cpp"}%
 #include "MicroDST/ObjectClonerAlg.h" #include "Event/RecHeader.h" //=============================================================================
Line: 43 to 43
 typedef MicroDST::ObjectClonerAlg<LHCb::RecHeader> CopyRecHeader; // Declaration of the Algorithm Factory DECLARE_NAMED_ALGORITHM_FACTORY( CopyRecHeader )
Changed:
<
<
>
>
%ENDSYNTAX%
 

MicroDST::KeyedContainerClonerAlg

This template class is intended to clone all the KeyedObject in a KeyedContainer in the TES into the MicroDST TES. It also allows for the cloning of related KeyedObject, either from SmartRefs or pointers. The algorithm contains a pointer to a pure virtual cloner tool.
Line: 56 to 56
 
LHCb::ProtoParticle CopyProtoParticles  
LHCb::RecVertex CopyPrimaryVertices  
The following example shows how the LHCb::Particle cloning algorithm is defined:
Changed:
<
<
>
>
%SYNTAX{syntax="cpp"}%
 #include "MicroDST/KeyedContainerClonerAlg.h" #include <MicroDST/ICloneParticle.h> #include "MicroDST/BindType2ClonerDef.h"
Line: 82 to 82
 typedef MicroDST::KeyedContainerClonerAlg<LHCb::Particle> CopyParticles; DECLARE_NAMED_ALGORITHM_FACTORY( CopyParticles ); #endif // COPYPRIMARYVERTICES_H
Changed:
<
<
>
>
%ENDSYNTAX%
 

MicroDST::RelationsClonerAlg

This template takes as input a Relations table T, seeks the T::From objects from the MicroDST transient event store, and copies them and the corresponding T::To. It also creates and copies a new relations table to the MicroDST. T::To that are not already in the MicroDST TES are not cloned. This requires that the user create the necessary relations and clone the To before running the RelationsClonerAlg. A typical example is the cloning of the MCParticles related to a set of Particles. A typical job would run a selection, run some Monte Carlo association resulting in the production of a relations table, and clone the selected Particles to the MicroDST. Running the RelationsClonerAlg would take care of the MCParticles and the Relations table in one action. There exist implementations for
Line: 92 to 92
 

The MicroDSTTools

There are a series of virtual tool interfaces with signatures
Changed:
<
<
>
>
%SYNTAX{syntax="cpp"}%
 T* operator()(const T*); T* clone(const T*);
Changed:
<
<
>
>
%ENDSYNTAX%
 for all the event model types requiring non-trivial cloning actions. Each implementation controls the cloning action, so it is possible to affect the contents of a MicroDST by the choice of different implementations of a single interface. The interfaces and available implementations are summarised below. Each of the implementation inherits common functionality via the MicroDSTTool common base class.

Interface MicroDSTTool implementation Master MicroDSTAlgorithm or MicroDSTTool
Line: 194 to 194
  Ex/MicroDSTExample/python/MicroDSTReadingExample.py (replace "python" in the path for "scripts" from DaVinci v23r1p1 onwards) is a script reads a MicroDST and does some very basic analysis. It is possible to steer its behavior via arguments:
Changed:
<
<
>
>
%SYNTAX{syntax="python"}%
 Usage: python -i MicroDSTReadingExample [options] Options:
Line: 204 to 204
  Default 'DC06selBs2JpsiPhi_unbiased' --root TES root of everything. Default 'Event/microDST'
Changed:
<
<
>
>
%ENDSYNTAX%
  Ex/MicroDSTExample uses modules AnalysisPython.Helpers, AnalysisPython.Functors, and AnalysisPython.HistoUtils (the latter borrowed from GaudiPython) to aid in the reading of MictoDSTs and DSTs. The MicroDSTReadingExample.py script uses these extensively.

Revision 482009-11-20 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Revision 472009-10-14 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Line: 212 to 212
 The file contains the initialisation required, the only step needed by the user is to correctly set the environment using SetupProject. It is intended simply to provide an example on how the contents can be retrieved, including complex objects like LHCb::Particle to LHCb::RecVertex and LHCb::MCParticle relations. It also includes an example on how to fit the proper time of a particle from the standard and "best" primary vertex.

C++

Changed:
<
<
When creating the MicroDST, the algorithms replicate the structure of the transient event store (TES) in a new branch and store the information about the relevant particles, etc. there. By default, this new branch has the prefix 'MicroDST', so everything is stored under '/Event/MicroDST'. Since DValgorithm and PhysDesktop use partial TES locatins for various things, it is better to use the RootInTES property instead of setting the input locations with full paths.
>
>
When creating the MicroDST, the algorithms replicate the structure of the transient event store (TES) in a new branch and store the information about the relevant particles, etc. there. By default, this new branch has the prefix 'MicroDST', so everything is stored under '/Event/MicroDST'. Since DValgorithm and PhysDesktop use partial TES locations for various things, it is better to use the RootInTES property instead of setting the input locations with full paths.
  Besides this, the EventClockSvc needs to know where to get the ODIN bank for in order to decode the time. Since it is not part of a sequence, it is necessary to set this separately (it is actually the OdinTimeDecoder tool of the EventClockSvc that needs this information):
Line: 225 to 225
  There are a few important points to note:
  • Beware of tools that require more information than is stored on the MicroDST. Some DaVinci tools might require more information than appears at first sight.
Changed:
<
<
  • You might have to add some further RootInTES instructions depending on your algorithms and sequences.
>
>
  • You can specify the RootInTES of a GaudiSequencer. This will propagate to all the members.
  To specify the input location for the DVALgorithm, use
<!-- SyntaxHighlightingPlugin -->
YourAlg.RootInTES = '/Event/MicroDST/'
YourAlg.InputLocations = ["Path/to/Your/orig/Alg"]
<!-- end SyntaxHighlightingPlugin -->
Changed:
<
<
>
>
However, if the algorithm is in a sequencer,
<!-- SyntaxHighlightingPlugin -->
YourAlg0 = ...
YourAlg0.InputLocations = ["Path/to/Your/orig/Alg0"]
YourAlg1 = ...
YourAlg1.InputLocations = ["Path/to/Your/orig/Alg1"]
YourSequencer = GaudiSequencer("MySeq", Members = [YourAlg0, YourAlg1])
YourSequencer.RootInTES = '/Event/MicroDST'
<!-- end SyntaxHighlightingPlugin -->
 If needed/wanted, you can then also add the ROOT/POOL catalogue to the options file, e.g.
PoolDbCacheSvc.Catalog = {
Line: 260 to 268
  from Configurables import FilterDesktop _bsFilter = FilterDesktop('_bsFilter',
Changed:
<
<
Code = 'ALL', RootInTES = '/Event/MicroDST/')
>
>
Code = 'ALL')
  BsFilterSel = Selection('HelloWorld', Algorithm = _bsFilter, RequiredSelections = [BsSel] )

BsSeq = SelectionSequence('Bs', TopSelection = BsFilterSel)

Changed:
<
<
seq = BsSeq.sequence() seq.RootInTES = "/Event/MicroDST/"
>
>
seq = BsSeq.sequence() # this is a GaudiSequencer seq.RootInTES = "/Event/MicroDST/" # All members of the sequence get their RootInTES set
 seq.OutputLevel=4

dv=DaVinci()

Revision 462009-10-14 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Line: 135 to 135
  %SYNTAX{syntax="python"}% from Gaudi.Configuration import *
Changed:
<
<
from Configurables import DaVinci, PhysMicroDST, MicroDSTWriter
>
>
from Configurables import DaVinci, MicroDSTWriter
  importOptions("$STDOPTS/PreloadUnits.opts") # Grab a selection from a selection module.
Line: 163 to 163
 %ENDSYNTAX%

2. Configure and add MicroDSTAlgorithms to selection sequence

Manually set up all the necessary MicroDSTAlgorithms and MicroDSTTools, plus populate the TES with the objects that one wants to store. This is the approach taken in Ex/MicroDSTExample/options/TestMicroDSTMake.py, which is an opitons file covering a "typical" physics analysis scenario. It is well worth looking at this example to understand what is really going on and to see the clear distinction between running algorithms that populate the TES with objects, and running MicroDSTAlgorithms that clone these objects onto the MicroDST TES. The doxygen documentation of the relevant MicroDSTAlgorithms has explanations on how to use them. Besides that, the general structure of a MicroDST-making job is something like
Changed:
<
<
>
>
%SYNTAX{syntax="python"}%
 from Gaudi.Configuration import * from Configurables import OutputStream from Configurables import SomeOfTheMicroDSTAlgorithms
Line: 186 to 186
 copyPV.OutputLevel = 4 MySelection.Members += [copyPV] # and so on
Changed:
<
<
>
>
%ENDSYNTAX%
 

Read a MicroDST

Revision 452009-10-13 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Line: 129 to 129
 

Creating a MicroDST

There are two ways to create a MicroDST:
Changed:
<
<

1. Configure and add MicroDSTAlgorithms to selection sequence

>
>

1. Use the new MicroDSTWriter configurable.

The MicroDSTWriter configurable is a light-weight configurable designed for a "typical" physics use-case, but has limited flexibility and hides away many details of what is going on. There is an example script in Ex/MicroDSTExample/options/TestMicroDSTMakeNewConf.py, where one can clearly see how it is standalone and requires some interplay with another LHCb application configurable, in this case, DaVinci(). The MicroDSTWriter requires that the selections that are to be used to write the MicroDST be in the form of SelectionSequences. For more details on how to write selections like that, see the Particle Selections page and DaVinci Tutorial 4.

<!-- SyntaxHighlightingPlugin -->
from Gaudi.Configuration import *
from Configurables import DaVinci, PhysMicroDST, MicroDSTWriter

importOptions("$STDOPTS/PreloadUnits.opts")
# Grab a selection from a selection module.
from MicroDSTExample.Selections import SeqBs2Jpsi2MuMuPhi2KK
selSequence = SeqBs2Jpsi2MuMuPhi2KK.SeqBs2Jpsi2MuMuPhi2KK # this is a SelectionSequence object.

conf = MicroDSTWriter("MicroDST0")
conf.OutputFileSuffix = "Test"
conf.CopyProtoParticles = False
conf.SelectionSequences = [selSequence]
conf.CopyL0DUReport = False
conf.CopyHltDecReports = False
conf.CopyMCTruth = True
conf.CopyBTags = True
microDST0Seq = conf.sequence() # this is a GaudiSequencer with everything that is needed.
<!-- end SyntaxHighlightingPlugin -->
The above configures the writer and makes a GaudiSequencer available via the MicroDSTWriter.sequence() method. This is then picked up by the application:
<!-- SyntaxHighlightingPlugin -->
dv = DaVinci()
dv.DataType = 'DC06'
dv.EvtMax = 100
dv.UserAlgorithms = [microDST0Seq]
dv.Input =  [ ....... ]
<!-- end SyntaxHighlightingPlugin -->

2. Configure and add MicroDSTAlgorithms to selection sequence

 Manually set up all the necessary MicroDSTAlgorithms and MicroDSTTools, plus populate the TES with the objects that one wants to store. This is the approach taken in Ex/MicroDSTExample/options/TestMicroDSTMake.py, which is an opitons file covering a "typical" physics analysis scenario. It is well worth looking at this example to understand what is really going on and to see the clear distinction between running algorithms that populate the TES with objects, and running MicroDSTAlgorithms that clone these objects onto the MicroDST TES. The doxygen documentation of the relevant MicroDSTAlgorithms has explanations on how to use them. Besides that, the general structure of a MicroDST-making job is something like
from Gaudi.Configuration import *
Line: 155 to 187
 MySelection.Members += [copyPV] # and so on
Deleted:
<
<

2. Use the new PhysMicroDST configurable.

The PhysMicroDST configurable. It is a light-weight configurable designed for a "typical" physics use-case, but has limited flexibility and hides away many details of what is going on. There is an example script in Ex/MicroDSTExample/options/TestMicroDSTMakeNewConf.py, where one can clearly see how it is standalone and requires some interplay with another LHCb application configurable, in this case, DaVinci().
from Gaudi.Configuration import *
from Configurables import DaVinci, PhysMicroDST
importOptions("$STDOPTS/PreloadUnits.opts")
importOptions( "SomeSelection.py" )
MySelection = GaudiSequencer("SelectionSequencerName")

# configure Micro-DST maker
conf = PhysMicroDST()
conf.OutputPrefix = "MicroDST"
conf.MicroDSTFile = "MyTestMDST_MC_newConf.dst"
conf.MicroDSTSelectionAlg = "SelectionAlgorithmName"
conf.CopyL0DUReport = False
conf.CopyHltDecReports = False
conf.CopyMCTruth = True
conf.CopyBTags = True

# Configure DaVinci
dv = DaVinci()
dv.EvtMax = 500
dv.UserAlgorithms = [MySelection]
dv.Input =  [
"DATAFILE='PFN:castor:/castor/cern.ch/grid/lhcb/production/DC06/phys-v4-lumi2/00002146/DST/0000/00002146_00000001_5.dst' TYP='POOL_ROOTTREE' OPT='READ'",
"DATAFILE='PFN:castor:/castor/cern.ch/grid/lhcb/production/DC06/phys-v4-lumi2/00002146/DST/0000/00002146_00000002_5.dst' TYP='POOL_ROOTTREE' OPT='READ'"]
 

Read a MicroDST

Line: 201 to 206
  Default 'Event/microDST'
Changed:
<
<
Ex/MicroDSTExample also contains modules MicroDSTExample.Helpers, MicroDSTExample.Functors, and MicroDSTExample.HistoUtils (borrowed from GaudiPython) to aid in the reading of MictoDSTs and DSTs. The MicroDSTReadingExample.py script uses these extensively.
>
>
Ex/MicroDSTExample uses modules AnalysisPython.Helpers, AnalysisPython.Functors, and AnalysisPython.HistoUtils (the latter borrowed from GaudiPython) to aid in the reading of MictoDSTs and DSTs. The MicroDSTReadingExample.py script uses these extensively.
  Note that here we do not use the DVAlgorithm and its vast tool set to do anything. Tasks can be performed by creating our own instances of tools. The file contains the initialisation required, the only step needed by the user is to correctly set the environment using SetupProject. It is intended simply to provide an example on how the contents can be retrieved, including complex objects like LHCb::Particle to LHCb::RecVertex and LHCb::MCParticle relations. It also includes an example on how to fit the proper time of a particle from the standard and "best" primary vertex.

C++

Changed:
<
<
When creating the MicroDST, the algorithms replicate the structure of the transient event store (TES) in a new branch and store the information about the relevant particles, etc. there. By default, this new branch has the prefix microDST. To use the MicroDST for analysis, root directory on the TES needs to point to this branch which can be done via the parameter RootOnTES. Add the following to your options file
DaVinciMainSeq.RootOnTES  = "/Event/microDST/"
YourSeq.RootOnTES  = "/Event/microDST/"
TestSeq.Members += ["YourAlg"]
YourAlg.PhysDesktop.InputPrimaryVertices = "microDST/Rec/Vertex/Primary"
YourAlg.PhysDesktop.OnOfflineTool.OfflinePVLocation  = "/Event/microDST/Rec/Vertex/Primary"
YourAlg.Particle2MCLinks.ChargedPPLocation = ["/Event/microDST/Rec/ProtoP/Charged", 
               "/Event/microDST/Rec/ProtoP/Upstream"]
>
>
When creating the MicroDST, the algorithms replicate the structure of the transient event store (TES) in a new branch and store the information about the relevant particles, etc. there. By default, this new branch has the prefix 'MicroDST', so everything is stored under '/Event/MicroDST'. Since DValgorithm and PhysDesktop use partial TES locatins for various things, it is better to use the RootInTES property instead of setting the input locations with full paths.
  Besides this, the EventClockSvc needs to know where to get the ODIN bank for in order to decode the time. Since it is not part of a sequence, it is necessary to set this separately (it is actually the OdinTimeDecoder tool of the EventClockSvc that needs this information):
Changed:
<
<
>
>
%SYNTAX{syntax="python"}%
 from Configurables import OdinTimeDecoder, EventClockSvc ecs = EventClockSvc() ecs.addTool(OdinTimeDecoder)
Changed:
<
<
ecs.OdinTimeDecoder.RootInTES = "/Event/microDST/"
>
>
ecs.OdinTimeDecoder.RootInTES = "/Event/MicroDST/" %ENDSYNTAX%
  There are a few important points to note:
  • Beware of tools that require more information than is stored on the MicroDST. Some DaVinci tools might require more information than appears at first sight.
  • You might have to add some further RootInTES instructions depending on your algorithms and sequences.

To specify the input location for the DVALgorithm, use

Changed:
<
<
YourAlg.InputLocations = ["/Event/microDST/Path/to/Your/orig/Alg"]
>
>
<!-- SyntaxHighlightingPlugin -->
YourAlg.RootInTES = '/Event/MicroDST/'
YourAlg.InputLocations = ["Path/to/Your/orig/Alg"]
<!-- end SyntaxHighlightingPlugin -->
  If needed/wanted, you can then also add the ROOT/POOL catalogue to the options file, e.g.
Line: 250 to 245
 microDST to access the information stored on the full DST (even if it is not available on the MicroDST). A possible use-case for this could be that you want to update the file or some plot with further information or that you have the full DST available but instead of reading in the complete event, only the relevant information (e.g. the B candidate) is read from the microDST and further information (say, track hits) are then obtained via the SmartRefs.
Changed:
<
<

Example of reading microDST in C++

An example of reading microDST in C++ within the DaVinci framework can be found from MultiVariateSelections page.
>
>

Examples of reading microDST in C++

Examples of reading MicroDST in C++ within the DaVinci framework can be found from MultiVariateSelections page and in Ex/MicroDSTExample/options/TestMicroDSTRead.py, which forms the basis for the example below. Note that here we create a SelectionSequence on the fly, and we use the SelectionSequence used to make the MicroDST just to get the particles' location.

<!-- SyntaxHighlightingPlugin -->
from MicroDSTExample.Selections import SeqBs2Jpsi2MuMuPhi2KK
selSequence = SeqBs2Jpsi2MuMuPhi2KK.SeqBs2Jpsi2MuMuPhi2KK
particleLocation = selSequence.outputLocation()

from PhysSelPython.Wrappers import DataOnDemand, Selection, SelectionSequence

BsSel = DataOnDemand('DOD_Bs',
                     Location = particleLocation)

from Configurables import FilterDesktop
_bsFilter = FilterDesktop('_bsFilter',
                          Code = 'ALL',
                          RootInTES = '/Event/MicroDST/')

BsFilterSel = Selection('HelloWorld',
                        Algorithm = _bsFilter,
                        RequiredSelections = [BsSel] )

BsSeq = SelectionSequence('Bs', TopSelection = BsFilterSel)
seq = BsSeq.sequence()
seq.RootInTES = "/Event/MicroDST/"
seq.OutputLevel=4

dv=DaVinci()
dv.EvtMax=-1
dv.DataType="MC09"
dv.UserAlgorithms = [seq]
dv.InputType='MDST'

# some necessary framework stuff
ecs = EventClockSvc()
ecs.addTool(OdinTimeDecoder)
ecs.OdinTimeDecoder.RootInTES = "/Event/MicroDST/"
<!-- end SyntaxHighlightingPlugin -->

Note that this requires that DaVinci() recognise InputType='MDST', which will only be possible with DaVinci v24r4 or higher.

 

Work in progress and known issues

  • Storage of neutral proto-particles (and associated MC truth) is not yet tested

Revision 442009-08-06 - SajanEaso

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Revision 432009-08-06 - SajanEaso

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Line: 250 to 250
 microDST to access the information stored on the full DST (even if it is not available on the MicroDST). A possible use-case for this could be that you want to update the file or some plot with further information or that you have the full DST available but instead of reading in the complete event, only the relevant information (e.g. the B candidate) is read from the microDST and further information (say, track hits) are then obtained via the SmartRefs.
Added:
>
>

Example of reading microDST in C++

An example of reading microDST in C++ within the DaVinci framework can be found from MultiVariateSelections page.
 

Work in progress and known issues

  • Storage of neutral proto-particles (and associated MC truth) is not yet tested

Revision 422009-07-30 - JuanPalacios

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Changed:
<
<

MicroDST

>
>

MicroDST

  The purpose of the MicroDST is to select and store a sub-set of data of interest from a file in the LHCb DST format, in a smaller file of the same format. Storing only the information of interest to a given analysis significantly reduces the size of the DST. The amount of space saved depends of course on the given analysis, i.e. it is beneficial to have a dedicated MicroDST per given analysis or per group of (closely) related analyses. The MicroDST code is written with flexibility and configurability in mind, such that it should be easy to define the contents of a MicroDST and configure a program that creates it. Since the MicroDST is in the LHCb DST format, it is even possible to run on a MicroDST and produce an even smaller file in the same format, ad infinitum. Furthermore, it is possible, at each stage of "reduction", to add new information not present in the original DST or MicroDST, as long as this information is in the form of supported Event Model classes.
Line: 254 to 254
 

Work in progress and known issues

  • Storage of neutral proto-particles (and associated MC truth) is not yet tested
  • ProcStatus information
Deleted:
<
<
  • RecVertex cloning with tracks needs testing
 

Presentations

Revision 412009-07-21 - JuanPalacios

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Line: 220 to 220
  "/Event/microDST/Rec/ProtoP/Upstream"]
Added:
>
>
Besides this, the EventClockSvc needs to know where to get the ODIN bank for in order to decode the time. Since it is not part of a sequence, it is necessary to set this separately (it is actually the OdinTimeDecoder tool of the EventClockSvc that needs this information):

from Configurables import OdinTimeDecoder, EventClockSvc
ecs = EventClockSvc()
ecs.addTool(OdinTimeDecoder)
ecs.OdinTimeDecoder.RootInTES = "/Event/microDST/"
 There are a few important points to note:
  • Beware of tools that require more information than is stored on the MicroDST. Some DaVinci tools might require more information than appears at first sight.
Changed:
<
<
  • You might have to add some further RootOnTES instructions depending on your algorithms and sequences.
>
>
  • You might have to add some further RootInTES instructions depending on your algorithms and sequences.
  To specify the input location for the DVALgorithm, use

Revision 402009-06-15 - JuanPalacios

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

The purpose of the MicroDST is to select and store a sub-set of data of interest from a file in the LHCb DST format, in a smaller file of the same format. Storing only the information of interest to a given analysis significantly reduces the size of the DST.

Changed:
<
<
The amount of space saved depends of course on the given analysis, i.e. it is beneficial to have a dedicated MicroDST per given analysis or per group of (closely) related analyses. The MicroDST code is written with flexibility and configurability in mind, such that it should be easy to define the contents of a MicroDST and configure a program that creates it. Since the MicroDST is in the LHCb DST format, it is even possible to run on a MicroDST and produce an even smaller file in the same format, ad infinitum. Furthermore, it is possible, at each stage of "reduction", to add new information not present in the original DST or MicroDST, as long as this information is in the form of supported Event Model classes.
>
>
The amount of space saved depends of course on the given analysis, i.e. it is beneficial to have a dedicated MicroDST per given analysis or per group of (closely) related analyses. The MicroDST code is written with flexibility and configurability in mind, such that it should be easy to define the contents of a MicroDST and configure a program that creates it. Since the MicroDST is in the LHCb DST format, it is even possible to run on a MicroDST and produce an even smaller file in the same format, ad infinitum. Furthermore, it is possible, at each stage of "reduction", to add new information not present in the original DST or MicroDST, as long as this information is in the form of supported Event Model classes.
 
Line: 108 to 108
 
ICloneTrack TrackCloner ProtoParticleCloner, RecVertexClonerWithTracks
ICloneVertex VertexCloner ParticleCloner
Added:
>
>
 

List of supported Event Model classes

Event Model Class Cloning mechanisms
LHCb::HltDecReports: CopyHltDecReports
Line: 119 to 120
 
LHCb::RecVertex Controlled by CopyPrimaryVertices. This uses one of the two implementations of ICloneRecVertex: RecVertexCloner, which clones everything to the MicroDST, except for the LHCb::Tracks that make the vertex, for which it stores SmartRefs, or RecVertexClonerWithTracks which clones everything to the MicroDST, including the LHCb::Tracks making the vertex.
LHCb::Vertex Cloned as a by-product of the cloning of LHCb::Particles, when CopyParticles uses the ParticleCloner implementation of ICloneParticle. The actual cloning is delegated to implementations of !ICloneVertex, the only implementation provided by the framework being !VertexCloner.
LHCb::Track Individual LHCb::Tracks are cloned by implementations of the ICloneTrack interface. At the moment, MicroDSTTools ProtoParticleCloner and RecVertexClonerWithTracks are the only ones to use this.
Changed:
<
<
LHCb::ProtoParticle Cloned directly by [[http://lhcb-release-area.web.cern.ch/LHCb-release-area/DOC/davinci/releases/latest/doxygen/dd/d60/class_copy_proto_particles.html][CopyProtoParticles], or indirectly by the ParticleCloner implementation of the ICloneParticle interface. Both of these call an implementation of the ICloneProtoParticle interface. In the case where the CopyProtoParticles MicroDSTAlgorithm is used, all the LHCb::ProtoParticles stores in the input TES location are cloned and copied to the MicroDST. In the case of cloning via the ParticleCloner, only the ProtoParticle that is related to a given LHCb::Particle already being copied is cloned.
>
>
LHCb::ProtoParticle Cloned directly by CopyProtoParticles, or indirectly by the ParticleCloner implementation of the ICloneParticle interface. Both of these call an implementation of the ICloneProtoParticle interface. In the case where the CopyProtoParticles MicroDSTAlgorithm is used, all the LHCb::ProtoParticles stores in the input TES location are cloned and copied to the MicroDST. In the case of cloning via the ParticleCloner, only the ProtoParticle that is related to a given LHCb::Particle already being copied is cloned.
 
LHCb::FlavourTag Copied by CopyFlavourTag. This requires that the LHCb::FlavourTag objects be created and placed on the TES. Also, the pointer to the tagged candidate LHCb::Particle must be set correctly.
LHCb::MCParticle The action of cloning a single LHCb::MCParticle onto the MicroDST is performed by any implementation of the ICloneMCParticle interface. There are three MicroDSTAlgorithms or MicroDSTTools that use an ICloneMCParticle: CopyMCParticles, CopyParticle2MCRelations, and MCVertexCloner. At the moment, there is only one implementation of ICloneMCPrticle: the MCParticleCloner. As can be seen from the doxygen documentation, this in turn makes use of an ICloneMCVertex implementation to clone the decay end vertices.
LHCb::MCVertex Cloned as a by-product of the cloning of LHCb::MCParticles, where a simple copy of the particle's origin vertex is performed. If the MCParticleCloner implemementation of ICloneMCParticles is used, an !ICloneMCVertex pointer can be used to clone the decay vertices.

Revision 392009-06-14 - JuanPalacios

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Line: 16 to 16
 This covers the simplest case, a self-contained DataObject is cloned and stored on the MicroDST TES. Implementations exist for
Event Model Class MicroDSTAlgorithm Class Comments
LHCb::HltDecReports: CopyHltDecReports Python access to individual HltDecReports only via decision name. Iteration not implemented.
Changed:
<
<
LHCb::L0DUReport CopyL0DUReport LHCb::L0DUConfig pointer not copied to the MicroDST. If there is a use-case for this, please inform the MicroDST software people ASAP.
>
>
LHCb::L0DUReport CopyL0DUReport LHCb::L0DUConfig pointer not copied to the MicroDST. If there is a use-case for this, please inform the MicroDST software people ASAP.
 
LHCb::MCHeader: CopyMCHeader  
Changed:
<
<
LHCb::ODIN CopyODIN  
LHCb::RecHeader CopyRecHeader  
>
>
LHCb::ODIN CopyODIN  
LHCb::RecHeader CopyRecHeader  
  This is how one of these is now defined in C++ (doxygen comments removed for clarity):
Line: 50 to 50
  The depth and scope of the cloning operation is controlled by the implementation of this cloner tool. Specialisations exist for
Event Model Class MicroDSTALgorithm Class Comments
Changed:
<
<
LHCb::FlavourTag CopyFlavourTags  
>
>
LHCb::FlavourTag CopyFlavourTag  
 
LHCb::MCParticle CopyMCParticles  
LHCb::Particle CopyParticles  
LHCb::ProtoParticle CopyProtoParticles  
Line: 97 to 97
 T* clone(const T*); for all the event model types requiring non-trivial cloning actions. Each implementation controls the cloning action, so it is possible to affect the contents of a MicroDST by the choice of different implementations of a single interface. The interfaces and available implementations are summarised below. Each of the implementation inherits common functionality via the MicroDSTTool common base class.
Deleted:
<
<
Interface MicroDSTTool implementation Client MicroDSTAlgorithms
ICloneFlavourTag FlavourTagCloner  
ICloneMCParticle MCParticleCloner  
ICloneMCVertex MCVertexCloner  
ICloneParticle ParticleCloner  
ICloneProtoParticle ProtoParticleCloner  
ICloneRecVertex RecVertexCloner, RecVertexClonerWithTracks  
ICloneTrack TrackCloner  
ICloneVertex VertexCloner  

Create a MicroDST

There are two ways to create a MicroDST.

  1. Manually set up all the necessary MicroDSTAlgorithms and MicroDSTTools, plus populate the TES with the objects that one wants to store. This is the approach taken in Ex/MicroDSTExample/options/TestMicroDSTMake.py, which is an opitons file covering a "typical" physics analysis scenario. It is well worth looking at this example to understand what is really going on and to see the clear distinction between running algorithms that populate the TES with objects, and running MicroDSTAlgorithms that clone these objects onto the MicroDST TES.
  2. Use the new PhysMicroDST configurable. It is a light-weight configurable designed for a "typical" physics use-case, but has limited flexibility and hides away many details of what is going on. There is an example script in Ex/MicroDSTExample/options/TestMicroDSTMakeNewConf.py, where one can clearly see how it is standalone and requires some interplay with another LHCb application configurable, in this case, DaVinci().
 
Changed:
<
<
Each MicroDSTAlgorithm specialization also has examples of use via options in its doxygen documentation. We include three of these algorithms here:
>
>
Interface MicroDSTTool implementation Master MicroDSTAlgorithm or MicroDSTTool
ICloneFlavourTag FlavourTagCloner CopyFlavourTag
ICloneMCParticle MCParticleCloner CopyMCParticles, CopyParticle2MCRelations, MCVertexCloner
ICloneMCVertex MCVertexCloner CopyMCParticles, CopyParticle2MCRelations, MCParticleCloner
ICloneParticle ParticleCloner CopyParticles, VertexCloner
ICloneProtoParticle ProtoParticleCloner CopyProtoParticles, ParticleCloner
ICloneRecVertex RecVertexCloner, RecVertexClonerWithTracks CopyPrimaryVertices, CopyParticle2PVRelations
ICloneTrack TrackCloner ProtoParticleCloner, RecVertexClonerWithTracks
ICloneVertex VertexCloner ParticleCloner

List of supported Event Model classes

Event Model Class Cloning mechanisms
LHCb::HltDecReports: CopyHltDecReports
LHCb::L0DUReport CopyL0DUReport
LHCb::MCHeader: CopyMCHeader takes care of this, storing SmartRefs to the original primary LHCb::MCVertices. There is no runtime configurability implemented here.
LHCb::ODIN CopyODIN
LHCb::RecHeader CopyRecHeader
LHCb::Particle Performed by CopyParticles. By default, recursively clones particles and their descendants by using ParticleCloner, which in turn uses pointers to a ICloneVertex and an ICloneProtoParticle to store each LHCb::Particle's related LHCb::Vertices and LHCb::ProtoParticle. If these pointers are not set, then SmartRefs are stored instead.
LHCb::RecVertex Controlled by CopyPrimaryVertices. This uses one of the two implementations of ICloneRecVertex: RecVertexCloner, which clones everything to the MicroDST, except for the LHCb::Tracks that make the vertex, for which it stores SmartRefs, or RecVertexClonerWithTracks which clones everything to the MicroDST, including the LHCb::Tracks making the vertex.
LHCb::Vertex Cloned as a by-product of the cloning of LHCb::Particles, when CopyParticles uses the ParticleCloner implementation of ICloneParticle. The actual cloning is delegated to implementations of !ICloneVertex, the only implementation provided by the framework being !VertexCloner.
LHCb::Track Individual LHCb::Tracks are cloned by implementations of the ICloneTrack interface. At the moment, MicroDSTTools ProtoParticleCloner and RecVertexClonerWithTracks are the only ones to use this.
LHCb::ProtoParticle Cloned directly by [[http://lhcb-release-area.web.cern.ch/LHCb-release-area/DOC/davinci/releases/latest/doxygen/dd/d60/class_copy_proto_particles.html][CopyProtoParticles], or indirectly by the ParticleCloner implementation of the ICloneParticle interface. Both of these call an implementation of the ICloneProtoParticle interface. In the case where the CopyProtoParticles MicroDSTAlgorithm is used, all the LHCb::ProtoParticles stores in the input TES location are cloned and copied to the MicroDST. In the case of cloning via the ParticleCloner, only the ProtoParticle that is related to a given LHCb::Particle already being copied is cloned.
LHCb::FlavourTag Copied by CopyFlavourTag. This requires that the LHCb::FlavourTag objects be created and placed on the TES. Also, the pointer to the tagged candidate LHCb::Particle must be set correctly.
LHCb::MCParticle The action of cloning a single LHCb::MCParticle onto the MicroDST is performed by any implementation of the ICloneMCParticle interface. There are three MicroDSTAlgorithms or MicroDSTTools that use an ICloneMCParticle: CopyMCParticles, CopyParticle2MCRelations, and MCVertexCloner. At the moment, there is only one implementation of ICloneMCPrticle: the MCParticleCloner. As can be seen from the doxygen documentation, this in turn makes use of an ICloneMCVertex implementation to clone the decay end vertices.
LHCb::MCVertex Cloned as a by-product of the cloning of LHCb::MCParticles, where a simple copy of the particle's origin vertex is performed. If the MCParticleCloner implemementation of ICloneMCParticles is used, an !ICloneMCVertex pointer can be used to clone the decay vertices.
LHCb::Particle to LHCb::RecVertex association Copied using CopyParticle2PVRelations, which internally uses an ICloneRecVertex.
LHCb::Particle to LHCb::MCParticle association CopyParticle2MCRelations

Creating a MicroDST

There are two ways to create a MicroDST:

1. Configure and add MicroDSTAlgorithms to selection sequence

Manually set up all the necessary MicroDSTAlgorithms and MicroDSTTools, plus populate the TES with the objects that one wants to store. This is the approach taken in Ex/MicroDSTExample/options/TestMicroDSTMake.py, which is an opitons file covering a "typical" physics analysis scenario. It is well worth looking at this example to understand what is really going on and to see the clear distinction between running algorithms that populate the TES with objects, and running MicroDSTAlgorithms that clone these objects onto the MicroDST TES. The doxygen documentation of the relevant MicroDSTAlgorithms has explanations on how to use them. Besides that, the general structure of a MicroDST-making job is something like
 
from Gaudi.Configuration import *
Deleted:
<
<
from Gaudi.Configuration import FileCatalog from MicroDSTAlgorithm import * from Configurables import CopyRelatedMCParticles from Configurables import CopyParticle2PVLink from Configurables import MCParticleCloner from Configurables import MCVertexCloner from Configurables import VertexCloner from Configurables import ProtoParticleCloner from Configurables import PrintHeader
 from Configurables import OutputStream
Changed:
<
<
from Configurables import BTagging, BTaggingTool from Configurables import PhysDesktop from Configurables import PVReFitterAlg
>
>
from Configurables import SomeOfTheMicroDSTAlgorithms
 #=================================================================
Deleted:
<
<
importOptions( "$DAVINCIROOT/options/DaVinciCommon.opts" )
 # get some signal data.
Changed:
<
<
importOptions( "$MICRODSTEXAMPLEROOT/options/JpsiPhiDataLFN.py") #importOptions( "$MICRODSTEXAMPLEROOT/options/JpsiPhiDataPFN.py")
>
>
importOptions( "$MICRODSTEXAMPLEROOT/options/JpsiPhiDataPFN.py")
 # Import a selection and get it's sequencer
Changed:
<
<
importOptions( "$CCBARROOT/options/DoDC06SelBs2Jpsi2MuMuPhi2KK_lifetime_unbiased.opts" ) MySelection = GaudiSequencer('SeqDC06selBs2JpsiPhi')
>
>
importOptions( "SomeSelection.py" ) MySelection = GaudiSequencer('SelectionAlgoName')
 # base location of selection's Particles and Vertices
Changed:
<
<
mainLocation = "Phys/DC06selBs2JpsiPhi_unbiased"
>
>
mainLocation = "Phys/SelectionAlgoOutputLocation"
 #=================================================================
Changed:
<
<
# Setting up special MicroDST output stream
>
>
# Set up special MicroDST output stream
 importOptions('$MICRODSTOPTS/MicroDSTStream.py') MicroDSTStream=OutputStream('MicroDSTStream') ApplicationMgr().OutStream.append(MicroDSTStream)
Changed:
<
<
MicroDSTStream.Output = "DATAFILE='Bs2JpsiPhiTest.dst' TYP='POOL_ROOTTREE' OPT='REC'" OutputLevel=4; AcceptAlgs.append( MySelection.name() ) #=================================================================
MySelection.Members += [CopyRecHeader()]
copyParticles = CopyParticles('CopyParticles')
copyParticles.InputLocation = mainLocation+"/Particles"
copyParticles.addTool(VertexCloner(), name='VertexCloner')
copyParticles.addTool(ProtoParticleCloner(), name='ProtoParticleCloner')
copyParticles.OutputLevel=4
MySelection.Members += [copyParticles]
  • Copy the primary vertices of the selected events and the Particle -> PV links
>
>
MicroDSTStream.Output = "DATAFILE='MicroDSTTest.dst' TYP='POOL_ROOTTREE' OPT='REC'" # add MicroDTSAlgorithms to the sequencer
 copyPV=CopyPrimaryVertices('CopyPrimaryVertices') copyPV.OutputLevel = 4 MySelection.Members += [copyPV]
Changed:
<
<
copyP2PVLink = CopyParticle2PVLink() copyP2PVLink.InputLocation = mainLocation+"/Particle2VertexRelations" copyP2PVLink.OutputLevel=4; MySelection.Members += [copyP2PVLink]
  • Add some information that is NOT on the DST: re-fitted primary vertices, exluding tracks from the candidate B!
PVReFitter = PVReFitterAlg("PVReFitterAlg")
PVReFitter.ParticleInputLocation = mainLocation+"/Particles"
refittedPVLocation = mainLocation+"/RefittedVertices"
PVReFitter.VertexOutputLocation = refittedPVLocation
p2ReFitPVRelationsLoc = mainLocation+"/Particle2ReFittedVertexRelations"
PVReFitter.P2VRelationsOutputLocation = p2ReFitPVRelationsLoc
PVReFitter.OutputLevel=4
MySelection.Members += [PVReFitter]
# put the re-fitted vertices on the MicroDST
copyReFittedPVs = CopyPrimaryVertices('CopyReFittedPVs')
copyReFittedPVs.InputLocation = refittedPVLocation
MySelection.Members += [copyReFittedPVs]
# copy the Particle->PV relations table
copyP2RefitPVLink = CopyParticle2PVLink("CopyP2RefitPVLink")
copyP2RefitPVLink.InputLocation = p2ReFitPVRelationsLoc
copyP2RefitPVLink.OutputLevel=4
MySelection.Members += [copyP2RefitPVLink]
>
>
# and so on
 
Changed:
<
<
>
>

2. Use the new PhysMicroDST configurable.

The PhysMicroDST configurable. It is a light-weight configurable designed for a "typical" physics use-case, but has limited flexibility and hides away many details of what is going on. There is an example script in Ex/MicroDSTExample/options/TestMicroDSTMakeNewConf.py, where one can clearly see how it is standalone and requires some interplay with another LHCb application configurable, in this case, DaVinci().
 
Changed:
<
<
importOptions('$FLAVOURTAGGINGOPTS/BTaggingTool.opts') BTagAlgo = BTagging('BTagging') BTagAlgo.addTool(PhysDesktop) BTaggingTool("BTaggingTool").OutputLevel=4 InputLocations=[mainLocation] BTagLocation = mainLocation+"/Tagging" TagOutputLocation = BTagLocation MySelection.Members += [BTagAlgo] copyFlavTag = CopyFlavourTag() copyFlavTag.InputLocation = BTagLocation MySelection.Members += [copyFlavTag] * copy related MC particles of candidates plus daughters
copyMC = CopyRelatedMCParticles()
copyMC.InputLocation = mainLocation+'/Particles'
copyMC.addTool(MCParticleCloner(), name= 'MCParticleCloner')
copyMC.MCParticleCloner.addTool(MCVertexCloner(), name = 'ICloneMCVertex')
copyMC.OutputLevel=4;
MySelection.Members += [copyMC]
>
>
from Gaudi.Configuration import * from Configurables import DaVinci, PhysMicroDST importOptions("$STDOPTS/PreloadUnits.opts") importOptions( "SomeSelection.py" ) MySelection = GaudiSequencer("SelectionSequencerName")

# configure Micro-DST maker conf = PhysMicroDST() conf.OutputPrefix = "MicroDST" conf.MicroDSTFile = "MyTestMDST_MC_newConf.dst" conf.MicroDSTSelectionAlg = "SelectionAlgorithmName" conf.CopyL0DUReport = False conf.CopyHltDecReports = False conf.CopyMCTruth = True conf.CopyBTags = True

# Configure DaVinci dv = DaVinci() dv.EvtMax = 500 dv.UserAlgorithms = [MySelection] dv.Input = [ "DATAFILE='PFN:castor:/castor/cern.ch/grid/lhcb/production/DC06/phys-v4-lumi2/00002146/DST/0000/00002146_00000001_5.dst' TYP='POOL_ROOTTREE' OPT='READ'", "DATAFILE='PFN:castor:/castor/cern.ch/grid/lhcb/production/DC06/phys-v4-lumi2/00002146/DST/0000/00002146_00000002_5.dst' TYP='POOL_ROOTTREE' OPT='READ'"]

 

Read a MicroDST

Line: 252 to 218
 ChargedPPLocation = ["/Event/microDST/Rec/ProtoP/Charged", "/Event/microDST/Rec/ProtoP/Upstream"]
Deleted:
<
<
N.B. You might have to add some further RootOnTES instructions depending on your algorithms and sequences. N.B. In recent Gaudi / DaVinci versions, the option RootInTES was introduced which replaces RootOnTES N.B. You might need to try some variations of the input locations, e.g. in DaVinci v19r5 I found that I had to use RootInTES and then "Phys/MyAlg" as input location for the PhysDesktop (instead of the full path) N.B. it may be beneficial to run
Yourq.Members += ["MuonPIDsFromProtoParticlesAlg/CreateMuonPIDs"]
YouSeq.Members                               += [RichPIDsFromProtoParticlesAlg/CreateRichPIDs"]
not via the Data-on-Demand service but at the beginning of the event.
 
Changed:
<
<
To specify the input location for the PhysDesktop, use
>
>
There are a few important points to note:
  • Beware of tools that require more information than is stored on the MicroDST. Some DaVinci tools might require more information than appears at first sight.
  • You might have to add some further RootOnTES instructions depending on your algorithms and sequences.

To specify the input location for the DVALgorithm, use

 
YourAlg.InputLocations                  = ["/Event/microDST/Path/to/Your/orig/Alg"]
Line: 274 to 235
  "xmlcatalog_file:CatalogueOfInputFilesToMicroDST" };
Changed:
<
<
The first line again holds the information about the objects used in the M!icroDST, the further line(s) contain the catalogue(s)
>
>
The first line holds the information about the objects used in the MicroDST, the further line(s) contain the catalogue(s)
 for the input files from which the microDST was created. If you have access to these files, you can follow the SmartRefs on the microDST to access the information stored on the full DST (even if it is not available on the MicroDST). A possible use-case for this could be that you want to update the file or some plot with further information or that you have the full DST available but instead of reading in the complete event, only the relevant information (e.g. the B candidate) is read from the microDST and further information (say, track hits) are then obtained via the SmartRefs.
Deleted:
<
<

Storing data on the MicroDST

Everything placed in the "/Event/microDST" trunk of the TES is stored in the MicroDST. MicroDSTAlgorithms and MicroDSTTools have been written to store many LHCb Event Model types. It is also possible, and in general quite easy, to write MicroDSTAlgorithms and MicroDSTTools to store other Event Model types, if an analysis so requires. The Event Model types supported at the moment have a certain bias towards physics selections and analysis since that is the background in which the MicroDST code has been developed. The following list contains the Event Model types that are currently supported, together with links to the doxygen documentation of the MicroDSTAlgorithms and - MicroDSTTools where, appropriate - in charge of cloning and storing them to the MicroDST. The doxygen documentation of the MicroDSTAlgorithms should always be the ultimate reference. The documentation of the MicroDSTTools is somewhat deficient at the moment, but I hope to change this in the near future.

-- JuanPalacios - 21 Aug 2008

LHCb::Particles

The cloning and storing of LHCb::Particles is performed by the CopyParticles MicroDSTAlgorithm. This typedef to MicroDST::KeyedContainerClonerAlg<LHCb::Particle> recursively clones particles and their descendants by use of an implementation of ICloneParticle. By default this is ParticleCloner, which in turn uses pointers to a ICloneVertex and an ICloneProtoParticle to store each LHCb::Particle's related LHCb::Vertices and LHCb::ProtoParticle. If these pointers are not set, then SmartRefs are stored instead.

Primary Vertices (LHCb::RecVetrex)

The cloning and storing of primary vertices is controlled by the CopyPrimaryVertices MicroDSTAlgorithm. This template to MicroDST::KeyedContainerClonerAlg<LHCb::RecVertex> uses an instance of the ICloneRecVertex interface, of which there are two implementations:

* RecVertexCloner: Clones everything to the MicroDST, except for the LHCb::Tracks that make the vertex, for which it stores SmartRefs.

* RecVertexClonerWithTracks: Clones everything to the MicroDST, including the LHCb::Tracks making the vertex

LHCb::Particle to LHCb::RecVertex association

Often it is necessary to establish a link between a candidate LHCb::Particle and a primary vertex (in the form of an LHCb::RecVertex). If the associations have been stored on the TES, it is possible to put them in the MicroDST using the CopyParticle2PVLink MicroDSTAlgorithm.

LHCb::Vertices

LHCb::Vertices are cloned as a by-product of the cloning of LHCb::Particles, when the CopyParticles MicroDSTAlgorithm uses the ParticleCloner implementation of ICloneParticle. The actual cloning is delegated to implementations of !ICloneVertex, the only implementation provided by the framework being !VertexCloner.

LHCb::Tracks

Individual LHCb::Tracks are cloned by implementations of the ICloneTrack interface. At the moment, MicroDSTTools ProtoParticleCloner and RecVertexClonerWithTracks are the only ones to use this.

LHCb::ProtoParticles

ProtoParticles can be cloned directly via the CopyProtoParticles MicroDSTAlgorithm, or indirectly via the ParticleCloner implementation of the ICloneParticle interface. Both of these call an implementation of the ICloneProtoParticle interface. In the case where the CopyProtoParticles MicroDSTAlgorithm is used, all the LHCb::ProtoParticles stores in the input TES location are cloned and copied to the MicroDST. In the case of cloning via the ParticleCloner, only the ProtoParticle that is related to a given LHCb::Particle already being copied is cloned.

LHCb::FlavourTag

CopyFlavourTag. This MicroDST::KeyedContainerClonerAlg<LHCb::FlavourTag> required that the LHCb::FlavourTag objects be created and placed on the TES. Also, the pointer to the tagged candidate LHCb::Particle must be set correctly.

LHCb::MCParticles

The action of cloning a single LHCb::MCParticle onto the MicroDST is performed by any implementation of the ICloneMCVertex interface. There are three MicroDSTAlgorithms or MicroDSTTools that use an ICloneMCParticle:

* CopyMCParticles: this MicroDST::KeyedContainerClonerAlg<LHCb::MCParticle> takes all the LHCb::MCParticles in a TES location and clones them onto the MicroDST.

* CopyRelatedMCParticles: This specialised MicroDSTAlgorithm takes a set of LHCb::Particles from an input TES location, and clones all it's associated LHCb::MCParticles onto the MicroDST.

* MCVertexCloner: This implementation of ICloneMCVertex has an ICloneMCParticle pointer to clone the decay products of the LHCb::MCVertex.

At the moment, there is only one implementation of ICloneMCPrticle: the MCParticleCloner. As can be seen from the doxygen documentation, this in turn makes use of an ICloneMCVertex implementation to clone the decay end vertices.

LHCb::MCVertices

LHCb::MCVertices can be cloned as a by-product of the cloning of LHCb::MCParticles, where a simple copy of the particle's origin vertex is performed. Furthermore, if the MCParticleCloner implemementation of ICloneMCParticles is used, an !ICloneMCVertex pointer can be used to clone the decay vertices.

LHCb::MCHeader

The CopyMCHeader MicroDSTAlgorithm takes care of this, storing SmartRefs to the original primary LHCb::MCVertices. There is no runtime configurability implemented here.
 

Work in progress and known issues

  • Storage of neutral proto-particles (and associated MC truth) is not yet tested
Changed:
<
<
>
>
 
  • RecVertex cloning with tracks needs testing

Presentations

Revision 382009-06-12 - JuanPalacios

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Line: 98 to 98
  for all the event model types requiring non-trivial cloning actions. Each implementation controls the cloning action, so it is possible to affect the contents of a MicroDST by the choice of different implementations of a single interface. The interfaces and available implementations are summarised below. Each of the implementation inherits common functionality via the MicroDSTTool common base class.
Interface MicroDSTTool implementation Client MicroDSTAlgorithms
Changed:
<
<
ICloneFlavourTag    
ICloneMCParticle    
ICloneMCVertex    
ICloneParticle    
ICloneProtoParticle    
ICloneRecVertex    
ICloneTrack    
ICloneVertex    
>
>
ICloneFlavourTag FlavourTagCloner  
ICloneMCParticle MCParticleCloner  
ICloneMCVertex MCVertexCloner  
ICloneParticle ParticleCloner  
ICloneProtoParticle ProtoParticleCloner  
ICloneRecVertex RecVertexCloner, RecVertexClonerWithTracks  
ICloneTrack TrackCloner  
ICloneVertex VertexCloner  
 

Create a MicroDST

Line: 244 to 244
 By default, this new branch has the prefix microDST. To use the MicroDST for analysis, root directory on the TES needs to point to this branch which can be done via the parameter RootOnTES. Add the following to your options file
Changed:
<
<
RootOnTES = "/Event/microDST/"; RootOnTES = "/Event/microDST/"; TestSeq.Members += {"YourAlg"}; InputPrimaryVertices = "microDST/Rec/Vertex/Primary"; OfflinePVLocation = "/Event/microDST/Rec/Vertex/Primary"; ChargedPPLocation = {"/Event/microDST/Rec/ProtoP/Charged", "/Event/microDST/Rec/ProtoP/Upstream"};
>
>
RootOnTES = "/Event/microDST/" RootOnTES = "/Event/microDST/" TestSeq.Members += ["YourAlg"] InputPrimaryVertices = "microDST/Rec/Vertex/Primary" OfflinePVLocation = "/Event/microDST/Rec/Vertex/Primary" ChargedPPLocation = ["/Event/microDST/Rec/ProtoP/Charged", "/Event/microDST/Rec/ProtoP/Upstream"]
  N.B. You might have to add some further RootOnTES instructions depending on your algorithms and sequences. N.B. In recent Gaudi / DaVinci versions, the option RootInTES was introduced which replaces RootOnTES N.B. You might need to try some variations of the input locations, e.g. in DaVinci v19r5 I found that I had to use RootInTES and then "Phys/MyAlg" as input location for the PhysDesktop (instead of the full path) N.B. it may be beneficial to run
Changed:
<
<
Yourq.Members += {"MuonPIDsFromProtoParticlesAlg/CreateMuonPIDs"}; YouSeq.Members += {"RichPIDsFromProtoParticlesAlg/CreateRichPIDs"};
>
>
Yourq.Members += ["MuonPIDsFromProtoParticlesAlg/CreateMuonPIDs"] YouSeq.Members += [RichPIDsFromProtoParticlesAlg/CreateRichPIDs"]
  not via the Data-on-Demand service but at the beginning of the event.

To specify the input location for the PhysDesktop, use

Changed:
<
<
InputLocations = {"/Event/microDST/Path/to/Your/orig/Alg"};
>
>
InputLocations = ["/Event/microDST/Path/to/Your/orig/Alg"]
 

If needed/wanted, you can then also add the ROOT/POOL catalogue to the options file, e.g.

Revision 372009-06-12 - JuanPalacios

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Line: 13 to 13
 

The MicroDST algorithms

From DaVinci v23r0 onwards, all MicroDSTAlgorithms provided are specialisations of three generic template classes:

MicroDST::ObjectClonerAlg

Changed:
<
<
This covers the simplest case, a self-contained DataObject is cloned and stored on the MicroDST TES. Examples are the LHCb::RecHeader and LHCb::ODIN. This is how one of these is now defined in C++ (doxygen comments removed for clarity):
>
>
This covers the simplest case, a self-contained DataObject is cloned and stored on the MicroDST TES. Implementations exist for
Event Model Class MicroDSTAlgorithm Class Comments
LHCb::HltDecReports: CopyHltDecReports Python access to individual HltDecReports only via decision name. Iteration not implemented.
LHCb::L0DUReport CopyL0DUReport LHCb::L0DUConfig pointer not copied to the MicroDST. If there is a use-case for this, please inform the MicroDST software people ASAP.
LHCb::MCHeader: CopyMCHeader  
LHCb::ODIN CopyODIN  
LHCb::RecHeader CopyRecHeader  

This is how one of these is now defined in C++ (doxygen comments removed for clarity):

 
#include "MicroDST/ObjectClonerAlg.h"
#include "Event/RecHeader.h"
Line: 38 to 46
 

MicroDST::KeyedContainerClonerAlg

Changed:
<
<
This template class is intended to clone all the KeyedObject in a KeyedContainer in the TES into the MicroDST TES. It also allows for the cloning of related KeyedObject, either from SmartRefs or pointers. The algorithm contains a pointer to a pure virtual cloner tool that exports the methods
T* operator()(const T*);
T* clone(const T*);
The depth and scope of the cloning operation is controlled by the implementation of this cloner tool. The following example shows how the LHCb::Particle cloning algorithm is defined:
>
>
This template class is intended to clone all the KeyedObject in a KeyedContainer in the TES into the MicroDST TES. It also allows for the cloning of related KeyedObject, either from SmartRefs or pointers. The algorithm contains a pointer to a pure virtual cloner tool.

The depth and scope of the cloning operation is controlled by the implementation of this cloner tool. Specialisations exist for

Event Model Class MicroDSTALgorithm Class Comments
LHCb::FlavourTag CopyFlavourTags  
LHCb::MCParticle CopyMCParticles  
LHCb::Particle CopyParticles  
LHCb::ProtoParticle CopyProtoParticles  
LHCb::RecVertex CopyPrimaryVertices  
The following example shows how the LHCb::Particle cloning algorithm is defined:
 
#include "MicroDST/KeyedContainerClonerAlg.h"
#include <MicroDST/ICloneParticle.h>
Line: 73 to 85
 

MicroDST::RelationsClonerAlg

Changed:
<
<
This template takes as input a Relations table T, seeks the T::From objects from the MicroDST transient event store, and copies them and the corresponding T::To. It also creates and copies a new relations table to the MicroDST. T::To that are not already in the MicroDST TES are not cloned. This requires that the user create the necessary relations and clone the To before running the RelationsClonerAlg. A typical example is the cloning of the MCParticles related to a set of Particles. A typical job would run a selection, run some Monte Carlo association resulting in the production of a relations table, and clone the selected Particles to the MicroDST. Running the RelationsClonerAlg would take care of the MCParticles and the Relations table in one action.
>
>
This template takes as input a Relations table T, seeks the T::From objects from the MicroDST transient event store, and copies them and the corresponding T::To. It also creates and copies a new relations table to the MicroDST. T::To that are not already in the MicroDST TES are not cloned. This requires that the user create the necessary relations and clone the To before running the RelationsClonerAlg. A typical example is the cloning of the MCParticles related to a set of Particles. A typical job would run a selection, run some Monte Carlo association resulting in the production of a relations table, and clone the selected Particles to the MicroDST. Running the RelationsClonerAlg would take care of the MCParticles and the Relations table in one action. There exist implementations for
Event Model Class MicroDSTAlgorithm Class Comments
Particle2MCParticle::Table (LHCb::Relation1D<LHCb::Particle, LHCb::MCParticle >) CopyParticle2MCRelations For unweighted LHCb::Particle to LHCb::MCParticle associations
Particle2Vertex::Table (LHCb::RelationWeighted1D<LHCb::Particle, LHCb::VertexBase, double> ) CopyParticle2PVRelations For LHCb::Particle to primary vertex weighted relations. Used for "best vertex" finding.

The MicroDSTTools

There are a series of virtual tool interfaces with signatures
T* operator()(const T*);
T* clone(const T*);
for all the event model types requiring non-trivial cloning actions. Each implementation controls the cloning action, so it is possible to affect the contents of a MicroDST by the choice of different implementations of a single interface. The interfaces and available implementations are summarised below. Each of the implementation inherits common functionality via the MicroDSTTool common base class.
Interface MicroDSTTool implementation Client MicroDSTAlgorithms
ICloneFlavourTag    
ICloneMCParticle    
ICloneMCVertex    
ICloneParticle    
ICloneProtoParticle    
ICloneRecVertex    
ICloneTrack    
ICloneVertex    
 

Create a MicroDST

Line: 253 to 285
  -- JuanPalacios - 21 Aug 2008
Deleted:
<
<

Simple LHCb event types

The following simple LHCb event types T can be copied to the MicroDST via the corresponding MicroDST::ObjectClonerAlg MicroDSTAlgorithm specialisations:

* LHCb::RecHeader: CopyRecHeader

* LHCb::ODIN: CopyODIN

* LHCb::L0DUReport: CopyL0DUReport Note: At the moment, the LHCb::L0DUConfig pointer is not copied to the MicroDST. If there is a use-case for this, please inform the MicroDST software people ASAP.

* LHCb::HltDecReports: CopyHLTDecReports

 

LHCb::Particles

The cloning and storing of LHCb::Particles is performed by the CopyParticles MicroDSTAlgorithm. This typedef to MicroDST::KeyedContainerClonerAlg<LHCb::Particle> recursively clones particles and their descendants by use of an implementation of ICloneParticle. By default this is ParticleCloner, which in turn uses pointers to a ICloneVertex and an ICloneProtoParticle to store each LHCb::Particle's related LHCb::Vertices and LHCb::ProtoParticle. If these pointers are not set, then SmartRefs are stored instead.

Revision 362009-06-12 - JuanPalacios

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Line: 72 to 72
 #endif // COPYPRIMARYVERTICES_H
Changed:
<
<

!MicroDST::RelationsClonerAlg

This template takes as input a Relations table T, seeks the T::From objects from the MicroDST transient event store, and copies them and the corresponding T::To. It also creates and copies a new relations table to the MicroDST. =T::To=s that are not already in the MicroDST TES are not cloned. This requires that the user create the necessary relations and clone the =To=s before running the RelationsClonerAlg. A typical example is the cloning of the MCParticles related to a set of Particles. A typical job would run a selection, run some Monte Carlo association resulting in the production of a relations table, and clone the selected Particles to the MicroDST. Running the RelationsClonerAlg would take care of the MCParticles and the Relations table in one action.
>
>

MicroDST::RelationsClonerAlg

This template takes as input a Relations table T, seeks the T::From objects from the MicroDST transient event store, and copies them and the corresponding T::To. It also creates and copies a new relations table to the MicroDST. T::To that are not already in the MicroDST TES are not cloned. This requires that the user create the necessary relations and clone the To before running the RelationsClonerAlg. A typical example is the cloning of the MCParticles related to a set of Particles. A typical job would run a selection, run some Monte Carlo association resulting in the production of a relations table, and clone the selected Particles to the MicroDST. Running the RelationsClonerAlg would take care of the MCParticles and the Relations table in one action.
 

Create a MicroDST

Changed:
<
<
Ex/MicroDSTExample/options/TestMicroDSTMake.py is an opitons file covering a "typical" physics analysis scenario. Each MicroDSTAlgorithm specialization also has examples of use via options in its doxygen documentation. We include three of these algorithms here:
>
>
There are two ways to create a MicroDST.
  1. Manually set up all the necessary MicroDSTAlgorithms and MicroDSTTools, plus populate the TES with the objects that one wants to store. This is the approach taken in Ex/MicroDSTExample/options/TestMicroDSTMake.py, which is an opitons file covering a "typical" physics analysis scenario. It is well worth looking at this example to understand what is really going on and to see the clear distinction between running algorithms that populate the TES with objects, and running MicroDSTAlgorithms that clone these objects onto the MicroDST TES.
  2. Use the new PhysMicroDST configurable. It is a light-weight configurable designed for a "typical" physics use-case, but has limited flexibility and hides away many details of what is going on. There is an example script in Ex/MicroDSTExample/options/TestMicroDSTMakeNewConf.py, where one can clearly see how it is standalone and requires some interplay with another LHCb application configurable, in this case, DaVinci().

Each MicroDSTAlgorithm specialization also has examples of use via options in its doxygen documentation. We include three of these algorithms here:

 
from Gaudi.Configuration import *
from Gaudi.Configuration import FileCatalog
Line: 184 to 188
 

Python

Changed:
<
<
Ex/MicroDSTExample/python/MicroDSTReadingExample.py (replace "python" in the path for "scropts" from DaVinci v23r1p1 onwards) is a script reads a MicroDST and does some very basic analysis. It is possible to steer its behavior via arguments:
>
>
Ex/MicroDSTExample/python/MicroDSTReadingExample.py (replace "python" in the path for "scripts" from DaVinci v23r1p1 onwards) is a script reads a MicroDST and does some very basic analysis. It is possible to steer its behavior via arguments:
 
Usage: python -i MicroDSTReadingExample [options]
Options:

        --input         Input MicroDST file
Changed:
<
<
Default /afs/cern.ch/lhcb/group/davinci/vol1/DEV/data/DV_v20r2/testBs2JpsiPhi_5Kevt_pythonConfig.dst
>
>
Default ''
  --selection DV Selection run to make the MicroDST. Default 'DC06selBs2JpsiPhi_unbiased' --root TES root of everything. Default 'Event/microDST'
Changed:
<
<
Ex/MicroDSTExample also contains modules MicroDSTExample.Helpers and MicroDSTExample.Functors to aid in the reading of MictoDSTs and DSTs. The MicroDSTReadingExample.py script uses these extensively.
>
>
Ex/MicroDSTExample also contains modules MicroDSTExample.Helpers, MicroDSTExample.Functors, and MicroDSTExample.HistoUtils (borrowed from GaudiPython) to aid in the reading of MictoDSTs and DSTs. The MicroDSTReadingExample.py script uses these extensively.
  Note that here we do not use the DVAlgorithm and its vast tool set to do anything. Tasks can be performed by creating our own instances of tools. The file contains the initialisation required, the only step needed by the user is to correctly set the environment using SetupProject. It is intended simply to provide an example on how the contents can be retrieved, including complex objects like LHCb::Particle to LHCb::RecVertex and LHCb::MCParticle relations. It also includes an example on how to fit the proper time of a particle from the standard and "best" primary vertex.
Line: 231 to 235
 InputLocations = {"/Event/microDST/Path/to/Your/orig/Alg"};
Changed:
<
<
If needed/wanted, you can then also add the ROOT/POOL catalogue to the options file, e.g.
>
>
If needed/wanted, you can then also add the ROOT/POOL catalogue to the options file, e.g.
 
PoolDbCacheSvc.Catalog = {
  "xmlcatalog_file:DSTCatalog.xml",
Line: 250 to 254
 -- JuanPalacios - 21 Aug 2008

Simple LHCb event types

Changed:
<
<
The following simple LHCb event types can be copied to the MicroDST via the corresponding MicroDST::ObjectClonerAlg specialisations:
>
>
The following simple LHCb event types T can be copied to the MicroDST via the corresponding MicroDST::ObjectClonerAlg MicroDSTAlgorithm specialisations:
 
Changed:
<
<
* LHCb::RecHeader: CopyRecHeader MicroDSTAlgorithm. This is a typedef to MicroDST::ObjectClonerAlg<LHCb::RecHeader>.
>
>
* LHCb::RecHeader: CopyRecHeader
 
Changed:
<
<
* LHCb::ODIN: CopyODIN MicroDSTAlgorithm. Typedef to MicroDST::ObjectClonerAlg<LHCb::ODIN>
>
>
* LHCb::ODIN: CopyODIN
 
Changed:
<
<
* LHCb::L0DUReport: CopyL0DUReport MicroDSTAlgorithm. Note: At the moment, the LHCb::L0DUConfig pointer is not copied to the MicroDST. If there is a use-case for this, please inform the MicroDST software people ASAP. Typedef to MicroDST::ObjectClonerAlg<LHCb::L0DUReport>
>
>
* LHCb::L0DUReport: CopyL0DUReport Note: At the moment, the LHCb::L0DUConfig pointer is not copied to the MicroDST. If there is a use-case for this, please inform the MicroDST software people ASAP.
 
Changed:
<
<
* LHCb::HltSummary: CopyHLTSummary MicroDSTAlgorithm. Typedef to MicroDST::ObjectClonerAlg<LHCb::HltSummary>
>
>
* LHCb::HltDecReports: CopyHLTDecReports
 

LHCb::Particles

The cloning and storing of LHCb::Particles is performed by the CopyParticles MicroDSTAlgorithm. This typedef to MicroDST::KeyedContainerClonerAlg<LHCb::Particle> recursively clones particles and their descendants by use of an implementation of ICloneParticle. By default this is ParticleCloner, which in turn uses pointers to a ICloneVertex and an ICloneProtoParticle to store each LHCb::Particle's related LHCb::Vertices and LHCb::ProtoParticle. If these pointers are not set, then SmartRefs are stored instead.
Line: 303 to 307
 

LHCb::MCHeader

The CopyMCHeader MicroDSTAlgorithm takes care of this, storing SmartRefs to the original primary LHCb::MCVertices. There is no runtime configurability implemented here.
Deleted:
<
<

Miscellaneous

Generating a ROOT POOL catalogue

The ROOT/POOL catalogue stores links back to files containing further information about the objects used to create the input files and/or the MicroDST. This is necessary to resolve SmartRefs pointing to objects that are not stored on the MictoDST. These could be stored in its direct descendants (typically DSTs from Brunel), or further levels down info the Boole .digi or Gauss.sim files. The first line is used for read/write access, i.e. to create or update the information about the MicroDST, further lines are used for reading only and contain information about other files used.
from Gaudi.Configuration import FileCatalog
FileCatalog().Catalogs = ["xmlcatalog_file:MicroDSTCatalog.xml", "xmlcatalog_file:someInputCatalogue"]

Printing the content of the Transient Event Store (TES)

Including the following
from Configurables import StoreExplorerAlg
storeExp = StoreExplorerAlg()
ApplicationMgr().TopAlg += [storeExp]
storeExp.Load = 1
storeExp.PrintFreq = 1.0
storeExp.Outputlevel = 1
in your .py job file prints the current structure of the TES each event
 

Work in progress and known issues

  • Storage of neutral proto-particles (and associated MC truth) is not yet tested
  • ProcStatus information

Revision 352009-06-12 - JuanPalacios

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Deleted:
<
<
 

MicroDST

Deleted:
<
<

Overview

 The purpose of the MicroDST is to select and store a sub-set of data of interest from a file in the LHCb DST format, in a smaller file of the same format. Storing only the information of interest to a given analysis significantly reduces the size of the DST. The amount of space saved depends of course on the given analysis, i.e. it is beneficial to have a dedicated MicroDST per given analysis or per group of (closely) related analyses. The MicroDST code is written with flexibility and configurability in mind, such that it should be easy to define the contents of a MicroDST and configure a program that creates it. Since the MicroDST is in the LHCb DST format, it is even possible to run on a MicroDST and produce an even smaller file in the same format, ad infinitum. Furthermore, it is possible, at each stage of "reduction", to add new information not present in the original DST or MicroDST, as long as this information is in the form of supported Event Model classes.
Added:
>
>
 

Framework principles

Changed:
<
<
The MicroDST code is based on specialized tools and algorithms that clone Event Model objects from a TES location into a parallel location. By default, objects from TES location "/Event/XXX/YYY/ZZZ" are cloned into location "/Event/microDST/XXX/YYY/ZZZ". Standard job options are set such that everything under "/Event/microDST" is written to a file. The framework is split into a set of Gaudi tools (MicroDSTTool) that perform the cloning and storing operations, and parent Gaudi algorithms (MicroDSTAlgorithm) that control these tools and take care of complexities arising from inter-dependencies between objects in different TES locations. Technically, the software is split into link libraries containing common code, base classes, and public interfaces for the cloning tools, and component libraries containing implementations of the cloning tools and the controlling MicroDSTAlgorithms. For most cases, the user only needs to configure a Gaudi job via options or Configurables.
>
>
The MicroDST code is based on specialized tools and template algorithms that clone Event Model objects from a TES location into a parallel location. By default, objects from TES location "/Event/XXX/YYY/ZZZ" are cloned into location "/Event/microDST/XXX/YYY/ZZZ". Standard job options are set such that everything under "/Event/microDST" is written to a file. For clarity, this location shall be refered to as the "!MicroDST TES" in the following discussion. The framework is split into a set of Gaudi tools (MicroDSTTool) that perform the cloning and storing operations, and parent Gaudi algorithms (MicroDSTAlgorithm) that control these tools and take care of complexities arising from inter-dependencies between objects in different TES locations. Technically, the software is split into link libraries containing common code, base classes, and public interfaces for the cloning tools, and component libraries containing implementations of the cloning tools and the controlling MicroDSTAlgorithms. For most cases, the user only needs to configure a Gaudi job via options or Configurables. However, the generic and modular design of the software allows to easily add new types to the MicroDST and to change the cloning function of the cloner tools.
 
Changed:
<
<

Generic MicroDST algorithms

From DaVinci v20r2 onwards, two types of generic MicroDSTAlgorithms are provided. Most of the MicroDSTAlgorithms listed below have been re-written as simple typedefs of these generic classes. We distinguish two types:

MicroDST::ObjectClonerAlg

>
>

The MicroDST algorithms

From DaVinci v23r0 onwards, all MicroDSTAlgorithms provided are specialisations of three generic template classes:

MicroDST::ObjectClonerAlg

 This covers the simplest case, a self-contained DataObject is cloned and stored on the MicroDST TES. Examples are the LHCb::RecHeader and LHCb::ODIN. This is how one of these is now defined in C++ (doxygen comments removed for clarity):
#include "MicroDST/ObjectClonerAlg.h"
#include "Event/RecHeader.h"
//=============================================================================
Added:
>
>
// These lines are not necessary for the RecHeader, just here to show that you can // pick a cloner functor at compile time. BasicCopy is the default.
 template <> struct BindType2Cloner<LHCb::RecHeader> { typedef LHCb::RecHeader type;
Line: 33 to 34
 //============================================================================= typedef MicroDST::ObjectClonerAlg<LHCb::RecHeader> CopyRecHeader; // Declaration of the Algorithm Factory
Changed:
<
<
DECLARE_ALGORITHM_FACTORY( CopyRecHeader )
>
>
DECLARE_NAMED_ALGORITHM_FACTORY( CopyRecHeader )
 
Changed:
<
<

MicroDST::KeyedContainerClonerAlg

>
>

MicroDST::KeyedContainerClonerAlg

 This template class is intended to clone all the KeyedObject in a KeyedContainer in the TES into the MicroDST TES. It also allows for the cloning of related KeyedObject, either from SmartRefs or pointers. The algorithm contains a pointer to a pure virtual cloner tool that exports the methods
T* operator()(const T*);
Line: 67 to 68
 const std::string Location<LHCb::Particle>::Default = LHCb::ParticleLocation::Production; //============================================================================= typedef MicroDST::KeyedContainerClonerAlg<LHCb::Particle> CopyParticles;
Changed:
<
<
DECLARE_ALGORITHM_FACTORY( CopyParticles );
>
>
DECLARE_NAMED_ALGORITHM_FACTORY( CopyParticles );
 #endif // COPYPRIMARYVERTICES_H
Added:
>
>

!MicroDST::RelationsClonerAlg

This template takes as input a Relations table T, seeks the T::From objects from the MicroDST transient event store, and copies them and the corresponding T::To. It also creates and copies a new relations table to the MicroDST. =T::To=s that are not already in the MicroDST TES are not cloned. This requires that the user create the necessary relations and clone the =To=s before running the RelationsClonerAlg. A typical example is the cloning of the MCParticles related to a set of Particles. A typical job would run a selection, run some Monte Carlo association resulting in the production of a relations table, and clone the selected Particles to the MicroDST. Running the RelationsClonerAlg would take care of the MCParticles and the Relations table in one action.
 

Create a MicroDST

Ex/MicroDSTExample/options/TestMicroDSTMake.py is an opitons file covering a "typical" physics analysis scenario. Each MicroDSTAlgorithm specialization also has examples of use via options in its doxygen documentation. We include three of these algorithms here:

Line: 180 to 184
 

Python

Changed:
<
<
Ex/MicroDSTExample/python/MicroDSTReadingExample.py is a script reads a MicroDST and does some very basic analysis. It is possible to steer its behavior via arguments:
>
>
Ex/MicroDSTExample/python/MicroDSTReadingExample.py (replace "python" in the path for "scropts" from DaVinci v23r1p1 onwards) is a script reads a MicroDST and does some very basic analysis. It is possible to steer its behavior via arguments:
 
Usage: python -i MicroDSTReadingExample [options]

Revision 342009-04-14 - JuanPalacios

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Deleted:
<
<

IMPORTANT NEWS

DaVinci v21r0 has been released. A serious bug with flavour tag cloning has been fixed. All MicroDST making and reading functionality has been greatly enhanced. Users are encouraged to move to this version.

Bug fix to FlavourTag cloning (MicroDST/MicroDSTTools v1r3p1)

FlavourTag clones were not being copied to TES, hence not onto MicroDST. This is solved in DaVinci v21r0.

Python configuration module for templated MicroDSTAlgorithms

I have added a new package MicroDST/MicroDSTConf, with module MicroDST.MicroDSTAlgorithm. This contains shortcuts to the mangled names of the templated MicroDSTAlgorithms, for example:

from Configurables import MicroDST__ObjectClonerAlg_LHCb__RecHeader_ as CopyRecHeader

This makes writing MicroDST making jobs in Configurables much easier. However, the released version is broken (technically, there is no init.py file in the module directory so python can't find the module). To use it, it is necessary to getpack MicroDST/MicroDSTConf v1r0p1. The shortcuts are loaded thus:

from MicroDST.MicroDSTAlgorithm import *

Ex/MicroDST package extended and now included in release

The new package Ex/MicroDSTExample, is released with Analysis v2r3. It contains the files:
  • options/TestMicroDSTMake.py. This is an example to make a "typical" MicroDST from the lifetime unbiased Bs->J/Psi(MuMu)Phi(KK) selection.
  • python/MicroDSTReadingExample.py this script reads a MicroDST and does some very basic analysis. It is possible to steer its behavior via arguments:
And modules MicroDSTExample.Helpers and MicroDSTExample.Functors to aid in the reading of MictoDSTs and DSTs. The MicroDSTReadingExample.py script uses these extensively.

Usage: python -i MicroDSTReadingExample [options]
Options:

        --input         Input MicroDST file
                             Default /afs/cern.ch/lhcb/group/davinci/vol1/DEV/data/DV_v20r2/testBs2JpsiPhi_5Kevt_pythonConfig.dst
        --selection     DV Selection run to make the MicroDST.
                              Default 'DC06selBs2JpsiPhi_unbiased'
        --root          TES root of everything.
                            Default 'Event/microDST'

-- JuanPalacios - 10 Dec 2008

 

Overview

The purpose of the MicroDST is to select and store a sub-set of data of interest from a file in the LHCb DST format, in a smaller file of the same format. Storing only the information of interest to a given analysis significantly reduces the size of the DST. The amount of space saved depends of course on the given analysis, i.e. it is beneficial to have a dedicated MicroDST per given analysis or per group of (closely) related analyses. The MicroDST code is written with flexibility and configurability in mind, such that it should be easy to define the contents of a MicroDST and configure a program that creates it. Since the MicroDST is in the LHCb DST format, it is even possible to run on a MicroDST and produce an even smaller file in the same format, ad infinitum. Furthermore, it is possible, at each stage of "reduction", to add new information not present in the original DST or MicroDST, as long as this information is in the form of supported Event Model classes.
Line: 144 to 103
 # Setting up special MicroDST output stream importOptions('$MICRODSTOPTS/MicroDSTStream.py') MicroDSTStream=OutputStream('MicroDSTStream')
Added:
>
>
ApplicationMgr().OutStream.append(MicroDSTStream)
 MicroDSTStream.Output = "DATAFILE='Bs2JpsiPhiTest.dst' TYP='POOL_ROOTTREE' OPT='REC'" OutputLevel=4; AcceptAlgs.append( MySelection.name() )
Line: 196 to 156
 
importOptions('$FLAVOURTAGGINGOPTS/BTaggingTool.opts')
BTagAlgo = BTagging('BTagging')
Changed:
<
<
BTagAlgo.addTool(PhysDesktop())
>
>
BTagAlgo.addTool(PhysDesktop)
 BTaggingTool("BTaggingTool").OutputLevel=4 InputLocations=[mainLocation] BTagLocation = mainLocation+"/Tagging"
Line: 234 to 194
  Default 'Event/microDST'
Added:
>
>
Ex/MicroDSTExample also contains modules MicroDSTExample.Helpers and MicroDSTExample.Functors to aid in the reading of MictoDSTs and DSTs. The MicroDSTReadingExample.py script uses these extensively.
  Note that here we do not use the DVAlgorithm and its vast tool set to do anything. Tasks can be performed by creating our own instances of tools.
Changed:
<
<
The file contains the initialisation required, the only step needed by the user is to correctly set the environment using SetupProject. It is intended simply to provide an example on how the contents can be retrieved, including complex objects like LHCb::Particle to LHCb::RecVertex and LHCb::MCParticle relations. It also includes an example on how to fit the proper time of a particle from the standard and "best" primary vertex.
>
>
The file contains the initialisation required, the only step needed by the user is to correctly set the environment using SetupProject. It is intended simply to provide an example on how the contents can be retrieved, including complex objects like LHCb::Particle to LHCb::RecVertex and LHCb::MCParticle relations. It also includes an example on how to fit the proper time of a particle from the standard and "best" primary vertex.
 

C++

When creating the MicroDST, the algorithms replicate the structure of the transient event store (TES) in a new branch and store the information about the relevant particles, etc. there.
Line: 298 to 259
 

LHCb::Particles

The cloning and storing of LHCb::Particles is performed by the CopyParticles MicroDSTAlgorithm. This typedef to MicroDST::KeyedContainerClonerAlg<LHCb::Particle> recursively clones particles and their descendants by use of an implementation of ICloneParticle. By default this is ParticleCloner, which in turn uses pointers to a ICloneVertex and an ICloneProtoParticle to store each LHCb::Particle's related LHCb::Vertices and LHCb::ProtoParticle. If these pointers are not set, then SmartRefs are stored instead.
Deleted:
<
<

Storing user-data associated with a given particle on the MicroDST

The LHCb::Particle class allows to store user-data (of type double) via the extraInfo field. To add your data (e.g. measured lifetime,etc.), add the value to the particle via:
BCand->addInfo(LHCb::Particle::LastGlobal + 0, value);
where LHCb::Particle::LastGlobal is the last number reseved for official LHCb use. At the moment of writing, the framework only supports the copying of extraInfo that are already present in the original LHCb::Particles to be cloned.
 

Primary Vertices (LHCb::RecVetrex)

The cloning and storing of primary vertices is controlled by the CopyPrimaryVertices MicroDSTAlgorithm. This template to MicroDST::KeyedContainerClonerAlg<LHCb::RecVertex> uses an instance of the ICloneRecVertex interface, of which there are two implementations:

Revision 332009-01-12 - JuanPalacios

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Line: 218 to 218
 

Read a MicroDST

Added:
>
>

Python

 Ex/MicroDSTExample/python/MicroDSTReadingExample.py is a script reads a MicroDST and does some very basic analysis. It is possible to steer its behavior via arguments:
Line: 232 to 234
  Default 'Event/microDST'
Changed:
<
<

Using the MicroDST

Python

The file MicroDSTReadingExample.py gives a simple example of how to read a MicroDST created with the options file provided, and perform a few simple tasks with its contents. Note that here we do not use the DVAlgorithm and its vast tool set to do anything. Tasks can be performed by creating our own instances of tools.

The file contains the initialisation required, the only step needed by the user is to correctly set the environment using SetupProject. It is messy, simple and probably slow, but it is intended to provide an example on how the contents can be retrieved, including complex things like LHCb::Particle to LHCb::RecVertex and LHCb::MCParticle relations. It also includes an example on how to fit the proper time of a particle from the "best" primary vertex.

>
>
Note that here we do not use the DVAlgorithm and its vast tool set to do anything. Tasks can be performed by creating our own instances of tools. The file contains the initialisation required, the only step needed by the user is to correctly set the environment using SetupProject. It is intended simply to provide an example on how the contents can be retrieved, including complex objects like LHCb::Particle to LHCb::RecVertex and LHCb::MCParticle relations. It also includes an example on how to fit the proper time of a particle from the standard and "best" primary vertex.
 

C++

When creating the MicroDST, the algorithms replicate the structure of the transient event store (TES) in a new branch and store the information about the relevant particles, etc. there.

Revision 322008-12-11 - JuanPalacios

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Line: 232 to 232
  Default 'Event/microDST'
Deleted:
<
<

Generating the ROOT POOL catalogue

The ROOT/POOL catalogue stores information about which file contains (further) information about the objects used in the input file. The first line is used for read/write access, i.e. to create or update the information about the microDST, futher lines are used for reading only and contain information about other files used.
from Gaudi.Configuration import FileCatalog
FileCatalog().Catalogs = ["xmlcatalog_file:MicroDSTCatalog.xml", "xmlcatalog_file:someInputCatalogue"]
 

Using the MicroDST

Python

The file MicroDSTReadingExample.py gives a simple example of how to read a MicroDST created with the options file provided, and perform a few simple tasks with its contents. Note that here we do not use the DVAlgorithm and its vast tool set to do anything. Tasks can be performed by creating our own instances of tools.
Line: 357 to 347
 

LHCb::MCHeader

The CopyMCHeader MicroDSTAlgorithm takes care of this, storing SmartRefs to the original primary LHCb::MCVertices. There is no runtime configurability implemented here.
Changed:
<
<

Printing the content of the Transient Event Store (TES)

>
>

Miscellaneous

Generating a ROOT POOL catalogue

The ROOT/POOL catalogue stores links back to files containing further information about the objects used to create the input files and/or the MicroDST. This is necessary to resolve SmartRefs pointing to objects that are not stored on the MictoDST. These could be stored in its direct descendants (typically DSTs from Brunel), or further levels down info the Boole .digi or Gauss.sim files. The first line is used for read/write access, i.e. to create or update the information about the MicroDST, further lines are used for reading only and contain information about other files used.
from Gaudi.Configuration import FileCatalog
FileCatalog().Catalogs = ["xmlcatalog_file:MicroDSTCatalog.xml", "xmlcatalog_file:someInputCatalogue"]

Printing the content of the Transient Event Store (TES)

 Including the following
Changed:
<
<
TopAlg += { "StoreExplorerAlg" }; StoreExplorerAlg.Load = 1; PrintFreq = 1.0; StoreExplorerAlg.Outputlevel = 1;
>
>
from Configurables import StoreExplorerAlg storeExp = StoreExplorerAlg() ApplicationMgr().TopAlg += [storeExp] storeExp.Load = 1 storeExp.PrintFreq = 1.0 storeExp.Outputlevel = 1
 
Changed:
<
<
in your option files prints the current structure of the TES each event
>
>
in your .py job file prints the current structure of the TES each event
 

Work in progress and known issues

  • Storage of neutral proto-particles (and associated MC truth) is not yet tested
  • ProcStatus information
  • RecVertex cloning with tracks needs testing
Deleted:
<
<
  • Dictionaries for new templated classes.
 

Presentations

Revision 312008-12-10 - JuanPalacios

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Line: 114 to 114
 

Create a MicroDST

Changed:
<
<

Example options

DaVinci v20r2

TestMicroDSTMake.opts is an opitons file covering a "typical" physics analysis scenario. Each MicroDSTAlgorithm specialization also has examples of use via options in its doxygen documentation. We include three of these algorithms here:
>
>
Ex/MicroDSTExample/options/TestMicroDSTMake.py is an opitons file covering a "typical" physics analysis scenario. Each MicroDSTAlgorithm specialization also has examples of use via options in its doxygen documentation. We include three of these algorithms here:
 
Changed:
<
<
#include "$DAVINCIROOT/options/MicroDSTCommon.opts"
>
>
from Gaudi.Configuration import * from Gaudi.Configuration import FileCatalog from MicroDSTAlgorithm import * from Configurables import CopyRelatedMCParticles from Configurables import CopyParticle2PVLink from Configurables import MCParticleCloner from Configurables import MCVertexCloner from Configurables import VertexCloner from Configurables import ProtoParticleCloner from Configurables import PrintHeader from Configurables import OutputStream from Configurables import BTagging, BTaggingTool from Configurables import PhysDesktop from Configurables import PVReFitterAlg #================================================================= importOptions( "$DAVINCIROOT/options/DaVinciCommon.opts" ) # get some signal data. importOptions( "$MICRODSTEXAMPLEROOT/options/JpsiPhiDataLFN.py") #importOptions( "$MICRODSTEXAMPLEROOT/options/JpsiPhiDataPFN.py") # Import a selection and get it's sequencer importOptions( "$CCBARROOT/options/DoDC06SelBs2Jpsi2MuMuPhi2KK_lifetime_unbiased.opts" ) MySelection = GaudiSequencer('SeqDC06selBs2JpsiPhi') # base location of selection's Particles and Vertices mainLocation = "Phys/DC06selBs2JpsiPhi_unbiased" #================================================================= # Setting up special MicroDST output stream importOptions('$MICRODSTOPTS/MicroDSTStream.py') MicroDSTStream=OutputStream('MicroDSTStream') MicroDSTStream.Output = "DATAFILE='Bs2JpsiPhiTest.dst' TYP='POOL_ROOTTREE' OPT='REC'" OutputLevel=4; AcceptAlgs.append( MySelection.name() ) #=================================================================
MySelection.Members += [CopyRecHeader()]
 
Changed:
<
<
SeqDC06selBs2Jpsi2MuMu_Phi2KK.Members += {"MicroDST::KeyedContainerClonerAlg<LHCb::Particle>/CopyParticles"}; InputLocation = "Phys/DC06selBs2Jpsi2MuMu_Phi2KK/Particles"; // Chose an ICloneProtoParticles to also clone each Particle's ProtoParticle // ("NONE" results in no ProtoParticles being copied ICloneProtoParticle = "ProtoParticleCloner"; OutputPrefix = "microDST"; // This is the default
  • Copy the primary vertices of the selected events
   SeqDC06selBs2Jpsi2MuMu_Phi2KK.Members   += 
{""MicroDST::KeyedContainerClonerAlg<LHCb::RecVertex>/CopyPV"};
   CopyPV.InputLocation = "";
>
>
copyParticles = CopyParticles('CopyParticles') copyParticles.InputLocation = mainLocation+"/Particles" copyParticles.addTool(VertexCloner(), name='VertexCloner') copyParticles.addTool(ProtoParticleCloner(), name='ProtoParticleCloner') copyParticles.OutputLevel=4 MySelection.Members += [copyParticles]
  • Copy the primary vertices of the selected events and the Particle -> PV links
copyPV=CopyPrimaryVertices('CopyPrimaryVertices')
copyPV.OutputLevel = 4
MySelection.Members += [copyPV]
copyP2PVLink = CopyParticle2PVLink()
copyP2PVLink.InputLocation = mainLocation+"/Particle2VertexRelations"
copyP2PVLink.OutputLevel=4;
MySelection.Members += [copyP2PVLink]
 
  • Add some information that is NOT on the DST: re-fitted primary vertices, exluding tracks from the candidate B!
Changed:
<
<
// First, put re-fitted vertices and Particle->Vertex relations on the TES SeqDC06selBs2Jpsi2MuMu_Phi2KK.Members += {"PVReFitterAlg"}; ParticleInputLocation = "Phys/DC06selBs2Jpsi2MuMu_Phi2KK/Particles"; P2VRelationsOutputLocation = "Phys/DC06selBs2Jpsi2MuMu_Phi2KK/Particle2ReFittedVertexRelations"; VertexOutputLocation = "Phys/DC06selBs2Jpsi2MuMu_Phi2KK/RefittedVertices"; // Now clone the re-fitted vertices to the microDST TES SeqDC06selBs2Jpsi2MuMu_Phi2KK.Members += {"MicroDST::KeyedContainerClonerAlg<LHCb::RecVertex>/CopyRefittedPV"}; InputLocation = "Phys/DC06selBs2Jpsi2MuMu_Phi2KK/RefittedVertices"; // and clone the Particle->Re-fitted vertex relations SeqDC06selBs2Jpsi2MuMu_Phi2KK.Members += {"CopyParticle2PVLink/P2RefitPVLink"}; InputLocation = "Phys/DC06selBs2Jpsi2MuMu_Phi2KK/Particle2ReFittedVertexRelations"; Note that the full name of the template specialisaitons has to be passed in the options, since the Gaudi algorithm factory does not understand the typedefs at the time being. Extensions to the Gaudi framwork are in place, so for future releases it should be possible to use the typedefs directly.

Ex/MicroDSTExample

This package contains MicroDST making options and configurables, and a MicroDST reading python script It will be released with DaVinci v20r3. In the meantime it is necessary to getpack v1r1 in order to use it. It contains the files
>
>
PVReFitter = PVReFitterAlg("PVReFitterAlg") ParticleInputLocation = mainLocation+"/Particles" refittedPVLocation = mainLocation+"/RefittedVertices" VertexOutputLocation = refittedPVLocation p2ReFitPVRelationsLoc = mainLocation+"/Particle2ReFittedVertexRelations" P2VRelationsOutputLocation = p2ReFitPVRelationsLoc OutputLevel=4 MySelection.Members += [PVReFitter] # put the re-fitted vertices on the MicroDST copyReFittedPVs = CopyPrimaryVertices('CopyReFittedPVs') copyReFittedPVs.InputLocation = refittedPVLocation MySelection.Members += [copyReFittedPVs] # copy the Particle->PV relations table copyP2RefitPVLink = CopyParticle2PVLink("CopyP2RefitPVLink") copyP2RefitPVLink.InputLocation = p2ReFitPVRelationsLoc copyP2RefitPVLink.OutputLevel=4 MySelection.Members += [copyP2RefitPVLink]
importOptions('$FLAVOURTAGGINGOPTS/BTaggingTool.opts')
BTagAlgo = BTagging('BTagging')
BTagAlgo.addTool(PhysDesktop())
BTaggingTool("BTaggingTool").OutputLevel=4
BTagAlgo.PhysDesktop.InputLocations=[mainLocation]
BTagLocation = mainLocation+"/Tagging"
BTagAlgo.TagOutputLocation = BTagLocation
MySelection.Members += [BTagAlgo]
copyFlavTag = CopyFlavourTag()
copyFlavTag.InputLocation = BTagLocation
MySelection.Members += [copyFlavTag]
* copy related MC particles of candidates plus daughters
copyMC = CopyRelatedMCParticles()
copyMC.InputLocation = mainLocation+'/Particles'
copyMC.addTool(MCParticleCloner(), name= 'MCParticleCloner')
copyMC.MCParticleCloner.addTool(MCVertexCloner(), name = 'ICloneMCVertex')
copyMC.OutputLevel=4;
MySelection.Members += [copyMC]
 
Changed:
<
<
  • options/TestMicroDSTMake.opts, options/TestMicroDSTMake.py. These both make a "typical" MicroDST from the lifetime unbiased Bs->J/Psi(MuMu)Phi(KK) selection. Both are corrected for the change described above.
  • python/MicroDSTReadingExample.py this script reads a MicroDST and does some very basic analysis. It is possible to steer its behavior via arguments:
>
>

Read a MicroDST

Ex/MicroDSTExample/python/MicroDSTReadingExample.py is a script reads a MicroDST and does some very basic analysis. It is possible to steer its behavior via arguments:

 
Usage: python -i MicroDSTReadingExample [options]
Line: 178 to 238
 in the input file. The first line is used for read/write access, i.e. to create or update the information about the microDST, futher lines are used for reading only and contain information about other files used.
Changed:
<
<
PoolDbCacheSvc.Catalog = { "xmlcatalog_file:DSTCatalog.xml", "xmlcatalog_file:someOtherInputCatalogue" };
>
>
from Gaudi.Configuration import FileCatalog FileCatalog().Catalogs = ["xmlcatalog_file:MicroDSTCatalog.xml", "xmlcatalog_file:someInputCatalogue"]
 

Using the MicroDST

Revision 302008-12-10 - JuanPalacios

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Line: 6 to 6
 

IMPORTANT NEWS

Changed:
<
<
Since the release of DaVinci v20r2 and the software week presentations there have been some MicroDST developments which I think are worth mentioning:
>
>
DaVinci v21r0 has been released. A serious bug with flavour tag cloning has been fixed. All MicroDST making and reading functionality has been greatly enhanced. Users are encouraged to move to this version.
 
Changed:
<
<

Particle -> MCParticle association broken

For some obscure reason, DaVinciAssociators were failing to find associated MCParticles when the Particles are in the MicroDST TES location. The problem has been circumvented by getting the relations from the original particles, cloning the MCParticles, and then linking them to the clones of the Particles. To get the fix, it is necessary to
CopyMC.InputLocation = "microDST/Phys/DC06selBs2JpsiPhi_unbiased/Particles";
should now be
CopyMC.InputLocation = "Phys/DC06selBs2JpsiPhi_unbiased/Particles";
>
>

Bug fix to FlavourTag cloning (MicroDST/MicroDSTTools v1r3p1)

FlavourTag clones were not being copied to TES, hence not onto MicroDST. This is solved in DaVinci v21r0.
 

Python configuration module for templated MicroDSTAlgorithms

Changed:
<
<
I have added to DaVinci the modile MicroDSTAlgorithm. This contains shortcuts to the mangled names of the templated MictoDSTAlgorithms, for example:
>
>
I have added a new package MicroDST/MicroDSTConf, with module MicroDST.MicroDSTAlgorithm. This contains shortcuts to the mangled names of the templated MicroDSTAlgorithms, for example:
 
from Configurables import MicroDST__ObjectClonerAlg_LHCb__RecHeader_ as CopyRecHeader
Changed:
<
<
This makes writing MicroDST making jobs in Configurables much easier. To get this functionality, it is necessary to get the jpalac_20080925 tag of DaVinci
>
>
This makes writing MicroDST making jobs in Configurables much easier. However, the released version is broken (technically, there is no init.py file in the module directory so python can't find the module). To use it, it is necessary to getpack MicroDST/MicroDSTConf v1r0p1. The shortcuts are loaded thus:
 
Changed:
<
<

Examples package with making options and configurables, and a MicroDST reading python script

The new package Ex/MicroDSTExample has files:
  • options/TestMicroDSTMake.opts, options/TestMicroDSTMake.py. These both make a "typical" MicroDST from the lifetime unbiased Bs->J/Psi(MuMu)Phi(KK) selection. Both are corrected for the change described above.
>
>
from MicroDST.MicroDSTAlgorithm import *

Ex/MicroDST package extended and now included in release

The new package Ex/MicroDSTExample, is released with Analysis v2r3. It contains the files:
  • options/TestMicroDSTMake.py. This is an example to make a "typical" MicroDST from the lifetime unbiased Bs->J/Psi(MuMu)Phi(KK) selection.
 
  • python/MicroDSTReadingExample.py this script reads a MicroDST and does some very basic analysis. It is possible to steer its behavior via arguments:
Added:
>
>
And modules MicroDSTExample.Helpers and MicroDSTExample.Functors to aid in the reading of MictoDSTs and DSTs. The MicroDSTReadingExample.py script uses these extensively.
 
Usage: python -i MicroDSTReadingExample [options]
Line: 47 to 43
  Default 'Event/microDST'
Changed:
<
<
To use it is necessary to getpack Ex/!MicroDSTExample v1r1

All these changes will be in the next release of DaVinci.

-- JuanPalacios - 01 Oct 2008

>
>
-- JuanPalacios - 10 Dec 2008
 

Overview

The purpose of the MicroDST is to select and store a sub-set of data of interest from a file in the LHCb DST format, in a smaller file of the same format. Storing only the information of interest to a given analysis significantly reduces the size of the DST.
Line: 336 to 326
 
META FILEATTACHMENT attachment="InitialisePyGaudi.py.txt" attr="" comment="Initialise Gaudi - Python session" date="1187090879" name="InitialisePyGaudi.py.txt" path="InitialisePyGaudi.py" size="1848" stream="InitialisePyGaudi.py" user="Main.ukerzel" version="1"
META FILEATTACHMENT attachment="InitGaudi.py.txt" attr="" comment="Initialise Gaudi" date="1207298383" name="InitGaudi.py.txt" path="InitGaudi.py.txt" size="3514" stream="InitGaudi.py.txt" user="Main.UlrichKerzel" version="1"
META FILEATTACHMENT attachment="AnalysisMicroDST.py.txt" attr="" comment="Simple analysis script" date="1207298418" name="AnalysisMicroDST.py.txt" path="AnalysisMicroDST.py.txt" size="2116" stream="AnalysisMicroDST.py.txt" user="Main.UlrichKerzel" version="1"
Deleted:
<
<
META FILEATTACHMENT attachment="TestMicroDSTMake.opts" attr="" comment="Options file to make a %22typical%22 MicroDST" date="1222097947" name="TestMicroDSTMake.opts" path="TestMicroDSTMake.opts" size="6365" stream="TestMicroDSTMake.opts" user="Main.JuanPalacios" version="1"
 
META FILEATTACHMENT attachment="JpsiPhiData.opts" attr="" comment="Data card containing Bs -> J/Psi Phi MC signal. Used by TestMicroDSTMake.opts" date="1222098039" name="JpsiPhiData.opts" path="JpsiPhiData.opts" size="31309" stream="JpsiPhiData.opts" user="Main.JuanPalacios" version="1"
META FILEATTACHMENT attachment="MicroDSTReadingExample.py.txt" attr="" comment="Simple (and messy) python script to read a %22typical%22 MicroDST and make some plots." date="1222098220" name="MicroDSTReadingExample.py.txt" path="MicroDSTReadingExample.py" size="6081" stream="MicroDSTReadingExample.py" user="Main.JuanPalacios" version="1"

Revision 292008-10-01 - JuanPalacios

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

MicroDST

Added:
>
>

IMPORTANT NEWS

Since the release of DaVinci v20r2 and the software week presentations there have been some MicroDST developments which I think are worth mentioning:

Particle -> MCParticle association broken

For some obscure reason, DaVinciAssociators were failing to find associated MCParticles when the Particles are in the MicroDST TES location. The problem has been circumvented by getting the relations from the original particles, cloning the MCParticles, and then linking them to the clones of the Particles. To get the fix, it is necessary to
CopyMC.InputLocation = "microDST/Phys/DC06selBs2JpsiPhi_unbiased/Particles";
should now be
CopyMC.InputLocation = "Phys/DC06selBs2JpsiPhi_unbiased/Particles";

Python configuration module for templated MicroDSTAlgorithms

I have added to DaVinci the modile MicroDSTAlgorithm. This contains shortcuts to the mangled names of the templated MictoDSTAlgorithms, for example:
from Configurables import MicroDST__ObjectClonerAlg_LHCb__RecHeader_ as CopyRecHeader

This makes writing MicroDST making jobs in Configurables much easier. To get this functionality, it is necessary to get the jpalac_20080925 tag of DaVinci

Examples package with making options and configurables, and a MicroDST reading python script

The new package Ex/MicroDSTExample has files:
  • options/TestMicroDSTMake.opts, options/TestMicroDSTMake.py. These both make a "typical" MicroDST from the lifetime unbiased Bs->J/Psi(MuMu)Phi(KK) selection. Both are corrected for the change described above.
  • python/MicroDSTReadingExample.py this script reads a MicroDST and does some very basic analysis. It is possible to steer its behavior via arguments:

Usage: python -i MicroDSTReadingExample [options]
Options:

        --input         Input MicroDST file
                             Default /afs/cern.ch/lhcb/group/davinci/vol1/DEV/data/DV_v20r2/testBs2JpsiPhi_5Kevt_pythonConfig.dst
        --selection     DV Selection run to make the MicroDST.
                              Default 'DC06selBs2JpsiPhi_unbiased'
        --root          TES root of everything.
                            Default 'Event/microDST'

To use it is necessary to getpack Ex/!MicroDSTExample v1r1

All these changes will be in the next release of DaVinci.

-- JuanPalacios - 01 Oct 2008

 

Overview

The purpose of the MicroDST is to select and store a sub-set of data of interest from a file in the LHCb DST format, in a smaller file of the same format. Storing only the information of interest to a given analysis significantly reduces the size of the DST.
Line: 74 to 124
 

Create a MicroDST

Changed:
<
<

Example options (DaVinci v20r2 onwards)

>
>

Example options

DaVinci v20r2

 TestMicroDSTMake.opts is an opitons file covering a "typical" physics analysis scenario. Each MicroDSTAlgorithm specialization also has examples of use via options in its doxygen documentation. We include three of these algorithms here:
   #include "$DAVINCIROOT/options/MicroDSTCommon.opts"
Line: 111 to 162
  Note that the full name of the template specialisaitons has to be passed in the options, since the Gaudi algorithm factory does not understand the typedefs at the time being. Extensions to the Gaudi framwork are in place, so for future releases it should be possible to use the typedefs directly.
Added:
>
>

Ex/MicroDSTExample

This package contains MicroDST making options and configurables, and a MicroDST reading python script It will be released with DaVinci v20r3. In the meantime it is necessary to getpack v1r1 in order to use it. It contains the files

  • options/TestMicroDSTMake.opts, options/TestMicroDSTMake.py. These both make a "typical" MicroDST from the lifetime unbiased Bs->J/Psi(MuMu)Phi(KK) selection. Both are corrected for the change described above.
  • python/MicroDSTReadingExample.py this script reads a MicroDST and does some very basic analysis. It is possible to steer its behavior via arguments:

Usage: python -i MicroDSTReadingExample [options]
Options:

        --input         Input MicroDST file
                             Default /afs/cern.ch/lhcb/group/davinci/vol1/DEV/data/DV_v20r2/testBs2JpsiPhi_5Kevt_pythonConfig.dst
        --selection     DV Selection run to make the MicroDST.
                              Default 'DC06selBs2JpsiPhi_unbiased'
        --root          TES root of everything.
                            Default 'Event/microDST'
 

Generating the ROOT POOL catalogue

The ROOT/POOL catalogue stores information about which file contains (further) information about the objects used in the input file. The first line is used for read/write access, i.e. to create or update the information about the microDST,

Revision 282008-09-22 - JuanPalacios

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Line: 74 to 74
 

Create a MicroDST

Changed:
<
<

Accessing the code

The packages that constitute the MicroDST code are released in the Analysis project. Hence any environment that uses the Analysis project gives access to the MicroDST software. There is no need to obtain and build a copy of the code, it is simply used via job configuration. However, at the moment there is reasonable job options file released, since the options define the MicroDST content, and the content is analysis specific. The following example should serve as an illustration on how to select and store various pieces of information onto the MicroDST.

Example options

Each MicroDSTAlgorithm specialization has examples of use via options in its doxygen documentation. We include three of them here.
  • To create a MicroDST, the following has to be added to your options file to set up and configure the OutputStream:
   #include "$MICRODSTALGORITHMROOT/options/microDST.opt"
Or, from DaVinci v20r1 onwards,
>
>

Example options (DaVinci v20r2 onwards)

TestMicroDSTMake.opts is an opitons file covering a "typical" physics analysis scenario. Each MicroDSTAlgorithm specialization also has examples of use via options in its doxygen documentation. We include three of these algorithms here:
 
   #include "$DAVINCIROOT/options/MicroDSTCommon.opts"
Line: 130 to 122
 };
Deleted:
<
<
This then creates the MicroDST with the filename microDST.dst in the current working directory.
 

Using the MicroDST

Added:
>
>

Python

The file MicroDSTReadingExample.py gives a simple example of how to read a MicroDST created with the options file provided, and perform a few simple tasks with its contents. Note that here we do not use the DVAlgorithm and its vast tool set to do anything. Tasks can be performed by creating our own instances of tools.

The file contains the initialisation required, the only step needed by the user is to correctly set the environment using SetupProject. It is messy, simple and probably slow, but it is intended to provide an example on how the contents can be retrieved, including complex things like LHCb::Particle to LHCb::RecVertex and LHCb::MCParticle relations. It also includes an example on how to fit the proper time of a particle from the "best" primary vertex.

C++

 When creating the MicroDST, the algorithms replicate the structure of the transient event store (TES) in a new branch and store the information about the relevant particles, etc. there. By default, this new branch has the prefix microDST. To use the MicroDST for analysis, root directory on the TES needs to point to this branch which can be done via the parameter RootOnTES. Add the following to your options file
Line: 256 to 253
 
  • RecVertex cloning with tracks needs testing
  • Dictionaries for new templated classes.
Changed:
<
<

Using Python to work with the MicroDST

Python hints

IPython

IPython is a powerfull Python environment with many features, short-cuts, etc. which is also used by Ganga. To use it with the "right" Python environment, setup e.g. DaVinci or Panoramix and call python /lhcb/Ganga/external/ipython/0.6.13/slc3_gcc323/bin/ipython (or accordingly, where Ganga points to)

Working with the microDST and Python

I found it convenient to have two Python scripts when working with the microDST:
  • InitialisePyGaudi.py initialises the Gaudi session, loads all setups, etc. (see attached file)
  • Ana.py is then the script which holds the actual analysis code to loop over particles, fill histograms, etc.

-- JuanPalacios - 21 Aug 2008

>
>

Presentations

-- JuanPalacios - 22 Sep 2008
 
META FILEATTACHMENT attachment="InitialisePyGaudi.py.txt" attr="" comment="Initialise Gaudi - Python session" date="1187090879" name="InitialisePyGaudi.py.txt" path="InitialisePyGaudi.py" size="1848" stream="InitialisePyGaudi.py" user="Main.ukerzel" version="1"
META FILEATTACHMENT attachment="InitGaudi.py.txt" attr="" comment="Initialise Gaudi" date="1207298383" name="InitGaudi.py.txt" path="InitGaudi.py.txt" size="3514" stream="InitGaudi.py.txt" user="Main.UlrichKerzel" version="1"
META FILEATTACHMENT attachment="AnalysisMicroDST.py.txt" attr="" comment="Simple analysis script" date="1207298418" name="AnalysisMicroDST.py.txt" path="AnalysisMicroDST.py.txt" size="2116" stream="AnalysisMicroDST.py.txt" user="Main.UlrichKerzel" version="1"
Added:
>
>
META FILEATTACHMENT attachment="TestMicroDSTMake.opts" attr="" comment="Options file to make a %22typical%22 MicroDST" date="1222097947" name="TestMicroDSTMake.opts" path="TestMicroDSTMake.opts" size="6365" stream="TestMicroDSTMake.opts" user="Main.JuanPalacios" version="1"
META FILEATTACHMENT attachment="JpsiPhiData.opts" attr="" comment="Data card containing Bs -> J/Psi Phi MC signal. Used by TestMicroDSTMake.opts" date="1222098039" name="JpsiPhiData.opts" path="JpsiPhiData.opts" size="31309" stream="JpsiPhiData.opts" user="Main.JuanPalacios" version="1"
META FILEATTACHMENT attachment="MicroDSTReadingExample.py.txt" attr="" comment="Simple (and messy) python script to read a %22typical%22 MicroDST and make some plots." date="1222098220" name="MicroDSTReadingExample.py.txt" path="MicroDSTReadingExample.py" size="6081" stream="MicroDSTReadingExample.py" user="Main.JuanPalacios" version="1"

Revision 272008-09-15 - JuanPalacios

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Line: 10 to 10
 The amount of space saved depends of course on the given analysis, i.e. it is beneficial to have a dedicated MicroDST per given analysis or per group of (closely) related analyses. The MicroDST code is written with flexibility and configurability in mind, such that it should be easy to define the contents of a MicroDST and configure a program that creates it. Since the MicroDST is in the LHCb DST format, it is even possible to run on a MicroDST and produce an even smaller file in the same format, ad infinitum. Furthermore, it is possible, at each stage of "reduction", to add new information not present in the original DST or MicroDST, as long as this information is in the form of supported Event Model classes.

Framework principles

Changed:
<
<
The MicroDST code is based on specialized tools and algorithms that clone Event Model objects from a TES location into a parallel location. By default, objects from TES location "/Event/XXX/YYY/ZZZ" are cloned into location "/Event/microDST/XXX/YYY/ZZZ". Standard job options are set such that everything under "/Event/microDST" is written to a file. The framework is split into a set of Gaudi tools (!MicroDSTTool) that perform the cloning and storing operations, and parent Gaudi algorithms (MicroDSTAlgorithm) that control these tools and take care of complexities arising from inter-dependencies between objects in different TES locations. Technically, the software is split into link libraries containing common code, base classes, and public interfaces for the cloning tools, and component libraries containing implementations of the cloning tools and the controlling MicroDSTAlgorithms. For most cases, the user only needs to configure a Gaudi job via options or Configurables.
>
>
The MicroDST code is based on specialized tools and algorithms that clone Event Model objects from a TES location into a parallel location. By default, objects from TES location "/Event/XXX/YYY/ZZZ" are cloned into location "/Event/microDST/XXX/YYY/ZZZ". Standard job options are set such that everything under "/Event/microDST" is written to a file. The framework is split into a set of Gaudi tools (MicroDSTTool) that perform the cloning and storing operations, and parent Gaudi algorithms (MicroDSTAlgorithm) that control these tools and take care of complexities arising from inter-dependencies between objects in different TES locations. Technically, the software is split into link libraries containing common code, base classes, and public interfaces for the cloning tools, and component libraries containing implementations of the cloning tools and the controlling MicroDSTAlgorithms. For most cases, the user only needs to configure a Gaudi job via options or Configurables.
 

Generic MicroDST algorithms

From DaVinci v20r2 onwards, two types of generic MicroDSTAlgorithms are provided. Most of the MicroDSTAlgorithms listed below have been re-written as simple typedefs of these generic classes. We distinguish two types:
Line: 37 to 37
 DECLARE_ALGORITHM_FACTORY( CopyRecHeader )
Changed:
<
<

MicroDST::KeyedContainerClonerAlg

>
>

MicroDST::KeyedContainerClonerAlg

 This template class is intended to clone all the KeyedObject in a KeyedContainer in the TES into the MicroDST TES. It also allows for the cloning of related KeyedObject, either from SmartRefs or pointers. The algorithm contains a pointer to a pure virtual cloner tool that exports the methods
T* operator()(const T*);
Line: 78 to 78
 The packages that constitute the MicroDST code are released in the Analysis project. Hence any environment that uses the Analysis project gives access to the MicroDST software. There is no need to obtain and build a copy of the code, it is simply used via job configuration. However, at the moment there is reasonable job options file released, since the options define the MicroDST content, and the content is analysis specific. The following example should serve as an illustration on how to select and store various pieces of information onto the MicroDST.

Example options

Added:
>
>
Each MicroDSTAlgorithm specialization has examples of use via options in its doxygen documentation. We include three of them here.
 
  • To create a MicroDST, the following has to be added to your options file to set up and configure the OutputStream:
   #include "$MICRODSTALGORITHMROOT/options/microDST.opt"
Line: 110 to 111
  P2VRelationsOutputLocation = "Phys/DC06selBs2Jpsi2MuMu_Phi2KK/Particle2ReFittedVertexRelations"; VertexOutputLocation = "Phys/DC06selBs2Jpsi2MuMu_Phi2KK/RefittedVertices"; // Now clone the re-fitted vertices to the microDST TES
Changed:
<
<
SeqDC06selBs2Jpsi2MuMu_Phi2KK.Members += {""MicroDST::KeyedContainerClonerAlg<LHCb::RecVertex>/CopyRefittedPV"};
>
>
SeqDC06selBs2Jpsi2MuMu_Phi2KK.Members += {"MicroDST::KeyedContainerClonerAlg<LHCb::RecVertex>/CopyRefittedPV"};
  InputLocation = "Phys/DC06selBs2Jpsi2MuMu_Phi2KK/RefittedVertices"; // and clone the Particle->Re-fitted vertex relations SeqDC06selBs2Jpsi2MuMu_Phi2KK.Members += {"CopyParticle2PVLink/P2RefitPVLink"}; InputLocation = "Phys/DC06selBs2Jpsi2MuMu_Phi2KK/Particle2ReFittedVertexRelations";
Added:
>
>
Note that the full name of the template specialisaitons has to be passed in the options, since the Gaudi algorithm factory does not understand the typedefs at the time being. Extensions to the Gaudi framwork are in place, so for future releases it should be possible to use the typedefs directly.
 

Generating the ROOT POOL catalogue

The ROOT/POOL catalogue stores information about which file contains (further) information about the objects used in the input file. The first line is used for read/write access, i.e. to create or update the information about the microDST,
Line: 168 to 171
 for the input files from which the microDST was created. If you have access to these files, you can follow the SmartRefs on the microDST to access the information stored on the full DST (even if it is not available on the MicroDST). A possible use-case for this could be that you want to update the file or some plot with further information or that you have the
Changed:
<
<
full DST available but instead of reading in the complete event, only the relevant information (e.g. the B candidate) is read from the microDST and further information (say, track hits) are then obtained via the SmartRefs.
>
>
full DST available but instead of reading in the complete event, only the relevant information (e.g. the B candidate) is read from the microDST and further information (say, track hits) are then obtained via the SmartRefs.
 

Storing data on the MicroDST

Everything placed in the "/Event/microDST" trunk of the TES is stored in the MicroDST. MicroDSTAlgorithms and MicroDSTTools have been written to store many LHCb Event Model types. It is also possible, and in general quite easy, to write MicroDSTAlgorithms and MicroDSTTools to store other Event Model types, if an analysis so requires. The Event Model types supported at the moment have a certain bias towards physics selections and analysis since that is the background in which the MicroDST code has been developed. The following list contains the Event Model types that are currently supported, together with links to the doxygen documentation of the MicroDSTAlgorithms and - MicroDSTTools where, appropriate - in charge of cloning and storing them to the MicroDST. The doxygen documentation of the MicroDSTAlgorithms should always be the ultimate reference. The documentation of the MicroDSTTools is somewhat deficient at the moment, but I hope to change this in the near future.
Line: 177 to 179
 -- JuanPalacios - 21 Aug 2008

Simple LHCb event types

Changed:
<
<
The following simple LHCb event types can be copied to the MicroDST via the corresponding MicroDSTAlgorithms:
>
>
The following simple LHCb event types can be copied to the MicroDST via the corresponding MicroDST::ObjectClonerAlg specialisations:
 
Changed:
<
<
* LHCb::RecHeader: CopyRecHeader MicroDSTAlgorithm
>
>
* LHCb::RecHeader: CopyRecHeader MicroDSTAlgorithm. This is a typedef to MicroDST::ObjectClonerAlg<LHCb::RecHeader>.
 
Changed:
<
<
* LHCb::ODIN: CopyODIN MicroDSTAlgorithm
>
>
* LHCb::ODIN: CopyODIN MicroDSTAlgorithm. Typedef to MicroDST::ObjectClonerAlg<LHCb::ODIN>
 
Changed:
<
<
* LHCb::L0DUReport: CopyL0DUReport MicroDSTAlgorithm. Note: At the moment, the LHCb::L0DUConfig pointer is not copied to the MicroDST. If there is a use-case for this, please inform the MicroDST software people ASAP.
>
>
* LHCb::L0DUReport: CopyL0DUReport MicroDSTAlgorithm. Note: At the moment, the LHCb::L0DUConfig pointer is not copied to the MicroDST. If there is a use-case for this, please inform the MicroDST software people ASAP. Typedef to MicroDST::ObjectClonerAlg<LHCb::L0DUReport>
 
Changed:
<
<
* LHCb::HltSummary: CopyHLTSummary MicroDSTAlgorithm
>
>
* LHCb::HltSummary: CopyHLTSummary MicroDSTAlgorithm. Typedef to MicroDST::ObjectClonerAlg<LHCb::HltSummary>
 

LHCb::Particles

Changed:
<
<
The cloning and storing of LHCb::Particles is performed by the CopyParticles MicroDSTAlgorithm. This recursively clones particles and their descendants by use of an implementation of ICloneParticle. By default this is ParticleCloner, which in turn uses pointers to a ICloneVertex and an ICloneProtoParticle to store each LHCb::Particle's related LHCb::Vertices and LHCb::ProtoParticle. If these pointers are not set, then SmartRefs are stored instead.
>
>
The cloning and storing of LHCb::Particles is performed by the CopyParticles MicroDSTAlgorithm. This typedef to MicroDST::KeyedContainerClonerAlg<LHCb::Particle> recursively clones particles and their descendants by use of an implementation of ICloneParticle. By default this is ParticleCloner, which in turn uses pointers to a ICloneVertex and an ICloneProtoParticle to store each LHCb::Particle's related LHCb::Vertices and LHCb::ProtoParticle. If these pointers are not set, then SmartRefs are stored instead.
 

Storing user-data associated with a given particle on the MicroDST

The LHCb::Particle class allows to store user-data (of type double) via the extraInfo field. To add your data (e.g. measured lifetime,etc.),
Line: 196 to 198
 
BCand->addInfo(LHCb::Particle::LastGlobal + 0, value);
Changed:
<
<
where LHCb::Particle::LastGlobal is the last number reseved for official LHCb use.
>
>
where LHCb::Particle::LastGlobal is the last number reseved for official LHCb use. At the moment of writing, the framework only supports the copying of extraInfo that are already present in the original LHCb::Particles to be cloned.
 

Primary Vertices (LHCb::RecVetrex)

Changed:
<
<
The cloning and storing of primary vertices is controlled by the CopyPrimaryVertices MicroDSTAlgorithm. This uses an instance of the ICloneRecVertex interface, of which there are two implementations:
>
>
The cloning and storing of primary vertices is controlled by the CopyPrimaryVertices MicroDSTAlgorithm. This template to MicroDST::KeyedContainerClonerAlg<LHCb::RecVertex> uses an instance of the ICloneRecVertex interface, of which there are two implementations:
  * RecVertexCloner: Clones everything to the MicroDST, except for the LHCb::Tracks that make the vertex, for which it stores SmartRefs.
Line: 210 to 212
 Often it is necessary to establish a link between a candidate LHCb::Particle and a primary vertex (in the form of an LHCb::RecVertex). If the associations have been stored on the TES, it is possible to put them in the MicroDST using the CopyParticle2PVLink MicroDSTAlgorithm.

LHCb::Vertices

Added:
>
>
LHCb::Vertices are cloned as a by-product of the cloning of LHCb::Particles, when the CopyParticles MicroDSTAlgorithm uses the ParticleCloner implementation of ICloneParticle. The actual cloning is delegated to implementations of !ICloneVertex, the only implementation provided by the framework being !VertexCloner.
 

LHCb::Tracks

Individual LHCb::Tracks are cloned by implementations of the ICloneTrack interface. At the moment, MicroDSTTools ProtoParticleCloner and RecVertexClonerWithTracks are the only ones to use this.
Line: 218 to 221
 ProtoParticles can be cloned directly via the CopyProtoParticles MicroDSTAlgorithm, or indirectly via the ParticleCloner implementation of the ICloneParticle interface. Both of these call an implementation of the ICloneProtoParticle interface. In the case where the CopyProtoParticles MicroDSTAlgorithm is used, all the LHCb::ProtoParticles stores in the input TES location are cloned and copied to the MicroDST. In the case of cloning via the ParticleCloner, only the ProtoParticle that is related to a given LHCb::Particle already being copied is cloned.

LHCb::FlavourTag

Changed:
<
<
CopyFlavourTag
>
>
CopyFlavourTag. This MicroDST::KeyedContainerClonerAlg<LHCb::FlavourTag> required that the LHCb::FlavourTag objects be created and placed on the TES. Also, the pointer to the tagged candidate LHCb::Particle must be set correctly.
 

LHCb::MCParticles

The action of cloning a single LHCb::MCParticle onto the MicroDST is performed by any implementation of the ICloneMCVertex interface. There are three MicroDSTAlgorithms or MicroDSTTools that use an ICloneMCParticle:
Changed:
<
<
* CopyMCParticles: this MicroDSTAlgorithm takes all the LHCb::MCParticles in a TES location and clones them onto the MicroDST.
>
>
* CopyMCParticles: this MicroDST::KeyedContainerClonerAlg<LHCb::MCParticle> takes all the LHCb::MCParticles in a TES location and clones them onto the MicroDST.
 
Changed:
<
<
* CopyRelatedMCParticles: This MicroDSTAlgorithm takes a set of LHCb::Particles from an input TES location, and clones all it's associated LHCb::MCParticles onto the MicroDST.
>
>
* CopyRelatedMCParticles: This specialised MicroDSTAlgorithm takes a set of LHCb::Particles from an input TES location, and clones all it's associated LHCb::MCParticles onto the MicroDST.
  * MCVertexCloner: This implementation of ICloneMCVertex has an ICloneMCParticle pointer to clone the decay products of the LHCb::MCVertex.

At the moment, there is only one implementation of ICloneMCPrticle: the MCParticleCloner. As can be seen from the doxygen documentation, this in turn makes use of an ICloneMCVertex implementation to clone the decay end vertices.

LHCb::MCVertices

Changed:
<
<
LHCb::MCVertices can be cloned as a by-product of the cloning of LHCb::MCParticles, where a simple copy of the particle's origin vertex is performed. Furthermore, if the MCParticleCloner implemementation of ICloneMCParticles is used, an [[][ICloneMCVertexPointer]] can be used to clone the decay vertices.
>
>
LHCb::MCVertices can be cloned as a by-product of the cloning of LHCb::MCParticles, where a simple copy of the particle's origin vertex is performed. Furthermore, if the MCParticleCloner implemementation of ICloneMCParticles is used, an !ICloneMCVertex pointer can be used to clone the decay vertices.
 

LHCb::MCHeader

Changed:
<
<
The CopyMCHeader MicroDSTAlgorithm takes care of this, storing SmartRefs to the original primary LHCb::MCVertices. There is no configurability implemented here.
>
>
The CopyMCHeader MicroDSTAlgorithm takes care of this, storing SmartRefs to the original primary LHCb::MCVertices. There is no runtime configurability implemented here.
 

Printing the content of the Transient Event Store (TES)

Including the following
Line: 251 to 254
 
  • Storage of neutral proto-particles (and associated MC truth) is not yet tested
  • ProcStatus information
  • RecVertex cloning with tracks needs testing
Added:
>
>
  • Dictionaries for new templated classes.
 

Using Python to work with the MicroDST

Revision 262008-09-15 - JuanPalacios

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Line: 12 to 12
 

Framework principles

The MicroDST code is based on specialized tools and algorithms that clone Event Model objects from a TES location into a parallel location. By default, objects from TES location "/Event/XXX/YYY/ZZZ" are cloned into location "/Event/microDST/XXX/YYY/ZZZ". Standard job options are set such that everything under "/Event/microDST" is written to a file. The framework is split into a set of Gaudi tools (!MicroDSTTool) that perform the cloning and storing operations, and parent Gaudi algorithms (MicroDSTAlgorithm) that control these tools and take care of complexities arising from inter-dependencies between objects in different TES locations. Technically, the software is split into link libraries containing common code, base classes, and public interfaces for the cloning tools, and component libraries containing implementations of the cloning tools and the controlling MicroDSTAlgorithms. For most cases, the user only needs to configure a Gaudi job via options or Configurables.
Added:
>
>

Generic MicroDST algorithms

From DaVinci v20r2 onwards, two types of generic MicroDSTAlgorithms are provided. Most of the MicroDSTAlgorithms listed below have been re-written as simple typedefs of these generic classes. We distinguish two types:

MicroDST::ObjectClonerAlg

This covers the simplest case, a self-contained DataObject is cloned and stored on the MicroDST TES. Examples are the LHCb::RecHeader and LHCb::ODIN. This is how one of these is now defined in C++ (doxygen comments removed for clarity):
#include "MicroDST/ObjectClonerAlg.h"
#include "Event/RecHeader.h"
//=============================================================================
template <> struct BindType2Cloner<LHCb::RecHeader> 
{
  typedef LHCb::RecHeader type;
  typedef MicroDST::BasicCopy<LHCb::RecHeader> cloner;
};
//=============================================================================
template <> struct Location<LHCb::RecHeader> 
{
  const static std::string Default;
};
const std::string Location<LHCb::RecHeader>::Default = LHCb::RecHeaderLocation::Default;
//=============================================================================
typedef MicroDST::ObjectClonerAlg<LHCb::RecHeader> CopyRecHeader;
// Declaration of the Algorithm Factory
DECLARE_ALGORITHM_FACTORY( CopyRecHeader )

MicroDST::KeyedContainerClonerAlg

This template class is intended to clone all the KeyedObject in a KeyedContainer in the TES into the MicroDST TES. It also allows for the cloning of related KeyedObject, either from SmartRefs or pointers. The algorithm contains a pointer to a pure virtual cloner tool that exports the methods
T* operator()(const T*);
T* clone(const T*);
The depth and scope of the cloning operation is controlled by the implementation of this cloner tool. The following example shows how the LHCb::Particle cloning algorithm is defined:
#include "MicroDST/KeyedContainerClonerAlg.h"
#include <MicroDST/ICloneParticle.h>
#include "MicroDST/BindType2ClonerDef.h"
//=============================================================================
template <> struct BindType2Cloner<LHCb::Particle> 
{
  typedef LHCb::Particle type;
  typedef ICloneParticle cloner;
};
//=============================================================================
template<> struct Defaults<LHCb::Particle>
{
  const static std::string clonerType;
};
const std::string Defaults<LHCb::Particle>::clonerType = "ParticleCloner";
//=============================================================================
template<> struct Location<LHCb::Particle>
{
  const static std::string Default;
};
const std::string Location<LHCb::Particle>::Default = LHCb::ParticleLocation::Production;
//=============================================================================
typedef MicroDST::KeyedContainerClonerAlg<LHCb::Particle> CopyParticles;
DECLARE_ALGORITHM_FACTORY( CopyParticles );
#endif // COPYPRIMARYVERTICES_H
 

Create a MicroDST

Accessing the code

Line: 28 to 88
 
Changed:
<
<
SeqDC06selBs2Jpsi2MuMu_Phi2KK.Members += {"CopyParticles"};
>
>
SeqDC06selBs2Jpsi2MuMu_Phi2KK.Members += {"MicroDST::KeyedContainerClonerAlg<LHCb::Particle>/CopyParticles"};
  InputLocation = "Phys/DC06selBs2Jpsi2MuMu_Phi2KK/Particles"; // Chose an ICloneProtoParticles to also clone each Particle's ProtoParticle // ("NONE" results in no ProtoParticles being copied
Line: 37 to 98
 
  • Copy the primary vertices of the selected events
Changed:
<
<
SeqDC06selBs2Jpsi2MuMu_Phi2KK.Members += {"CopyPrimaryVertices/CopyPV"};
>
>
SeqDC06selBs2Jpsi2MuMu_Phi2KK.Members += {""MicroDST::KeyedContainerClonerAlg<LHCb::RecVertex>/CopyPV"};
  InputLocation = "";
  • Add some information that is NOT on the DST: re-fitted primary vertices, exluding tracks from the candidate B!
Line: 48 to 110
  P2VRelationsOutputLocation = "Phys/DC06selBs2Jpsi2MuMu_Phi2KK/Particle2ReFittedVertexRelations"; VertexOutputLocation = "Phys/DC06selBs2Jpsi2MuMu_Phi2KK/RefittedVertices"; // Now clone the re-fitted vertices to the microDST TES
Changed:
<
<
SeqDC06selBs2Jpsi2MuMu_Phi2KK.Members += {"CopyPrimaryVertices/CopyRefittedPV"};
>
>
SeqDC06selBs2Jpsi2MuMu_Phi2KK.Members += {""MicroDST::KeyedContainerClonerAlg<LHCb::RecVertex>/CopyRefittedPV"};
  InputLocation = "Phys/DC06selBs2Jpsi2MuMu_Phi2KK/RefittedVertices"; // and clone the Particle->Re-fitted vertex relations SeqDC06selBs2Jpsi2MuMu_Phi2KK.Members += {"CopyParticle2PVLink/P2RefitPVLink"};

Revision 252008-08-26 - JuanPalacios

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Line: 18 to 18
 The packages that constitute the MicroDST code are released in the Analysis project. Hence any environment that uses the Analysis project gives access to the MicroDST software. There is no need to obtain and build a copy of the code, it is simply used via job configuration. However, at the moment there is reasonable job options file released, since the options define the MicroDST content, and the content is analysis specific. The following example should serve as an illustration on how to select and store various pieces of information onto the MicroDST.

Example options

Changed:
<
<
  • To create a MicroDST, the following has to be added to your options file:
>
>
  • To create a MicroDST, the following has to be added to your options file to set up and configure the OutputStream:
 
   #include "$MICRODSTALGORITHMROOT/options/microDST.opt"
Added:
>
>
Or, from DaVinci v20r1 onwards,
   #include "$DAVINCIROOT/options/MicroDSTCommon.opts"
 
   SeqDC06selBs2Jpsi2MuMu_Phi2KK.Members   += {"CopyParticles"};

Revision 242008-08-22 - JuanPalacios

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Line: 110 to 110
  -- JuanPalacios - 21 Aug 2008
Changed:
<
<

LHCb::RecHeader

CopyRecHeader MicroDSTAlgorithm

LHCb::ODIN

CopyODIN MicroDSTAlgorithm

LHCb::L0DUReport

CopyL0DUReport MicroDSTAlgorithm

LHCb::HltSummary

CopyHLTSummary MicroDSTAlgorithm
>
>

Simple LHCb event types

The following simple LHCb event types can be copied to the MicroDST via the corresponding MicroDSTAlgorithms:

* LHCb::RecHeader: CopyRecHeader MicroDSTAlgorithm

* LHCb::ODIN: CopyODIN MicroDSTAlgorithm

* LHCb::L0DUReport: CopyL0DUReport MicroDSTAlgorithm. Note: At the moment, the LHCb::L0DUConfig pointer is not copied to the MicroDST. If there is a use-case for this, please inform the MicroDST software people ASAP.

* LHCb::HltSummary: CopyHLTSummary MicroDSTAlgorithm

 

LHCb::Particles

Changed:
<
<
The cloning and storing of LHCb::Particles is performed by the CopyParticles MicroDSTAlgorithm.
>
>
The cloning and storing of LHCb::Particles is performed by the CopyParticles MicroDSTAlgorithm. This recursively clones particles and their descendants by use of an implementation of ICloneParticle. By default this is ParticleCloner, which in turn uses pointers to a ICloneVertex and an ICloneProtoParticle to store each LHCb::Particle's related LHCb::Vertices and LHCb::ProtoParticle. If these pointers are not set, then SmartRefs are stored instead.
 

Storing user-data associated with a given particle on the MicroDST

The LHCb::Particle class allows to store user-data (of type double) via the extraInfo field. To add your data (e.g. measured lifetime,etc.),
Line: 143 to 146
 

LHCb::Vertices

LHCb::Tracks

Added:
>
>
Individual LHCb::Tracks are cloned by implementations of the ICloneTrack interface. At the moment, MicroDSTTools ProtoParticleCloner and RecVertexClonerWithTracks are the only ones to use this.
 

LHCb::ProtoParticles

ProtoParticles can be cloned directly via the CopyProtoParticles MicroDSTAlgorithm, or indirectly via the ParticleCloner implementation of the ICloneParticle interface. Both of these call an implementation of the ICloneProtoParticle interface. In the case where the CopyProtoParticles MicroDSTAlgorithm is used, all the LHCb::ProtoParticles stores in the input TES location are cloned and copied to the MicroDST. In the case of cloning via the ParticleCloner, only the ProtoParticle that is related to a given LHCb::Particle already being copied is cloned.
Line: 165 to 169
 LHCb::MCVertices can be cloned as a by-product of the cloning of LHCb::MCParticles, where a simple copy of the particle's origin vertex is performed. Furthermore, if the MCParticleCloner implemementation of ICloneMCParticles is used, an [[][ICloneMCVertexPointer]] can be used to clone the decay vertices.

LHCb::MCHeader

Changed:
<
<
The [[][CopyMCHeader]] MicroDSTAlgorithm takes care of this, storing SmartRefs to the original primary LHCb::MCVertices. There is no configurability implemented here.
>
>
The CopyMCHeader MicroDSTAlgorithm takes care of this, storing SmartRefs to the original primary LHCb::MCVertices. There is no configurability implemented here.
 

Printing the content of the Transient Event Store (TES)

Including the following

Revision 232008-08-22 - JuanPalacios

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Line: 105 to 105
 full DST available but instead of reading in the complete event, only the relevant information (e.g. the B candidate) is read from the microDST and further information (say, track hits) are then obtained via the SmartRefs.
Changed:
<
<

Possible MicroDST content

>
>

Storing data on the MicroDST

 Everything placed in the "/Event/microDST" trunk of the TES is stored in the MicroDST. MicroDSTAlgorithms and MicroDSTTools have been written to store many LHCb Event Model types. It is also possible, and in general quite easy, to write MicroDSTAlgorithms and MicroDSTTools to store other Event Model types, if an analysis so requires. The Event Model types supported at the moment have a certain bias towards physics selections and analysis since that is the background in which the MicroDST code has been developed. The following list contains the Event Model types that are currently supported, together with links to the doxygen documentation of the MicroDSTAlgorithms and - MicroDSTTools where, appropriate - in charge of cloning and storing them to the MicroDST. The doxygen documentation of the MicroDSTAlgorithms should always be the ultimate reference. The documentation of the MicroDSTTools is somewhat deficient at the moment, but I hope to change this in the near future.

-- JuanPalacios - 21 Aug 2008

Line: 151 to 151
 CopyFlavourTag

LHCb::MCParticles

Changed:
<
<
The action of cloning a single LHCb::MCParticle onto the MicroDST is performed by any implementation of the ICloneMCVertex interface. There are three MicroDSTAlgorithms or MicroDSTTools that use an ICloneMCVertex:
>
>
The action of cloning a single LHCb::MCParticle onto the MicroDST is performed by any implementation of the ICloneMCVertex interface. There are three MicroDSTAlgorithms or MicroDSTTools that use an ICloneMCParticle:
 
Changed:
<
<
CopyMCParticles
>
>
* CopyMCParticles: this MicroDSTAlgorithm takes all the LHCb::MCParticles in a TES location and clones them onto the MicroDST.
 
Changed:
<
<
CopyRelatedMCParticles
>
>
* CopyRelatedMCParticles: This MicroDSTAlgorithm takes a set of LHCb::Particles from an input TES location, and clones all it's associated LHCb::MCParticles onto the MicroDST.

* MCVertexCloner: This implementation of ICloneMCVertex has an ICloneMCParticle pointer to clone the decay products of the LHCb::MCVertex.

At the moment, there is only one implementation of ICloneMCPrticle: the MCParticleCloner. As can be seen from the doxygen documentation, this in turn makes use of an ICloneMCVertex implementation to clone the decay end vertices.

 
Deleted:
<
<
MCVertexCloner
 

LHCb::MCVertices

Added:
>
>
LHCb::MCVertices can be cloned as a by-product of the cloning of LHCb::MCParticles, where a simple copy of the particle's origin vertex is performed. Furthermore, if the MCParticleCloner implemementation of ICloneMCParticles is used, an [[][ICloneMCVertexPointer]] can be used to clone the decay vertices.
 

LHCb::MCHeader

Changed:
<
<
The MC header is stored to the microDST as well, preserving the SmartRefs to the original (true) primary vertices.
>
>
The [[][CopyMCHeader]] MicroDSTAlgorithm takes care of this, storing SmartRefs to the original primary LHCb::MCVertices. There is no configurability implemented here.
 

Printing the content of the Transient Event Store (TES)

Including the following

Revision 222008-08-22 - JuanPalacios

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Line: 148 to 148
 ProtoParticles can be cloned directly via the CopyProtoParticles MicroDSTAlgorithm, or indirectly via the ParticleCloner implementation of the ICloneParticle interface. Both of these call an implementation of the ICloneProtoParticle interface. In the case where the CopyProtoParticles MicroDSTAlgorithm is used, all the LHCb::ProtoParticles stores in the input TES location are cloned and copied to the MicroDST. In the case of cloning via the ParticleCloner, only the ProtoParticle that is related to a given LHCb::Particle already being copied is cloned.

LHCb::FlavourTag

Added:
>
>
CopyFlavourTag
 

LHCb::MCParticles

Added:
>
>
The action of cloning a single LHCb::MCParticle onto the MicroDST is performed by any implementation of the ICloneMCVertex interface. There are three MicroDSTAlgorithms or MicroDSTTools that use an ICloneMCVertex:
 
Added:
>
>
CopyMCParticles

CopyRelatedMCParticles

MCVertexCloner

 

LHCb::MCVertices

LHCb::MCHeader

Revision 212008-08-21 - JuanPalacios

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Line: 105 to 105
 full DST available but instead of reading in the complete event, only the relevant information (e.g. the B candidate) is read from the microDST and further information (say, track hits) are then obtained via the SmartRefs.
Changed:
<
<

Information stored on the MicroDST

Everything placed in the "/Event/microDST" trunk of the TES is stored in the MicroDST.
>
>

Possible MicroDST content

Everything placed in the "/Event/microDST" trunk of the TES is stored in the MicroDST. MicroDSTAlgorithms and MicroDSTTools have been written to store many LHCb Event Model types. It is also possible, and in general quite easy, to write MicroDSTAlgorithms and MicroDSTTools to store other Event Model types, if an analysis so requires. The Event Model types supported at the moment have a certain bias towards physics selections and analysis since that is the background in which the MicroDST code has been developed. The following list contains the Event Model types that are currently supported, together with links to the doxygen documentation of the MicroDSTAlgorithms and - MicroDSTTools where, appropriate - in charge of cloning and storing them to the MicroDST. The doxygen documentation of the MicroDSTAlgorithms should always be the ultimate reference. The documentation of the MicroDSTTools is somewhat deficient at the moment, but I hope to change this in the near future.
 
Changed:
<
<

Particles

>
>
-- JuanPalacios - 21 Aug 2008

LHCb::RecHeader

CopyRecHeader MicroDSTAlgorithm

LHCb::ODIN

CopyODIN MicroDSTAlgorithm

LHCb::L0DUReport

CopyL0DUReport MicroDSTAlgorithm

LHCb::HltSummary

CopyHLTSummary MicroDSTAlgorithm

LHCb::Particles

 The cloning and storing of LHCb::Particles is performed by the CopyParticles MicroDSTAlgorithm.
Added:
>
>

Storing user-data associated with a given particle on the MicroDST

The LHCb::Particle class allows to store user-data (of type double) via the extraInfo field. To add your data (e.g. measured lifetime,etc.), add the value to the particle via:
BCand->addInfo(LHCb::Particle::LastGlobal + 0, value);
where LHCb::Particle::LastGlobal is the last number reseved for official LHCb use.
 

Primary Vertices (LHCb::RecVetrex)

The cloning and storing of primary vertices is controlled by the CopyPrimaryVertices MicroDSTAlgorithm. This uses an instance of the ICloneRecVertex interface, of which there are two implementations:

* RecVertexCloner: Clones everything to the MicroDST, except for the LHCb::Tracks that make the vertex, for which it stores SmartRefs.

Changed:
<
<
* RecVertexClonerWithTracks: UNDER CONSTRUCTION Clones everything to the MicroDST, including the LHCb::Tracks making the vertex
>
>
* RecVertexClonerWithTracks: Clones everything to the MicroDST, including the LHCb::Tracks making the vertex
 
Changed:
<
<

Storing user-data associated with a given particle on the MicroDST

The LHCb::Particle class allows to store user-data (of type double) via the extraInfo field. To add your data (e.g. measured lifetime,etc.), add the value to the particle via:
BCand->addInfo(LHCb::Particle::LastGlobal + 0, value);
where LHCb::Particle::LastGlobal is the last number reseved for official LHCb use.
>
>

LHCb::Particle to LHCb::RecVertex association

Often it is necessary to establish a link between a candidate LHCb::Particle and a primary vertex (in the form of an LHCb::RecVertex). If the associations have been stored on the TES, it is possible to put them in the MicroDST using the CopyParticle2PVLink MicroDSTAlgorithm.

LHCb::Vertices

LHCb::Tracks

LHCb::ProtoParticles

ProtoParticles can be cloned directly via the CopyProtoParticles MicroDSTAlgorithm, or indirectly via the ParticleCloner implementation of the ICloneParticle interface. Both of these call an implementation of the ICloneProtoParticle interface. In the case where the CopyProtoParticles MicroDSTAlgorithm is used, all the LHCb::ProtoParticles stores in the input TES location are cloned and copied to the MicroDST. In the case of cloning via the ParticleCloner, only the ProtoParticle that is related to a given LHCb::Particle already being copied is cloned.

LHCb::FlavourTag

LHCb::MCParticles

LHCb::MCVertices

 
Changed:
<
<

MC (truth) information

>
>

LHCb::MCHeader

 The MC header is stored to the microDST as well, preserving the SmartRefs to the original (true) primary vertices.

Printing the content of the Transient Event Store (TES)

Line: 144 to 169
 

Work in progress and known issues

  • Storage of neutral proto-particles (and associated MC truth) is not yet tested
  • ProcStatus information
Changed:
<
<
  • RecVertex cloning with tracks
>
>
  • RecVertex cloning with tracks needs testing
 

Using Python to work with the MicroDST

Revision 202008-08-21 - JuanPalacios

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Changed:
<
<

microDST

>
>

MicroDST

 

Overview

Changed:
<
<
The purpose of the MicroDST is to select and store a sub-set of data of interest from a file in the LHCb DST format, in a smaller file of the same format. Storing only the information of interest to a given analysis significantly reduces the size of the DST. The amount of space saved depends of course on the given analysis, i.e. it is beneficial to have a dedicated microDST per given analysis or per group of (closely) related analyses. The microDST code is written with flexibility and configurability in mind, such that it should be easy to define the contents of a microDST and configure a program that creates it. Since the microDST is in theLHCb DST format, it is even possible to run on a microDST and produce an even smaller file in the same format, ad infinitum. Furthermore, it is possible, at each stage of "reduction", to add new information not present in the original DST or MicroDST, as long as this information is in the form of supported Event Model classes.
>
>
The purpose of the MicroDST is to select and store a sub-set of data of interest from a file in the LHCb DST format, in a smaller file of the same format. Storing only the information of interest to a given analysis significantly reduces the size of the DST. The amount of space saved depends of course on the given analysis, i.e. it is beneficial to have a dedicated MicroDST per given analysis or per group of (closely) related analyses. The MicroDST code is written with flexibility and configurability in mind, such that it should be easy to define the contents of a MicroDST and configure a program that creates it. Since the MicroDST is in the LHCb DST format, it is even possible to run on a MicroDST and produce an even smaller file in the same format, ad infinitum. Furthermore, it is possible, at each stage of "reduction", to add new information not present in the original DST or MicroDST, as long as this information is in the form of supported Event Model classes.
 

Framework principles

Changed:
<
<
The MicroDST code is based on specialized tools and algorithms that clone Event Model objects from a TES location into a parallel location. By default, objects from TES location "/Event/XXX/YYY/ZZZ" are cloned into location "/Event/microDST/XXX/YYY/ZZZ". Standard job options are set such that everything under "/Event/microDST" is written to a file. The framework is split into a set of Gaudi tools (MicroDSTTool) that perform the cloning and storing operations, and parent Gaudi algorithms (MicroDSTALgorithm) that control these tools and take care of complexities arising from interdependencies between objects in different TES locations. Technically, the software is split into link libraries containing common code, base classes, and public interfaces for the cloning tools, and component libraries containing implementations of the cloning tools and the controlling MicroDSTAlgorithms. For most cases, the user only needs to configure a Gaudi job via options or Configurables.
>
>
The MicroDST code is based on specialized tools and algorithms that clone Event Model objects from a TES location into a parallel location. By default, objects from TES location "/Event/XXX/YYY/ZZZ" are cloned into location "/Event/microDST/XXX/YYY/ZZZ". Standard job options are set such that everything under "/Event/microDST" is written to a file. The framework is split into a set of Gaudi tools (!MicroDSTTool) that perform the cloning and storing operations, and parent Gaudi algorithms (MicroDSTAlgorithm) that control these tools and take care of complexities arising from inter-dependencies between objects in different TES locations. Technically, the software is split into link libraries containing common code, base classes, and public interfaces for the cloning tools, and component libraries containing implementations of the cloning tools and the controlling MicroDSTAlgorithms. For most cases, the user only needs to configure a Gaudi job via options or Configurables.
 
Changed:
<
<

Create a microDST

>
>

Create a MicroDST

 

Accessing the code

Changed:
<
<
The packages that constitute the microDST code are released in the Analysis project. Hence any environment that uses the Analysis project gives access to the MicroDST software. There is no need to obtain and build a copy of the code, it is simply used via job configuration. However, at the moment there is reasonable job options file released, since the options define the MicroDST content, and the content is analysis specific. The following example should serve as an illustration on how to select and store various pieces of information onto the MicroDST.
>
>
The packages that constitute the MicroDST code are released in the Analysis project. Hence any environment that uses the Analysis project gives access to the MicroDST software. There is no need to obtain and build a copy of the code, it is simply used via job configuration. However, at the moment there is reasonable job options file released, since the options define the MicroDST content, and the content is analysis specific. The following example should serve as an illustration on how to select and store various pieces of information onto the MicroDST.
 

Example options

Changed:
<
<
  • To create a microDST, the following has to be added to your options file:
>
>
  • To create a MicroDST, the following has to be added to your options file:
 
   #include "$MICRODSTALGORITHMROOT/options/microDST.opt"
Line: 61 to 61
 };
Changed:
<
<
This then creates the microDST with the filename microDST.dst in the current working directory.
>
>
This then creates the MicroDST with the filename microDST.dst in the current working directory.
 
Changed:
<
<

Using the microDST

When creating the microDST, the algorithms replicate the structure of the transient event store (TES) in a new branch and store the information about the relevant particles, etc. there.
>
>

Using the MicroDST

When creating the MicroDST, the algorithms replicate the structure of the transient event store (TES) in a new branch and store the information about the relevant particles, etc. there.
 By default, this new branch has the prefix microDST.
Changed:
<
<
To use the microDST for analysis, root directory on the TES needs to point to this branch which can be done via the parameter RootOnTES. Add the following to your options file
>
>
To use the MicroDST for analysis, root directory on the TES needs to point to this branch which can be done via the parameter RootOnTES. Add the following to your options file
 
DaVinciMainSeq.RootOnTES                                                  = "/Event/microDST/";
YourSeq.RootOnTES                                                              = "/Event/microDST/";
Line: 98 to 98
  "xmlcatalog_file:CatalogueOfInputFilesToMicroDST" };
Changed:
<
<
The first line again holds the information about the objects used in the microDST, the further line(s) contain the catalogue(s) for the input files from which the microDST was created. If you have access to these files, you can follow the SmartRefs on the microDST to access the information stored on the full DST (even if it is not available on the microDST).
>
>
The first line again holds the information about the objects used in the M!icroDST, the further line(s) contain the catalogue(s) for the input files from which the microDST was created. If you have access to these files, you can follow the SmartRefs on the microDST to access the information stored on the full DST (even if it is not available on the MicroDST).
 A possible use-case for this could be that you want to update the file or some plot with further information or that you have the full DST available but instead of reading in the complete event, only the relevant information (e.g. the B candidate) is read from the microDST and
Changed:
<
<
further information (say, track hits) are then obtained via the SmartRefs.
>
>
further information (say, track hits) are then obtained via the SmartRefs.
 
Changed:
<
<

Information stored on the microDST

The microDST stores all the information related to the particles given as input to the PhysDesktop of the CreateMicroDSTAlg. Typically, these would be the candidate particles resulting from a selection or pre-selection. For example, if a Bs -> J/psi (-> mu mu) Phi (-> KK) is selected by an algorithm, the microDST would contain the Bs candidate, the J/psi with the muons, the Phi with the kaons, the decay vertices of the Bs, the J/psi and the phi, as well as the underlying proto-particles and tracks for the 'basic' particles (muons and kaons in this case). In addition, the ODIN bank and the Event/Rec header are stored as well to have access to run-number, event-time and number, etc.

-- JuanPalacios - 30 Sep 2007

Primary Vertices on the microDST

Information about the primary vertices is also stored, e.g. position and covariance matrix, number of tracks etc. However, to save space, the tracks attached to the primary vertex (and further detailed information) is not stored. This makes it possible to use information about the primary vertex on the microDST, however, refitting the primary vertex is not possible (as the tracks are not available). If this is needed or wanted, this should be done when producing the microDST.

Storing user-created primary vertices on the microDST

The microDST supports primary vertices created by the user. This can be useful when measuring the lifetime of a given particle: To avoid a potential bias, the primary vertex may be re-fitted excluding the tracks which are used to build the signal candidate. If the user-PV is stored at the location: yourAlg/PrimaryVertices (e.g. /Event/Phys/DC06selBs2Jpsi2MuMu_Phi2KK/PrimaryVertices), they will be automatically stored on the microDST as well.
>
>

Information stored on the MicroDST

Everything placed in the "/Event/microDST" trunk of the TES is stored in the MicroDST.
 
Changed:
<
<

Storing user-data associated with a given particle on the microDST

>
>

Particles

The cloning and storing of LHCb::Particles is performed by the CopyParticles MicroDSTAlgorithm.

Primary Vertices (LHCb::RecVetrex)

The cloning and storing of primary vertices is controlled by the CopyPrimaryVertices MicroDSTAlgorithm. This uses an instance of the ICloneRecVertex interface, of which there are two implementations:

* RecVertexCloner: Clones everything to the MicroDST, except for the LHCb::Tracks that make the vertex, for which it stores SmartRefs.

* RecVertexClonerWithTracks: UNDER CONSTRUCTION Clones everything to the MicroDST, including the LHCb::Tracks making the vertex

Storing user-data associated with a given particle on the MicroDST

 The LHCb::Particle class allows to store user-data (of type double) via the extraInfo field. To add your data (e.g. measured lifetime,etc.), add the value to the particle via:
Line: 132 to 129
 where LHCb::Particle::LastGlobal is the last number reseved for official LHCb use.

MC (truth) information

Deleted:
<
<
If the CreateMicroDSTMCAlg algorithm is executed as well, MC (truth) information is also stored on the microDST.

* treatment of 'basic' particles (without daughthers, e.g. muons) The Particle2MCLinker(this,Particle2MCMethod::Links,"") associator is used to find the MC (truth) particle corresponding to the reconstructed (basic) particle (e.g. the muons and kaons in the example of Bs -> J/psi (-> mu mu) Phi (-> KK) ). The link between the underlying (reconstructed) object and the corresponding MC (truth) particle is stored on the microDST such that the above associator should work if running on the microDST.

* treatment of 'composite' particles (e.g. a B candidate) Composite particles are found via the Particle2MCLinker(this,Particle2MCMethod::Composite,"") associator. However, due to the way the composite associator works, this association cannot be stored on the microDST in a way that the default composite associator will be able to establish this association when operating on the microDST. Instead, a direct link between the composite reconstructed particle and the corresponding MC particle is saved on the microDST. In order to retrieve this link, execute the following (see TestDST.cpp and TestDST.h). It might be possible to extend the default composite associator to handle this case, this is currently under investigation.

       std::string locTES = TestDST::objectLocation(particle->parent() );
       verbose() << "try to get direct link for particle on TES " << locTES << endmsg;
       Part2MCPart p2pAssociator(eventSvc(), locTES);
       const Part2MCPartTable *p2pTable = p2pAssociator.direct();
       if (p2pTable) {
       Part2MCPartRange p2pRange =  p2pTable->relations(particle);
       verbose() << "found #relations " << p2pRange.size() << endmsg;
      for (Part2MCPartIterator iRel = p2pRange.begin(); iRel != p2pRange.end(); iRel++) {
         const LHCb::MCParticle *mcPart = iRel->to();
         double                  weight = iRel->weight();
         verbose() << "particle related to MCPart with pid " << mcPart->particleID().pid() 
                        << " weight " << weight 
                      << " on TES " <<  TestDST::objectLocation(mcPart->parent())
                    << endmsg;     
        }//for iRel
       } else {
      verbose() << "table for direct association not found " << endmsg;
     }// if table
       

N.B. This extension has been committed to Phys/DaVinciAssociators/.../src/CompositeParticle2MCLinks.cpp and should be available in the default installation soon

 The MC header is stored to the microDST as well, preserving the SmartRefs to the original (true) primary vertices.
Deleted:
<
<

Information not stored on the microDST

In order to avoid loss of information, SmartRefs are set in case a given quantity is not stored on the microDST. Thus, if one has access to the full DST (and the corresponding XML catalogue), all quantities can be accessed via the SmartRefs from the microDST. This has the advantage that one can always operate on the microDST (and hence loop over only the candidates of interest) even if further information is required. Information not stored:
  • tracks attached to primary vertex
  • raw data-banks
  • other particles than the selected (signal) candidates and the attached daughters, etc.
  • detailed MC truth information (e.g. Bremsstrahlungs photons and corresponding vertices, etc. of an electron, ...
 

Printing the content of the Transient Event Store (TES)

Including the following
Line: 193 to 142
 in your option files prints the current structure of the TES each event

Work in progress and known issues

Changed:
<
<
  • storage of neutral proto-particles (and associated MC truth) is not yet working/tested
  • using the composite associator to retrieve information about a composite MC particle, work in progress
  • adding flavour tagging information to the microDST
  • adding trigger summary information to the microDST
  • ProcStatus information (once copy-constructor is available)
>
>
  • Storage of neutral proto-particles (and associated MC truth) is not yet tested
  • ProcStatus information
  • RecVertex cloning with tracks
 
Changed:
<
<

Using Python to work with the microDST

>
>

Using Python to work with the MicroDST

 

Python hints

Line: 215 to 162
 
Changed:
<
<
-- JuanPalacios - 30 Sep 2007
>
>
-- JuanPalacios - 21 Aug 2008
 

Revision 192008-08-20 - JuanPalacios

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Line: 6 to 6
 

Overview

Changed:
<
<
The idea of the microDST is to store all relevant information about the particles of interest by creating a new file (the microDST) in the standard LHCb DST format. Storing only the candidates of interest significantly reduces the size of the DST file used by concentrating on the information relevant to the given analysis. The amount of space saved depends of course on the given analysis, i.e. it is beneficial to have a dedicated microDST per given analysis or per group of (closely) related analyses. Two algorithms are used to create the microDST:
  • CreateMicroDSTAlg handles reconstructed particles and the associated vertices, the reconstructed primary vertex, etc. This algorithm is intended also to be used on data.
  • CreateMicroDSTMCAlg handles MC (truth) particles and vertices, also establishes links between the reconstructed particles and the associated MC particles.
A further algorithm provided by the package (TestDST) can be used to investigate the content of the created microDST and allows to print the content of the various containers, etc. stored on the microDST.
>
>
The purpose of the MicroDST is to select and store a sub-set of data of interest from a file in the LHCb DST format, in a smaller file of the same format. Storing only the information of interest to a given analysis significantly reduces the size of the DST. The amount of space saved depends of course on the given analysis, i.e. it is beneficial to have a dedicated microDST per given analysis or per group of (closely) related analyses. The microDST code is written with flexibility and configurability in mind, such that it should be easy to define the contents of a microDST and configure a program that creates it. Since the microDST is in theLHCb DST format, it is even possible to run on a microDST and produce an even smaller file in the same format, ad infinitum. Furthermore, it is possible, at each stage of "reduction", to add new information not present in the original DST or MicroDST, as long as this information is in the form of supported Event Model classes.

Framework principles

The MicroDST code is based on specialized tools and algorithms that clone Event Model objects from a TES location into a parallel location. By default, objects from TES location "/Event/XXX/YYY/ZZZ" are cloned into location "/Event/microDST/XXX/YYY/ZZZ". Standard job options are set such that everything under "/Event/microDST" is written to a file. The framework is split into a set of Gaudi tools (MicroDSTTool) that perform the cloning and storing operations, and parent Gaudi algorithms (MicroDSTALgorithm) that control these tools and take care of complexities arising from interdependencies between objects in different TES locations. Technically, the software is split into link libraries containing common code, base classes, and public interfaces for the cloning tools, and component libraries containing implementations of the cloning tools and the controlling MicroDSTAlgorithms. For most cases, the user only needs to configure a Gaudi job via options or Configurables.
 

Create a microDST

Changed:
<
<

Getting the code

The microDST code is available from the central LHCb software repository (CVS) and can be obtained via getpack MicroDST/MicroDSTAlgorithm. During the current development and test phase, the head-revision should be used.

Options

To create a microDST, the following has to be added to your options file:
  • #include "$MICRODSTALGORITHMROOT/options/microDST.opt"
  • Tell the microDST algorithm which particles are to be stored, e.g. for Bs -> J/psi Phi:
>
>

Accessing the code

The packages that constitute the microDST code are released in the Analysis project. Hence any environment that uses the Analysis project gives access to the MicroDST software. There is no need to obtain and build a copy of the code, it is simply used via job configuration. However, at the moment there is reasonable job options file released, since the options define the MicroDST content, and the content is analysis specific. The following example should serve as an illustration on how to select and store various pieces of information onto the MicroDST.

Example options

  • To create a microDST, the following has to be added to your options file:
 
Changed:
<
<
SeqDC06selBs2Jpsi2MuMu_Phi2KK.Members += {"CreateMicroDSTAlg/microDST"}; SeqDC06selBs2Jpsi2MuMu_Phi2KK.Members += {"CreateMicroDSTMCAlg/microDSTMC"}; microDST.PhysDesktop.InputLocations = {"/Event/Phys/DC06selBs2Jpsi2MuMu_Phi2KK"}; microDSTMC.PhysDesktop.InputLocations = {"/Event/Phys/DC06selBs2Jpsi2MuMu_Phi2KK"};
>
>
#include "$MICRODSTALGORITHMROOT/options/microDST.opt"
   SeqDC06selBs2Jpsi2MuMu_Phi2KK.Members   += {"CopyParticles"};
   CopyParticles.InputLocation = "Phys/DC06selBs2Jpsi2MuMu_Phi2KK/Particles";
   // Chose an ICloneProtoParticles to also clone each Particle's ProtoParticle 
   // ("NONE" results in no ProtoParticles being copied
   CopyParticles.ParticleCloner.ICloneProtoParticle = "ProtoParticleCloner";
   CopyParticles.OutputPrefix = "microDST"; // This is the default
  • Copy the primary vertices of the selected events
   SeqDC06selBs2Jpsi2MuMu_Phi2KK.Members   += {"CopyPrimaryVertices/CopyPV"};
   CopyPV.InputLocation = "";
  • Add some information that is NOT on the DST: re-fitted primary vertices, exluding tracks from the candidate B!
   // First, put re-fitted vertices and Particle->Vertex relations on the TES
   SeqDC06selBs2Jpsi2MuMu_Phi2KK.Members += {"PVReFitterAlg"};
   PVReFitterAlg.ParticleInputLocation = "Phys/DC06selBs2Jpsi2MuMu_Phi2KK/Particles";
   PVReFitterAlg.P2VRelationsOutputLocation = "Phys/DC06selBs2Jpsi2MuMu_Phi2KK/Particle2ReFittedVertexRelations";
   PVReFitterAlg.VertexOutputLocation = "Phys/DC06selBs2Jpsi2MuMu_Phi2KK/RefittedVertices";
   // Now clone the re-fitted vertices to the microDST TES
   SeqDC06selBs2Jpsi2MuMu_Phi2KK.Members += {"CopyPrimaryVertices/CopyRefittedPV"};
   CopyRefittedPV.InputLocation = "Phys/DC06selBs2Jpsi2MuMu_Phi2KK/RefittedVertices";
   // and clone the Particle->Re-fitted vertex relations
   SeqDC06selBs2Jpsi2MuMu_Phi2KK.Members += {"CopyParticle2PVLink/P2RefitPVLink"};
   P2RefitPVLink.InputLocation = "Phys/DC06selBs2Jpsi2MuMu_Phi2KK/Particle2ReFittedVertexRelations";
 
Deleted:
<
<
 

Generating the ROOT POOL catalogue

The ROOT/POOL catalogue stores information about which file contains (further) information about the objects used in the input file. The first line is used for read/write access, i.e. to create or update the information about the microDST,

Revision 182008-04-04 - UlrichKerzel

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Line: 200 to 200
  -- JuanPalacios - 30 Sep 2007
Added:
>
>

 
META FILEATTACHMENT attachment="InitialisePyGaudi.py.txt" attr="" comment="Initialise Gaudi - Python session" date="1187090879" name="InitialisePyGaudi.py.txt" path="InitialisePyGaudi.py" size="1848" stream="InitialisePyGaudi.py" user="Main.ukerzel" version="1"
Added:
>
>
META FILEATTACHMENT attachment="InitGaudi.py.txt" attr="" comment="Initialise Gaudi" date="1207298383" name="InitGaudi.py.txt" path="InitGaudi.py.txt" size="3514" stream="InitGaudi.py.txt" user="Main.UlrichKerzel" version="1"
META FILEATTACHMENT attachment="AnalysisMicroDST.py.txt" attr="" comment="Simple analysis script" date="1207298418" name="AnalysisMicroDST.py.txt" path="AnalysisMicroDST.py.txt" size="2116" stream="AnalysisMicroDST.py.txt" user="Main.UlrichKerzel" version="1"

Revision 172007-11-13 - UlrichKerzel

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Line: 60 to 60
  "/Event/microDST/Rec/ProtoP/Upstream"}; N.B. You might have to add some further RootOnTES instructions depending on your algorithms and sequences.
Added:
>
>
N.B. In recent Gaudi / DaVinci versions, the option RootInTES was introduced which replaces RootOnTES N.B. You might need to try some variations of the input locations, e.g. in DaVinci v19r5 I found that I had to use RootInTES and then "Phys/MyAlg" as input location for the PhysDesktop (instead of the full path)
 N.B. it may be beneficial to run
Yourq.Members                                 += {"MuonPIDsFromProtoParticlesAlg/CreateMuonPIDs"};

Revision 162007-09-30 - JuanPalacios

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Line: 87 to 87
 further information (say, track hits) are then obtained via the SmartRefs.

Information stored on the microDST

Changed:
<
<
The microDST is intended to store all information about your selected (signal) candidates. For example, if a Bs -> J/psi (-> mu mu) Phi (-> KK) was selected in your algorithm, the microDST would contain the Bs candidate, the J/psi with the muons, the Phi with the kaons as well as the underlying proto-particles and tracks for the 'basic' particles (muons and kaons in this case).
>
>
The microDST stores all the information related to the particles given as input to the PhysDesktop of the CreateMicroDSTAlg. Typically, these would be the candidate particles resulting from a selection or pre-selection. For example, if a Bs -> J/psi (-> mu mu) Phi (-> KK) is selected by an algorithm, the microDST would contain the Bs candidate, the J/psi with the muons, the Phi with the kaons, the decay vertices of the Bs, the J/psi and the phi, as well as the underlying proto-particles and tracks for the 'basic' particles (muons and kaons in this case).
 In addition, the ODIN bank and the Event/Rec header are stored as well to have access to run-number, event-time and number, etc.
Added:
>
>
-- JuanPalacios - 30 Sep 2007
 

Primary Vertices on the microDST

Information about the primary vertices is also stored, e.g. position and covariance matrix, number of tracks etc. However, to save space, the tracks attached to the primary vertex (and further detailed information) is not stored.
Line: 176 to 176
 

Work in progress and known issues

  • storage of neutral proto-particles (and associated MC truth) is not yet working/tested
  • using the composite associator to retrieve information about a composite MC particle, work in progress
Deleted:
<
<
  • the outgoing particles and/or daughters returned by a particle was not always correct, fixed in version 1.20 of Phys/VertexFit/src/OfflineVertexFitter.cpp
 
  • adding flavour tagging information to the microDST
  • adding trigger summary information to the microDST
  • ProcStatus information (once copy-constructor is available)
Line: 196 to 194
 
  • InitialisePyGaudi.py initialises the Gaudi session, loads all setups, etc. (see attached file)
  • Ana.py is then the script which holds the actual analysis code to loop over particles, fill histograms, etc.
Deleted:
<
<
-- Main.ukerzel - 11 Apr 2007
 
Added:
>
>
-- JuanPalacios - 30 Sep 2007
 
META FILEATTACHMENT attachment="InitialisePyGaudi.py.txt" attr="" comment="Initialise Gaudi - Python session" date="1187090879" name="InitialisePyGaudi.py.txt" path="InitialisePyGaudi.py" size="1848" stream="InitialisePyGaudi.py" user="Main.ukerzel" version="1"

Revision 152007-08-14 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Line: 182 to 182
 
  • adding trigger summary information to the microDST
  • ProcStatus information (once copy-constructor is available)
Added:
>
>

Using Python to work with the microDST

Python hints

IPython

IPython is a powerfull Python environment with many features, short-cuts, etc. which is also used by Ganga. To use it with the "right" Python environment, setup e.g. DaVinci or Panoramix and call python /lhcb/Ganga/external/ipython/0.6.13/slc3_gcc323/bin/ipython (or accordingly, where Ganga points to)

Working with the microDST and Python

I found it convenient to have two Python scripts when working with the microDST:
  • InitialisePyGaudi.py initialises the Gaudi session, loads all setups, etc. (see attached file)
  • Ana.py is then the script which holds the actual analysis code to loop over particles, fill histograms, etc.
 -- Main.ukerzel - 11 Apr 2007
Added:
>
>

META FILEATTACHMENT attachment="InitialisePyGaudi.py.txt" attr="" comment="Initialise Gaudi - Python session" date="1187090879" name="InitialisePyGaudi.py.txt" path="InitialisePyGaudi.py" size="1848" stream="InitialisePyGaudi.py" user="Main.ukerzel" version="1"

Revision 142007-05-16 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Line: 13 to 13
 Two algorithms are used to create the microDST:
  • CreateMicroDSTAlg handles reconstructed particles and the associated vertices, the reconstructed primary vertex, etc. This algorithm is intended also to be used on data.
  • CreateMicroDSTMCAlg handles MC (truth) particles and vertices, also establishes links between the reconstructed particles and the associated MC particles.
Added:
>
>
A further algorithm provided by the package (TestDST) can be used to investigate the content of the created microDST and allows to print the content of the various containers, etc. stored on the microDST.
 

Create a microDST

Line: 69 to 71
 
YourAlg.PhysDesktop.InputLocations                  = {"/Event/microDST/Path/to/Your/orig/Alg"};
Deleted:
<
<
N.B. Unlike the case when creating the microDST, only the last (highest-level) algorithm should be specified here, e.g. in the case of Bs -> J/psi Phi, use
YourAlg.PhysDesktop.InputLocations                  = {"/Event/microDST/Phys/DC06selBs2Jpsi2MuMu_Phi2KK"};
and not the ones for J/psi -> mu mu or Phi -> KK as well.
  If needed/wanted, you can then also add the ROOT/POOL catalogue to the options file, e.g.
Line: 164 to 161
 
  • tracks attached to primary vertex
  • raw data-banks
  • other particles than the selected (signal) candidates and the attached daughters, etc.
Added:
>
>
  • detailed MC truth information (e.g. Bremsstrahlungs photons and corresponding vertices, etc. of an electron, ...
 

Printing the content of the Transient Event Store (TES)

Including the following

Revision 132007-05-11 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Line: 150 to 150
  }// if table
Changed:
<
<
N.B. work in progress
>
>
N.B. This extension has been committed to Phys/DaVinciAssociators/.../src/CompositeParticle2MCLinks.cpp and should be available in the default installation soon
  The MC header is stored to the microDST as well, preserving the SmartRefs to the original (true) primary vertices.

Revision 122007-04-19 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Line: 152 to 152
  N.B. work in progress
Added:
>
>
The MC header is stored to the microDST as well, preserving the SmartRefs to the original (true) primary vertices.
 

Information not stored on the microDST

In order to avoid loss of information, SmartRefs are set in case a given quantity is not stored on the microDST. Thus, if one has access to the full DST (and the corresponding XML catalogue), all quantities can be accessed via the SmartRefs from the microDST. This has the advantage that one can always operate on the
Line: 178 to 180
  fixed in version 1.20 of Phys/VertexFit/src/OfflineVertexFitter.cpp
  • adding flavour tagging information to the microDST
  • adding trigger summary information to the microDST
Added:
>
>
  • ProcStatus information (once copy-constructor is available)
  -- Main.ukerzel - 11 Apr 2007 \ No newline at end of file

Revision 112007-04-19 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Line: 93 to 93
 The microDST is intended to store all information about your selected (signal) candidates. For example, if a Bs -> J/psi (-> mu mu) Phi (-> KK) was selected in your algorithm, the microDST would contain the Bs candidate, the J/psi with the muons, the Phi with the kaons as well as the underlying proto-particles and tracks for the 'basic' particles (muons and kaons in this case).
Changed:
<
<
In addition, the ODIN bank is stored as well to have access to run-number, event-time and number, etc.
>
>
In addition, the ODIN bank and the Event/Rec header are stored as well to have access to run-number, event-time and number, etc.
 

Primary Vertices on the microDST

Information about the primary vertices is also stored, e.g. position and covariance matrix, number of tracks etc.

Revision 102007-04-18 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Line: 176 to 176
 
  • using the composite associator to retrieve information about a composite MC particle, work in progress
  • the outgoing particles and/or daughters returned by a particle was not always correct, fixed in version 1.20 of Phys/VertexFit/src/OfflineVertexFitter.cpp
Added:
>
>
  • adding flavour tagging information to the microDST
  • adding trigger summary information to the microDST
  -- Main.ukerzel - 11 Apr 2007 \ No newline at end of file

Revision 92007-04-17 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Line: 27 to 27
 
      SeqDC06selBs2Jpsi2MuMu_Phi2KK.Members   += {"CreateMicroDSTAlg/microDST"};
      SeqDC06selBs2Jpsi2MuMu_Phi2KK.Members   += {"CreateMicroDSTMCAlg/microDSTMC"};
Changed:
<
<
microDST.PhysDesktop.InputLocations = {"/Event/Phys/DC06selBs2Jpsi2MuMu_Phi2KK", "/Event/Phys/Jpsi2MuMuForDC06selBs2Jpsi2MuMu_Phi2KK", "/Event/Phys/Phi2KKForDC06selBs2Jpsi2MuMu_Phi2KK" };

microDSTMC.PhysDesktop.InputLocations = {"/Event/Phys/DC06selBs2Jpsi2MuMu_Phi2KK", "/Event/Phys/Jpsi2MuMuForDC06selBs2Jpsi2MuMu_Phi2KK", "/Event/Phys/Phi2KKForDC06selBs2Jpsi2MuMu_Phi2KK"}; N.B Due to the way the PhysicsDesktop works, all locations for the particles used need to be specified. E.g.: If you operate on the B candidate in YourAlg, but use as input e.g. J/psi candidates created by MakeResonances earlier in the chain, you need to specifiy these locations as well, as they 'own' the J/psi on the physics desktop.

>
>
microDST.PhysDesktop.InputLocations = {"/Event/Phys/DC06selBs2Jpsi2MuMu_Phi2KK"}; microDSTMC.PhysDesktop.InputLocations = {"/Event/Phys/DC06selBs2Jpsi2MuMu_Phi2KK"};
 

Generating the ROOT POOL catalogue

The ROOT/POOL catalogue stores information about which file contains (further) information about the objects used
Line: 59 to 50
 To use the microDST for analysis, root directory on the TES needs to point to this branch which can be done via the parameter RootOnTES. Add the following to your options file
DaVinciMainSeq.RootOnTES               = "/Event/microDST/";
Deleted:
<
<
RootOnTES = "/Event/microDST/";
 RootOnTES = "/Event/microDST/"; TestSeq.Members += {"YourAlg"}; InputPrimaryVertices = "microDST/Rec/Vertex/Primary"; OfflinePVLocation = "/Event/microDST/Rec/Vertex/Primary";
Deleted:
<
<
OfflinePVLocation = "/Event/microDST/Rec/Vertex/Primary";
 ChargedPPLocation = {"/Event/microDST/Rec/ProtoP/Charged", "/Event/microDST/Rec/ProtoP/Upstream"};
Deleted:
<
<
RootOnTES = "microDST/";
  N.B. You might have to add some further RootOnTES instructions depending on your algorithms and sequences.
Added:
>
>
N.B. it may be beneficial to run
Yourq.Members                                 += {"MuonPIDsFromProtoParticlesAlg/CreateMuonPIDs"};
YouSeq.Members                                 += {"RichPIDsFromProtoParticlesAlg/CreateRichPIDs"};
not via the Data-on-Demand service but at the beginning of the event.
  To specify the input location for the PhysDesktop, use
Line: 179 to 173
 

Work in progress and known issues

  • storage of neutral proto-particles (and associated MC truth) is not yet working/tested
Added:
>
>
  • using the composite associator to retrieve information about a composite MC particle, work in progress
 
  • the outgoing particles and/or daughters returned by a particle was not always correct, fixed in version 1.20 of Phys/VertexFit/src/OfflineVertexFitter.cpp

Revision 82007-04-16 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Line: 179 to 179
 

Work in progress and known issues

  • storage of neutral proto-particles (and associated MC truth) is not yet working/tested
Changed:
<
<
>
>
  • the outgoing particles and/or daughters returned by a particle was not always correct, fixed in version 1.20 of Phys/VertexFit/src/OfflineVertexFitter.cpp
  -- Main.ukerzel - 11 Apr 2007 \ No newline at end of file

Revision 72007-04-16 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Line: 15 to 15
 
  • CreateMicroDSTMCAlg handles MC (truth) particles and vertices, also establishes links between the reconstructed particles and the associated MC particles.

Create a microDST

Added:
>
>

Getting the code

The microDST code is available from the central LHCb software repository (CVS) and can be obtained via getpack MicroDST/MicroDSTAlgorithm. During the current development and test phase, the head-revision should be used.
 

Options

To create a microDST, the following has to be added to your options file:
  • #include "$MICRODSTALGORITHMROOT/options/microDST.opt"

Revision 62007-04-16 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Line: 14 to 14
 
  • =CreateMicroDSTAlg=handles reconstructed particles and the associated vertices, the reconstructed primary vertex, etc. This algorithm is intended also to be used on data.
  • CreateMicroDSTMCAlg handles MC (truth) particles and vertices, also establishes links between the reconstructed particles and the associated MC particles.
Changed:
<
<

Options to create a microDST

>
>

Create a microDST

Options

 To create a microDST, the following has to be added to your options file:
Changed:
<
<
  • =#include "$MICRODSTALGORITHMROOT/options/microDST.opt"*
>
>
  • #include "$MICRODSTALGORITHMROOT/options/microDST.opt"
 
  • Tell the microDST algorithm which particles are to be stored, e.g. for Bs -> J/psi Phi:
          SeqDC06selBs2Jpsi2MuMu_Phi2KK.Members   += {"CreateMicroDSTAlg/microDST"};
Line: 34 to 35
  MakeResonances earlier in the chain, you need to specifiy these locations as well, as they 'own' the J/psi on the physics desktop.
Changed:
<
<
This then creates the microDST with the filename 'microDST.dst' in the current working directory.
>
>

Generating the ROOT POOL catalogue

The ROOT/POOL catalogue stores information about which file contains (further) information about the objects used in the input file. The first line is used for read/write access, i.e. to create or update the information about the microDST, futher lines are used for reading only and contain information about other files used.
PoolDbCacheSvc.Catalog = {
  "xmlcatalog_file:DSTCatalog.xml",
  "xmlcatalog_file:someOtherInputCatalogue"
};

This then creates the microDST with the filename microDST.dst in the current working directory.

 
Deleted:
<
<
CATALOGUE
 

Using the microDST

Added:
>
>
When creating the microDST, the algorithms replicate the structure of the transient event store (TES) in a new branch and store the information about the relevant particles, etc. there. By default, this new branch has the prefix microDST. To use the microDST for analysis, root directory on the TES needs to point to this branch which can be done via the parameter RootOnTES. Add the following to your options file
DaVinciMainSeq.RootOnTES               = "/Event/microDST/";
ProtoPRecalibration.RootOnTES          = "/Event/microDST/";
YourSeq.RootOnTES                             = "/Event/microDST/";
TestSeq.Members                                += {"YourAlg"};
YourAlg.PhysDesktop.InputPrimaryVertices         = "microDST/Rec/Vertex/Primary";
YourAlg.PhysDesktop.OnOfflineTool.OfflinePVLocation          = "/Event/microDST/Rec/Vertex/Primary";
YourAlg.RelatedPVFinder.ContextTool.OnOfflineTool.OfflinePVLocation = "/Event/microDST/Rec/Vertex/Primary";
YourAlg.Particle2MCLinks.ChargedPPLocation              = {"/Event/microDST/Rec/ProtoP/Charged",
                                                                                         "/Event/microDST/Rec/ProtoP/Upstream"};
YourAlg.CompositeParticle2MCLinks.RootOnTES          = "microDST/";
N.B. You might have to add some further RootOnTES instructions depending on your algorithms and sequences.

To specify the input location for the PhysDesktop, use

YourAlg.PhysDesktop.InputLocations                  = {"/Event/microDST/Path/to/Your/orig/Alg"};
N.B. Unlike the case when creating the microDST, only the last (highest-level) algorithm should be specified here, e.g. in the case of Bs -> J/psi Phi, use
YourAlg.PhysDesktop.InputLocations                  = {"/Event/microDST/Phys/DC06selBs2Jpsi2MuMu_Phi2KK"};
and not the ones for J/psi -> mu mu or Phi -> KK as well.

If needed/wanted, you can then also add the ROOT/POOL catalogue to the options file, e.g.

PoolDbCacheSvc.Catalog = {
  "xmlcatalog_file:DSTCatalog.xml",
  "xmlcatalog_file:CatalogueOfInputFilesToMicroDST"
};
The first line again holds the information about the objects used in the microDST, the further line(s) contain the catalogue(s) for the input files from which the microDST was created. If you have access to these files, you can follow the SmartRefs on the microDST to access the information stored on the full DST (even if it is not available on the microDST). A possible use-case for this could be that you want to update the file or some plot with further information or that you have the full DST available but instead of reading in the complete event, only the relevant information (e.g. the B candidate) is read from the microDST and further information (say, track hits) are then obtained via the SmartRefs.
 

Information stored on the microDST

The microDST is intended to store all information about your selected (signal) candidates. For example, if a Bs -> J/psi (-> mu mu) Phi (-> KK) was selected in your algorithm,
Line: 121 to 172
  in your option files prints the current structure of the TES each event
Added:
>
>

Work in progress and known issues

  • storage of neutral proto-particles (and associated MC truth) is not yet working/tested
 -- Main.ukerzel - 11 Apr 2007

Revision 52007-04-13 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Line: 36 to 36
  This then creates the microDST with the filename 'microDST.dst' in the current working directory.
Added:
>
>
CATALOGUE
 

Using the microDST

Information stored on the microDST

Line: 110 to 111
 
  • raw data-banks
  • other particles than the selected (signal) candidates and the attached daughters, etc.
Added:
>
>

Printing the content of the Transient Event Store (TES)

Including the following
ApplicationMgr.TopAlg                += { "StoreExplorerAlg" };
StoreExplorerAlg.Load                  = 1;
StoreExplorerAlg.PrintFreq            = 1.0;
StoreExplorerAlg.Outputlevel         = 1;
in your option files prints the current structure of the TES each event
  -- Main.ukerzel - 11 Apr 2007

Revision 42007-04-13 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Line: 50 to 50
 This makes it possible to use information about the primary vertex on the microDST, however, refitting the primary vertex is not possible (as the tracks are not available). If this is needed or wanted, this should be done when producing the microDST.
Added:
>
>

Storing user-created primary vertices on the microDST

 The microDST supports primary vertices created by the user. This can be useful when measuring the lifetime of a given particle: To avoid a potential bias, the primary vertex may be re-fitted excluding the tracks which are used to build the signal candidate. If the user-PV is stored at the location: yourAlg/PrimaryVertices (e.g. /Event/Phys/DC06selBs2Jpsi2MuMu_Phi2KK/PrimaryVertices), they will be automatically stored on the microDST as well.
Added:
>
>

Storing user-data associated with a given particle on the microDST

The LHCb::Particle class allows to store user-data (of type double) via the extraInfo field. To add your data (e.g. measured lifetime,etc.), add the value to the particle via:
BCand->addInfo(LHCb::Particle::LastGlobal + 0, value);
where LHCb::Particle::LastGlobal is the last number reseved for official LHCb use.
 

MC (truth) information

If the CreateMicroDSTMCAlg algorithm is executed as well, MC (truth) information is also stored on the microDST.

Revision 32007-04-13 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Line: 11 to 11
 The amount of space saved depends of course on the given analysis, i.e. it is beneficial to have a dedicated microDST per given analysis or per group of (closely) related analyses. Two algorithms are used to create the microDST:
Changed:
<
<
  • CreateMicroDSTAlg: handles reconstructed particles and the associated vertices, the reconstructed primary vertex, etc. This algorithm is intended also to be used on data.
  • CreateMicroDSTMCAlg: handles MC (truth) particles and vertices, also establishes links between the reconstructed particles and the associated MC particles.
>
>
  • =CreateMicroDSTAlg=handles reconstructed particles and the associated vertices, the reconstructed primary vertex, etc. This algorithm is intended also to be used on data.
  • CreateMicroDSTMCAlg handles MC (truth) particles and vertices, also establishes links between the reconstructed particles and the associated MC particles.
 
Changed:
<
<

Information stored on the microDST

>
>

Options to create a microDST

To create a microDST, the following has to be added to your options file:
  • =#include "$MICRODSTALGORITHMROOT/options/microDST.opt"*
  • Tell the microDST algorithm which particles are to be stored, e.g. for Bs -> J/psi Phi:
          SeqDC06selBs2Jpsi2MuMu_Phi2KK.Members   += {"CreateMicroDSTAlg/microDST"};
          SeqDC06selBs2Jpsi2MuMu_Phi2KK.Members   += {"CreateMicroDSTMCAlg/microDSTMC"};
          microDST.PhysDesktop.InputLocations      = {"/Event/Phys/DC06selBs2Jpsi2MuMu_Phi2KK",
                                                                                  "/Event/Phys/Jpsi2MuMuForDC06selBs2Jpsi2MuMu_Phi2KK",
                                                                                  "/Event/Phys/Phi2KKForDC06selBs2Jpsi2MuMu_Phi2KK" };
    
         microDSTMC.PhysDesktop.InputLocations    = {"/Event/Phys/DC06selBs2Jpsi2MuMu_Phi2KK",
                                                                                    "/Event/Phys/Jpsi2MuMuForDC06selBs2Jpsi2MuMu_Phi2KK",
                                                                                   "/Event/Phys/Phi2KKForDC06selBs2Jpsi2MuMu_Phi2KK"};
         
    N.B Due to the way the PhysicsDesktop works, all locations for the particles used need to be specified. E.g.: If you operate on the B candidate in YourAlg, but use as input e.g. J/psi candidates created by MakeResonances earlier in the chain, you need to specifiy these locations as well, as they 'own' the J/psi on the physics desktop.

This then creates the microDST with the filename 'microDST.dst' in the current working directory.

 
Changed:
<
<

Information not stored on the microDST

>
>

Using the microDST

 
Added:
>
>

Information stored on the microDST

The microDST is intended to store all information about your selected (signal) candidates. For example, if a Bs -> J/psi (-> mu mu) Phi (-> KK) was selected in your algorithm, the microDST would contain the Bs candidate, the J/psi with the muons, the Phi with the kaons as well as the underlying proto-particles and tracks for the 'basic' particles (muons and kaons in this case). In addition, the ODIN bank is stored as well to have access to run-number, event-time and number, etc.

Primary Vertices on the microDST

Information about the primary vertices is also stored, e.g. position and covariance matrix, number of tracks etc. However, to save space, the tracks attached to the primary vertex (and further detailed information) is not stored. This makes it possible to use information about the primary vertex on the microDST, however, refitting the primary vertex is not possible (as the tracks are not available). If this is needed or wanted, this should be done when producing the microDST.

The microDST supports primary vertices created by the user. This can be useful when measuring the lifetime of a given particle: To avoid a potential bias, the primary vertex may be re-fitted excluding the tracks which are used to build the signal candidate. If the user-PV is stored at the location: yourAlg/PrimaryVertices (e.g. /Event/Phys/DC06selBs2Jpsi2MuMu_Phi2KK/PrimaryVertices), they will be automatically stored on the microDST as well.

MC (truth) information

If the CreateMicroDSTMCAlg algorithm is executed as well, MC (truth) information is also stored on the microDST.

* treatment of 'basic' particles (without daughthers, e.g. muons) The Particle2MCLinker(this,Particle2MCMethod::Links,"") associator is used to find the MC (truth) particle corresponding to the reconstructed (basic) particle (e.g. the muons and kaons in the example of Bs -> J/psi (-> mu mu) Phi (-> KK) ). The link between the underlying (reconstructed) object and the corresponding MC (truth) particle is stored on the microDST such that the above associator should work if running on the microDST.

* treatment of 'composite' particles (e.g. a B candidate) Composite particles are found via the Particle2MCLinker(this,Particle2MCMethod::Composite,"") associator. However, due to the way the composite associator works, this association cannot be stored on the microDST in a way that the default composite associator will be able to establish this association when operating on the microDST. Instead, a direct link between the composite reconstructed particle and the corresponding MC particle is saved on the microDST. In order to retrieve this link, execute the following (see TestDST.cpp and TestDST.h). It might be possible to extend the default composite associator to handle this case, this is currently under investigation.

       std::string locTES = TestDST::objectLocation(particle->parent() );
       verbose() << "try to get direct link for particle on TES " << locTES << endmsg;
       Part2MCPart p2pAssociator(eventSvc(), locTES);
       const Part2MCPartTable *p2pTable = p2pAssociator.direct();
       if (p2pTable) {
       Part2MCPartRange p2pRange =  p2pTable->relations(particle);
       verbose() << "found #relations " << p2pRange.size() << endmsg;
      for (Part2MCPartIterator iRel = p2pRange.begin(); iRel != p2pRange.end(); iRel++) {
         const LHCb::MCParticle *mcPart = iRel->to();
         double                  weight = iRel->weight();
         verbose() << "particle related to MCPart with pid " << mcPart->particleID().pid() 
                        << " weight " << weight 
                      << " on TES " <<  TestDST::objectLocation(mcPart->parent())
                    << endmsg;     
        }//for iRel
       } else {
      verbose() << "table for direct association not found " << endmsg;
     }// if table
       

N.B. work in progress

Information not stored on the microDST

In order to avoid loss of information, SmartRefs are set in case a given quantity is not stored on the microDST. Thus, if one has access to the full DST (and the corresponding XML catalogue), all quantities can be accessed via the SmartRefs from the microDST. This has the advantage that one can always operate on the microDST (and hence loop over only the candidates of interest) even if further information is required. Information not stored:
  • tracks attached to primary vertex
  • raw data-banks
  • other particles than the selected (signal) candidates and the attached daughters, etc.
 

-- Main.ukerzel - 11 Apr 2007

Revision 22007-04-11 - unknown

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Line: 6 to 6
 

Overview

Added:
>
>
The idea of the microDST is to store all relevant information about the particles of interest by creating a new file (the microDST) in the standard LHCb DST format. Storing only the candidates of interest significantly reduces the size of the DST file used by concentrating on the information relevant to the given analysis. The amount of space saved depends of course on the given analysis, i.e. it is beneficial to have a dedicated microDST per given analysis or per group of (closely) related analyses. Two algorithms are used to create the microDST:
  • CreateMicroDSTAlg: handles reconstructed particles and the associated vertices, the reconstructed primary vertex, etc. This algorithm is intended also to be used on data.
  • CreateMicroDSTMCAlg: handles MC (truth) particles and vertices, also establishes links between the reconstructed particles and the associated MC particles.
 
Added:
>
>

Information stored on the microDST

 
Added:
>
>

Information not stored on the microDST

 
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback