This page documents Transient-Persistent (TP) separation in the trigger Event Data Model (EDM). The page contains guidelines for developers needing to introduce new classes or evolve old classes. Since much of the trigger EDM in in the TrigEvent packages, this twiki uses TrigEvent examples.

What is TP Separation?

Transient-Persistent (TP) separation means that the classes stored on disk (ie. the persistent classes xxx_p1 etc.) are different from the classes the user interacts with (the transient classes). When someone asks to read an object from POOL, StoreGate uses the "persistent to transient" converter to read the instance of persistent class from disk and deliver an instance of the transient class to the user. When someone asks to write an object to POOL, StoreGate uses the "transient to persistent" converter to create a persistent instance of the class and write it to disk. The primary functionality of TP separation is to enable backwards compatibility, so that data written with ATHENA release N can be read with ATHENA release N+1.

A complete description of the TP separation framework can be found on Marcin Nowak's TP separation description: TransientPersistentSeparation.

Introducing a New TrigEvent Class

If you would like to persistify (ie. write to ESD/AOD/DPD) a new class into the TrigEvent EDM (any class in Trigger/TrigEvent) you will need to include suitable persistent classes and converters in the Trigger/TrigEvent/TrigEventTPCnv and Trigger/TrigEvent/TrigEventAthenaPool packages. The TrigEventAthenPool classes (AthenaPoolConverters) act as the interface between POOL and your persistent classes and converters - this is the code that decides which persistent class (ie p1, p2, tlp1, etc) should be read from, or written to, disk. The persistent classes and their corresponding converters are found in the TrigEventTPCnv package.

Example: The TrigEventAthenaPool/src/TrigElectronContainerCnv class describes the POOL interface to the TrigElectronContainer converters. The TrigEventTPCnv package contains the TrigElectron_p1 and TrigElectronCnv_p1 classes which describe a persistent class and it's corresponding converter. There are other TrigElectron related classes that are described below.

The following is a loose "step-by-step" set of instructions you can follow to create introduce your new TrigEvent class. It does not explain all details or use cases, but should be used as a starting point for understanding what to do:

  • Write the persistent version of your class in the TrigEventTPCnv package. The name will be YourClassName_p1, the "_p1" reflecting that it is the first persistent version of the class. The persistent class should contain private data members with the information you want to persistify, but no accessor methods (since the persistent class will never be accessed by the user). If the persistent class contains a data member of a non-basic type, then you must decide to either "embed" or "link" (using an ElementLink) the contained class. If you "embed" the object then you will be making a copy of the object inside your class (likely wasting disk space and reducing flexibility), while if you "link" the object to your class then you must be sure to persistify the object linked to so that the link points to! Any base classes (P4EtaPhiM for example) must be embedded as a data member of the persistent class. Note that there should be no pointer data members in the persistent class since this duplicates the information pointed to.
    • Example: TrigElectron_p2 has links to the TrigEMCluster and TrigInDetTrack, and embeds the P4EtaPhiM base class. The p2 here represents the fact that it is actually the second persistent TrigElectron version.

  • Write the converter for your class in the TrigEventTPCnv package, the name should be YourClassNameCnv_p1. The converter will simply inherit from T_AthenaPoolTPCnvBase and contain a persToTrans and transToPers methods which do the conversion from the current transient version of the class to the p1 persistent version.
    • Example: TrigElectronCnv_p2 shows many types of conversions. Note that basic types, embedded classes, base classes, and ElementLinks are all dealt with differently, see TransentPersistentSeparation for more details.

  • Write a persistent container and the corresponding container converter in the TrigEventTPCnv package, YourClassNameContainer_p1 and YourClassNameContainerCnv_p1. Normally, the container only requires a header file, no cxx file, because the implementation comes from the inheritance. Note that every TrigEvent class that can have more than one instance per event must be contained in a container (or collection). This is due to the underlying structure of the trigger EDM and navigation. If your transient class does not have a container (or collection) you will have to add one.
    • Example: See TrigElectronCollection_p2 and TrigElectronCollectionCnv_p2

  • Write the "Top Level" persistent class and converter for your class in the TrigEventTPCnv package, YourClassNameContainer_tlp1 and YourClassNameContainerCnv_tlp1. This "!TLP" class is used to eliminate the chain effect caused when contained classes evolve. For example, if you embedded a P4EtaPhiM_p1 class into your class, you would need to change this if P4EtaPhiM was evolved to P4EtaPhiM_p2. This is obviously not maintainable in the long term. Therefore, the TLP allows you to specify classes by reference, called a TPObjRef, and you do not need to change your class if a base class evolves under you.
    • Example: See TrigElectronCollection_tlp2 and TrigElectronCollectionCnv_tlp2

  • Include your class in the TrigEventTPCnv/TrigEventTPCnvDict.h

  • Include your class in the TrigEventTPCnv/selection.xml. The id number needs to be unique for each class and all letters must be capital. You get a unique id by doing uuidgen | tr 'a-z' 'A-Z' (the tr part of the command converts the id to all capitols).

  • Write the AthenaPoolConverters in the TrigEventAthenaPool package, YourClassNameContainerCnv. These converters are used to 'steer' the user request for an object to the correct persistent version to be read or written. The id values from the TrigEventTPCnv/selection.xml are used in this converter to identify the persistent version present on disk (using the compareClassGuid method)

Instructions to Schema Evolve (change the EDM)

The actual converters and persistent representations (_pN objects) are located in the TrigEventTPCnv package. Every persistified object (as well as every data member and object in the inheritence structure of the persistified object) must have both a converter and a persistent class definition. For example, TrigElectronContainer is a DataVector of TrigElectron objects, TrigElectron objects inherit from P4PtEtaPhiM objects. Therefore we will need the persistent classes: TrigElectronContainer_p1, TrigElectron_p1, and P4PtEtaPhiM_p1 and their converters TrigElectronContainerCnv_p1, TrigElectronCnv_p1, and P4PtEtaPhiMCnv_p1.

Because the structure of TrigElectron_p1 depends on P4PtEtaPhiM_p1, if P4PtEtaPhiM_p1 evolves to P4PtEtaPhiM_p2, then TrigElectron_p1 must also be evolved to TrigElectron_p2. This can create an undesirable chain effect in classes with large or complex data member and inheritence structure. To get around this, we define "Top Level" converters and persistent classes, denoted by _tlpN (these do not replace the pN objects, they are in addition to the pN objects). The top-level converter classes (denoted with *Cnv_tlpN) contain data members of all _pN converters used in the converter class, while the top-level persistent classes (denoted with _tlpN) contain std::vector data members of every _pN class used in the top-level class. For example, this means that TrigElectronContainer has a TrigElectronContainer_tlp1 class (containing std::vectors of TrigElectronContainer_p1, TrigElectron_p1, and P4PtEtaPhiM_p1 types), and a TrigElectronContainerCnv_tlp1 class (containing data members of type TrigElectronContainerCnv_p1, TrigElectronCnv_p1, and P4PtEtaPhiMCnv_p1.

So, as an example, here is what you need to do to schema evolve TrigElectron (note that you are adding the _p2 classes, you are not replacing the _p1 classes):

  • you will need to modify two packages Trigger/TrigEvent/TrigEventAthenaPool and Trigger/TrigEvent/TrigEventTPCnv
  • in TrigEventTPCnv:
    • create a new file TrigEventTPCnv/TrigElectron_p2.h containing the new TrigElectron persistent class (obviously, use TrigElectron_p1 as a template)
    • create the new converter in TrigEventTPCnv/TrigElectronCnv_p2.h and src/TrigElectronCnv_p2.cxx (use _p1 as template)
    • electrons are in "containers", so you need to create TrigEventTPCnv/TrigElectronContainer_p2.h, TrigEventTPCnv/TrigElectronContainerCnv_p2.h (inheritance takes care of the content of the converter, so in this case TrigElectronContainerCnv_p2.cxx is not needed)
    • create the new "top level object" TrigEventTPCnv/TrigElectronContainerCnv_tlp2.h and src/TrigElectronContainerCnv_tlp2.cxx
    • add entries into TrigEventTPCnv/selection.xml for the new classes (to get the a id do uuidgen | tr 'a-z' 'A-Z'), follow examples already in selection.xml
    • add headers and dummy vectors into TrigEventTPCnv/TrigEventTPCnvDict.h

  • in TrigEventAthenaPool:
    • note that all header and source files are in TrigEventAthenaPool/src
    • edit TrigElectronContainerCnv.cxx to contain another if statement for your new guid (the id from the TrigEventTPCnv/selection.xml file)
    • edit the TrigElectronContainerCnv.h to typedef the new _tlp2 object, and create a new m_TPConverter data member (probably with a name something like m_TPConverter_tlp2)

  • now you should have a functioning converter which produces an _tlp2 object when you write an AOD and reads either a _tlp1 or _tlp2 object when you read an AOD depending on which object exists in the AOD.

To create a converter for a new class you will need to add the class to TrigEventAthenaPool/cmt/requirements. For example:

-s=${TrigInDetEvent_root}/TrigInDetEvent TrigInDetTrackCollection.h TrigVertexCollection.h TrigTauTracksInfo.h \ 

If you have trouble, please email me, I will be glad to help, AndrewHamilton

Below this point are notes for myself, developers trying to evolve a TrigEvent class need to read no further!

Memory Leaks in 13.0.40

Large memory leaks seen in black hole events due to a conditional ownership problem. Leaks found in:

TrigEFBphysCnv_p1 In transient class TrigEFBphys.h, the pointer m_secondaryDecay is not deleted - conditional ownership.

TrigL2BphysCnv_p1 In transient class TrigL2Bphys.h, the pointers m_pVertex and m_secondaryDecay are not deleted - conditional ownership.

TrigPhotonCnv_p1 In transient class TrigPhoton.h, the pointer to m_cluster is not deleted - conditional ownership problem...

TrigVertexCnv_p1 In transient class TrigVertex.h, the pointer to m_tracks is deleted, but the tracks pointed to in the list are not deleted - conditional ownership.

The conditional ownership problem arises because during the initial creation of a transient class, say TrigPhoton, the object does not own the object pointed to, TrigEMCluster. But, when the transient objects are created with the T/P converters, the TrigPhoton does own the TrigEMCluster, thus should delete it.

Scott Snyder's fix for the conditional ownership (in TrigPhoton example):

  • add a bool m_ownsCluster to transient class TrigPhoton
  • initialize m_ownsCluster to false in constructors of TrigPhoton (need to be careful in copy constructor)
  • in the destructor of transient class, if(m_ownsCluster) delete m_cluster
  • in the persToTrans of the converter, set transObj->m_ownsCluster to true
  • therefore, TrigPhotons created by the T/P converter will have m_ownsCluster = true, while TrigPhotons created elsewhere will have m_ownsCluster = false

All Except TrigInDetTruthMap

The work started from TrigEventAthenaPool-00-01-08 and TrigEventTPCnv-00-00-00, so to remove all TP separation of TrigEvent one can use those two tags.


  • produce and AOD from RDO
  • dump variables of TrigEvent classes to log file
    • using the AnalysisSkeleton 'framework' and the modified following files: dumpAOD.tgz
    • also add the following to the requirments file:
      • use TrigEventAthenaPool       TrigEventAthenaPool-*           Trigger/!TrigEvent
      • use TrigEventTPCnv            TrigEventTPCnv-*                Trigger/!TrigEvent
      • use TrigSteeringEvent         TrigSteeringEvent-*             Trigger/!TrigEvent
      • use TrigCaloEvent             TrigCaloEvent-*                 Trigger/!TrigEvent

  • check variables are consistent compared to non-TP split AOD
    • the following variables are not initialized, so differences are expected in TP to non-TP split AODs:
      • TrigL2Bphys->isValid
      • TrigMissingET->RoIword
      • pVertex->energyFraction
      • pVertex->dist
      • TrigL2Bphys->dist
    • use egrep -v "persToTrans|create|tlp1|AthenaSealSvc|AthenaPool|TrigMissingET->RoIword|pVertex->energyFraction|pVertex->dist|TrigL2Bphys->dist|pSecondDecay->dist" dumpAll.noTPSep.log > dumpAll.noTPSep.trim to remove these benign log file differences
    • rounding differences due to double -> float conversion are expected in:
      • TrigInDetTrackFitPar->ea0
      • TrigInDetTrackFitPar->ez0
      • TrigInDetTrackFitPar->ephi0
      • TrigInDetTrackFitPar->eeta0
      • TrigInDetTrackFitPar->epT
      • pVertex->x
      • pVertex->y
      • pVertex->z
      • pVertex->chi2
      • pVertex->massVariance

Notes to myself:

  • dump files in: unidisk/trigger/noTPSep/PhysicsAnalysis/AnalysisCommon/UserAnalysis/run

Topic attachments
I AttachmentSorted ascending History Action Size Date Who Comment
Compressed Zip archivetgz dumpAOD.tgz r1 manage 8.0 K 2007-11-16 - 14:50 AndrewHamilton modified files from AnalysisSkeleton to dump the AOD

This topic: Main > TWikiUsers > AndrewHamilton > TriggerEDMCoordination > TrigEventTPSeparation
Topic revision: r34 - 2008-08-12 - AndrewHamilton
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback