DDMarlinPandora : The bridge between DD4hep and PandoraPFA

DDMarlinPandora is a package that facilitates the translation of the detector geometry provided by DD4hep, as well as the LCIO objects in the event (tracks, calorimeter hits, ...) to a format compatible with the PandoraPFA Particle Flow Reconstruction framework. It is a direct copy of the previous MarlinPandora package which was based on GEAR.

Introduction for DDMarlinPandora developers

Please note that often when you have a local checkout of DDMarlinPandora built against an ILCSOFT installation and a local DD4hep installation, this will result on a segmentation fault when running Marlin with DDMarlinPandora. This situation occurs because DD4hep dependencies creep into your installation through other packages, namely MarlinDD4hep, which also needs to be built locally, if DD4hep changes. The best solution would be to always build DDMarlinPandora against a recent complete HEAD release of ILCSOFT. See for example here.

Another thing to have in mind is that DDMarlinPandora is implemented in Marlin as a Marlin Processor. It is therefore necessary to be published to Marlin via the MARLIN_DLL environment variable. Usually, it's just needed to add the path to your library's .so file to the variable. However, Marlin does not allow multiple versions of the same library (or at least same processor names). It is therefore necessary to remove the old reference to libDDMarlinPandora.so that comes with the ILCSOFT installation. The following example script does exactly that, for any locally installed package. See inline comments for more information:

Show script... Hide script

#!/bin/bash
#store PWD so we can get back
realOldPwd=$OLDPWD
myPwd=$PWD

#either source base ILCSOFT installation before executing or uncomment following line
#source /afs/cern.ch/eng/clic/work/ilcsoft/HEAD-2016-02-10/init_ilcsoft.sh

#Area where you check out your packages for local modification
LOCALILCSOFTAREA=/afs/cern.ch/user/n/nikiforo/DDRecTest

#Create a map with all the locally installed packages
#For now, just DDMArlinPandora
declare -A local_packages
#syntax:
#local_packages["PACKAGE_NAME"]="PACKAGE_DIRECTORY_NAME"
local_packages["DDMarlinPandora"]="${LOCALILCSOFTAREA}/DDMarlinPandora-trunk"

#Set MARLIN_DLL, LD_LIBRARY_PATH and PATH if needed, for each package in the map
for package in ${!local_packages[@]}; do 

        #Fancy grep and sed manipulation just to get rid of package library installed with ILCSOFT
        export MARLIN_DLL=`echo $MARLIN_DLL|tr ":" "\n"|grep -v "${package}"|tr '\n' ':'|sed 's/::\+$//'`
        export LD_LIBRARY_PATH=`echo $LD_LIBRARY_PATH|tr ":" "\n"|grep -v "${package}"|tr '\n' ':'|sed 's/::\+$//'`
        export ${package}="${local_packages[${package}]}"
        export LD_LIBRARY_PATH="${local_packages[${package}]}/lib:$LD_LIBRARY_PATH"
        export MARLIN_DLL="${local_packages[${package}]}/lib/lib${package}.so:"$MARLIN_DLL
        
        #set path if needed
        
        if [ -d "${local_packages[${package}]}/bin" ]; 
        then
            export PATH="${local_packages[${package}]}/bin":$PATH
        fi
done

cd $myPwd
export OLDPWD=${realOldPwd}

#need to use this when ssh-ing from ubuntu machine
unset LIBGL_ALWAYS_INDIRECT

DDMarlinPandora description

Limitations, Open Issues, and Things to Fix

Here is a list of things that you should be aware of or try to fix:

Remove processor parameters that access subdetectors by name

Processor parameters like ECalBarrelDetectorName, ECalEndcapDetectorName, ECalOtherDetectorNames and simirarly for the HCal, should be removed and instead the detector elements should be accessed by detector type flags. These flags are a new addition to DD4hep and the detector drivers and allow you to "brand" each detector with a combination of flags like TRACKER, CALORIMETER, BARREL, ELECTROMAGNETIC, etc. Conversely, you can use combinations of these flags to access all the detector elements that match the requirements. Once this logic is there, the string parameters are not necessary and should be removed. Once the processor parameters are not needed, they should be removed from DDPandoraPFANewProcessor. In addition, there should be no more need to keep this information in Settings objects for each of the classes in DDMarlinPandora.

• Calorimeters: Can almost directly be replaced, except the need for additional flags for the ECal plug, HCal ring, etc
• Vertex/Trackers: Already removed use of string constants in DDTrackCreatorCLIC. Need to do same for DDTrackCreatorILD and then remove TrackerBarrelDetectorNames, etc and associated Settings object variables.

Also fix accessing subdetectors by name for digitizers

The two simple digitizers shipped with DDMarlinPandora still access subdetectors by name. This should be changed.

Proper handling of ECal Plug, HCal ring etc...

Pandora allows for a handful of "default" subdetectors:

pandora::ECAL_BARREL
pandora::ECAL_ENDCAP
pandora::HCAL_BARREL
pandora::HCAL_ENDCAP
pandora::MUON_BARREL
pandora::MUON_ENDCAP

/afs/cern.ch/eng/clic/work/ilcsoft/HEAD-2016-02-10/PandoraPFANew/HEAD $ grep -rI "ECAL_ENDCAP" *
LCContent-master/src/LCParticleId/MuonReconstructionAlgorithm.cc:    const float eCalEndCapInnerR(pGeometryManager->GetSubDetector(ECAL_ENDCAP).GetInnerRCoordinate());
LCContent-master/src/LCPlugins/LCPseudoLayerPlugin.cc:    this->StoreLayerPositions(pGeometryManager->GetSubDetector(ECAL_ENDCAP), m_endCapLayerPositions);
LCContent-master/src/LCPlugins/LCPseudoLayerPlugin.cc:    m_endCapEdgeZ = (std::max(std::fabs(pGeometryManager->GetSubDetector(ECAL_ENDCAP).GetOuterZCoordinate()), std::max(
LCContent-master/src/LCPlugins/LCPseudoLayerPlugin.cc:    m_endCapInnerZ = std::fabs(pGeometryManager->GetSubDetector(ECAL_ENDCAP).GetInnerZCoordinate());
LCContent-master/src/LCPlugins/LCPseudoLayerPlugin.cc:    const float endCapOuterR = pGeometryManager->GetSubDetector(ECAL_ENDCAP).GetOuterRCoordinate();
PandoraMonitoring-master/src/PandoraMonitoring.cc:    try {pTEveTrackPropagator->SetMaxZ(std::fabs(m_pPandora->GetGeometry()->GetSubDetector(ECAL_ENDCAP).GetOuterZCoordinate()) * m_scalingFactor);}

• It should be checked what the impact of this is in the algorithms.
• We should also consider merging the two component (Endcap+Plug) geometries into one pandora::ECAL_ENDCAP pandora geometry object in DDGeometryCreator , though this maybe hard without loss of generality.
  • We could try and see whether DD4hep can provide this merging (via detector assemblies?)
• * One other option would be to modify the algorithms above (and any others than need access to special subdetectors) to access the subdetector by name*. We can agree on a convention in DDMarlinPandora (e.g. "HCalRing" and "ECalPlug") and then use this convention in the PandoraAPI algorithms (in LCContent).

Consolidation of helper functions

The various classes in DDMarlinPandora use several static helper/auxiliary accessor functions defined in the main processor class (DDPandoraPFANewProcessor.cc) which are then inelegantly defined as forward declarations in each of the class .cc files. For example:

//forward declarations. See in DDPandoraPFANewProcessor.cc
double getFieldFromLCDD(); 
DD4hep::DDRec::LayeredCalorimeterData * getExtension(unsigned int includeFlag, unsigned int excludeFlag=0);

std::vector<double> getTrackingRegionExtent();

Check if the distances and other variables are meaningful as input to Pandora

• Coil dimensions: For example, the outer radius is obtained by a DD4hep::DDRec::LayeredCalorimeterData structure attached to the Solenoid_o1_v01 driver ( SCoil02 for ILD). The structure is used for easy implementation (even though it is an overkill). In the CLIC case, the Solenoid even has layers which are built inside the envelope of the solenoid. The main part is further complemented by endcaps implemented as DiskTracker (?). The solenoid is put together as an assembly. However, the dimensions are attached to and accessed from the barrel.

Proper digitization/handling of Yoke hits

Right now, DDSimpleMuonDigi is used, a copy from the ILD digitizer. However, ILD are using scintillators for the muon system I believe, therefore the response is completely different. Probably we will need to treat RPC hits as digital hits and handle them with a standalone algorithm as was done for SiD/CLIC_SiD. We need to make sure DDMarlinPandora will be able to accommodate allow this scenario.

Synchronization with MarlinPandora

MarlinPandora is being maintained by Cambridge (hosted on github) independently of DDMarlinPandora (hosted on DESY svn). Unfortunately there is no easy way to merge changes to MalrinPandora into DDMarlinPandora, not only since the latter is not hosted on github, but also since the initial fork (around July 2015), the packages have diverged: All the class and file names have "DD" prepended, references to "TPC" and GEAR have been removed, etc. The last git commit to Marlinpadnora before the fork should be on June 9th, 2015 (SHA 0f744b96d6e9872598788ef59c535953e03e57c5). The latest synchronization was performed on February 17, 2016 (MarlinPandora git SHA 7c88d10c636f23c107f94238531bc7130df5a9df) mainly introducing changes to DDPfoCreator and to DDTrackCreatorBase.

Retraining of Photon Reconstruction

Probably this is not so much a (DD)MarlinPandora issue as it is an issue of the difference of our detector model compared to ILD_o1_v05. Given the different implementation of our ECal in DD4hep and also due to different layer composition and layout, the photon reconstruction in Pandora needs to be retrained. This translates to a different photon likelihood xml data file. Bono Xu from Cambridge is the expert in this case. According to his instructions here is the procedure:

• Simulate Z->uds events at 500 GeV (at least 10k, uniformly distributed).
• Run PandoraPFA through the DDMarlinPandora processor, using PandoraSettingsPhotonTraining.xml for the pandora configuration settings file.
• Pandora will produce a new file, by default named as PandoraLikelihoodData9EBin.xml.
• You will need to run this procedure for every detector model you are using and obtain this file to be used in PandoraSettingsDefault.xml while running later your physics event reconstruction.

-- NikiforosNikiforou - 2016-01-25