C++ Interface Description for Stream Modules

Complete: 4

Contacts

Base Class

Each of the three fundamental types of modules, EDProducer, EDFilter and EDAnalyzer, all have their own base class which reside in the edm::stream namespace. The base classes are named
  • edm::stream::EDProducer<>
  • edm::stream::EDFilter<>
  • edm::stream::EDAnalyzer<>
All the base classes are templatized and take optional template arguments. The optional template arguments are used to extend the interface and therefore abilities of the base classes. Each of the allowed template arguments are described later in this page.

The interface available to all modules inheriting from the base classes (with or without giving additional template arguments) is described below.

void beginStream(edm::StreamID)
Called only once when a Stream is going to begin processing data.
void beginRun(edm::Run const&, edm::EventSetup const&)
Called when the Stream is switching from one Run to a new Run. It is possible for the Source to start a new Run and for some of the Streams to never see that Run if there are less Events in that Run then there are Streams in the job. In the case where there are no LuminosityBlocks in a Run, one and only one Stream will see a beginRun/endRun transition pair.
void beginLuminosityBlock(edm::LuminosityBlock const&, edm::EventSetup const&)
Called when the Stream is switching from one LuminosityBlock to a new LuminosityBlock. It is possible for the Source to start a new LuminosityBlock and for some of the Streams to never see that Run if there are less Events in that LuminosityBlock then there are Streams in the job. In the case where there are no Events in a LuminosityBlock, one and only one Stream will see a beginLuminosityBlock/endLuminosityBlock transition pair.
void produce(edm::Event&, edm::EventSetup const&)
This is called each Event seen by the Stream. This is only available to edm::stream::EDProducer<>
bool filter(edm::Event&, edm::EventSetup const&)
This is called each Event seen by the Stream. The return value decides whether the modules following the filter on its Paths will be processed this Event. This is only available to edm::stream::EDFilter<>
void analyze(edm::Event& const, edm::EventSetup const&)
This is called each Event seen by the Stream. This is only available to edm::stream::EDAnalyzer<>
void endLuminosityBlock(edm::LuminosityBlock const&, edm::EventSetup const&)
Called when the Stream has finished processing all data from a LuminosityBlock and is switching to a new LuminosityBlock.
void endRun(edm::Run const&, edm::EventSetup const&)
Called when the Stream has finished processing all data from a Run and is switching to a new Run.
void endStream()
Called only once when a Stream has finished processing data.
void respondToOpenInputFile(edm::FileBlock const&)
Called when a new file is opened by the Source. A module only gets this call if it will process data from this file. NOTE: I'm not certain we will actually make this interface available since in the old system it basically went unused.
void respondToCloseInputFile(edm::FileBlock const&)
Called when the Source closes a file. NOTE: I'm not certain we will actually make this interface available since in the old system it basically went unused.

Template Arguments

Each of the base classes can take multiple template arguments. These template arguments allow one to extend the interface in order to get additional information about the state of the system or to share information across the collection of related Stream modules.

edm::GlobalCache<T>

This extention allows all related instances of modules across Streams to share 'Global Cache' information. The actual information is up to the developer and is specified to the system based on the template parameter of the edm::GlobalCache class. The only restriction is all const interactions with the class must be thread safe. The easiest way to guarantee that is to not allow modification of the data once it has been set in the constructor. However, the use to std::atomic<> or other serialization mechanisms is also allowed. Note that the template argument void is an allowed value and can be used in the case where you just want to be informed about the globalEndJob and do not need to share any information across the Stream instances.

The extended interface is as follows:

static std::unique_ptr<T> initializeGlobalCache(edm::ParameterSet const&)
This static function is called before the module's constructor is called. The function is passed the module's ParameterSet and is expected to return an instance of the class which holds all the Global Cache information. Returning a nullptr is an allowed value.
static void globalEndJob(T*)
This static function is called only once at the end of the job. It is passed the Global Cache object which was created in the initializeGlobalCache function. If the Global Cache object was used to accumulate information in a thread-safe manner, one can then make use of that fully accumulated information at this point.
Constructor(edm::ParameterSet const&, T const*)
The signature of the constructor is changed when using this extension. In addition to the standard ParameterSet the Global Cache created in the call to initializeGlobalCache is passed in. There is no need for you to hold onto the T const* since that object is already made available through the globalCache method.
T const* globalCache() const
Returns the object created by the call to initializeGlobalCache.

example usage: counting events in a job

Below is shown an example where we are keeping a count of all events which are ever seen by all instances of the Stream modules. This is accomplished by sharing an std::atomic<unsigned int> across all the instances. We must use an std::atomic<unsigned int> instead of just a simple unsigned int because the value will be updated simultaneously by multiple Streams.

#include <atomic>

//use an anonymous namespace since there is no need for any other file to see these class names
namespace {
struct Count { 
   Count():value(0) {}
   //Using mutable since we want to update the value.
   mutable std::atomic<unsigned int> value;
};

class MyAnalyzer : public edm::stream::EDAnalyzer<edm::GlobalCache<Count>> {
public:
   MyAnalyzer(edm::ParameterSet const&, Count const*) {}
   
   static std::unique_ptr<Count> initializeGlobalCache(edm::ParameterSet const&) {
       return std::unique_ptr<Count>(new Count());
   }

   void analyze(edm::Event const&, edm::EventSetup const&) {
      //This can safely be updated from multiple Streams because we are using an std::atomic
      ++(globalCache()->value);
   }

   static void globalEndJob(Count const* iCount) {
      std::cout <<"Number of events seen "<<iCount->value<<std::endl;
   }
};
} //ends namespace

edm::RunCache<T>

This extension allows all related instances of modules across Streams to share information pertaining to one Run. The actual information is up to the developer and is specified to the system based on the template parameter of the edm::RunCache<T> class. The only restriction is all const interactions with the class must be thread safe. Note that the template argument void is an allowed value and can be used in the case where you just want to be informed about the globalEndRun and do not need to share any information across the Stream instances.

The main reason to use this extension is if the item (or items) to be shared either requires a lot of memory or a lot of CPU time to construct. If it only takes modest amounts of both it is better to just add it as member data to the class and have a per Stream instance of that data. Another reason to use this extension is if you must accumulate simple information for a Run and that information can easily be made thread safe. If it is not trivial to make the information thread-safe, use the RunSummary extension instead.

The extended interface is as follows:

static std::shared_ptr<T> globalBeginRun(edm::Run const&, edm::EventSetup const&, GlobalCache const*)
This method is called each time the Source sees a new Run. This method is guaranteed to finish before any Stream calls beginRun for that same Run. Therefore it is safe to call runCache() from within beginRun. The GlobalCache is a typedef from the base class which is either the template argument from edm::GlobalCache<T> or a void depending on if the module uses the edm::GlobalCache<T> extension. The return value is a std::shared_ptr<T> rather than a T* in order to facilitate the possibility of reusing the created object. (See example below.)
T const* runCache() const
Returns the approriate run cache object created from globalBeginRun.
static void globalEndRun(edm::Run const&, edm::EventSetup const&, RunContext const*)
This static method is called after all Streams have finished processing a given Run (i.e. endRun for all Streams have completed). The RunContext is a typedef from the base class for a GlobalAndRunContext<G,R> instance where G is either the value from GlobalCache<G> or void and R is the value from edm::RunCache<R>.

example usage: counting events in a run

The following example counts the number of events in a Run. It is crucial to use a std::atomic<unsigned int> rather than an unsigned int since we need to safely update the value from multiple Streams.

#include <atomic>

//use an anonymous namespace since there is no need for any other file to see these class names
namespace {
struct Count { 
   Count():value(0) {}
   //Using mutable since we want to update the value.
   mutable std::atomic<unsigned int> value;
};

class MyAnalyzer : public edm::stream::EDAnalyzer<edm::RunCache<Count>> {
public:
   MyAnalyzer(edm::ParameterSet const&) {}
   
   static std::shared_ptr<Count> globalBeginRun(edm::Run const&, edm::EventSetup const&, GlobalCache const*) {
       return std::shared_ptr<Count>(new Count());
   }

   void analyze(edm::Event const&, edm::EventSetup const&) {
      //This can safely be called from multiple Streams because we are using an std::atomic
      ++(runCache()->value);
   }

   static void globalEndRun(edm::Run const& iRun, edm::EventSetup const&, RunContext const* iContext) {
      std::cout <<"Number of events seen in Run "<<iRun.runNumber()<<" = "<<iContext->run()->value<<std::endl;
   }
};
} //ends namespace

example usage: reusing a heavy weight object that gets updated each run

The following example guarantees that we have a unique instance of a heavy weight object (TrackFitter) for each Run. By using a edm::ReusableObjectHolder<> the system only creates new instances of the TrackFitter if an unused older instance is not available. This guarantees the minimum number of TrackFitters in the job needed without causing the system to wait on Run boundaries. The edm::ReusuableObjectHolder<> is maintained in the Global Cache since the ability to re-use a TrackFitter crosses Run boundaries.

#include "FWCore/Utilities/interface/ReusableObjectHolder.h"

//use an anonymous namespace since there is no need for any other file to see these class names
namespace {

class FitterGlobalCache {
public:
  FitterGlobalCache(edm::ParameterSet const& iPSet); //This remembers all the parameters needed to create a TrackFitter
  
  std::shared_ptr<TrackFitter> getTrackFitter(edm::Run const&, edm::EventSetup& iES) const;

private:
  TrackFitter* makeNewTrackFitter() const; //helper function that creates a new TrackFitter based on the saved parameters

  mutable edm::ReusuableObjectHolder<TrackFitter> m_holder;

  //This is where all the data is pulled from the PSet
  ...
};

std::shared_ptr<TrackFitter> FitterGlobalCache::getTrackFitter(edm::Run const& iRun, edm::EventSetup const& iES) const {
   //use a lambda to generate a new TrackFitter only if there isn't an old one available
   auto trackFitter = m_holder->makeOrGet( [this] () { return this->makeNewTrackFitter();} );
   trackFitter->update(iRun,iES);
   return trackFitter;
}

class ATrackFitter : public edm::stream::EDProducer<edm::GlobalCache<FitterGlobalCache>, edm::RunCache<TrackFitter> > {
public:

   static std::unique_ptr<FitterGlobalCache> initializeGlobalCache(edm::ParameterSet const& iPSet) {
       return std::unique_ptr<FitterGlobalCache>(new FitterGlobalCache(iPSet));
   }

   ATrackFitter(edm::ParameterSet const&, FitterGlobalCache const*) {}
   
   static std::shared_ptr<TrackFitter> globalBeginRun(edm::Run const&, edm::EventSetup const& iES, GlobalCache const* iCache) {
       return iCache->getTrackFitter(iRun, iES);
   }

   void producer(edm::Event& iEvent, edm::EventSetup const&) {
      //Get the track you are interested in
      ...
      //Refit the track using the Run specific TrackFitter
      runCache()->fit(track);

      //Do what ever else is needed
      ...
   }

   static void globalEndRun(edm::Run const& iRun, edm::EventSetup const&, RunContext const* iContext) {
      //Do nothing
   }

   static void endJob(FitterGlobalCache const*) {
      //Do nothing
   }

};
} //ends namespace

edm::LuminosityBlockCache<T>

This extension allows all related instances of modules across Streams to share information pertaining to one LuminosityBlock. The actual information is up to the developer and is specified to the system based on the template parameter of the edm::LuminosityBlockCache<T> class. The only restriction is all const interactions with the class must be thread safe. Note that the template argument void is an allowed value and can be used in the case where you just want to be informed about the globalEndLuminosityBlock and do not need to share any information across the Stream instances.

The main reason to use this extension is if the item (or items) to be shared either requires a lot of memory or a lot of CPU time to construct. If it only takes modest amounts of both it is better to just add it as member data to the class and have a per Stream instance of that data. Another reason to use this extension is if you must accumulate simple information for a LuminosityBlock and that information can easily be made thread safe. If it is not trivial to make the information thread-safe, use the LuminosityBlockSummary extension instead.

The extended interface is as follows:

static std::shared_ptr<T> globalBeginLuminosityBlock(edm::LuminosityBlock const&, edm::EventSetup const&, RunContext const*)
This method is called each time the Source sees a new LuminosityBlock. This method is guaranteed to finish before any Stream calls beginLuminosityBlock for that same LuminosityBlock. Therefore it is safe to call luminosityBlockCache() from within beginLuminosityBlock.The RunContext is a typedef from the base class for a GlobalAndRunContext<G,R> instance where G is either the value from GlobalCache<G> or void and R is the value from edm::RunCache<R>. The return value is a std::shared_ptr<T> rather than a T* in order to facilitate the possibility of reusing the created object.
T const* luminosityBlockCache() const
Returns the approriate luminosity block cache object created from globalBeginLuminosityBlock.
static void globalEndLuminosityBlock(edm::LuminosityBlock const&, edm::EventSetup const&, LuminosityBlockContext const*)
This static method is called after all Streams have finished processing a given LuminosityBlock (i.e. endLuminosityBlock for all Streams have completed). The LuminosityBlockContext is a typedef from the base class for a GlobalRunLumiContext<G,R,L> instance where G is either the value from GlobalCache<G> or void and R is either the value from edm::RunCache<R> or void and L is the value from edm::LuminosityBlockCache<L>.

example usage: updating a helper object only if a condition actually changed

The following example updates a helper class, PositionFinder, only if a particular condition has changed. This code assumes that the condition in question is only allowed to change on LuminosityBlock boundaries (which is the common case). Like the Run example, we will use an edm::ReusableObjectHolder<> object to allow us to reuse old instances of the PositionFinder. In addition, we need to use an edm::SerialTaskQueue when determining if the condition has changed from one LuminosityBlock to the next since the static function globalBeginLuminosityBlock may be called simultaneously from different Streams because multiple new LuminositBlocks have entered the system. If that happens, we need to use the edm::SerialTaskQueue in order to guarantee that one Stream doesn't update m_mostRecentPostionFinder while a different Stream is reading the old value.

NOTE: The design presented in this example should only be used if all of the following apply

  1. The change of conditions requires a length recalculation and/or the creation of a large memory structure
  2. The structure created is only useful to this module.
If you just need to have access to an object in the edm::EventSetup it is better just to ask for that item once each Event. Fetching data from the edm::EventSetup is very fast. If, for some reason it is not practicle to fetch the data each Event then consider fetching it each call to beginLuminosityBlock and just caching the value per Stream. Further, if the data structure is not modified each Event, consider moving it into the EventSetup directly and let the framework manage the proper updating of the structure.

#include "FWCore/Utilities/interface/ReusableObjectHolder.h"
#include "FWCore/Utilities/interface/SerialTaskQueue.h"

//use an anonymous namespace since there is no need for any other file to see these class names
namespace {

class MoverGlobalCache {
public:
  MoverGlobalCache(edm::ParameterSet const& iPSet); //This remembers all the parameters needed to create a PositionFinder
  
  std::shared_ptr<PositionFinder> getPositionFinder(edm::EventSetup& iES) const;

private:
  TrackFitter* makeNewPositionFinder() const; //helper function that creates a new PositionFinder based on the saved parameters

  mutable edm::ReusuableObjectHolder<PositionFinder> m_holder;
  mutable edm::SerialTaskQueue m_queue;
  //the access to the two following member data are controlled by the queue
  mutable edm::ESWatcher<AlignmentRcd> m_watcher; 
  mutable std::shared_ptr<PositionFinder> m_mostRecentPostionFinder;

  //This is where all the data is pulled from the PSet
  ...
};

std::shared_ptr<PositionFinder> MoverGlobalCache::getPositionFinder(edm::EventSetup const& iES) const {
   // use the serial queue to guarantee serial read/write of m_watcher and m_mostRecentPositionFinder
   std::shared_ptr<PositionFinder> returnValue;
   m_queue.pushAndWait([&]() {
      if(!this->m_watcher.check(iES)) {
         //the condition hasn't changed so we can just use our old value
         returnValue = m_mostRecentPositionFinder;
      } else {
         //the condition has changed so we need to update
         //use a lambda to generate a new PositionFinder only if there isn't an old one available
         returnValue = m_holder->makeOrGet( [this]() { return this->makeNewPositionFinder();} );
         returnValue->update(iES.get<AlignmentRcd>());
         m_mostRecentPositionFinder = returnValue;
      }
   });
   return returnValue;
}

class MovedItemProducer : public edm::stream::EDProducer<edm::GlobalCache<MoverGlobalCache>, 
                                                         edm::LuminosityBlockCache<PositionFinder> > {
public:

   static std::unique_ptr<MoverGlobalCache> initializeGlobalCache(edm::ParameterSet const& iPSet) {
       return std::unique_ptr<MoverGlobalCache>(new MoverGlobalCache(iPSet));
   }

   MovedItemProducer(edm::ParameterSet const&, MoverGlobalCache const*) {}
   
   static std::shared_ptr<PositionFinder> globalBeginLuminosityBlock(edm::LuminosityBlock const&, 
                                                                     edm::EventSetup const& iES, 
                                                                     RunContext const* iContext) {
       return iContext->getPositionFinder(iES);
   }

   void produce(edm::Event& iEvent, edm::EventSetup const&) {
      //Get the item you need to move
      ...
      //Apply the move
      luminosityBlockCache()->move(item);

      //Do what ever else is needed
      ...
   }

   static void endLuminosityBlock(edm::Run const&, edm::EventSetup const&, LuminosityBlockContext const*) {
      //Do nothing
   }

   static void endJob(MoverGlobalCache const*) {
      //Do nothing
   }

};
} //ends namespace

Should we also allow edm::JobSummaryCache?

edm::RunSummaryCache<T>

This extension allows all related instances of modules across Streams to each accumulate a partial sum of information about a Run and then when all of them have finished with that Run the partial sums can be accumulated into a final sum. The actual summary information is up to the developer and is specified to the system based on the template parameter T of the edm:RunSummaryCache<T> class. The system guarantees serial access to T so its member data does not need to be thread-safe.

The extended interface is as follows:

static std::shared_ptr<T> globalBeginRunSummary(edm::Run const&, edm::EventSetup const&, RunContext const*)
This method is called each time the Source sees a new Run. This method is guaranteed to be called after globalBeginRun in the case where the class also uses the edm::RunCache extension. The RunContext is a typedef from the base class for a GlobalAndRunContext<G,R> instance where G is either the value from GlobalCache<G> or void and R is either the value from edm::RunCache<R> or void. The return value is a std::shared_ptr<T> rather than a T* in order to facilitate the possibility of reusing the created object.
void endRunSummary(edm::Run const&, edm::EventSetup const&, T*) const
Called when a Stream has finished processing a Run. The framework guarantees that only one endRunSummary for all related module instances is called at a time. Therefore it is safe to modify the summary information which is passes in as the argument T *. The framework guarantees that endRun is called before endRunSummary.
static void globalEndRunSummary(edm::Run const&, edm::EventSetup const&, RunContext const*, T*)
This static method is called after the endRunSummary method for all Streams have finished processing a given Run. The RunContext is a typedef from the base class for a GlobalAndRunContext<G,R> instance where G is either the value from GlobalCache<G> or void and R is either the value from edm::RunCache<R> or void.

example usage: filling a histogram for each run

This example uses a fictitious histogramming library to create histograms that are per Run and filled with information obtained from the Event. The fictitious histogram library is assumed to have the following properties
  1. It is safe to do operations simultaneously on two different object instances. This holds for different HistFile, MemoryHist1D and StoredHist1D.
  2. Simultaneous operations on the same object instance are not safe (i.e. the classes are not thread-safe).
  3. Class methods do not affect any global state of the system.
The design for this example is as follows
  1. We have a GlobalCache which holds the HistFile as well as a SerialTaskQueue which is used to make all interactions with the HistFile thread safe.
  2. Each Run gets its own new StoredHist1D which is attached to the shared HistFile.
  3. Each module stream instance has its own MemoryHist1D which is used to accumulate information from the Events that the particular Stream sees.
  4. Once a Stream finishes with a Run its MemoryHist1D is added to the StoredHist1D.
  5. When all Streams have finished with a Run the StoredHist1D is written out to the file and then the StoredHist1D is removed from the HistFile object's data structure so its memory can be reclaimed.

#include "FWCore/Utilities/interface/SerialTaskQueue.h"

//use an anonymous namespace since there is no need for any other file to see these class names
namespace {

class HistMaker {
public:
   HistMaker(std::string const& iName): m_file(new HistFile(iName));

   std::shared_ptr<StoredHist1D> makeHist1D(unsigned int iRunNumber, ...) const {
      std::shared_ptr<StoredHist1D> returnValue;

      //make sure only one thread is asking for a new histogram at a time
      m_queue.pushAndWait([&]() {
         //use iRunNumber to create a unique name
         std::string name = ...;
         returnValue = this->m_file->bookHist1D(name, ...);
      });
      return returnValue;
   }

   void writeAndReleaseFromFile(StoredHist1D const* iHist) const {
      //need to serialize writing to the file and updating the file's internal data structures
      m_queue.pushAndWait([&]() {
         this->m_file->write(iHist);
         this->m_file->release(iHist);
      });
   }

   //This is called only once so it doesn't have to be made thread safe
   void closeFile() { m_file->close();}
private:
   mutable std::unique_ptr<HistFile> m_file;
   mutable SerialTaskQueue m_queue;
};

class HistFillerAnalyzer : public edm::stream::EDAnalyzer<edm::GlobalCache<HistMaker>, 
                                                          edm::RunSummaryCache<StoredHist1D> > {
public:

   static std::unique_ptr<HistMaker> initializeGlobalCache(edm::ParameterSet const& iPSet) {
       return std::unique_ptr<HistMaker>( new HistMaker(iPSet.getParameter<std::string>("histFileName"))
   }

   HistFillerAnalyzer(edm::ParameterSet const&, HistMaker const*): m_streamHist(...) {}
   
   static std::shared_ptr<StoredHist1D> globalBeginRunSummary(edm::Run const& iRun, edm::EventSetup const&,
                                                              RunContext const* iContext) {
       return iContext->globalCache()->makeHist1D(iRun.runNumber());
   }

   void analyze(edm::Event const& iEvent, edm::EventSetup const&) {
      //Get the information you are interested in
      ...
      //fill the histogram for this stream
      m_streamHist.fill(value);

      //Do what ever else is needed
      ...
   }
   void endRunSummary(edm::Run const& iRun, edm::EventSetup const&, StoredHist1D* iStoredHist) {
      //Add the values seen in this Stream to the total for this Run
      iStoredHist->add(m_streamHist);

      //now clear in order to be ready for the next Run
      m_streamHist.clear();
   }

   static void globalEndRunSummary(edm::Run const& iRun, edm::EventSetup const&, 
                                   RunContext const* iContext, StoredHist1D* iStoredHist) {
      iContext->globalCache()->writeAndReleaseFromFile(iStoredHist);
   }

   static void endJob(HistMaker* iMaker) {
      iMaker->closeFile();
   }
private:
   MemoryHist1D m_streamHist;
};
} //ends namespace

edm::LuminosityBlockSummaryCache<T>

This extension allows all related instances of modules across Streams to each accumulate a partial sum of information about a LuminosityBlock and then when all of them have finished with that LuminosityBlock the partial sums can be accumulated into a final sum. The actual summary information is up to the developer and is specified to the system based on the template parameter T of the edm:LuminosityBlockSummaryCache<T> class. The system guarantees serial access to T so its member data does not need to be thread-safe.

The extended interface is as follows:

static std::shared_ptr<T> globalBeginLuminosityBlockSummary(edm::LuminosityBlock const&, edm::EventSetup const&, LuminosityBlockContext const*)
This method is called each time the Source sees a new LuminosityBlock. This method is guaranteed to be called after globalBeginLuminosityBlock in the case where the class also uses the edm::LuminosityBlockCache extension. The LuminosityBlockContext is a typedef from the base class for a GlobalRunLumiContext<G,R,L> instance where G is either the value from GlobalCache<G> or void, R is either the value from edm::RunCache<R> or void and L is either the value from edm::LuminosityBlockCache<L> or void. The return value is a std::shared_ptr<T> rather than a T* in order to facilitate the possibility of reusing the created object.
void endLuminosityBlockSummary(edm::LuminosityBlock const&, edm::EventSetup const&, T*) const
Called when a Stream has finished processing a Run. The framework guarantees that only one endLuminosityBlockSummary for all related module instances is called at a time. Therefore it is safe to modify the summary information which is passes in as the argument T *. The framework guarantees that endLuminosityBlock is called before endLuminosityBlockSummary.
static void globalEndLuminosityBlockSummary(edm::LuminosityBlock const&, edm::EventSetup const&, LuminosityBlockContext const*, T*)
This static method is called after the endLuminosityBlockSummary method for all Streams have finished processing a given LuminosityBlock. The LuminosityBlockContext is a typedef from the base class for a GlobalRunLumiContext<G,R,L> instance where G is either the value from GlobalCache<G> or void, R is either the value from edm::RunCache<R> or void and L is either the value from edm::LuminosityBlockCache<L> or void.

example usage: calculating per luminosity block statistics

This example calculates statistics for the number of tracks in all events in a luminosity block. This requires keeping a list of the number of tracks in all events in a given luminosity block.

#include <vector>

//use an anonymous namespace since there is no need for any other file to see these class names
namespace {

class TrackStatisticsAnalyzer : public edm::stream::EDAnalyzer<edm::LuminosityBlockSummaryCache<std::vector<unsigned int>> > {
public:

   TrackStatisticsAnalyzer(edm::ParameterSet const& iPSet): m_trackTag(iPSet.getParameter<edm::InputTag>("trackTag")) {}
   
   static std::shared_ptr<std::vector<unsigned int>> globalBeginLuminosityBlockSummary(edm::LuminosityBlock const&, 
                                                                                       edm::EventSetup const&, 
                                                                                       LuminosityBlockContext const*) {
       return new std::vector<unsigned int>();
   }

   void analyze(edm::Event const& iEvent, edm::EventSetup const&) {
      //Get the track info
      edm::Handle<std::vector<reco::Track>> hTracks;
      iEvent.getByLabel(m_trackTag,hTracks);
  
      m_trackListSizes.push_back(hTracks->size());
   }
   
   void endLuminosityBlockSummary(edm::LuminosityBlock const& iRun, 
                                  edm::EventSetup const&, 
                                  std::vector<unsigned int>* iSummary) {
      //add the Stream's partial information to the full information
      iSummary->insert(m_trackListSizes.begin(),m_trackListSizes.end());

      //now clear in order to be ready for the next LuminosityBlock
      m_trackListSizes.clear();
   }

   static void globalEndLuminosityBlockSummary(edm::LuminosityBlock const&, 
                                               edm::EventSetup const&, 
                                               LuminosityBlockContext const* iContext, 
                                               std::vector<unsigned int>* iStoredHist) {
     //calculate lots of statistically interesting quantities and print them
     ...
   }

private:

   edm::InputTag m_trackTag;
   std::vector<unsigned int> m_trackListSizes;
};
} //ends namespace

edm::BeginRunProducer

This extension allows an EDProducer to put data into the Run at the begin of Run transition. Since an edm::Run is shared across all Streams processing that Run the interface for this extension only involves a static method. Items placed during the begin of Run transition are available to get for any method that directly or indirectly provides a edm::Run.

The extended interface is as follows:

static void globalBeginRunProduce(edm::Run&, edm::EventSetup const&, RunContext const*)
This method is called each time the Source sees a new Run. This method is guaranteed to be called after globalBeginRun in the case where the class also uses the edm::RunCache extension. The RunContext is a typedef from the base class for a GlobalAndRunContext<G,R> instance where G is either the value from GlobalCache<G> or void and R is either the value from edm::RunCache<R> or void.

example usage: pull data from a database and put it into the run

This example will use a fictitious thread-safe database interface to retrieve data and then place it into the Run.

//use an anonymous namespace since there is no need for any other file to see these class names
namespace {

class LHCStatusProducer : public edm::stream::EDProducer<edm::GlobalCache<DBConnection>, edm::BeginRunProducer> {
public:

   static std::unique_ptr<DBConnection> initializeGlobalCache(edm::ParameterSet const& iPSet) {
       return std::unique_ptr<DBConnection>( new DBConnection(iPSet.getParameter<std::string>("dbURL"));
   }

   LHCStatusProducer(edm::ParameterSet const&, DBConnection const*) {
      //we need to register the fact that we will make the LHCStatus object
      produces<LHCStatus, edm::Transition::BeginRun>();
   }
   
   void produce(edm::Event& iEvent, edm::EventSetup const&) {
      //Do nothing
   }

   static void globalBeginRunProduce(edm::Run& iRun, edm::EventSetup const&, RunContext const*) {
      std::auto_ptr<LHCStatus> status(new LHCStatus);
      globalCache()->fill(*status);
 
      iRun.put(status);
   }

   static void endJob(DBConnection* iDB) {
      iDB->closeConnection();
   }
};
} //ends namespace

edm::BeginLuminosityBlockProducer

This extension allows an EDProducer to put data into the LuminosityBlock at the begin of LuminosityBlock transition. Items placed during the begin of LuminosityBlock transition are available to get for any method that directly or indirectly provides a edm::LuminosityBlock.

The extended interface is as follows:

static void globalBeginLuminosityBlockProduce(edm::LuminosityBlock&, edm::EventSetup const&, LuminosityBlockContext const*)
This method is called each time the Source sees a new LuminosityBlock. This method is guaranteed to be called after globalBeginLuminosityBlock in the case where the class also uses the edm::LuminosityBlockCache extension. The LuminosityBlockContext is a typedef from the base class for a GlobalRunLumiContext<G,R,L> instance where G is either the value from GlobalCache<G> or void, R is either the value from edm::RunCache<R> or void and L is either the value from edm::LuminosityBlockCache<L> or void.

example usage: pull data from a database and put it into the luminosity block

This example will use a fictitious thread-safe database interface to retrieve data and then place it into the LuminosityBlock.

//use an anonymous namespace since there is no need for any other file to see these class names
namespace {

class LumiSummaryProducer : public edm::stream::EDProducer<edm::GlobalCache<DBConnection>, edm::BeginLuminosityBlockProducer> {
public:

   static std::unique_ptr<DBConnection> initializeGlobalCache(edm::ParameterSet const& iPSet) {
       return std::unique_ptr<DBConnection>( new DBConnection(iPSet.getParameter<std::string>("dbURL"));
   }

   LumiSummaryProducer(edm::ParameterSet const&, DBConnection const*) {
      //we need to register the fact that we will make the LumiSummary object
      produces<LumiSummary, edm::Transition::BeginLuminosityBlock>();
   }
   
   void produce(edm::Event& iEvent, edm::EventSetup const&) {
      //Do nothing
   }

   static void globalBeginLuminosityBlockProduce(edm::LuminosityBlock& iLumi, edm::EventSetup const&, LuminosityBlockContext const*) {
      std::auto_ptr<LumiSummary> summary(new LumiSummary);
      globalCache()->fill(iLumi.runNumber(), iLumi.luminosityBlockNumber(),*summary);
 
      iLumi.put(summary);
   }

   static void endJob(DBConnection* iDB) {
      iDB->closeConnection();
   }
};
} //ends namespace

edm::EndRunProducer

This extension allows an EDProducer to put data into the Run at the end of Run transition. Items placed during the end of Run transition are only available during globalEndRun calls and are not available to the standard Stream method endRun. The reason for this restriction is the need to make all Run products invariant for the entire processing of the Run.

The normal extended interface is as follows:

static void globalEndRunProduce(edm::Run&, edm::EventSetup const&, RunContext const*)
This method is called once all Streams processing a Run are done with the Run. This method is guaranteed to be called before globalEndRun in the case where the class also uses the edm::RunCache extension. The RunContext is a typedef from the base class for a GlobalAndRunContext<G,R> instance where G is either the value from GlobalCache<G> or void and R is either the value from edm::RunCache<R> or void.
If the module also uses the edm::RunSummaryCache<T> extension then the interfaces is modified to give access to the summary object
static void globalEndRunProduce(edm::Run&, edm::EventSetup const&, RunContext const*, T const*)
Same as above except also gives access to Run summary information via T const*. This method is guaranteed to be called after globalEndRunSummary so that the Run summary object is complete.

example usage: calculate a Run average efficiency

#include <atomic>
#include "DataFormats/Something/interface/Efficiency.h"

//use an anonymous namespace since there is no need for any other file to see these class names
namespace {

struct TempEfficiency {
   mutable std::atomic<unsigned long> m_numerator=0;
   mutable std::atomic<unsigned long> m_denominator=0;
};

class RunEfficiencyProducer : public edm::stream::EDProducer< edm::RunCache<TempEfficiency>, edm::EndRunProducer > {
public:

   RunEfficiencyProducer(edm::ParameterSet const&) {
      //have to register what we will make
      produces<Efficiency, edm::Transition::EndRun>();
   }
   
   static std::shared_ptr<Efficiency> globalBeginRun(edm::Run const&, edm::EventSetup const&, RunContext const*) {
       return new Efficiency;
   }

   void produce(edm::Event& iEvent, edm::EventSetup const&) {
      //Get the information you are interested in and see if it passes your requirement
      ...
      ++(runCache()->m_denominator);
      if(passesCuts) {
         ++(runCache()->m_numerator);
      }
   }
   static void globalEndRun(edm::Run const&, edm::EventSetup const&, RunContext const*) {
      //don't need to do anything
   }

   static void globalEndRunProduce(edm::Run const& iRun, edm::EventSetup const&, RunContext const* iContext) {
      std::auto_ptr<Efficiency> eff(new Efficiency(iContext->run()->m_numerator, iContext->run()->m_denominator));
      iRun.put(eff);
   }
private:
};
} //ends namespace

edm::EndLuminosityBlockProducer

This extension allows an EDProducer to put data into the LuminosityBlock at the end of a LuminosityBlock transition. Items placed during the end of LuminosityBlock transition are only available during globalEndLuminosityBlock calls and are not available to the standard Stream method endLuminosityBlock. The reason for this restriction is the final product cannot be determined until all the Streams have completed their processing of that LuminosityBlock.

The normal extended interface is as follows:

static void globalEndLuminosityBlockProduce(edm::LuminosityBlock&, edm::EventSetup const&, LuminosityContext const*)
This method is called once all Streams processing a LuminosityBlock are done with the LuminosityBlock. This method is guaranteed to be called before globalEndLuminosityBlock in the case where the class also uses the edm::LuminosityBlockCache extension. The LuminosityBlockContext is a typedef from the base class for a GlobalRunLumiContext<G,R,L> instance where G is either the value from GlobalCache<G> or void, R is either the value from edm::RunCache<R> or void and L is either the value from edm::LuminosityBlockCache<L> or void.
If the module also uses the edm::LuminosityBlockSummaryCache<T> extension then the interfaces is modified to give access to the summary object
static void globalEndLuminosityBlockProduce(edm::LuminosityBlock&, edm::EventSetup const&, LuminosityBlockContext const*, T const*)
Same as above except also gives access to LuminosityBlock summary information via T const*. This method is guaranteed to be called after globalEndLuminosityBlockSummary so that the LuminosityBlock summary object is complete.

example usage: find a luminosity block average beam spot

This example keeps track of all vertices for all events in a luminosity block and then when the luminosity block ends we form a beam spot. We accumulate a list of points for each Stream independently. When a Stream finishes processing a luminosity block its lists of points is added to the global luminosity block point list.

//use an anonymous namespace since there is no need for any other file to see these class names
namespace {

class BeamSpotProducer : public edm::stream::EDProducer<edm::LuminosityBlockSummaryCache<std::vector<Point3D>>, 
                                                        edm::EndLuminosityBlockProducer> {
public:

   BeamSpotProducer(edm::ParameterSet const& iPSet) : m_tag(iPSet.getParameter<edm::InputTag>("vertexTag"){
      
      //we need to register the fact that we will make the BeamSpot object
      produces<BeamSpot, edm::Transition::EndLuminosityBlock>();
   }
   
   static std::shared_ptr<std::vector<Point3D>> globalBeginLuminosityBlockSummary(edm::LuminosityBlock const&, 
                                                                                  edm::EventSetup const&, 
                                                                                  LuminosityBlockContext const*) {
       return new std::vector<Point3D>();
   }

   void produce(edm::Event& iEvent, edm::EventSetup const&) {
      //Get vertices
      edm::Handle<std::vector<Vertex>> vertices;
      iEvent.getByLabel(m_tag, vertices);

      m_points.reserve(m_points.size()+vertices->size());
      for( auto v : *vertices) {
         m_points.push_back(v.point());
      }
   }

   void endLuminosityBlockSummary(edm::LuminosityBlock const& iRun, edm::EventSetup const&, std::vector<Point3D>* iPoints) {
      //add the Stream's partial information to the full information
      iPoints->insert(m_points.begin(),m_points.end());

      //now clear in order to be ready for the next LuminosityBlock
      m_points.clear();
   }

   static void globalEndLuminosityBlockSummary(edm::LuminosityBlock const&, 
                                               edm::EventSetup const&, 
                                               LuminosityBlockContext const* iContext, 
                                               std::vector<Point3D>*) {
      //Nothing to do
   }

   static void globalEndLuminosityBlockProduce(edm::LuminosityBlock& iLumi, 
                                         edm::EventSetup const&, 
                                         LuminosityBlockContext const* iContext, 
                                         std::vector<Point3D> const* iPoints) {
      //calculate beam spot from the points
      
      std::auto_ptr<BeamSpot> spot( makeBeamSpotFromPoints(*iPoints) );
 
      iLumi.put(spot);
   }
private:
   //Each Stream will have its own list of points
   std::vector<Point3D> m_points;   
};
} //ends namespace

edm::Accumulator

This extension is usually used with EDProducers that put products in the Run or LuminosityBlock but do not put products in the Event. Although they do not put products in the Event, these EDProducers need data from Events for the products they put in the Run or LuminosityBlock. Without this extension and if the EDProducer is configured to run unscheduled, its event method will never run because nothing ever requests a product it produces. With this extension, these modules will run their event method on every event and accumulate the data they need.

The declaration of an EDProducer with the Accumulator feature looks like this:

class MyProducer : public edm::stream::EDProducer<edm::Accumulator>

This only works with the EDProducer base class. It does not work with EDAnalyzers, EDFilters, or OutputModules.

When you use this feature you must replace the produce function with the accumulate function. It will not compile if you do not define the accumulate function. It will not compile if you define the produce function. The accumulate function will not allow products to be put into the Event because the Event argument is declared const.

void accumulate(edm::Event const&, edm::EventSetup const&) override

You can put modules with this feature on a Path or EndPath. The special behavior does not occur then. In that case, they run on the Path or EndPath the same as if they did not have the Accumulate extension. If the module is on a Path after an EDFilter that does not pass the event, then the module will not run.

Usually this feature extension will be used with other feature extensions. For example, it might be used with LuminosityBlockSummaryCache and EndLuminosityBlockProducer.

This extension was first added in release 10_1_0 (in 2018), it cannot be used in earlier releases.

edm::ExternalWork

This extension was added to provide Framework support for future modules that run in new ways. For example, modules that use GPUs or FPGAs might use this extension. Initially, this extension may be used for prototypes and tests of such modules. As of this writing (January 2018), none of these modules exist and the only things using this extension are Framework tests.

This extension affects the event processing part of the module and divides it into 3 parts. Without this extension there is one function called produce or filter. With this extension, there is a new acquire function added that is defined in the module. Its purpose is to get data from the Event and communicate to "something external to the module". One possible example of "something external to the module" would be a service that runs GPU processing. The acquire method also passes to this external thing an edm::WaitingTaskWithArenaHolder. Then the acquire method returns and is not left on the stack. It is important that it is not left on the stack because that eliminates the potential problem that it might block its thread or deadlock. Then the external thing performs the work. The third part is that the Framework calls the produce or filter method of the module. The external thing is not part of the Framework and there are very few requirements on what it is or how it functions. It must exchange data with the module in some way so that the acquire function can pass it input and the produce/filter method can use the results it produces. It also must call the doneWaiting function of the WaitingTaskWithArenaHolder. This lets the Framework know when it can call the produce or filter function. Beyond that, the Framework places no requirements on the design of the external work entity. There is great freedom in how it is designed and how it works.

One defines a class with this extension as follows:

class MyProducer : public edm::stream::EDProducer<edm::ExternalWork>

This will work for both EDProducers and EDFilters, but not for EDAnalyzers or OutputModules.

One must define one additional function in the module when this extension used:

void acquire(edm::Event const&, edm::EventSetup const&, edm::WaitingTaskWithArenaHolder) override;

This extension was first added in release 10_0_0 (in 2017), it cannot be used in earlier releases.

Review status

Reviewer/Editor and Date Comments
Last reviewed by: ChrisDJones - 14-Nov-2012 Created the page

Responsible: ChrisDJones
Last reviewed by:

Edit | Attach | Watch | Print version | History: r26 < r25 < r24 < r23 < r22 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r26 - 2018-07-12 - DavidDagenhart
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback