The CMSSW MixingModule

Complete: 4

Goal of page

This page outlines the purpose and use of the MixingModule.

For previous than 3_1_0 releases, please, see the information on the twiki page: https://twiki.cern.ch/twiki/bin/view/CMS/SWGuideMixingModule?rev=32.

The mixing module was extensively modified in release 6_0_0, so that tracker hits (except the Muon Trackers) and calorimeter hits will be digitized by accumulators that are run by the mixing module, rather than by external producers. These changes are only partially documented here.

INTRODUCTION

The primary goal of the MixingModule is to superimpose pileup events to a signal event. This includes out-of-time pileup and therefore the mixing module has to deal with a sequence of bunch crossings so to properly simulate the contributions affecting the in-time bunch crossing. This has been extended to other kind of secondary stream whose events can be added to signal events, such as cosmic and beam halo events. The superposition is done at the hit level. The hit level detector information as well as the Monte Carlo truth from the different streams are superimposed, resulting in a new EdProduct, CrossingFrame, (one CrossingFrame per object from release 170 on), containing the information of the different crossings. The objects for which mixing with the secondary input streams are done are: PSimHits, PCaloHits, SimTracks, SimVertices, HepMCProduct . There have been several new features introduced progressively in the previous releases:

  • from release 170, the monolithic CrossingFrame object has become templated (cf details in Access to the result), and mixing for HepMCProducts was introduced.
  • from release 180 on, the MixingModule allows to redo identically previously dropped CrossingFrames (cf playback), and multiple secondary input sources are implemented (cf secondary input sources).
  • from release 210 on, there is the possibility to select the type of objects to be mixed (cf Selection_of_objects).Possible objects are PSimHits, PCaloHits, SimTracks, SimVertices and HepMCProducts.
  • from release 600 on, the mixed hits (except for those in the Muon trackers) are digitized by accumulator modules that are run internally by the mixing module, although the accumulators are not part of the mixing module. The production of each crossing frame is now optional and controlled by the configuration. By default, the production of each crossing frame is enabled.

The code of the CrossingFrame and the MixingModule are respectively in the packages :

  • SimDataFormats/CrossingFrame
and
  • Mixing/Base
  • SimGeneral/MixingModule.

IMPORTANT:

  • since your code (after the MM) will find all the event information in the CrossingFrame(s), it is compulsory to put the MixingModule in your path, even if you don't want pileup, and to get the objects via the CrossingFrame(s). In case you don't want to superimpose pileup or any secondary stream events, you can configure the MM to do so, see below in chapter configuration.
  • starting from the 3_1_0 release, the CrossingFrame is transient in order to reduce the memory consumption. In this case it is not possible to write the CrossingFrame into the output root file by running only the MixingModule, you should run also the producer CFWriter, see an example of how to configure it in the paragraph Reference Guide/Configuration.
  • starting from the 6_0_0 release, each CrossingFrame, except for those for the Muon trackers, is optional. To save memory, please disable the production of unneeded crossing frames. Crossing frames are specified in SimGeneral/MixingModule/python/mixObjects_cfi.py.

USER GUIDE

The Mixing module simulates the full detector response. What it does exactly for each event is the following:

  • it copies the signal data into the CrossingFrame(s) at bunchcrossing 0
  • it loops over all bunchcrossings (as given by the configuration)
  • for each bunchcrossing it decides (also according to the configuration) which number of events from which source should be added. These events are delivered by the secondary input services, more details are given below.
  • for each bunchcrossing and for each type of data (PSimHits, PCaloHits, SimTracks, SimVertices, HepMCProducts) it adds the corresponding objects from the events read from the secondary streams. At the same time, the time information is shifted according to the bunchcrossing value, and EncodedEventIds are filled in. In more details:
  • for SimTracks: vertex index is shifted, the rest of information is kept, EncodedEventId is added
  • for SimVertices: ToF is shifted, the rest is kept unchanged, EncodedEventId is added
  • for SimHits and CaloHits: ToF shifted, rest is kept unchanged, EncodedEventId is added

Starting from the 3_2_X release, through the 5_X_X release cycle, the MixingModule can be run in two different modes :

  • A. Standard mode: the secondary event streams is superimposed to a signal event stream
  • B. Multistep mode: the different secondary event streams are firstly merged together, then the result is merged with the signal event stream ;
    • B1. Step1: production of the mixed data ; the results of the mixing module, the CrossingFrames, are recorded into a root file as objects named PCrossingFrames, in these mode the signal data is not added to the CrossingFrames
    • B2. Step2: mixing using the mixed source data from the Step1, the CrossingFrame is superimposed to the signal event stream

Starting from the 6_0_0 release, multistep mode is no longer supported. :

Access to the result (CrossingFrame)

The CrossingFrame contains all signal and pileup information for the different subdetectors present in the ProductRegistry at the start of the MixingModule. From release 170 on, the CrossingFrame is a templated class, and there will be one CrossingFrame for SimTracks, for SimVertices and HepMCProducts, and as many for PCaloHits and PSimHits as subdetectors for these were present in the ProductRegistry. The subdetector name is given as a label. Example: the branchname for mixed PSimHits, for ROU TrackerHitsPixelBarrelHighTof will be something like
PSimHitCrossingFrame_mix_TrackerHitsPixelBarrelHighTof_PROD

The only supported way to access the CrossingFrame information is via the helper class MixCollection. Here is an example:

  #include "SimDataFormats/CrossingFrame/interface/MixCollection.h"
  ...
  const std::string subdet("TrackerHitsTECHighTof");
  edm::Handle<CrossingFrame<PSimHit> > cf_simhit;
  iEvent.getByLabel("mix",subdet,cf_simhit);
  auto col = std::make_unique<PSimHit>(cf_simhit.product());
  MixCollection<PSimHit>::iterator cfi;
...
For more detailed examples, please have a look at SimGeneral/MixingModule/plugins/TestMix.cc. Since the CrossingFrames are transient in order to access them you should run your analyzer in the same python file just after the MixingModule.

....
process.p = cms.Path(process.mix+process.yourAnalyzer)

IMPORTANT: Please note that the MixCollection was not adapted to be used with the PCrossingFrames, i.e. it is not possible to analyse the CrossingFrames information using the PCrossingFrames written in the root file by the CFWriter. The root files containig the PCrossingFrames could be read only as a secondary source.

How to specify number of events to be added

The number of events from the secondary stream to add per bunch crossing to each signal event can be either randomly chosen according to a Poissonian distribution, or fixed. See the chapter describing the configuration for configuration details. The number of secondary events to be added per bunch crossing is configured separately for each of the secondary sources. It can be specified
  • by giving an average number explicitly
                               nbPileupEvents = cms.PSet(
                               averageNumber = cms.untracked.double(1.0)
                               )
  • by computing it from the provided luminosity, inelastic cross section and bunchspace, according to:

Rate/crossing = luminosity x sigma_inelastic x bunchspace / f

where

  • the factor f takes into account the non complete filling of the LHC machine, f=2808/3564,
  • luminosity is expressed in units of E33 cm-2s-1,
  • the inelastic cross section is given in mb,
  • the bunch space is in ns, and can be evaluated from the number Nbx of filled bunches by bunchspace = 25*2808/Nbx

Python config style: Example (secondary source):

                             nbPileupEvents  =cms.untracked.PSet(
                                Lumi=cms.untracked.double(0.5), # in units of E33 cm-2s-1
                                sigmaTot=cms.untracked.double(80) #   sigma inelastic in mb
                             )
and for the MM
                bunchspace = cms.untracked.int32(-5)

 

  • by giving an histogram distribution
                             nbPileupEvents  =cms.untracked.PSet(
                                fileName=cms.untracked.string('histo.root'),
                                histoName=cms.untracked.string('h') 
                             )
The "type" of the source should be set to "histo". The histogram you use should be normalized, the bins' width should be 1 and the edges should be integers.

* by giving a probability function

                             nbPileupEvents  =cms.untracked.PSet(
                                probFunctionVariable=cms.vint32(0,1,2,3,.....),
                                probValue=cms.vdouble(prob1value, prob2value,prob3value.....),
                                histoFileName=cms.untracked.string('histoProbFunction.root')
                             )
In this case you should precise the "type" = "probFunction". The probability function is defined giving the variables (probFunctionVariable) and the corresponding probability values (probValue). The first variable should be 0, and the difference between two variables should be 1! The code will create an histogram from the probability function and it will be saved into the root file with the name defined by the histoFileName.

In the case of a Poissonian distribution this number is interpreted as the average number, ie the single parameter of the Poissonian distribution. In case the distribution choosen is "fixed", this number will be used for each event as the constant number of secondary strem events to be added.

Selection of objects

By default, all possible objects (PSimHits, PCaloHits, SimTracks, SimVertices, HepMCProducts) are added. This can be changed if needed by modifying mixObjects_cfi.py.

If it becomes necessary to mix other types of objects, SimGeneral/MixingModule/plugins/MixingModule.cc will also need to be modified. Refer to the existing code in this file supporting the currently supported types to see what needs to be done.

Default configurations

Several default set of parameters have been prepared for the superposition of pileup according to the different configurations of the LHC machine as described in http://lhc-commissioning.web.cern.ch/lhc-commissioning/? They can be found in SimGeneral/MixingModule/python/ :

playback option

From release 180 on, the MM is able to restore, on an event per event basis, exactly the same CrossingFrames as in a previous job (this may be useful since in production the CrossingFrames are normally dropped, but may be needed for special studies). The idea is to have a first job executing the mixing, with parameter playback=false. This job will write a minimal information on the output file (CrossingFramePlaybackInfo) ( http://cmssw.cvs.cern.ch/cgi-bin/cmssw.cgi/CMSSW/SimGeneral/MixingModule/test/mm_store_cfg.py?revision=1.3&view=markup ). In a subsequent job this output file will be reread as input, and the fact of executing the MixingModule with parameter playback=true will reproduce the CrossingFrames identically (for more information the example http://cmssw.cvs.cern.ch/cgi-bin/cmssw.cgi/CMSSW/SimGeneral/MixingModule/test/mm_back_cfg.py?revision=1.2&view=markup).

Important note:

The configuration for the mixing in the second job must be strictly the same as for the first one, in particular you should give the same PileUp data files, in the same order.

From the 3_5 release a new class CrossingFramePlaybackInfoExtended has been implemented in order to can use the playback mecanism in a real production. Then, with the new MixingModule in a playback mode it will be possible to use the RelVal data with PileUp to recreate the CrossingFrame.

Secondary input sources

From release 180 on, the possibility of several secondary sources has been implemented in order to extend the addition of pileup events to the more general case of any kind of background events. The names of the available sources are (sourcetype is used in the MixCollection interface):

input , sourcetype =0, to be used for pileup superposition

cosmics , sourcetype =1

beamhalo_plus , sourcetype =2

beamhalo_minus, sourcetype=3

For each type, a parameterset 'secsource' has to be given. An example configuration file using 4 sources is given in SimGeneral/MixingModule/test/mix_4s_cfg.py , which uses !SimGeneral/MixingModule/python/mixLowLumPU_4sources_cfi.py.

In order not to merge events from these possible sources, one has to set the type to "none".

In the Step2 mode, there is no special name for the mixed source, it should be configured as a input source.

Random choice of pileup events

Here we explain how the pileup events are chosen randomly. This task is not part of the mixing module, rather here the MM uses functionalities provided by the framework.

First, a file is chosen randomly with each input file being equiprobable. Then an event is chosen in that file, with each event equiprobable. Then the events in that file are read, beginning at the randomly chosen one. After the last event in the file, reading wraps around to the first event in the file. This continues until every event in the file has been read exactly once. This two steps randomization process is repeated as long as needed to produce the sequences of secondary events needed for all requested signal events .

The framework provides also a sequential way to read the pileup events. In this case the events are read sequentially starting from the first event in the first file. It is used in the Step2 of the Multistep mode of the MM and for debugging purposes.

The number of pileup events, type of distribution and the random seed as well as the files names containing the pileup events should be given as parameters for the secsource (see the paragraph “How to specify number of events to be added”):

     ...
     input = cms.SecSource("PoolSource",
         nbPileupEvents = cms.PSet(
            sigmaInel = cms.double(80.0),
            Lumi = cms.double(2.8)
        ),
        seed = cms.int32(1234567),
        type = cms.string('poisson'),
        sequential = cms.untracked.bool(False),
        fileNames = cms.untracked.vstring(‘file.root’)
),
     .....

Please, note that if "playBack" parameter is also specified as true, the "sequential" parameter will be ignored.

REFERENCE GUIDE

Configuration (python)

Example of the MixingModule configuration file to be run in the standard mode

from SimGeneral.MixingModule.mixObjects_cfi import *
mix = cms.EDProducer("MixingModule",
    LabelPlayback = cms.string(''),
    maxBunch = cms.int32(3),
    minBunch = cms.int32(-5), ## in terms of 25 ns

    bunchspace = cms.int32(25), ## nsec
    mixProdStep1 = cms.bool(False),
    mixProdStep2 = cms.bool(False),

    playback = cms.untracked.bool(False),
    useCurrentProcessOnly = cms.bool(False),
                   
    input = cms.SecSource("PoolSource",
       nbPileupEvents = cms.PSet(
            sigmaInel = cms.double(80.0),
            Lumi = cms.double(2.8)
        ),
        seed = cms.int32(1234567),
        type = cms.string('poisson'),
        sequential = cms.untracked.bool(False),
        fileNames = cms.untracked.vstring('data.root')
    ),
    mixObjects = cms.PSet(
        mixCH = cms.PSet(
            mixCaloHits
        ),
        mixTracks = cms.PSet(
            mixSimTracks
        ),
        mixVertices = cms.PSet(
            mixSimVertices
        ),
        mixSH = cms.PSet(
            mixSimHits
        ),
        mixHepMC = cms.PSet(
            mixHepMCProducts
        )
    )
)

Example of the configuration file for CFWriter (used to write the CrossingFrames into the output root file)

Please, note that the CFWriter should be configured with the same mixObjects which are mixed by the MixingModule.

process = cms.Process("CFWRITER")
process.load("SimGeneral.MixingModule.mixLowLumPU_cfi")
..........
process.CFWriter = cms.EDProducer("CFWriter",
    maxBunch = cms.int32(3),
    minBunch = cms.int32(-5),
    
    mixObjects = cms.PSet(
    mixCH = cms.PSet(
      mixCaloHits
    ),
    mixTracks = cms.PSet(
      mixSimTracks
    ),
   mixVertices = cms.PSet(
      mixSimVertices
    ),
    mixSH = cms.PSet(
      mixSimHits
    ),
    mixHepMC = cms.PSet(
      mixHepMCProducts
    )
    )
)
.......
process.p = cms.Path(process.mix+process.CFWriter)

Parameters

  • for each secondary source:
    • untracked vstring fileNames
    • string type may be 'poisson' for Poissonian distribution, 'fixed' for a constant number of secondary events to superimpose, 'histo' for histogram given by the user, 'probFunction' for probability function given by the user, or 'none' if ones do not want to add any secondary event
    • PSet nbPileupEvents can be specified via luminosity (shown above) or by giving directly an average number (double)
    • int32 seed
    • untracked bool sequential: this parameter allows to configure the random choice of pileup events, its default value is false
  • for the MM itself:
    • int32 minBunch , int32 maxBunch : given in terms of 25 ns, specifies the time range where detector is sensitive
    • int32 bunchspace : in nsec, specifies the space between the bunch crossings.
    • string Label: label for selecting input objects to be treated, default ""
    • bool mixProdStep1: label for selecting to run the MixingModule in the Step1 mode
    • bool mixProdStep2: label for selecting to run the MixingModule in the Step2 mode
    • untracked bool playback: see paragraph on playback option, default false

The minimum/maximum bunch crossings to be filled by the MM will be minBunch*25/bunchspace and maxBunch*25/bunchspace .

How to run the MixingModule

in the standard mode

Run the configuration file /SimGeneral/MixingModule/test/mm_cfg.py

By default, this configuraiton file calls the /SimGeneral/MixingModule/python/mixLowLumPU_cfg.py. If you want to use the HighLumi configuration, please include the configuration file /SimGeneral/MixingModule/python/mixHighLumPU_cfg.py.

in the Step1 of the Multistep mode

Run the configuration file /SimGeneral/MixingModule/test/mm_CFWriter_cfg.py. For the LowLumi, please, include /SimGeneral/MixingModule/python/mixLowLumPU_mixProdStep1_cfg.py, for the HighLumi, /SimGeneral/MixingModule/python/mixHighLumPU_mixProdStep1_cfg.py.

in the Step2 of the Multistep mode

Run the configuration file /SimGeneral/MixingModule/test/mm_cfg.py where you should replace process.load("SimGeneral.MixingModule.mixLowLumPU_cfi") by process.load("SimGeneral.MixingModule.mixProdStep2_cfi").

The particularities are:

 
    maxBunch = cms.int32(0),
    minBunch = cms.int32(0),
   ...........
    mixProdStep2 = cms.bool(True),
    .........
    input = cms.SecSource("PoolSource",
      type = cms.string('fixed'),
   nbPileupEvents = cms.PSet(
        averageNumber = cms.double(1.0)
       ),
   sequential = cms.untracked.bool(True),
        .............
    )

  mixObjects = cms.PSet(
        # Objects for Step2
   mixPCFCH = cms.PSet(
            mixPCFCaloHits
        ),
        mixPCFTracks = cms.PSet(
            mixPCFSimTracks
        ),
        mixPCFVertices = cms.PSet(
            mixPCFSimVertices
        ),
        mixPCFSH = cms.PSet(
            mixPCFSimHits
        ),
        mixPCFHepMC = cms.PSet(
            mixPCFHepMCProducts
        ),
   .............
)

MixCollection

  • Constructor:
        MixCollection(const CrossingFrame<T> *cf, const range bunchRange =range(-999,999));
        MixCollection(std::vector<const CrossingFrame<T> *> cfs, const range bunchRange =range(-999,999));
        

  • methods of MixCollection:
           int size()    - returns the number of objects from signal + secondary source(s)
           int sizePileup()  - returns the number of objects from the secondary source _input_
           int sizeSignal()  - returns number of signal objects
          MixCollection<T>::iterator begin() - returns iterator to first element
          MixCollection<T>::iterator end() - returns iterator to last element
       

  • iterator (usual iterator interface):
         MixCollection<T>::iterator it;
       

  • methods of the iterator:
          int bunch() const              - returns the number of the current bunchcrossing
          bool getTrigger()              - returns =true for signal, =false otherwise
          int getSourceType()          - returns the type of source: pileup=0, cosmics=1, beam halo+ =2, beam halo- =3
       

Validation suite

The validation suite includes two kind of validation: a validation of the mechanics of time shifts and filling of bunch crossings and distributions to check the global behavior of events with pileup (or any other secondary stream) added.

To verify that bunch crossings are correctly filled by the MM, and that values like the vertex/track pointers and times etc are correctly shifted, 3 steps are needed:

  • execute the MM for bunchcrossings -5 to 3, restricting the bunch range to this crossing only, and adding exactly 1 and the same single particle event as used for the signal as pileup event.
  • reread the output of the MM with the analysis module TestSuite( src/TestSuite.cc). This produces a root-file
containing histograms ("histos.root"), which can be looked at interactively with the root macro lookAtHistos.C for example.

The histograms produced for each bunch are:

  • Number of tracks per bunch crossings
  • Number of tracks to vertex indices per bunch crossings
  • Number of vertices per bunch crossings
  • Number of vertices to track indices per bunch crossings
  • Time of flight for the tracker
  • Time of flight for the Ecal
  • Time of flight for the Hcal

Each type of histogram is produced separately for the pileup and for the signal.

To check further the global behaviour of the mixing module results, global distributions are looked at. These are:

  • Number of tracks for the pileup events
  • Number of vertices for the pileup events
  • Particle id for the tracks
  • Sum of the energy deposit into the ECAL Barrel
  • Sum of the energy deposit into the ECAL Endcap

The histograms are produced separately for each bunch crossing.

For the DQM validation we have, especially dedicated for this purpose, MixCollectionValidation module, written by Fabio Cossutti (see Validation/Mixing/src/MixCollectionValidation.cc). The produced histograms are:

  • Number of GenParticle in HepMCProduct
  • Number of SimTrack
  • Number of SimVertex
  • Number of PSimHit
  • Number of PCaloHit
  • Time of PSimHit
  • Time of PCaloHit

To execute the validation suite:

  • In the test-directory, type chmod +x testsuite.sh to make the shell script executable
  • type ./testsuite.sh to execute it. You should find root-files Cum_-5.root ,...,Cum_3.root
  • start a root-session and type .x testsuite.C. You should see a certain number of Canvasses containing histograms

Plots done by the validation:

  • SimTracks: bcr numbers, vertex indices
  • SimVertices: bcr numbers, ToF
  • SimHits, CaloHits: ToF

Review Status

Reviewer/Editor and Date Describe latest features
Main.uberthon - 09 Feb 2006 page last content editor
UrsulaBerthon - 11 Dec 2007 releases 170 and 180
EmiliaBecheva - 3 Sep 2009 new features starting from the release 310
WilliamTanenbaum - 24 Apr 2013 new features starting from the release 600
WilliamTanenbaum - 24 Apr 2013 How to add new types of products to mix
WilliamTanenbaum - 17 Sep 2016 Replace auto_ptr with unique_ptr

Responsible: Main.ebecheva
Last reviewed by: Sudhir Malik- 24 January 2009

Edit | Attach | Watch | Print version | History: r60 < r59 < r58 < r57 < r56 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r60 - 2016-09-17 - WilliamTanenbaum



 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback