Proposal for Handling Coherent Changes in a Configuration

Complete: 4


Purpose of Page

This page presents alternate proposals of handling coherent changes to cmsRun configurations. The presentation is meant to further discussion on the topic.


cmsRun configurations are broken into a broad set of processing steps: HLT, GEN, SIM, RawToDIGI, RECO, DQM, Validation, Analysis, etc. However, within a processing step (or even between processing steps) we can have a series of variations. Some variations are differences in processing between data taking periods ( 2011 vs 2012 vs 2015 ), between machine coditions ( 7TeV vs 8TeV , 50ns vs 25ns , 40 pileup vs 70 pileup ), between input data ( online vs data vs MC , fast sim vs full sim , minimum bias data set vs scouting dataset ), standard performance variations ( tight electron cuts vs loose electron cuts ) as well as different possible future detector redesigns. In all these cases we want to share a common setup for the bulk of the configuration and make it easy to apply a perturbation in a standard way which is guaranteed to modify the configuration in a coherent way.

Proposal: Instances of a Modifier class

We could add new class types to the configuration language which are carry out a configuration modification only if requested. These modifiers could operate on a particular object or the entire process. The modifiers would take as input a key/value pair and use that information to decide exactly what should be changed.

Object Modifier

All configuration objects, e.g. EDProducer or Path, would gain a new addModifier_ method which would take a function as argument.

# Object with default
foo = cms.EDProducer(“FooMaker”,bar = cms.int32(6))

# function that handles modification
def _modify_foo(obj, year=0, **kw):
  if 2017 == year: = 12

Removal of an object from a configuration could also be supported by having the system pass in a 'remover' method as an argument which the modifier could then use

# Object with default
foo = cms.EDProducer(“FooMaker”)

# function that handles modification
def _modify_foo(obj, year=0, remover=None, **kw):
  if 2017 == year:

NOTE: Does removing an object from a configuration really belong with the object or with a higher level cff?

CFF Modifier

If a more systemic change is needed, like adding new producers to a job, this can be accomplished by creating a cms.Modifier object which the Process will load.

# standard configuration

# function that handles modification
def _modify_stuff(process, year=0, **kw):
  if 2017 == year:
     from Some.Place.fastbar_cfi barFast = barFast

modify_stuff = cms.Modifier(_modify_stuff)

Running the modifications

To request a modification to be run, you call the modifyWith method of Process and pass in the key/value pairs. Process will pass the key/value pairs to each object the Process holds and these objects will execute any modifier they hold. Once that is done the Process will pass the key/value pairs to any cms.Modifier object it has picked up from the configuration.

# Do regular setup of process
# switch to a new ‘version’

Multiple keys

This design supports multiple orthogonal keys

# Different subdetectors have different geometries
                   expectedPileup = 200)

The modifiers themselves only have to know about the keys they care about and can ignore the others

# Only cares about tracker
def _modify_tracker_geom(obj,process,trackerGeom=None, **kw):
   if trackerGeom is not None:
In the above the default for trackerGeom is required to handle the case where that key is not requested by the user. The **kw 'eats' any key/value pairs passed to the function for which the function wishes to ignore.

Design attributes




C++ ParmeterSet validation compatibility

Affect on clones

Proposal: Modifier Singletons

Edit | Attach | Watch | Print version | History: r4 < r3 < r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r1 - 2013-05-21 - ChrisDJones
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback