Proposal for Handling Coherent Changes in a Configuration

Complete: 4

Contacts

Purpose of Page

This page presents alternate proposals of handling coherent changes to cmsRun configurations. The presentation is meant to further discussion on the topic.

Introduction

cmsRun configurations are broken into a broad set of processing steps: HLT, GEN, SIM, RawToDIGI, RECO, DQM, Validation, Analysis, etc. However, within a processing step (or even between processing steps) we can have a series of variations. Some variations are differences in processing between data taking periods ( 2011 vs 2012 vs 2015 ), between machine coditions ( 7TeV vs 8TeV , 50ns vs 25ns , 40 pileup vs 70 pileup ), between input data ( online vs data vs MC , fast sim vs full sim , minimum bias data set vs scouting dataset ), standard performance variations ( tight electron cuts vs loose electron cuts ) as well as different possible future detector redesigns. In all these cases we want to share a common setup for the bulk of the configuration and make it easy to apply a perturbation in a standard way which is guaranteed to modify the configuration in a coherent way.

Proposal: Instances of a Modifier class

We could add new class types to the configuration language which are carry out a configuration modification only if requested. These modifiers could operate on a particular object or the entire process. The modifiers would take as input a key/value pair and use that information to decide exactly what should be changed.

Object Modifier

All configuration objects, e.g. EDProducer or Path, would gain a new addModifier_ method which would take a function as argument.

# Object with default
foo = cms.EDProducer(“FooMaker”,bar = cms.int32(6))

# function that handles modification
def _modify_foo(obj, year=0, **kw):
  if 2017 == year:
     obj.bar = 12
foo.addModifier_(_modify_foo)

Removal of an object from a configuration could also be supported by having the system pass in a 'remover' method as an argument which the modifier could then use

# Object with default
foo = cms.EDProducer(“FooMaker”)

# function that handles modification
def _modify_foo(obj, year=0, remover=None, **kw):
  if 2017 == year:
     remover()
foo.addModifier_(_modify_foo)

NOTE: Does removing an object from a configuration really belong with the object or with a higher level cff?

CFF Modifier

If a more systemic change is needed, like adding new producers to a job, this can be accomplished by creating a cms.Modifier object which the Process will load.

# standard configuration
...

# function that handles modification
def _modify_stuff(process, year=0, **kw):
  if 2017 == year:
     from Some.Place.fastbar_cfi barFast
     process.bar = barFast

modify_stuff = cms.Modifier(_modify_stuff)

Running the modifications

To request a modification to be run, you call the modifyWith method of Process and pass in the key/value pairs. Process will pass the key/value pairs to each object the Process holds and these objects will execute any modifier they hold. Once that is done the Process will pass the key/value pairs to any cms.Modifier object it has picked up from the configuration.

# Do regular setup of process
process.load(“Special/Foo/foo_cfi”)
...
# switch to a new ‘version’
process.modifyWith(year=2017)

Multiple keys

This design supports multiple orthogonal keys

# Different subdetectors have different geometries
process.modifyWith(hcalGeom=”bold”,
                   trackerGeom=”radical”,
                   expectedPileup = 200)

The modifiers themselves only have to know about the keys they care about and can ignore the others

# Only cares about tracker
def _modify_tracker_geom(obj,process,trackerGeom=None, **kw):
   if trackerGeom is not None:
      ...
In the above the default for trackerGeom is required to handle the case where that key is not requested by the user. The **kw 'eats' any key/value pairs passed to the function for which the function wishes to ignore.

Design attributes

Maintenance

Discoverability

Traceability

C++ ParmeterSet validation compatibility

Affect on clones

Proposal: Modifier Singletons

Edit | Attach | Watch | Print version | History: r4 < r3 < r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r1 - 2013-05-21 - ChrisDJones
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback