Parameters for Modules

Complete: 3

Goal of this page

This page lists and describes configurable parameters in important EDM modules.

The options Parameter Set

One can define a parameter set named options at the top level that is attached directly to the process object which will affect event processing for all modules. All the parameters within "options" are optional as is the parameter set itself. For example, it might look like this in a top level configuration file.
process.options = cms.untracked.PSet(
  fileMode = cms.untracked.string('FULLMERGE')
)
name type description
fileMode untracked string The four possible values are FULLMERGE, FULLLUMIMERGE, MERGE, and NOMERGE. The default mode is FULLMERGE. See additional comments on these modes below.
wantSummary untracked bool If true, a trigger summary report is written at the end of the job (to the MessageLogger FwkSummary category, which goes to std::cerr with the MessageLogger defaults). Default is false.
allowUnscheduled untracked bool If true, the Framework will execute Producer modules which were declared in the configuration but not scheduled in a path and which produce products requested by other modules (warning, this feature is fully implemented but has not been used much in CMS yet, please report any bugs you notice). Defaults to false.
makeTriggerResults untracked bool An obsolete parameter. Has no effect but as a matter of cleanup should be deleted from any configurations where it occurs.
The values the next five parameters hold are exception categories. See additional comments below.
Rethrow untracked vstring If an exception with this category is thrown, Event processing stops immediately. If caught, exceptions are rethrown so that they rise to the top level, then an error message prints, and the process stops. There is an attempt to call endLuminosityBlock, endRun, close open files, and perform other cleanup (although each cleanup step is attempted only once and may also fail).
SkipEvent untracked vstring If an exception with this category is thrown, it is caught and not rethrown. Processing is immediately stopped for the current path and no other regular paths are executed. TriggerResults is inserted into the Event (in TriggerResults the current path has the status Exception and paths not run have the status Ready). EndPaths are run, because this is necessary to keep the ROOT "Events" tree balanced in the event that fast cloning is taking place. Processing then proceeds normally with the next event or whatever comes next. However, see notes 1 and 2 below.
FailPath untracked vstring If an exception with this category is thrown, it is caught and not rethrown. The current regular path is recorded as failed in TriggerResults as if a filter had returned false (unless the filter was vetoed, in that case true) and stopped processing on that path in the same position as the module that threw the exception. No more modules are executed on the path. Event processing continues with the next path. If the module is encountered again on a different path for the same event, it will not be rerun and the same exception will be rethrown and handled the same way. However, see notes 1 and 2 below
FailModule untracked vstring If an exception with this category is thrown, it is caught and not rethrown. Rather, the module is recorded as failing, and processing proceeds with the next module on the same path or endpath. However, see note 1 below.
IgnoreCompletely untracked vstring An attempt is made to ignore the exception and proceed. As this is not possible for some exceptions, the exception may still cause the module to fail. However, see note 1 below.

The fileMode parameter exists only to control memory usage related to merging of Run and LuminosityBlock products. Run and LuminosityBlock products can have a method named mergeProduct which allows the Framework to automatically merge products related to the same Run or LuminosityBlock from different input files into a single product for each output file. For example, a Run product might contain histograms related to a Run that are summed.

  • FULLMERGE mode. Runs and Luminosity blocks in the input files need not be in order or contiguous. Products in each Run or LuminosityBlock will be automatically merged. Because the order is arbitrary, this merging requires all these products be cached in memory until the next time that all output files are closed. The memory used is proportional to the size of the products and number of Runs or LuminosityBlocks. The problem with this mode is memory can be exhausted if the total size of the products is large.
  • FULLLUMIMERGE mode. Requires that each Run be contiguous in the input files. If this is not the case, an exception will be thrown. Luminosity blocks in the same Run need not be in order or contiguous. Products in each Run or LuminosityBlock will be automatically merged. Only products in the current run need be cached, so this mode is less likely to exhaust memory. Note that within a single input file, processing always occurs in order sorted by Run. It is only when multiple input files are processed that contiguity of each Run becomes an issue. In some cases, it is impossible to order the input files so that each Run is contiguous. For example, there may be two input files that each contain events from both run 1 and run 2.
  • MERGE mode. Each Run and LuminosityBlock must be contiguous in the input files. If this is not the case, an exception will be thrown. Products in each Run and LuminosityBlock will be automatically merged. Caching is not necessary, so there is no memory problem. Note that within a single input file, processing always occurs in order sorted by Run and LuminosityBlock. It is only when multiple input files are processed that contiguity of each Run and each luminosityBlock becomes an issue. In some cases, it is impossible to order the input files so that each Run and LuminosityBlock is contiguous. For example, there may be two input files that each contain events from both run 1 and run 2.
  • NOMERGE mode. Each output file is closed every time a new input file is encountered. Products are not merged. Therefore, there is no need to cache them and no memory problem. One necessary side effect of NOMERGE mode is that endLuminosityBlock, endRun, beginRun, beginLuminosityBlock all are called each time a new input file is encountered, even if the run number and luminosity block number do not change.

There is a simple alternative to using the fileMode. If the Run and LuminosityBlock products using the memory are "dropped on input", then the memory problem will be avoided. There may be situations where this is the only alternative.

NOTE 1: The behavior after exceptions described above only applies during a call to a user module or source function while processing an Event (InputSource::readEvent, EDProducer::produce, EDFilter::filter, EDAnalyzer::analyze or OutputModule::write). At all other times the behavior is similar to the Rethrow behavior described above.

NOTE 2: The behavior described above for SkipEvent and FailPath does not apply if the exception is thrown while processing an event in a module on an EndPath. In this case, SkipEvent and FailPath behave as FailModule. This is because all modules on an EndPath are considered independent.

The default for all exception categories is Rethrow except the following categories default to SkipEvent: "ProductNotFound", "InvalidReference", "NullPointerError", "EventTimeout", "EventCorruption", and "NotFound" (This is correct as of the time this documentation was written, look in FWCore/Framework/src/Actions.cc for the actual defaults for a given release).

SWGuideInputSources

Generated Input Sources (e.g. EmptySource)

name type description
firstRun untracked uint32 first run number to generate. Defaults to 1.
firstLuminosityBlock untracked uint32 first luminosity block number to generate. Defaults to 1.
firstEvent untracked uint32 first event number to generate. Defaults to 1.
numberEventsInRun untracked uint32 number of events to generate per run. Defaults to maxEvents.
numberEventsInLuminosityBlock untracked uint32 number of events to generate per lumi block. Defaults to maxEvents.
firstTime untracked uint32 value before first event. Defaults to 1.
timeBetweenEvents untracked uint32 increment before each Event. Defaults to 1.
eventCreationDelay untracked uint32 real-time delay between events, in microseconds. Defaults to 0.
processingMode untracked string "Runs" (i.e. don't process lumis or events), "RunsAndLumis" (i.e. don't process events), or "RunsLumisAndEvents". defaults to "RunsLumisAndEvents"

External Input Sources (e.g. Test Beam sources)

name type description
firstRun untracked uint32 first run number to generate. Defaults to 1.
firstLuminosityBlock untracked uint32 first luminosity block number to generate. Defaults to 1.
firstEvent untracked uint32 first event number to generate. Defaults to 1.
numberEventsInRun untracked uint32 number of events to generate per run. Defaults to maxEvents.
numberEventsInLuminosityBlock untracked uint32 number of events to generate per lumi block. Defaults to maxEvents.
firstTime untracked uint32 value before first event. Defaults to 1.
timeBetweenEvents untracked uint32 increment before each Event. Defaults to 1.
eventCreationDelay untracked uint32 real-time delay between events, in microseconds. Defaults to 0.
processingMode untracked string "Runs" (i.e. don't process lumis or events), "RunsAndLumis" (i.e. don't process events), or "RunsLumisAndEvents". defaults to "RunsLumisAndEvents"
fileNames untracked vstring file names, URL's or other input entities. Required

PoolSource

name type description
fileNames untracked vstring physical or logical file names. Required
processingMode untracked string "Runs" (i.e. don't process lumis or events), "RunsAndLumis" (i.e. don't process events), or "RunsLumisAndEvents". defaults to "RunsLumisAndEvents"
secondaryFileNames untracked vstring secondary physical or logical file names. defaults to empty
needSecondaryFileNames untracked bool throw if secondaryFileNames is not specified or empty. defaults to false
parametersMustMatch untracked string match requirement for parameter values for multiple input files. "permissive" or "strict". defaults to "permissive"
branchesMustMatch untracked string match requirement for branches for multiple input files. "permissive" or "strict". defaults to "permissive"
setRunNumber untracked uint32 overrides run number in file (usable only with generated data). defaults to 0 (no override)
skipBadFiles untracked bool skip missing or inaccessible files. defaults to false
skipEvents untracked uint32 Number of initial events to skip. Does not skip Runs or LuminosityBlocks. Does not count as skipped events, events that would have been skipped for some other reason. Defaults to 0
eventsToSkip untracked VEventRange Vector of EventRanges to skip. Skips the Events in those EventRanges. Defaults to empty vector
lumisToSkip untracked VLuminosityBlockRange Vector of LuminosityBlockRanges to skip. Skips the LuminosityBlocks and the Events in those LuminosityBlockRanges. Defaults to empty vector
eventsToProcess untracked VEventRange Event ranges to process. Other events are not processed. Within a specific input file, only Runs and LuminosityBlocks associated with events that are both in ranges on the list and in that input file are processed. Default is an empty vector and this parameter is ignored.
lumisToProcess untracked VLuminosityBlockRange LuminosityBlock ranges to process. Other luminosity blocks, and the events in them, are not processed. Within a specific input file, only Runs associated with luminosity that are both in ranges on the list and in that input file are processed. Default is an empty vector and this parameter is ignored.
noEventSort untracked bool Events are processed in order of run number and luminosity block number always. If this parameter is true, then within a luminosity block the events will be processed in the order they appear in the ROOT file TTree. If this parameter is false, then within a luminosity block the events will be processed in event number order. Defaults to false.
duplicateCheckMode untracked string Four possible values: "noDuplicateCheck", "checkEachFile", "checkEachRealDataFile", and "checkAllFilesOpened". In the first mode, no duplicate checking is done. In the other modes, when a duplicate Event is found it is skipped. An event is duplicate if it has the same run number, lumi number, and event number as an event already encountered. In mode "checkEachFile", each event is compared with the other events in the same input file. In the "checkEachRealDataFile", this is done only for real data files, nothing is done for simulation. In the final mode, events are compared with all other events in all the input files. The default is "checkAllFilesOpened".
firstRun untracked uint32 Runs, LuminosityBlocks, and Events associated with run numbers less than firstRun are not processed. Defaults to 1.
firstLuminosityBlock untracked uint32 LuminosityBlocks and Events associated with firstRun and with luminosity block number less than firstLuminosityBlock are not processed. Defaults to 1.
firstEvent untracked uint32 Events associated with firstRun and with event number less than firstEvent are not processed. Defaults to 1.
inputCommands untracked vstring Used to drop selected branches on input. Defaults to "keep *"
dropDescendantsOfDroppedBranches untracked bool If a branch is dropped on input, drop other branches derived from this branch. Defaults to true.

See Example 6 in SWGuidePoolInputSources for an example of how to use inputCommands to implement drop on input.

inputCommands was not supported prior to CMSSW_2_1_0.

processingMode was not supported prior to CMSSW_2_1_2.

dropDescendantsOfDroppedBranches was not supported prior to CMSSW_2_2_1.

duplicateCheckMode was not supported prior to CMSSW_2_2_0. Prior to CMSSW_3_0_0_pre9, the default was "checkEachRealDataFile". Prior to 3_1_0_pre2 the check was based only on run number and event number (lumi number was ignored).

noEventSort was not supported prior to CMSSW_2_1_5.

eventsToProcess was not supported prior to CMSSW 2_0_0. Prior to CMSSW_3_1_0_pre2, it was a VEventID, rather than a VEventRange.

eventsToSkip was not supported prior to CMSSW 2_0_0. Prior to CMSSW_3_1_0_pre2, it was a VEventID, rather than a VEventRange.

secondaryFileNames was not supported prior to CMSSW_2_0_0.

needSeconfaryFIleNames was not supported prior to CMSSW_3_1_0_pre4

parametersMustMatch and branchesMustMatch See SWGuideMultipleFiles for more information

parametersMustMatch was called fileMatchMode prior to CMSSW_2_2_4

branchesMustMatch was not supported prior to CMSSW_2_2_4.

setRunNumber was not supported prior to CMSSW_1_7_0.

skipBadFIles was not supported prior to CMSSW_1_8_0.

firstLuminosityBlock was not supported prior to CMSSW_1_8_0.

lumisToProcess was not supported prior to CMSSW_3_1_0_pre2.

lumisToSkip was not supported prior to CMSSW_2_0_0. Prior to CMSSW_3_1_0_pre2, it was a VLuminosityBlockID, rather than a VLuminosityBlockRange.

See also:

SWGuidePoolInputSources - How to specify a POOL/ROOT file as a source of events.

SWGuideMultipleFiles - How to read multiple POOL/ROOT files in one job.

OutputModules

PoolOutputModule

name type description
fileName untracked string output file or URL. Required
catalog untracked string output catalog name. Default: file:PoolFileCatalog.xml
logicalFileName untracked string logical file name. Defaults to empty string.
compressionLevel untracked int32 ROOT compression level (0-9). Default: 7.
basketSize untracked int32 default ROOT buffer size in bytes. Default: 16384.
splitLevel untracked int32 default ROOT split level (0-99). Default: 99.
overrideInputFileSplitLevels untracked bool for copied branches, override ROOT split levels and basket sizes in input files. Default: false
maxSize untracked int32 Max file size before starting new file, in kilobytes. Default: ~2TB.
fastCloning untracked bool If false, fast copying is disabled. Default: true.
sortBaskets untracked string "sortbasketsbyoffset", "sortbasketsbyentry" or "sortbasketsbybranch". Default: "sortbasketsbyoffset". Passed directly to ROOT when the Events tree is fast cloned.
dropMetaData untracked string "NONE", "DROPPED", "PRIOR", or "ALL" Default: "NONE".
outputCommands untracked vstring Used to drop selected branches. Defaults to "keep *"
SelectEvents untracked PSet Used to select only events satisfying certain triggers. See Paths and trigger bits -

The outputCommands and SelectEvents parameters are supported for all output modules, not just PoolOutputModule.

For a more detailed explanation of outputCommands, see SWGuideSelectingBranchesForOutput - How to configure output modules to write specific branches

If dropMetaData is "ALL", all per product per event metadata will be dropped. If dropMetaData is "PRIOR", all per product per event metadata for products produced in prior processes will be dropped. If dropMetaData is "DROPPED", all per product per event metadata for dropped products produced in prior processes will be dropped.

sortBaskets was not supported prior to CMSSW_3_2_0. Its value is passed directly to ROOT, and controls the order of ROOT baskets output when the Events tree is fast cloned. Performance studies will be done to determine the optimal default value.

dropMetaData was not supported prior to CMSSW_3_0_0_pre6. The value of "DROPPED" was not supported prior to CMSSW_3_1_0_pre5.

In release CMSSW_2_2_0, setting the now obsolete bool parameter dropMetaDataForDroppedData to true was roughly equivalent to dropMetaData being set to DROPPED.

The maxSize parameter is checked only when a primary input file is closed. So, events from the same input file will always go into the same output file, regardless of the value of maxSize.

overrideInputFileSplitLevels was not supported prior to CMSSW_3_1_0_pre11. Before this prerelease, the split level of a branch in the input file was always ignored (i.e. overridden) in determining the split level of the branch when copied to an output file. Now, by default, the split level of copied branches remain unchanged.

The catalog and logicalFileName parameters are passed to the job report, and are otherwise unused.

See also Paths and trigger bits - How to configure output modules to write specific events

EDProducers

EDAnalyzers

EventContentAnalyzer

This module prints to standard out a list of all products within the Event. A 'verbose' option will actually print the contents of the requested products.
name type description
indentation untracked string string prepended to output (defaults to "++")
verbose untracked bool when true it prints the contents of Event products. Default is false.
verboseForModuleLabels untracked vstring prints the contents of the products made by those modules. If 'verbose=true' and this is empty, print contents of all products. Default is empty.
verboseIndentation untracked string string used to indent verbose information (defaults to " ")

EventSetupRecordDataGetter

This module can be configured to get any data from any EventSetup Record.
name type description
verbose untracked bool when true it prints a statement for each successful get. Default is false.
toGet VPSet holds records and data to get

Parameters expected in toGet parameter

name type description
record string name of the record holding the data
data vstring names of the data types to get from the record. If a label is needed, give the type followed by '/' and then ending with the label.

EDFilters

ESSources

EmptyESSource

This module creates empty EventSetup Records using the IOVs provided.
name type description
recordName string name of the Record which should be created
iovIsRunNotTime bool ?
firstValid vuint32 a list of the run numbers for the start of each IOV transition. Set to '1' if you want valid for all time

Services

Tracer

Prints 'tracing' information to standard out which tells you want cmsRun is doing.
name type description
indention untracked string string prepended to tracing output (defaults to "++")

Review Status

Reviewer/Editor and Date (copy from screen) Comments
WilliamTanenbaum - 22 jun 2009 updating
WilliamTanenbaum - 05 jun 2009 updating
WilliamTanenbaum - 30 Apr 2009 updating
WilliamTanenbaum - 09 Apr 2009 updating
WilliamTanenbaum - 20 Jan 2009 updating
WilliamTanenbaum - 15 Jan 2007 page last content editor
JennyWilliams - 31 Jan 2007 editing to include in SWGuide

Responsible: WilliamTanenbaum
Last reviewed by: Reviewer

Edit | Attach | Watch | Print version | History: r74 | r53 < r52 < r51 < r50 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r51 - 2010-07-06 - WilliamTanenbaum



 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback