Framework Release Notes for 3_1_X

These notes describe the differences in the framework between 3_1_X and 2_2_X that are visible to users.

New functionality

EDM

A PtrVector can now be used with Views. This completes the functionality that we can provide to migrate RefToBaseVector to PtrVector. We would like to encourage users to do this migration as RefToBase and associated classes are deprecated.

Parameter Set

  • Added new parameter types EventRange, VEventRange, LuminosityBlockRange, VLuminosityBlockRange, ESInputTag, and VESInputTag.

  • Added new public member functions of ParameterSet class:

    • getParameterSet()
    • getUntrackedParameterSet(),
    • getParameterSetVector(),
    • getUntrackedParameterSetvector(),

which return parameter sets or parameter set vectors by const reference instead of by value. These types can be large so this is a performance improvement.

  • getParameterAsString() is a new utility function that can return the parameter in string form.

  • copyFrom() is another utility function that copies a parameter from another parameter set.

Parameter Set Verification

This is the first release in which a new FWCore sub-system for Parameter Set Verification appears. It is being developed to solve two problems with the Parameter Set system that need addressing in the long term.

  1. Ease the task of validating configurations. At the moment there is no "spell checker" for parameter set names. If you use the wrong name of a parameter there is no way for a module writer to warn you that it doesn't understand something you were trying to tell it. There is also no way of specifying what the legal space of values for a parameter are. This is an opt-in system. The validation system is designed so that it can be deployed incrementally, one module at a time. If the fillDescription function for a module has not been defined, then the validation will do nothing for that module and the Parameter Set system will behave as it has in all earlier CMSSW releases.
  2. Reduce the size of large configurations, like those needed to configure the HLT event processing applications.

This is a short summary of the new system:

Each module can describe the parameters it allows or requires to be in its configuration. This description will appear in a function named fillDescriptions in the plugin's C++ code. When this function has been implemented, several things will be possible.

1. cmsRun will validate the configuration for each module at the beginning of a job. If the validation determines that a parameter required by the description is missing, then it will insert the parameter into the configuration if it has a default. For other types of configuration errors, validation will throw an exception. For example, validation will throw if a parameter is in the configuration that is not in the description (protection against spelling errors). 1. The executable edmPluginHelp will printout human readable documentation about the configuration of a module. 1. CFI python files can be automatically generated based on the information in the description. This will be part of the build and can also be run standalone with the edmWriteConfig executable. 1. Currently, the system is supported only for modules(i.e. EDProducers, EDFilters, EDAnalyzers, or OutputModules). It will eventually be supported for other types of plugins, such as sources or services.


FWLite

A member function, getTFile(), was added to fwlite::Event and fwlite::ChainEvent to get the associated TFile.

A MultiChainEvent class was implemented to allow FWLite to read from two input files (i.e. the "one file two file" feature). Not yet accepted in 3_1_X.

A command line parser and histogram storage was provided for those wishing to write stand-alone tools to use FWLite. Backported to 2_2_X. Not yet accepted in 3_1_X.


MessageLogger and related items:

The MessageLogger is now single threaded by default. A multithreaded version can still be specified as a cmsRun option (-t or --multithread)

A new producer, the LogErrorHarvester, was added. If any error or warning messages are issued while processing an event, this producer puts the current collection of error summary entries into the event.

The symbol for messages produced by LogDebug was changed from %MSG-! to %MSG-d.

Examples of configuration files for use of the Message Logger were added to FWCore/MessageLogger/examples.


ROOT/EDM File IO in general

If fast cloning must be disabled for any reason, a LogInfo message will be issued indicating all the reasons that fast cloning was not possible.


PoolSource

The default for the "duplicateCheckMode" parameter was changed from "checkEachRealDataFile" in 2_2_X to "checkAllFilesOpened" in 3_1_X. By default, duplicate events will be checked for in all input files across file boundaries.

The "eventsToProcess" parameter was modified to be a vector of EventID ranges, rather than a vector of EventIDs.

The new "eventsToSkip" parameter was added. It is a vector of EventID ranges.

The new "lumisToProcess" parameter was added. It is a vector of LuminosityBlockID ranges.

The "lumisToSkip" parameter was modified to be a vector of LuminosityBlockID ranges, rather than a vector of LuminosityBlockIDs.

The new "needSecondaryFileNames" parameter (a bool) was added to guarantee that the "secondaryFileNames" parameter is used and non-empty when required.

Added full support for environment variable expansion and tilde expansion in physical file path names.

The new "branchesMustMatch" parameter, if set to "strict" allows merging only if the product branches of all input files match. (also backported to 2_2_X).


PoolOutputModule

The following configurable parameters are new for 3_1_X:

The new "dropMetaData" parameter allows the user to drop all or some of the per product per event metadata from the output files. It replaces the less functional "dropMetaDataForDroppedBranches" parameter that was supported in 2_2_X.

In 2_2_X, the ROOT split levels and basket sizes of branches in ROOT/EDM input files were ignored in determining the split levels and basket sizes of copied branches in output files. In 3_1_X, by default, the output file split levels and basket sizes of branches copied from an input file are the same as in the input file. The new "overrideInputFileSplitLevels" parameter can be used to override the default and obtain the old behavior.

The new "sortBaskets" parameter (not yet accepted in 3_1_X) can be used to set the order of baskets when branches are fast cloned. The value used can have performance implications, which are still under study.

The behavior of the existing "splitLevel" parameter has been modified slightly from 2_2_X. A value of "0" should be used to default to non-split products in 3_1_X. In 2_2_X, the correct value for this purpose was "1", because of the Wrappers. In 3_1_X, the Wrappers are accounted for internally.


Streamer

Some redundant information was removed from the file header and from streamed products in streamer files, to decrease the size of streamer files.

Added "inputFileTransitionsEachEvent" parameter to streamer input source. (also backported to 2_2_X) This tells the Finite State Machine to treat each event as an independent file. This parameter is on by default for the online HTTP stream reader and off by default for other streamer input sources.

Added support for refactored storage manager (also backported to 2_2_X).


Stand alone tools

edmFileUtil
Option for Adler32 check sum added.


Services

New "CPU" service added for CPU information.

Enhanced information available from Memory and Timing services.


Modules

Enhanced printout from EventContentAnalyzer.


ROOT/EDM file size reduction

The provenance/metadata has been radically redesigned to eliminate redundancy and reduce file size. The following are some of the main changes:

1) The new "dropMetaData" parameter for PoolOutputModule allows the user to drop all or some of the per product per event metadata.

2) The ParameterSet registry will store nested parameter sets by reference (i.e. by ID) rather than by value.

3) The hash values (i.e. ID's) of registry entries will not be written to ROOT/EDM files, except in the case of the ParameterSet registry.

4) Unreferenced entries in the Parentage Registry will not be written to ROOT/EDM files.

5) Redundant data in the Product Registry will not be written to ROOT/EDM files.


Performance and memory improvements

Remove unnecessary class templating. In a few cases, class templates were used for obsolete reasons, so the templates could be converted to simple classes.

Remove unnecessary function template bloat. Many member functions of class templates that did not depend on template parameter(s) were modified to become member functions of a non-templated base class instead.

Remove unnecessary inlining of destructors by specifying non-inlined explicit destructors where desired.

Where this was a major problem, remove unnecessary inlining of char* to string conversions by providing non-inlined functions that take char* arguments directly. This was a major problem for getParameter() and getUntrackedParameter(), which typically get passed char* arguments (C literal strings).

Modified internal Parameter Set code to eliminate returning of strings by value whenever possible.

The duplicate event checker was optimized for performance and memory usage.

Numerous performance improvements for get* calls, in particular, for getByLabel(), and even more so, for getByLabel(InputTag).

Performance improvement for Event::put() call.

-- WilliamTanenbaum - 13 Jul 2009 -- ElizabethSextonKennedy - 09 Aug 2009

Edit | Attach | Watch | Print version | History: r4 < r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r4 - 2009-08-10 - ElizabethSextonKennedy
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback