Configuration Validation and Help

Goal: We need a way for users to determine the allowed parameters for a module and to enforce that only those allowed parameters are passed to the module (no more accidently mistyping parameter names and not noticing). The configuration validation system described on this page solves these problems.

Introduction

Each plugin can describe the parameters it allows or requires to be in its configuration. This description will appear in a function named fillDescriptions in the plugin's C++ code. When this function has been implemented, several things will happen automatically.

  • cmsRun will validate the configuration for each plugin at the beginning of a job. If the validation determines that a parameter required by the description is missing, then it will insert the parameter into the configuration if it has a default. For other types of configuration errors, validation will throw an exception. For example, validation will throw if a parameter is in the configuration that is not in the description (protection against spelling errors).
  • The description of the parameters that a plugin requires or allows will be accessible from the executable edmPluginHelp in a human readable format.
  • CFI python files will be generated automatically after giving the command "scram b". The python files are generated along with the libraries and other things created as part of the build. These python files are distributed as part of standard releases and will be built in local areas if a package is checked out. Also you can generate the python files manually with the edmWriteConfigs executable.
  • This part is not implemented yet, but we imagine a C++ interface that allows other applications to access the details of the description.

This validation system is designed so that it can be deployed incrementally, one plugin at a time. If the "fillDescriptions" function for a plugin has not been defined, then validation will do nothing for that plugin.

The next two sections contain all the information needed for the majority of CMS plugins. Some readers may want to stop there and only use the rest of this TWIKI page as a reference when needed.

Describing the configuration in C++

You can describe the configuration parameters a plugin can use by adding a single function to the C++ code of the plugin. The declaration of this function in the header file of the class will always look like this:

namespace edm {
   class ConfigurationDescriptions;
}
class MuonProducer : public edm::EDProducer {
...
public:
   static void fillDescriptions(edm::ConfigurationDescriptions & descriptions);
...
};

The class name MuonProducer is just used as an example. It needs to be replaced with the C++ class name for the plugin. The declaration of the function is the same for EDProducer's, EDFilter's, EDAnalyzer's, OutputModule's, InputSource's, ESSource's, ESProducer's and Service's.

An Example

Let's assume this hypothetical MuonProducer module could use 3 parameters. The first parameter is named "algorithm", has type int, and is tracked. The second parameter is named "ptCut", has type double, and is tracked. The third parameter is named "debugLevel", has type unsigned int, and is untracked. In this case, the fillDescriptions function should be defined as follows:

#include "FWCore/ParameterSet/interface/ConfigurationDescriptions.h"
#include "FWCore/ParameterSet/interface/ParameterSetDescription.h"

void MuonProducer::fillDescriptions(edm::ConfigurationDescriptions & descriptions) {
   edm::ParameterSetDescription desc;
   desc.add<int>("algorithm", 1);
   desc.add<double>("ptCut", 20.0);
   desc.addUntracked<unsigned int>("debugLevel", 1U);
   descriptions.add("produceMuons", desc);
}

During validation, an exception would be thrown and the process would stop if any parameters other than "algorithm", "ptCut", or "debugLevel" were in the configuration. This protects the user against spelling errors and misuse of parameters in the configuration. An exception would also be thrown if a parameter had the wrong type or was tracked when it should have been untracked or vice versa. If any of the three parameters above were missing in the configuration, then the validation algorithm would insert them into the ParameterSet that is passed to the plugin constructor with default values which are provided in the second argument to the "add" function above.

During the build, a CFI file would be automatically generated with the following path (substituting something like slc6_amd64_gcc700 for CurrentArchitecture, and the actual directories holding the associated code for the SubsystemName/PackageName with the filename based on the module label passed as an argument in the last line of the code above):

cfipython/currentArchitecture/SubsystemName/PackageName/produceMuons_cfi.py
This file would contain the following:

import FWCore.ParameterSet.Config as cms

produceMuons = cms.EDProducer('MuonProducer',
  algorithm = cms.int32(1),
  ptCut = cms.double(20),
  debugLevel = cms.untracked.uint32(1)
)

If you gave the following command at the UNIX prompt in a CMSSW working release area, you would get the following output.

% edmPluginHelp -b -p MuonProducer
1  MuonProducer  pluginTestMuonProducer.so
  1.1 produceMuons
    algorithm  int32             1
    ptCut      double            20
    debugLevel untracked uint32  1

Note that the interfaces for adding the types ParameterSet and vector<ParameterSet> are slightly different than in the example above. These are described in the following sections.

Note that the Framework will automatically add extra parameters in special cases. For example "mightGet" is added for producer modules and "appendToDataLabel" is added for EventSetup producer modules. Even though not explicitly appearing in fillDescriptions functions, these special parameters have the same effect as if they were, they affect validation and show up in generated cfi files and the output of edmPluginHelp.

One point above is worth repeating. If any parameters are missing from the python configuration, then during validation the parameters will be inserted into the ParameterSet before it is passed to the plugin constructor. This means that any defaults that reside in other parts of the plugin's code will no longer be used for anything. We recommend these other defaults be deleted because they might cause confusion.

Converting an Existing CFI file into a fillDescriptions Function

There is a Python executable that will convert an existing CFI file into a fillDescriptions function. Run it as follows:

  edmPythonConfigToCppValidation [configuration file name]

This is very useful if you already have CFI file defined and want to create a fillDescriptions function. It could also be useful if you are more familiar with writing CFI functions in Python than writing the syntax used in fillDescriptions functions in C++.

In most simple cases, the output will be exactly what you need. You only need to cut and paste it into the C++ code of your module. But the fillDescriptions functions can describe requirements and relationships between parameters that cannot be expressed in a cfi file. In complex cases, the output may need additional editing to get the desired fillDescriptions function.

Note when you use a CFI file in a configuration passed to cmsRun, a manually written CFI file in the python subdirectory will override a generated CFI file with the same file name, so you probably want to remove the existing CFI file from the CMSSW repository when you create a fillDescriptions function that generates a CFI file with the same name.

Types Allowed for Arguments

In the examples on this TWIKI, a literal string is shown for the argument for the parameter name and some other kinds of names. In almost all these cases, the function will accept either a "char const*" or a "std::string const&".

In the examples on this TWIKI, some simple type like "int" is used for the parameter type to improve readability, but the description code already fully supports all the types listed below. The only one allowed in a python configuration that is not supported at the moment is "SecSource" (and we will probably add support for that as soon as we find the time).

int32
vector<int32>
uint32
vector<uint32>
int64
vector<int64>
uint64
vector<uint64>
double
vector<double>
bool
string
vector<string>
EventID
vector<EventID>
LuminosityBlockID
vector<LuminosityBlockID>
InputTag
vector<InputTag>
ESInputTag
vector<ESInputTag>
FileInPath
LuminosityBlockRange
vector<LuminosityBlockRange>
EventRange
vector<EventRange>
ParameterSet
vector<ParameterSet>

A Plugin Which Allows No Parameters

The simplest case is a plugin that does have any configurable parameters. The fillDescriptions function is implemented as shown below. In this case, the validation will throw an exception if the user tries to include any parameters in the python configuration for this plugin.

#include "FWCore/ParameterSet/interface/ConfigurationDescriptions.h"
#include "FWCore/ParameterSet/interface/ParameterSetDescription.h"

void MuonProducer::fillDescriptions(edm::ConfigurationDescriptions & descriptions) {
   edm::ParameterSetDescription desc;
   descriptions.add("aModuleLabel", desc);
}

ParameterSets that Contain ParameterSets

A ParameterSet can contain other ParameterSet's. This is implemented in the configuration by making the type of a parameter be ParameterSet. One adds one of these parameters to the description in almost the same way as other types of parameters, but there are a few subtle differences. Here is an example:

edm::ParameterSetDescription desc;
 
edm::ParameterSetDescription descNested;
descNested.add<unsigned int>("x", 5);

desc.add<edm::ParameterSetDescription>("nameForDescNested", descNested);

The template parameter of the add function is ParameterSetDescription. It is not ParameterSet. The second argument of the add function is also of type ParameterSetDescription. It describes the nested ParameterSet. This description may include the defaults for the parameters it contains, but it is more than just a default value. This second argument must always be present when adding a parameter of type ParameterSet to the description. For other types of parameters, it is possible to add them without defaults (described below in another section). This is not possible for parameters of type ParameterSet.

Validation will recursively descend into nested ParameterSet's using the provided ParameterSetDescription to validate the contents of the nested ParameterSet from the configuration. If a required nested ParameterSet is missing, first the validation will insert an empty ParameterSet and then use the description to validate that.

ParameterSets that Contain Vectors of ParameterSets

A ParameterSet can contain a vector of ParameterSet's. This is implemented in the configuration by making the type of a parameter be vector<ParameterSet>. One adds one of these parameters to the description using a function named "addVPSet" instead of "add". Here is an example.

#include "FWCore/ParameterSet/interface/ConfigurationDescriptions.h"
#include "FWCore/ParameterSet/interface/ParameterSetDescription.h"
#include "FWCore/ParameterSet/interface/ParameterSet.h"
#include <vector>
...
edm::ParameterSetDescription desc;

edm::ParameterSetDescription validator;
validator.add<int>("x", 7);

std::vector<edm::ParameterSet> vDefaults;
edm::ParameterSet vDefaults0;
vDefaults.push_back(vDefaults0);
edm::ParameterSet vDefaults1;
vDefaults1.addParameter<int>("x", 100);
vDefaults.push_back(vDefaults1);

desc.addVPSet("nameForVPSet", validator, vDefaults);

The second argument to addVPSet is a ParameterSetDescription. This description is used to validate all ParameterSet's that are in the vector being validated. The third argument is a default that is used if the vector is missing and also when writing the CFI file. Note that validation will be run on each element of the default vector if it is ever used, so the second and third arguments must be consistent. In the example above, the 0th element of the default vector is an empty ParameterSet and validation will insert the parameter named "x" with the default value 7. The other element of the default vector already contains "x" so it will pass through validation unmodified.

There is an "addVPSetUntracked" function also that should be used instead of "addVPSet" if the parameter is untracked.

Optional Parameters

All the parameters described in the sections above are required parameters. In addition, there are functions analogous to the functions shown above that can be used to add optional parameters to a ParameterSetDescription. They are:

addOptional
addOptionalUntracked
addVPSetOptional
addVPSetOptionalUntracked

These are used in the same way as the functions above. There is only one difference in behavior when these are used. During validation if a required parameter is missing, then it is either inserted with its default value or an exception is thrown. During validation if an optional parameter is missing, then it is just ignored. Nothing is inserted and nothing is thrown. There are no other differences. Note that both optional and required parameters are written into CFI files in exactly the same manner.

It is the intent of the designers that optional parameters only be used in the case where the behavior of a plugin is different depending on whether a parameter exists in the ParameterSet used to configure the plugin. It is not intended to be used in places where defaults were provided in the code as a second argument to a getUntrackedParameter call. Those should be added to the description as required parameters and the default value should be deleted from the getUntrackedParameter function call.

Parameters Without Default Values

You can add a parameter without a default value as follows:

edm::ParameterSetDescription desc;
desc.add<int>("x");

The only difference between this and the case where a default is provided is eliminating the argument that provides the default. This works in every case described in the sections above, with one exception. When one is describing a ParameterSet nested inside a ParameterSet, one must always provide two arguments.

If no default is provided, then there are three consequences:

  • During validation, if a required parameter is missing and there is no default then an exception is thrown. If a default was provided, then it is inserted.
  • This is indicated in the output of edmPluginHelp.
  • The parameter will be written into the CFI file in a different way. It will appear as follows depending on whether the parameter is tracked or untracked and whether it is required or optional.

p1 = cms.required.int32,
p2 = cms.optional.int32,
p3 = cms.required.untracked.int32,
p4 = cms.optional.untracked.int32

Most of this TWIKI is discussing a level of validation that occurs when C++ code is being executed. There is a separate level of validation that occurs earlier while the Python code is being executed. While the Python is being executed and just before Python data structures are converted into C++ data structures the parameters are checked. A Python exception is thrown if required parameters as shown above have not been set to a value. Optional parameters that have not been set to a value are simply dropped.

If the parameters were set after the import of the generated CFI file, then they will be passed through to the C++ data structures as if they had been declared as follows:

p1 = cms.int32(value1),
p2 = cms.int32(value2),
p3 = cms.untracked.int32(value3),
p4 = cms.untracked.int32(value4)

In addition to "required" and "optional", there is another special Python type called "obsolete". This type will never be written into generated CFI files. During the Python level validation, those are simply ignored and not passed into the C++ ParameterSet objects.

Note that the behavior described above is new with release series 11_0_X. Prior to that parameters with no defaults were not written into CFI files at all.

Adding Comments

You can add comments to the description. These comments show up in the output of the edmPluginHelp utility. They do not affect validation or the CFI files. Comments are not necessary and not required. Here is an example:

     ParameterSetDescription desc;
     desc.addUntracked<bool>("verbose", false)->
     setComment("If true, the contents of products are printed using Reflex.");

And this is part of the output from edmPluginHelp related to the above code:

% edmPluginHelp -p EventContentAnalyzer
...
     verbose
                        type: untracked bool
                        default: False
                        If true, the contents of products are printed using Reflex.
...

Note that the addUntracked function above returns a pointer to a ParameterDescriptionNode. This is the base class for the class that hold the description of a single parameter. But it is also a base class for other things like the logical expressions which are the subject of the following sections. It is possible to call setComment on all of those objects. In addition, the classes ParameterSetDescription and ConfigurationDescriptions also have a setComment function. All of these comments will show up in different places in the output of edmPluginHelp.

There is a lot of flexibility in how these comments can be used. Our current thought is that it is best if they are kept brief, saving long explanations for TWIKI pages or other document formats. Also remember that correcting an error in documentation that is hard coded will not be fast. The correction will not reach users until the next software release is distributed, which could be a long time.

In release 9_3_X another alternative way to set comments will be added. Later sections will describe how a ParameterDescription can be explicitly constructed. In only those cases, one can add the comment as an additional argument to the constructor.

...
edm::ParameterDescription<std::string>("name", "defaultValue", true, edm::Comment("your comment"))
...

NEW A Plugin Module Using Another Helper Plugin NEW

This is a new feature first available in CMSSW_10_3_X_2018-10-04-1100 which should be included in CMSSW_10_3_0 when it is released.

Sometimes a module plugin may load and use one or more helper plugins. In this case, it is desirable that the helper plugin itself contains the description of its configuration parameters. This is good for several reasons. First of all there is just the general principal that a module plugin should not depend on the details of the implementation of a helper plugin it uses. The module plugin should only depend on the interface specified in the helper plugin's base class. Second, there may be many different helper plugin types that can be used and module plugin does not know in advance which will be used. Also, it should be possible to use a completely new type of helper plugin without modifying the module plugin. Third, a helper plugin might be used by many different module plugins and it would be wasteful to duplicate the description in all those different module plugins.

Putting the parameter description in the helper plugin requires special handling, because the compiled libraries containing the helper plugins will not be loaded when validation is started. And the executable should only load the plugin libraries that are actually used. This is not known at compile time. It only known at run time, because it depends on configuration parameters.

There are 6 places where you will need to do something special to use validation for a helper plugin.

  1. in the configuration
  2. in the helper plugin
  3. in fillDescriptions in the module plugin
  4. at factory registration
  5. at plugin registration
  6. in the module plugin constructor

The details of what needs to be done are discussed below.

1. The configuration of the module plugin would look something like the following:

foo = cms.EDProducer(“FooProd”,
    plugin = cms.PSet(
      type = cms.string(“BarHelper”),
      value = cms.int32(5)
    )
    # other module parameters allowed outside the PSet
    ...
)

It is important that there is a separate PSet defined that is contained inside the module's configuration. All the parameters used to configure the helper plugin should be in this separate PSet. This separate PSet should also contain a special parameter of type string whose value is the plugin name of the helper plugin (usually this is the type of the helper plugin or a name associated with that type).

Many of the names used in the example above are just made up names that would be appropriately replaced in an actual configuration. "foo" is the module name. "FooProd" is plugin name of the module. This will work for types other than EDProducer. "BarHelper" is an example plugin name for the helper type. The value parameter would be replaced with the parameters needed to configure the helper. The names "plugin" and "type" could be replaced with different names, but the names used in the configuration must match the names used in the C++ code defining the modules constructor and fillDescriptions function. If the module loaded more than one helper plugin, then the module would need to use a name other than "plugin" so the names would not conflict.

2. The following function must be defined inside the plugin helper class. It has to be a static function returning void named "fillPSetDescription" that takes a reference to a ParameterSetDescription as an argument. The content of the function would be replaced with code adding whatever parameters are appropriate for that helper plugin (the "value" parameter in the example should be replaced).

There is one addition important requirement. The string parameter that contains the plugin name of the helper type should not be included. It gets added to the description automatically.

static void
fillPSetDescription(edm::ParameterSetDescription& iDesc) {
    iDesc.add<int>(“value”,5);
}

3. The fillDescriptions function would need to be defined in the module as follows. If there is a helper plugin, then one needs to create a ParameterSetDescription devoted solely to that helper plugin. The ParameterSetDescription should have exactly one node added to it as its content. The node should be of type PluginDescription. The template parameter of PluginDescription should be the type of the factory that creates the plugin. The first argument of its constructor is the name of the parameter that contains the plugin name of the helper plugin type. The second argument indicates whether or not that helper plugin name is stored in a tracked parameter (false would indicate untracked).

void FooProd::fillDescriptions(edm:ConfigurationDescriptions& oDesc) {

    edm::ParameterSetDescription pluginDesc;   
    pluginDesc.addNode(edm::PluginDescription<HelperFactory>(“type”, true));

    edm::ParameterSetDescription top;
    top.add<edm::ParameterSetDescription>(“plugin”, pluginDesc);
    # other parameters can be added to top
    ...
    oDesc.addDefault(top);
}

The ParameterDescription class has an alternative contructor that takes 3 arguments. The first and last arguments are the same as above. There is an additional argument added in between the other two that allows specifying a default helper plugin name.

4. The factory that creates the creates the helper plugin objects needs to be registered with a special macro. If you are converting an existing helper plugin that does not support validation to support validation, then the macro listed below would replace the macro currently being used to register the factory.

#include “FWCore/ParameterSet/interface/ValidatedPluginFactoryMacros.h”
...
EDM_REGISTER_VALIDATED_PLUGINFACTORY(HelperFactory, “HelperFactory”);

5. The helper plugin type also needs to be registered with a special macro. If you are converting an existing helper plugin that does not support validation to support validation, then the macro listed below would replace the macro currently being used to register the helper plugin.

#include “FWCore/ParameterSet/interface/ValidatedPluginMacros.h”
...
DEFINE_EDM_VALIDATED_PLUGIN(HelperFactory,BarHelper, ”BarHelper”);

6. And finally the helper plugin would need to be created in the constructor of the module plugin in a certain way. Even without validation, helper plugins might be created this way. But to be able to validate the helper plugin's configuration, the helper must be created this way.

auto pluginPSet = pset.getParameter<edm::ParameterSet>("plugin");
   
//  helper_ is a data member, probably a unique_ptr<BarHelperBase>
helper_.reset(HelperFactory::get()->create(pluginPSet.getParameter<std::string>("type"), pluginPSet));

In the example above, a contained ParameterSet is retrieved from the module's ParameterSet. The type of the helper plugin is retrieved from the contained ParameterSet. It must be retrieved from there. Then both the helper plugin name and the contained ParameterSet are passed to the create method of the factory. The ParameterSet then gets passed through to the constructor of the helper plugin C++ class.

One final comment, the above prescription also works with a vector of ParameterSets in a straightforward way.

Let the ParameterSetDescription Allow Anything

It is possible to tell a ParameterSetDescription to allow anything.

  edm::ParameterSetDescription desc;
  desc.setAllowAnything();

When this function has been called illegal parameters that are in the configuration but not in the description will be ignored instead of an exception being thrown. Other than these exceptions not being thrown, validation will proceed normally. Missing parameters will be injected and other types of configuration errors can result in exceptions being thrown.

(Note that prior to release CMSSW_4_4_0, the behavior when setAllowAnything was called was different. In those earlier releases, when this was called validation was skipped entirely and missing parameter injection was not done.)

Multiple Descriptions for One Plugin

It is possible to add multiple ParameterSetDescriptions for one plugin if it is an EDProducer, EDFilter, EDAnalyzer, OutputModule, ESSource, or ESProducer.

#include "FWCore/ParameterSet/interface/ConfigurationDescriptions.h"
#include "FWCore/ParameterSet/interface/ParameterSetDescription.h"

void MuonProducer::fillDescriptions(edm::ConfigurationDescriptions & descriptions) {

   edm::ParameterSetDescription desc;
   desc.add<int>("algorithm", 1);
   descriptions.add("produceMuons", desc);

   edm::ParameterSetDescription desc1;
   desc1.add<int>("debugLevel", 7);
   descriptions.add("debugProduceMuons", desc1);

   edm::ParameterSetDescription desc2;
   desc2.add<double>("ptCut", 100.0);
   descriptions.addDefault(desc2);
}

Each description must be associated with a unique module label that is passed as an argument to the add function. Other than that the descriptions are completely independent. In addition, a default description can be added with the addDefault function. There can only be one default description per plugin.

When validating a plugin's configuration, a description is selected using the following steps.

  • If there is a description whose label exactly matches the label in the configuration, then that description is used for validation.
  • If there is not an exact match and there is a default description, then the default is used.
  • If there is no match and no default, the first description added in the fillDescriptions function is used.
  • If there are not any descriptions, then silently no validation is performed.

A CFI file will be generated for each description that was added with a module label. The module label is used in both the file name and the content of the file. A CFI file is not generated for a default description.

Uniqueness of CFI Filenames

Within a package, the filenames of all the generated CFI files must be unique because the CFI files are kept in the same directory. These filenames are based on the module labels passed to the ConfigurationDescriptions:add function. So these module labels must be unique within a package across the fillDescriptions functions for all the plugins in the package.

For plugins based on template classes, it is necessary to create a function specialization of the fillDescriptions function if a CFI file is generated. In this case, the CFI filenames will conflict if all the template instantiations share the same fillDescriptions function definition.

For plugins based on an inheritance hierarchy and generating CFI files, all derived classes will need to define their own fillDescriptions function. If they share that function from a base class and generate CFI files, then the filenames will conflict.

Sources and Services

Defining a ParameterSetDescription for a service or a source is almost the same as defining one for a plugin that is a module. There are a few minor differences. First of all, sources and services can only have one description associated with them. It can be one description with a name or one default description, but not both. If a name is associated with the description, then a CFI file will be generated. A CFI file will not be generated if it is a default description.

For sources the name passed to the add function must always be "source". Here is an example of how it is done for the PoolSource.

void
PoolSource::fillDescriptions(ConfigurationDescriptions & descriptions) {
  edm::ParameterSetDescription desc;
  ... // add parameters as usual
  descriptions.add("source", desc);
}

For services the name passed to the add function must always be the name of the plugin. Here is an example of how it is done for the Tracer plugin.

void
Tracer::fillDescriptions(edm::ConfigurationDescriptions & descriptions) { 
  edm::ParameterSetDescription desc;
  ... // add parameters as usual
  descriptions.add("Tracer", desc);
}

The python configuration syntax for cmsRun does not allow a source or a service to have a module label. That is why one is not allowed in the description either.

ESSource and ESProducer

Support for parameter set validation for ESSource's and ESProducer's was added in release 3_11_0. ESSource's and ESProducer's are handled almost identically to the other types of modules. There are two differences.

First, for every description for an ESSource or ESProducer, the parameter named "appendToDataLabel" will be automatically added to the description unless it was already explicitly added. If added automatically, it will have type string, be required, be tracked, and have for a default value an empty string. This automatic parameter addition did not get included in release 3_11_0, although it should be included in releases very soon after that.

The other difference is very minor. Most people can just ignore it and will never know the difference, but just to be complete I'll describe it here (I advise most readers to skip the rest of this subsection and not waste their time reading it). In the old configuration language that was used before we started using python to configure cmsRun, an ESSource or ESProducer could be labeled or unlabeled. This was not true for other modules like EDAnalyzer. Even in the old configuration language, they were always labeled. So there are two cases to consider, labeled and unlabeled.

The labeled case is easy. In this case, validation and parameter set descriptions are handled exactly the same for ESSource's and ESProducer's as for other modules.

The unlabeled ESSource's and ESProducer's really had no label in the old configuration language. In python, every object must have a name so the unlabeled ESSource's and ESProducer's actually have a label in python configuration files. We identify this special case by giving the module label the same value as the plugin name. For example,

process.DoodadESSource = cms.ESSource("DoodadESSource")

That is an 'unlabeled' ESSource. So you can see that 'unlabeled' is now a misnomer. Internally and in its interface the parameter set validation system treats this 'unlabeled' case as having module label equal to its plugin name.

The one place this affects the interface is when adding a ParameterSetDescription to a ConfigurationDescriptions object. Here is an example:

...
descriptions.add("DoodadESSource", desc1);
...
descriptions.add("myModuleLabel", desc2);

The first ParameterSetDescription added above has the same name as the plugin and so will be used for validation in the 'unlabeled' case. It is legal to add other descriptions with other labels.

As far as I know the user is never faced with an object that actually does not have a label. You have to look into the internal workings of the ParameterSet class to find anyplace where the label does not have a value equal to the plugin name. This is all hidden from the user.

Automatic module labels from plugin class names

In `10_0_0_pre3` it became possible to use a module label that is automatically derived from the plugin class name. This is useful mostly for templated plugins where a typedef is used as the plugin name. The automatic label can be used with

void MuonProducer::fillDescriptions(edm::ConfigurationDescriptions & descriptions) {
   edm::ParameterSetDescription desc;
   desc.add<int>("algorithm", 1);
   descriptions.addWithDefaultLabel(desc); // in this case the label would be "muonProducer"
}
The addWithDefaultLabel() works otherwise exactly as regular add().

Fixed size arrays

If you know the size of a vector of parameters at compile time there is a utility function that will check the size and return a std::array object containing the contents of the std::vector. If the size does not match the expected size, then an exception will be thrown. This function does not run as part of the validation. It runs later when getting the parameter. See FWCore/ParameterSet/interface/getFixedSizeArray.h. This was added in the 10_1_X release series.

More Complex Features Available for Use in a Description

The next several sections introduce several interesting features that can be used when writing a fillDescriptions function. These sections are intentionally brief and they do not explain all the details related to these features. With your intuition, you might be able to use the features without reading any additional documentation. After the brief sections there is additional documentation that explains in detail how these features work and can be used together.

The "Exclusive Or" Operator

Consider the following description.

edm::ParameterSetDescription desc;
desc.addNode( edm::ParameterDescription<std::string>("x", "11", true) xor 
              edm::ParameterDescription<unsigned int>("y", 11U, true) xor
              edm::ParameterDescription<double>("z", 11.0, true ));

This creates a group of 3 parameters named "x", "y", and "z". When validating a ParameterSet, this requires that exactly one parameter from that group is in the ParameterSet. If more than one of them exists in the ParameterSet, then an exception would be thrown. If none of them exist, then it would attempt to insert the first parameter in the group into the ParameterSet. In this particular case this would always succeed and the ParameterSet would pass validation. If exactly one of these parameters already existed in the ParameterSet, then it would pass validation without anything being inserted.

The "And" Operator

Consider the following description.

edm::ParameterSetDescription desc;
desc.addNode( edm::ParameterDescription<std::string>("x", "11", true) and
              edm::ParameterDescription<unsigned int>("y", 11U, true) and
              edm::ParameterDescription<double>("z", 11.0, true ));

This creates a group of 3 parameters named "x", "y", and "z". When validating a ParameterSet this requires that all three parameters must exist in the ParameterSet. If some or all of them are missing, they will be inserted when a ParameterSet is validated.

The "Or" Operator

Consider the following description.

edm::ParameterSetDescription desc;
desc.addNode( edm::ParameterDescription<std::string>("x", "11", true) or
              edm::ParameterDescription<unsigned int>("y", 11U, true) or
              edm::ParameterDescription<double>("z", 11.0, true ));

This creates a group of 3 parameters named "x", "y", and "z". When validating a ParameterSet this requires that at least one of the three parameters must exist in the ParameterSet. If one or more of them exists, then the ParameterSet will pass validation. If all three are not present, then the first one, "x", will be inserted into the ParameterSet.

A "Switch"

Consider the following description.

edm::ParameterSetDescription desc;
desc.ifValue( edm::ParameterDescription<std::string>("sswitch", "b", true),
              "a" >> edm::ParameterDescription<int>("x", 100, true) or
              "b" >> (edm::ParameterDescription<double>("y1", 101.0, true) and 
                      edm::ParameterDescription<double>("y2", 101.0, true)) or
              "c" >> edm::ParameterDescription<std::string>("z", "102", true) );

The parameter "sswitch" controls which other parameters are required to be in a ParameterSet. If "sswitch" has the value "a", then the parameter "x" is required. If "sswitch" has the value "b", then parameters "y1" and "y2" are both required. If "sswitch" has the value "c", then parameter "z" is required.

Wildcards

Consider the following description.

edm::ParameterSetDescription desc;
desc.addWildcard<int>("*");

A ParameterSet is allowed to have zero or more tracked parameters of type "int" and any name. There is also an "addWildcardUntracked" function if the allowed parameters are untracked instead of tracked.

Allowed Labels

Consider the following description.

edm::ParameterSetDescription desc;
desc.labelsFrom<int>("allowedLabels");

A ParameterSet is required to have a parameter whose name is "allowedLabels" (the argument above). This parameter is a tracked vector of strings. The strings are names of other parameters that are allowed to be in the ParameterSet. These other parameters must have type "int" (the template parameter above) and be tracked.

There is also a "labelsFromUntracked" function if the allowed parameters and vector are untracked instead of tracked.

IfExists

Consider the following description.

#include "FWCore/ParameterSet/interface/ConfigurationDescriptions.h"
#include "FWCore/ParameterSet/interface/ParameterSetDescription.h"
#include "FWCore/ParameterSet/interface/IfExistsDescription.h"
...
edm::ParameterSetDescription desc;
desc.ifExists( edm::ParameterDescription<unsigned int>("x", 11U, true),
               edm::ParameterDescription<std::string>("y", "11", true) );

If parameter "x" exists in a ParameterSet, then parameter "y" is required to also exist. If both do not exist it is OK and this will pass validation without any insertion of missing variables. If parameter "y" exists and parameter "x" does not exist, then "x" will be inserted into the ParameterSet.

Empty Description

Consider the following description.

#include "FWCore/ParameterSet/interface/ConfigurationDescriptions.h"
#include "FWCore/ParameterSet/interface/ParameterSetDescription.h"
#include "FWCore/ParameterSet/interface/EmptyGroupDescription.h"
...
edm::ParameterSetDescription desc;
edm::EmptyGroupDescription emptyGroup;
desc.addNode(emptyGroup);

This is something that requires nothing and allows nothing. When evaluated during the course of validation it always has the value true. By itself it is meaningless, but it can be nested in and combined together with many of the features described above. In that context, it can be useful.

Details Related to the More Complex Features

ParameterDescription

ParameterDescription is the C++ class name for objects that hold the description of a single parameter. Here is how one is created.

edm::ParameterDescription<std::string>("x", "11", true)
The template argument is the type of the parameter. The first function argument of the constructor is the parameter name. The second argument is the default value of the parameter and has the same type as the template argument. The third argument is a boolean that is true if the parameter is "tracked" and false if it is "untracked". There is another constructor that is the same except the argument providing the default value is omitted. When the constructor without a default is used, the parameter does not get written into CFI files and also cannot be inserted into ParameterSet's during validation. If the parameter is required and missing, then an exception gets thrown.

There are two types of parameters where the constructor is different. First, when the parameter type is ParameterSet:

edm::ParameterSetDescription desc;
... content gets added to desc
edm::ParameterDescription<edm::ParameterSetDescription>("x", desc, true)
In this case, the template argument and second argument are ParameterSetDescription instead of being ParameterSet, which is the actual type of the parameter. The second argument is also a more than a default, it is also used to validate the nested ParameterSet and it must always be present. The other special case occurs when the type of the parameter is vector of ParameterSet.
edm::ParameterSetDescription desc;
... content gets added to desc
std::vector<edm::ParameterSet> vectorOfPSets;
... content gets added to vectorOfPSets
edm::ParameterDescription<std::vector<edm::ParameterSet> >("x", desc, true, vectorOfPSets);
In this case, the second argument is used to validate every element of the vector of ParameterSet's. The fourth argument is the default and there is another constructor with the default omitted.

The base class of ParameterDescription is ParameterDescriptionNode. A single parameter is a ParameterDescriptionNode, but it is not the only kind of ParameterDescriptionNode. Because it is a node this works:

edm::ParameterSetDescription desc;
desc.addNode( edm::ParameterDescription<int>("algorithm", 1, true));

This is equivalent to

edm::ParameterSetDescription desc;
desc.add<int>("algorithm", 1);

Operators

The "xor", "and", and "or" operators described above are overloaded C++ operators. You can use the symbol "^" instead of "xor". You can use "||" instead of "or". You can use "&&" instead of "and". They have the precedence of those operators and one can use parentheses to group them.

When filling the description these operators are binary operators. They take two arguments of type "ParameterDescriptionNode const&" or "auto_ptr<ParameterDescriptionNode>" and return a value of type "auto_ptr<ParameterDescriptionNode>". This means that the result of any of the three operators can be fed as an argument to any of the three operators. This allows one to build long expressions. The node created by the operator contains two smart pointers to the nodes passed to it as arguments. Filling the description with a long expression creates a node tree in memory. ParameterDescriptionNode is a base class. The actual class of the object which is pointed to by the auto_ptr returned by the operator is either XORGroupDescription, ORGroupDescription, and ANDGroupDescription. The type of the returned node is the only difference in the behavior of the three operators while filling the description. The difference in the logic for "and", "or", and "xor" does make a difference until the node is used to validate a ParameterSet.

Validation of a ParameterSet is a different operation that occurs later than filling the description. For the "xor" operator, validation is not a binary operation at all. (The analogy between the standard behavior of the operator and its use in this case is not perfect.) Here what happens when an "xor" node is validated.

  • The validation algorithm starts at the top of the node tree and works down.
  • When a node is encountered that is an "xor" node, the algorithm searches down the node tree following both pointers until it encounters a node that is not an "xor" node on every branch. Then all of these nodes are dealt with as group. These could be simple ParameterDescription type nodes or "switch" nodes or "and" nodes or any type of node other than an "xor" node.
  • Then it checks whether each of those nodes evaluates to true or false considering only the parameters that already exist in the ParameterSet and without validating any of those nodes. (You might ask, what does it mean without validating any nodes? It means two things. First, without inserting any parameters. Second, the entire validation process for a ParameterSet records the name of each validated parameter as it is validated. Then at the end the names in the ParameterSet are compared to the list of validated names and if there are extra names an exception is thrown. Without validating a node means the names in that node are not recorded on the list of validated names yet).
  • Then it examines how many nodes evaluated to true.
    • If it finds more than one that is true, it just throws an exception.
    • If it finds one that is true, then the validation method is called on that one node. All the other nodes and whatever parameters they might contain are not validated and should not appear in the ParameterSet unless the selected node validates them also.
    • If it finds that none are true, then calls the validation method on the first node. If this were a single parameter and it had a default, it would be inserted. In this case, after the validation is complete it goes back and tests that the entire set of "xor" related nodes evaluates to true because validating one node might have had side effects on other nodes.
Here are some examples that might make some of the consequences of this procedure more clear. For clarity, I've simplified the notation and not given the full C++ syntax. Let A, B, C, D, and E represent some kind of node, they could be a single parameter in the description.
A ^ B ^ C ^ D ^ E
Exactly one of the parameters A,B,C,D, and E is required to be in the ParameterSet.
(A ^ B) && (C ^ D ^ E)
Exactly one of the parameters A and B is required to be in the ParameterSet and exactly one of the parameters C,D, and E is required to be in the ParameterSet.
A ^ B ^ (C && D) ^ E
Exactly one of the parameters A, B, and E are required to be in the ParameterSet, except that if none of those are then both parameters C and D are required to be in the ParameterSet.
A ^ B ^ C
A ^ (B ^ C)
Both of these expressions have the same identical meaning. The node tree they create is the same. Particularly note that both evaluate false and an exception will get thrown if all 3 parameters exist in the ParameterSet being validated using this node. The validation algorithm would not evaluate B^C as (True^True = False) and then A^False as True^False which is True. The algorithm will evaluate based on all three parameters at the same time and only 1 of the three is allowed.

The "and" operator behaves in a similar way. The difference is that the validation method is called on all the subnodes if the top level "and" node has its validation method called. Note that the testing procedure mentioned above is actually only relevant if the "and" node is optional (discussed later). In that case if all the nodes evaluate to false then none are validated.

The "or" operator also behaves in a similar way. The difference is that in the validation procedure only the nodes tested that evaluated as true have their validation procedure called, unless there are none. If there are none, the first node gets its validation procedure called.

A "Switch" - More Details

Consider this example again

edm::ParameterSetDescription desc;
desc.ifValue( edm::ParameterDescription<std::string>("sswitch", "b", true),
              "a" >> edm::ParameterDescription<int>("x", 100, true) or
              "b" >> (edm::ParameterDescription<double>("y1", 101.0, true) and 
                      edm::ParameterDescription<double>("y2", 101.0, true)) or
              "c" >> edm::ParameterDescription<std::string>("z", "102", true) );

The second argument to ifValue function is a fairly complicated thing. Each case is constructed using operator>> whose left argument identifies the case and matches the type of the switch variable and whose right argument is any kind of a ParameterDescriptionNode. Note that for the second case above, the right argument is an "and" node containing two parameters. These cases are connected with an "or" operator whose arguments are the cases. This is not the same "or" operator discussed in the previous section. It has different types for its arguments and return type. The type use to represent a case does not inherit from ParameterDescriptionNode. The only use of this "or" operator is to collect the cases together to pass them into the switch.

The "ifValue" function shown above is used to insert the switch directly into the ParameterSetDescription. The ParameterSetDescription contains a vector whose elements contain a pointer to a ParameterDescriptionNode. When added directly the switch node goes into that vector. It is also possible to add a switch node to some other type of node, for example as an argument to an "and" operator. In this case, the switch must be constructed explicitly. The arguments to the constructor are the same.

edm::ParameterSetDescription desc;
edm::ParameterSwitch<std::string> switchNode(
              edm::ParameterDescription<std::string>("sswitch", "b", true),
              "a" >> edm::ParameterDescription<int>("x", 100, true) or
              "b" >> (edm::ParameterDescription<double>("y1", 101.0, true) and 
                      edm::ParameterDescription<double>("y2", 101.0, true)) or
              "c" >> edm::ParameterDescription<std::string>("z", "102", true) );
desc.addNode(switchNode);

When validating a ParameterSet, if the value of the switch variable does not match one of the cases in the description, an exception is thrown.

During validation, a node sometimes need to be evaluated for a true or false value without actually validating it. For a ParameterSwitch, it will evaluate to true if its switch variable exists in the ParameterSet. When a required ParameterSwitch is validated and the switch parameter is missing, then it will be inserted. If that parameter has no default and cannot be inserted, then an exception is thrown. Whether it was inserted or not, the node corresponding to the selected case gets validated and nothing is done with the other nodes.

Wildcards - More Details

Consider this example again:

edm::ParameterSetDescription desc;
desc.addWildcard<int>("*");

The wildcard pattern used here is "*". This will match any parameter name in a ParameterSet. Currently, that is the only pattern allowed. We have discussed extending this to support regular expressions or globbing. If there were actually requests for this feature, it would motivate us to implement this faster. At the moment it is fairly low on the priority list.

The function above supports adding the node directly to the ParameterSetDescription, but one can also construct the node directly and add it to some other kind of node, for example as an operator argument or switch case.

edm::ParameterSetDescription desc;
edm::ParameterWildcard<int> wnode("*", edm::RequireExactlyOne, true);
desc.addNode(wnode);

The class ParameterWildcard inherits from ParameterDescriptionNode. In the example above, the template argument is the type which parameter types must match. The first argument is the pattern which parameter names must match. The second argument in an enumeration and there are 3 allowed values.

  • edm::RequireZeroOrMore
  • edm::RequireAtLeastOne,
  • edm::RequireExactlyOne
These are requirements on the number of matches that must exist for the wildcard node to pass validation. Also note that currently, the class ParameterWildcard does not allow defaults. There are never any parameters inserted for a wildcard. If the enum requires matches and they do not exist, an exception is thrown. Finally, the third argument is a boolean. If it is true then parameters must be tracked to match the wildcard. Otherwise they must be untracked.

If the type that the parameter must match is a ParameterSet or vector<ParameterSet> then the constructor is defined as follows. Notice the template parameter is different and there is extra argument which will be used to validate any ParameterSet's which match the wildcard. The extra argument can be omitted and if it is omitted the matching ParameterSet's are not validated.

edm::ParameterSetDescription desc;
edm::ParameterSetDescription val;
... add content to val ...
edm::ParameterWildcard<edm::ParameterSetDescription> wnode("*", edm::RequireExactlyOne, true, val);
desc.addNode(wnode);

edm::ParameterSetDescription desc;
edm::ParameterSetDescription val;
... add content to val ...
edm::ParameterWildcard<std::vector<edm::ParameterSet> > wnode("*", edm::RequireExactlyOne, true, val);
desc.addNode(wnode);

Allowed Labels - More Details

The labelsFrom function also creates a node that is added directly to the ParameterSetDescription.

edm::ParameterSetDescription desc;
desc.labelsFrom<int>("allowedLabels");

This node can also be explicitly constructed so that it can be added to other nodes. The class AllowedLabelsDescription inherits from ParameterDescriptionNode.

edm::ParameterSetDescription desc;
edm::AllowedLabelsDescription<int> node("allowedLabels", true);
desc.addNode(node);

AllowedLabelsDescription inherits from ParameterDescriptionNode. In the above example, the template parameter gives the type of the parameters that are allowed to exist. The first argument is the name of the parameter that holds the vector of strings which are the names of the parameters that are allowed to exist. The second argument is a boolean that is true if the allowed parameters must be tracked, otherwise they must be untracked.

These following two should be used in the special case that the allowed parameter type would be a ParameterSet or vector<ParameterSet>. The added second argument would be used validate the allowed ParameterSet's. That argument can be omitted and the nested ParameterSet's would not be validated.

edm::ParameterSetDescription desc;
edm::ParameterSetDescription val;
edm::AllowedLabelsDescription<edm::ParameterSetDescription> node("allowedLabels", val, true);
desc.addNode(node);

edm::ParameterSetDescription desc;
edm::ParameterSetDescription val;
edm::AllowedLabelsDescription<std::vector<edm::ParameterSet> > node("allowedLabels", val, true);
desc.addNode(node);

IfExists - More Details

The ifExists function also creates a node that is added directly to the ParameterSetDescription.

edm::ParameterSetDescription desc;
desc.ifExists( edm::ParameterDescription<unsigned int>("x", 11U, true),
               edm::ParameterDescription<std::string>("y", "11", true) );

This node can also be explicitly constructed so that it can be added to other nodes. The class IfExistsDescription inherits from ParameterDescriptionNode.

edm::ParameterSetDescription desc;
IfExistsDescription node( edm::ParameterDescription<unsigned int>("x", 11U, true),
                          edm::ParameterDescription<std::string>("y", "11", true) );
desc.addNode(node);

Optional Nodes

In a much earlier section above we discussed optional parameters. These are optional nodes where the node has type ParameterDescription. There is another way one can add these to a ParameterSetDescription.

edm::ParameterSetDescription desc;
desc.addOptionalNode( edm::ParameterDescription<int>("algorithm", 1, true),
                      true);

The first argument is the node. It does not have to be a ParameterDescription node, it can be any type inheriting from ParameterDescriptionNode. The second argument is something not available in the addNode function. It is a boolean. If it is false the node will not be written into the generated CFI file. If it is true and other factors allow it, then the node will get written into a CFI file.

In addition, the following functions are members of the ParameterSetDescription class and allow one to add optional nodes.

ifValueOptional
ifExistsOptional
labelsFromOptional
labelsFromOptionalUntracked

Here is how the behavior of the more complex types of nodes is affected if they are optional.

  • "and", "or", and "xor" nodes - If the parameters that are their subnodes (the arguments to their constructors) all do not exist or evaluate to false without parameter insertion, then nothing is done. Nothing is inserted, nothing is thrown, and the validation method is not called for their subnodes.
  • For a wildcard node, optional means nothing is thrown if nothing matches no matter what the criteria parameter is set to.
  • For a switch, if the switch parameter is missing, then it is not inserted, nothing is thrown, and none of the cases get validated.
  • If the ifExists node is optional, then the existence of the first node implies the second node is optional instead of implying that it is required.
  • For the AllowedLabelsDescription, it means the parameter containing the vector of names is optional. If it is not there, nothing is done during validation for that node.

An observant reader might notice that the interface only allows the top level nodes in a ParameterSetDescription to be optional. Nodes nested in other nodes cannot be optional. (Try imagining what it would mean for node inside an "xor" node to be optional ... That is only the beginning of the problems allowing optional nodes in nodes would bring. The logic would be insane, both for the user and implementer.)

Writing CFIs

Here is the behavior of the more complex nodes, when the CFI file is being generated.

  • For "and" nodes, both subnodes are written to the CFI file.
  • For "or" and "xor" nodes, only the left subnode is written to the CFI file.
  • For a switch node, the switch parameter is written to the CFI file. If that parameter has a default, it is used to select a case and the node associated with that default case is written to the CFI file.
  • For a wildcard node, nothing is written to the CFI file.
  • For an "AllowedLabels" node, the parameter that is the vector of allowed labels is written to the CFI file, but the vector is empty.
  • For an "IfExists" node, both subnodes are written to the CFI file.

Restrictions

The ParameterSetDescription system puts some restrictions on what is allowed in a description. The motivation for the restriction is that these rules prevent a problem that can occur when validating a ParameterSet. The problem is related to parameters that get injected into a ParameterSet. When this injection occurs one must worry that nodes other than the one being validated could be affected by the parameters that are being validated. In particular, nodes that have already passed validation could be put in a state where they no longer pass validation. With the flexibility the system allows, a vicious circle could be entered with great complications for the implementation. It might not even be possible to come up with a validation algorithm that would work in all cases. So there are restrictions that avoid the problem.

These restrictions have the additional benefit that the things they prohibit would tend to confuse a user trying to configure a module or a module developer writing the code to extract the parameters from a ParameterSet. These rules tend to prohibit bad design. Even if the original motivation for them did not exist, they may be a good set of rules to enforce.

Here are the restrictions:

  • The same parameter names cannot occur in different nodes of the same ParameterSetDescription. There are two exceptions to this. Nodes that are contained in different cases of a ParameterSwitch or different subnodes of an "exclusive or" are allowed to use the same names.

  • Within a single ParameterSetDescription a wildcard and a ParameterDescription cannot have the same type.

There is one related problem that can occur with the current code. If insertion is necessary to make an "exclusive or" node pass validation, then the insertion could make more than one of the possibilities evaluate true. This must be checked for after the insertions occur. The current behavior is to throw a Configuration exception if this problem is encountered. (Example: (A && B) ^ (A && C) where C already exists in the ParameterSet but A and B do not. A and B get inserted by the algorithm, because it tries to make the first possibility true when all fail without insertion. Then both parts of the "exclusive or" pass, which is a validation failure). We are discussing improvements which may affect which subnode of an "xor" node is selected which may reduce problems like this, although probably not completely eliminate them.

edmPluginHelp

The executable named edmPluginHelp will print out the configuration descriptions in a human readable format. For the most part, the output of edmPluginHelp is self explanatory. The command "edmPluginHelp -h" explains the available options and gives a little information about the format. Here are some more details.

  1. The output of edmPluginHelp includes comments that describe the output format. In addition to the generic comments edmPluginHelp automatically inserts, it is also possible for a plugin to add comments into its description that will appear in the edmPluginHelp output. These comments are optional, but could help to significantly improve the clarity of the output. We recommend such comments be brief saving long explanations for other formats.
  2. One option to the command will cause it to produce a brief format. This suppresses most comments and produces a more compact format. Other than the comments the information is the same. For simple parameters, the columns in the brief format are:
    • parameter name
    • type (possibly preceded by "untracked")
    • possibly the word "optional"
    • default values
  3. The printout for each PSet description will start with a series of top level parameters, references to groups of parameters, or wildcards that could represent many parameters. Below the top level list there can be a series of sections describing groups or default contents of large vectors. The sections are numbered and the reference to the section in the top level group will contain that number. In complex cases, one might find the search function on an editor useful to jump back and forth between a section and the reference to it.
  4. Nested ParameterSetDescriptions and vectors of them are analogous to the nested structures in ParameterSets. But other nested structures like AND Groups and switches will NOT appear in a PSet or python configuration files. They only help to describe which combinations of parameters are allowed or required to be in the configuration.
  5. Parameters can be "tracked" or "untracked". edmPluginHelp will explicitly print the word "untracked" just before the type for "untracked" parameters. It does NOT print "tracked" for the others. You just have to assume that all parameters not marked "untracked" are "tracked". "tracked" implies the parameter value will be saved in the output file of a cmsRun job in the provenance. (There is one exception to this. ParameterSet's used to configure services are not saved in the provenance at all, even if they contain tracked parameters.)
  6. Definition of "optional": When the validation encounters a required parameter that is missing, it either inserts a default value for the parameter into the PSet or throws an exception. Parameters are either required or optional. When the validation encounters an optional missing parameter, it does nothing. It does not insert the missing parameter, nor does it throw. It just silently ignores it and proceeds to validate the next parameter in the description. In the printed output of edmPluginHelp the word "optional" is printed following the type for optional parameters. The word "required" is not printed. One can assume anything not marked optional is required. Note that only the top level entries in a PSet description can be optional.
  7. Historically (before the validation and PSet description code existed), tracked parameters were supposed to always be required parameters. The most common function used to get a tracked parameter from a ParameterSet would and still will throw if the parameter is not there. But for a long time there have been ways around this used in many places in CMSSW. As far as the PSet description and validation is concerned "tracked" and "optional" are independent concepts. If a missing parameter will cause a module to throw, the module's description should declare that the parameter is required. However, the validation itself will be perfectly happy with tracked parameters that are missing if the module's description declares them optional.
  8. Values in vectors are preceded by the index in square brackets. For example, "[0] " precedes the first element of the vector.
  9. At the moment, the only allowed wildcard expression is simply "*", which matches any parameter label. We are thinking of extending this to either regular expressions or globbing. (If users requested this, this would be more likely to happen or happen sooner.) The wildcard pattern is matched against the parameter label. The algorithm only searches for matching parameters of a single specified type.

Features that still need to implemented

We are discussing whether and when to implement these features.

  • A C++ interface to allow other applications to access the description. For example, ConfDB might be populated this way. The description is already capable of storing the information. The main issue is defining the interface and its requirements.
  • Add support for the python parameter type SecSource. Currently one can treat this type as if it were a nested ParameterSet for validation purposes only, but when you do this the cfi files and edmPluginHelp output just print out as if the parameter were a ParameterSet and not the python SecSource type. This is incorrect behavior.
  • Defaults for wildcards.
  • Some discussion of eliminating CFI files entirely in favor of inserting missing parameters. (Probably very far in the future if ever)
  • Support the MessageLogger special case
  • Implement a reset function for the value of a single parameter description that makes it easier to define multiple descriptions for the same plugin
  • Improve error checking and messages printed after exceptions
  • Implement the fillDescriptions function in more plugins. Many plugins already have it, but not all.
  • Continue to improve this documentation
  • Add the ability for the validation to check if the value of a parameter is in a certain range, or is one of a limited number of allowed values.
  • In the case of parameters that are vectors, the validation might also check that the vector is of a certain size.

Note that the validation system has supported EDProducers, EDAnalyzers, EDFilters, and OutputModules since release 3_1_0. It started supporting sources and services in release 3_5_0. Support for ESSources and ESProducers has been implemented and checked into CVS and will probably be available starting in release 3_11_0. Support for module plugins loading helper plugins should be available in release 10_3_0.

An example

Here is an example of a job that will detect a parameter that is not allowed:

  cmsRun FWCore/Integration/test/testParameterSet_cfg.py

It will fail because the configuration has a parameter not in the description of the allowed parameters.

Review status

Reviewer/Editor and Date (copy from screen) Comments
DavidDagenhart - 18 May 2009 created page
DavidDagenhart - 22 April 2013 reviewed
DavidDagenhart - 5 October 2018 added section on helper plugins

Responsible: DavidDagenhart
Last reviewed by: DavidDagenhart - 8 December 2011

Edit | Attach | Watch | Print version | History: r50 < r49 < r48 < r47 < r46 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r50 - 2019-08-05 - DavidDagenhart



 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback