Behavior of Event Selector when Trigger Bits are not all Pass or Fail

Complete:

Introduction

The EventSelector class is used by standard Output Modules to decide which events are to be output. This decision is guided by Path Specifiers which are determined by SelectEvents blocks in the configuration, and Trigger Results which are created by the framework for each event based on the results reported by each module in the various paths configured.

The EventSelector is capable of deciding based on Pass or Fail states of various Trigger Results "bits", according to how it is set up at construction by the Output Module. The behavior in most cases is quite intuitive, but because the configuration may involve wildcard specifications, it is useful to know exactly what happens with these when, for example, no trigger name might match a wildcarded specification. In addition, the Trigger Results "bits" can be in four states: Pass, Fail, Exception, and Ready, and behavior when some relevant bits are Exception, and Ready has not previously been clearly described.

EventSelector also has several member functions other than acceptEvent(), and we will discuss the behavior of those member functions under the same sorts of unusual circumstances in the last part of this page.

Jargon

Filter is a module that can be configured to run as part of a path, and will provide Pass, or Fail information (or occasionally indicate an Exception).

Path is an ordered string of Filters. Ordinarily, the path passes if each of the filters in the path pass, and fails (without forcing subsequent filters to run) if one of them fails. However, the path can have negation indicators (for example, { A, B } would say that A must pass but B must fail) and "don't-care" indicators that cause a filter to run without affecting the result of the path. Each Path leads to formation of a Trigger Bit in the event.

Path Specifier, is a single string selecting some potential trigger bit, or possibly an expression involving wildcards which may select any number of trigger bits. The Path Specifier can begin with "!", requesting Fail state(s) on the specified bit(s), in which case we call it a "_negative_ path specifier".

Path Specifiers also called Trigger Path Specifications, or in code, pathspecs, is a vector of Path Specifier, and is used to initialize the EventSelector, telling it how to decide which events to accept.

Parsed Path is a Path Specifier, along with a Process Name. The SelectEvents syntax allows, for example, { p1:HLT } meaning to select on p1 but only as a result in the HLT process. The Parsed Path is a pair, with the first element being the path (p1) and the second being the process (HLT). If no process name is present, or the process name matches the current process, then the Trigger Results to compare with are simply those in the event. But if the process does not match the current process, then acceptEvent may need to do somethig special to obtain the correct Trigger Results. Parsed Path is not especially relevant for the behaviors discussed on this page, since by the time it gets to a Path Specifier, the process part has been stripped off.

Process Name is the name after the colon in the SelectEvents syntax { p1:HLT }. See Parsed Path.

Trigger Bit is a single result of some Path, which can be one of four values (thus making the name "bit" a tad mis-descriptive): Pass, Fail, Exception, or Ready. These correspond to the states defined in HLTenums.h, namely edm::hlt::Pass, edm::hlt::Fail, edm::hlt::Exception, and edm::hlt::Ready. Snice the

Trigger Name List is a vector of all available trigger names for events (in a given process). The Path Specifiers may be check for validity against a Trigger Name List.

Trigger Results, sometimes called Event Trigger Mask, is a vector of Trigger Bits each corresponding to a name in the Trigger Name List. The acceptEvent function compares the Trigger Bits with some decision masks it creates at initialization, to decide whether to accept or reject the event for output.

Wildcarded Path Specifier is a Path Specifier containing a wildcard charactoer (either "*" or "?"). Any Wildcarded Path Specifier other than "*", "!*", "exception@*" and "!exception@" will be refered to as a partially wildcarded Path Specifier.

Reminders of EventSelector Path Specifier Syntax

The Path Specifiers appear in the SelectEvents PSet of the output module configuration PSet. For example,

  module outp12 = PoolOutputModule {
    untracked string fileName = "savep12.root"
    untracked PSet SelectEvents = {
      vstring SelectEvents = { "p1", "p2" }
    }

Here we had two specifiers, "p1" and "p2", but there can be any positive number of specifiers.

Along with this simplest form of specifier, there are four sorts of extensions (the meanings of which will be discussed below):

1. Negation, as in "!p2", meaning that we select this if the p2 Trigger Bit is Fail.
2. Wildcards, as in "p*" or "!p*".
3. Exception demands, as in "exception@p1" or "!exception@p*".
4. Specfiers with a Process Name, as in "p1:HLT".

Path Specifiers with Process Names

A Process Name can be provided to indicate that one cares about matching the corresponding trigger bit only in that particular process. (See the jargon discussion of Parsed Path.) By the time the Path Specifier gets to the EventSelector, this Process Name will have been stripped off, so it is unimportant for the present discussion of EventSelector behavior.

One caveat, however, involves wildcards. We support wildcards in the Path Name only. Wildcards are not allowed in a Process Name. (Omitting the Process name has the meaning of "all processes.")

When is a Path Specifier Valid?

In order to catch typos before a potentially long job executes without doing proper output, the EventSelector constructor will detect Path Specifiers that make no sense, and throw an exception if it encounters one. The rules are:

1. "*" and "!*" are always OK.
2. A non-wildcarded Path Specifier, whether positive ("p1") or negative ("!p1"), and whether plain or exception ("exception@p1"), is valid if and only if that name matches one of the names on the Trigger Name List.
3. A partially wildcarded Path Specifer, to be valid, must match at least one name on the Trigger Name List. This applies to both positive ("p*") and negative ("!p*") specifiers.
4. Use of the "@" character other that in the specific sequence "exception@" or "!exception@" at the start of the specifier will be caught as an error.

Behavior For Wildcarded Path Specifiers

A positive Wildcarded Path Specifier accepts the event if any matching Trigger Bit is in the Pass state. (Note that since we can assume the specifier is valid, there must exist at least some matching Trigger Bits.) Thus "*" accepts all events with any Pass bits, while "p*x" would accept the event if the Trigger Bit for (for example) p345x is Pass.

A negative Wildcarded Path Specifier accepts the event if every matching Trigger Bit is in the Fail state. Thus "!*" accepts only events which fail every trigger, while "!p*" would accept the event if the all Trigger Bits for path names starting with a "p" are Fail.

Performance when Wildcarded Path Specifiers are Used

When only non-wildcarded Path Specifiers appear, the EventSelector uses a single "mask" saying that this bit matches if it is Pass, that one if it is Fail, and don't look at the other one at all. This is also the case if positive Wildcarded Path Specifiers are used, or if a negative Wildcarded Path Specifier that matches only one Trigger Name is specified.

However, a negative Wildcarded Path Specifier that matches multiple trigger names implies the and of multiple "this bit must *Fail*" conditions. Each such specifier is implemented as a separate "mask", and takes time comparable to that for the positive specifiers. (The acceptEvent logic is nonetheless quite fast, and probably will represent a negligible fraction of time even in skim or filter type jobs with multiple negative Wildcarded Path Specifiers.)

Behavior When Some Trigger Bits are Exception

Exception is not treated as Pass or Fail. Thus "!*" would not accept an event which has some Trigger Bits in the Exception state. The only way to accept events in the Exception state is to specify that in the Path Specifiers.

The Path Specifier "exception@p1" will accept an event if the Trigger Bit for p1 is *Exception". The Path Specifier "exception@p*" will accept an event if he Trigger Bit for any trigger starting with a "p" is *Exception".

Negative specifiers involving exception are sometimes useful. While the Path Specifiers {"!exception@p1"} is equivalent to { "p1", "!p1" }, the Path Specifiers {"exception@p*"}, if there are multiple trigger names matching "p*", cannot be mimicked in another way.

While the default if no SelectEvents configuration is present is to accept every event, one could equivalently express that by specifying SelectEvents = {"*", "!*", "exception@*"}, which covers all possibilities.

Behavior When Some Trigger Bits are in the Ready State

The Ready state is treated as if, for the purposes of just this event, that Trigger Bit were non-existant. Thus a bit in the Ready state can never cause an event to be accepted; nor can it eliminate an event by violating some negative Path Specifier (which requres several bits to be Fail).

The "No Triggers Exist" Situation

The question arises: What happens if so many Trigger Bits are Ready that absolutely all for the bits in a Path Specifier are to be treated as non-existant? Ordinarily, a Path Specifier refering to no bits cannot occur at the acceptEvent point, because it would be detected as invalid when the EventSelectorwas constructed. But the presence of the Ready state makes this situation possible.

This situation is, of course, ordinarily moot, since in the ordinary course of running, a Trigger Bit can't be in the Ready state by the time the Output Module is activated. But other code utlilizes the EventSelector in different ways, so we need to define what would happen.

When all the Path Specifier is positive or non-wildcarded, the desired behavior is clear: If none of the Trigger Bits relevant to the Path Specifier exist, then that Path Specifer does not accept the event. Thus, both "p" and "!p" would reject if the "p" bit is Ready, and "*" would reject if all bits are Ready. But what happens with a negative Wildcarded Path Specifier that matches multiple trigger names, all of which happen in this event to be in the Ready state?

The answer to this question is constrained by the desire to preserve the behavior under the original version of EventSelector, when "!*" (the only negative Wildcarded Path Specifier supported at that time) encounters every trigger bit Ready. The result would be to accpet the event since "none of these bits have passed." For the best consistency we can come up with, this leads to the following rule.

When all Trigger Bits matching a Path Specifier are in the Ready state:

1. If the Path Specifier is not a negative specifier (does not begin with "!") the event is rejected.
2. If the Path Specifier matches only one Trigger Name (either because it was not wildcarded, or because its wildcard expression happens to match only one name) the event is rejected.
3. If the Path Specifier is a negative specifier, and matches multiple Trigger Names all of which triggers are Ready, the event is accepted.

Other Member Functions in EventSelector

Four other public static member functions in EventSelector deal with DESCRIPTIOVE PHRASE HERE

1. selectionIsValid
2. testSelectionOverlap
3. maskTriggerResults
4. getEventSelectionVString

selectionIsValid

bool selectionIsValid (Strings const& pathspec,
                       Strings const& fullTriggerList)

Tests if the specified trigger selection list Path Specifiers) is valid in the context of the specified full trigger list (Trigger Name List). The Path Specifiers list is considered valid if for each element in that list it possible for some combination of trigger results among the fullTriggerList to satisfy that Path Specifier and cause the event to be accpeted.

A Path Specifier can never demand ANDS of specific results of trigger bits, nor require counts of numbers of Pass or Fail or Exception bits. Therefore, if a Path Specifier can be satisfied at all, it can be satisfied by just one trigger bit in the Pass, Fail, or Exception state, and all the other bits in the Ready state and thus ignored. This, in fact, is how the routine does its test. If any of the Path Specifiers can never be satisfied in this way, the return value is false.

At some level, this method tests whether the selection list is a "subset" of the full trigger list. That is, if any item in the selection list does not match some trigger bit, then selectionIsValid will return false; otherwise, it will return true.

edm::evtSel::OverlapResult
testSelectionOverlap(Strings const& pathspec1,
                     Strings const& pathspec2,
                     Strings const& fullTriggerList)

Tests if the two specified trigger selection lists (Path Specifiers) overlap, where "overlap" means that a single valid trigger result (among the full trigger list) could satisfy both selections, if all other trigger results were non-existant (or the Ready state). The method returns an OverlapResult, which is an enum defined in EventSelector.h with possible values

  InvalidSelection 
  NoOverlap
  PartialOverlap 
  ExactMatch

Note that when a negative Path Specifier with wildcards is part of one of the trigger selection lists, this definition in terms of all but one trigger bit as Ready leads to results that might be surprising. For example, { p1 } would partially overlap { q* } if the full trigger list were { p1, q1, q2 }, but not if the full trigger list were { p1, q1 }.

maskTriggerResults

boost::shared_ptr<TriggerResults>
maskTriggerResults(Strings const& pathspecs,
                   TriggerResults const& inputResults,
                   Strings const& fullTriggerList)

Applies a trigger selection mask to a specified trigger result object. Within the trigger result object, each path status is left unchanged if it satisfies the trigger selection (path specs) or cleared (set to Ready) if it does not satisfy the trigger selection. In this way, the resulting trigger result object contains only path status values that "pass" the selection criteria.

This method may not make sense it the context of negative Wildcarded Path Specifiers. In such a case, in may happen that several trigger bits early in the list cause the event to be rejected, and are therefore reset to Ready. When examining a later bit, one of the negative Wildcarded Path Specifiers might now be vaucuously satisfied (because all of its relevant bits are now Ready). In such a situation, the results of this method could be dependant on the order of definition of the fullTriggerList.

getEventSelectionVString

std::vector<std::string>
getEventSelectionVString(edm::ParameterSet const& pset)                

Returns the vector of strings(logically, this is a Pat Specifiers list) that correspond to the trigger selection request in the specified parameter set. That is, this extracts the ist of strings contained in the SelectEvents parameter.

Review Status

Reviewer/Editor and Date (copy from screen)	Comments
MarkFischler - 14 Feb 2008	Initial version

Responsible: Main.jbk
Last reviewed by: Reviewer