Run Concurrency

Contacts

• Contacts: W. David Dagenhart (FNAL),
• Hypernews forum: https://hypernews.cern.ch/HyperNews/CMS/get/edmFramework.html
• Documentation contact: David Dagenhart

Run Concurrency

Support for run concurrency was added to the Framework in release 12_6_0 in 2022 (see pull request 39673).

One configures the "numberOfConcurrentRuns" in the top level options parameter set. If it is 0 or greater than the number of concurrent lumis, then it will be reset to equal the number of concurrent lumis.

If an EventSetup IOV changes at a run boundary, then one also would need to configure concurrent IOVs for that record to two to actually have the runs on both sides of that run boundary process concurrently. Without that cmsRun would execute properly, but the IOVs would block concurrent execution. In addition, it is technically possible for the sequence of transitions beginLumi, endRun, beginRun, and beginLumi to all have different IOVs. An EventSetup record with such IOVs would need to be configured to allow 4 concurrent IOVs to process both runs concurrently across such a run boundary. Note that records can be individually configured with each allowing a different number of concurrent IOVs. One could configure a small number of records to allow concurrent IOVs and configure the rest to allow 1 (saving resources).

The initial pull request implementing support for concurrent runs in the Framework did NOT upgrade modules and services outside the Framework to support concurrent runs. We expect many of them will fail if the number of concurrent runs is configured to be more than one in an existing production configuration. We have not surveyed existing code to see which modules and services cannot support concurrent runs. Most should be OK because they do not depend on run transitions. But for example, a module designed to create per run histograms might have problems with concurrent runs. As of November 2022, no additional work had been done on run concurrency on modules and services outside the Framework. At some point in the future, we plan to work on this.

Review status

Responsible: DavidDagenhart
Last reviewed by: