Writing a Source That Supports cmsRun's Forking Ability

WARNING: THIS ENTIRE PAGE IS OBSOLETE. cmsRun no longer supports forking. Support for forking was removed after support for multithreading was added. We found multithreading the better alternative and supporting both forking and multithreading was causing difficult problems. Complete: 3



cmsRun has the ability to fork itself into a parent and multiple child processes. The parent process coordinates the child processes by telling them how how many events they should skip and how many events to process in a row before doing another skip. To make this work, a source must be specially written to listen to the parent.

Calling Sequence

The following is the methods of the source that will be called for each stage of forking as well as a description of the behavior of the source for each call. A working example can be found by looking at the code for ConfigurableInputSource

Before Forking

This case starts just like the non-forking case. cmsRun needs to find the first Run being processed by the source in order to prefetch all conditions information.
  1. InputSource::nextItemType : This must return the 'IsFile' declaration
  2. InputSource::readFile
  3. InputSource::nextItemType : This must return the 'IsRun' declaration.
  4. InputSource::runAuxiliary : Since the source was told to go to the next transition and that transition must be the run we now get the run information.
  5. InputSource::preForkReleaseResource : If the source did open a file, it must close it now in order to release the file descriptor. Any other external resources should also be released at this time.

Parent Process

There is nothing the source needs to do since the source will never be called in the parent.

Child Process

Each child process will see the following calls
  1. InputSource::postForkReaquireResources( boost::shared_ptr<edm::multicore::MessageReceiverForSource>) : The source should hold onto the receiver. The receiver is used to get how many events should be skipped and how many events are to be processed before skipping again. In addition, the source should call InputSource::rewind() to reset the source to its beginning (i.e. pre-open file condition). After the full rewind, note that InputSource::rewind() calls the virtual function InputSource::rewind_() , the source must skip forward the number of events specified by the receiver.
  2. From here on out the source will behave as normal except for the fact that it must count how many events it processed and compare that to the receivers value for how many to process. Once the two agree the source must call receiver.receive() to update its information from the parent process. Once the data is updated the source must skip the new number of events or if that exceeds the number of events in the source the source just stop. When told to 'skip' the source must be able to properly tell the framework to do 'endLumi' and 'beginLumi' if skipping causes the source to go to a new lumi and generate 'endRun' and 'beginRun' if skipping sends it to a new Run. If a skip causes the source to skip over a complete Lumi or Run then it does not need to generate the 'begin/end' transitions for that Lumi or Run.

Methods that must be overridden in the inheriting class

  • nextItemType_()
  • rewind_()
  • preForkReleaseResource()
  • postForkReaquireResources( boost::shared_ptr<edm::multicore::MessageReceiverForSource>)

Review status

Reviewer/Editor and Date Comments
Last reviewed by: ChrisDJones - 28-Aug-2012 created the page

Responsible: ChrisDJones
Last reviewed by: Sudhir Malik- 24 January 2009

Edit | Attach | Watch | Print version | History: r4 < r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r4 - 2019-10-11 - DavidDagenhart
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback