How to derive a production

Typical case: when a production is running for a given processing pass, and a new version of the application is released, which should substitute the current one. So, the input run range is the same, but we want that the input files that have not yet been processed by the current one, will be picked up by the new one. This will not create a new processing pass.

What happens when a production is derived?

At the moment when a new production is derived all "Unused" files of the parent will be picked up by the derived production. This happens ONLY at the moment of deriving the production, so this process is static. So in case a production needs to be derived e.g. because of an improved version of the application the procedure should be the following

  • Stop the "parent" production
    • for safety and clarity set the endrun of the parent production
  • Create a new duplicate production with the improved application version
    • Includes a new step for the application
    • Duplicate the "parent request" and replace the application step with the new one
    • Create a new production with startrun >= (parent endrun) + 1
  • Wait for the parent production to have no more files in Assigned - once this is achieved
    • Reset all files in the parent which are NOT in Status "Processed" (e.g. MaxReset) to "Unused"
    • Derive a new production with the same request as the duplicate production by setting the parent production in the Generate step

Now just complete the three productions above wink

Example

for Reco13a a first production launched with Brunel . Current request: 7854, current production: 18028.

A bug was found and a new Brunel version produced, with the relative step: 17423.

now, we want to derive a new production from the 18028, this means that the processing pass stays the same, for the input data query and for the output. The only change is the Brunel version.

What to do:

  • Stop production 18028 and set endrun
  • Create a new request, duplicating the old request (in this case 7854) and changing only the Brunel step, to use the new one (in this case 17423)
  • for the rest, keep all the same (same Bkk input query)
  • generate a new production from this request with startrun = (previous endrun + 1)
  • wait for production 18028 to have all files in Unused / MaxReset / Processed (no more Assigned)
  • reset any MaxReset files in 18028 with "dirac-transformation-reset-files 18028 MaxReset"
  • derive a new production from request 7854 with setting the 'ancestor production' the number of the old one: 18028

automatically, the old one will be changed to status 'Completing' and the new one 'Active'. The run range and all the other parameters, should be the same as the parent production. The files already assigned to the old one, will be processed by the old one. If a job fails, the input file is reset as Unused and will be picked up by the new one.

Important: if the old production has some files already in MaxReset status, then they should be set as Unused, so they will be picked up by the new production. This can be done using the command:

dirac-transformation-reset-files 18028 MaxReset

-- ElisaLanciotti - 09-May-2012

Edit | Attach | Watch | Print version | History: r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r2 - 2012-05-10 - StefanRoiser
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LHCb All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback