Difference: ProductionAPI (4 vs. 5)

Revision 52009-04-21 - AndrewCSmith

Line: 1 to 1
 
META TOPICPARENT name="ProductionProcedures"
-- StuartPaterson - 08 Apr 2009
Line: 201 to 201
 
    • Set to False this will not publish the production but allows printing of BK parameters / generation of BK script with default production ID
  • fileMask=''
    • Optional file mask (regular expression approach)
Changed:
<
<
  • groupSize=0
>
>
  • bkQuery = {}
    • Optional dictionary containing the BK query parameters
  • groupSize=1
 
    • Number of input files per job for data processing productions
Added:
>
>
  • derivedProduction
    • Optionally the ID of the production from which to derive the input data
 
  • bkScript=True
    • Set to True this will write a script that can be checked before publishing to the BK
    • Set to False this will automatically publish to the BK
The workflow XML is created regardless of the flags (with a default production ID of 12345 if publish=False).
Added:
>
>
It should be noted that as well as providing arguments to the create() it is also possible to use the following dedicated setting methods:
  • setInputFileMask(fileMask)
    • Where fileMask is the reg expression to match input files
  • setInputBKSelection(bkQuery)
    • Where bkQuery is the dictionary containing the BK query parameters
  • setJobFileGroupSize(files)
    • Where files is an integer containing the number of files
  • setAncestorProduction(prodID)
    • Where prodID is the ID of the production from which to derive the input data
When these methods are used their input overwrites what is supplied to create().

In addition to the explicit setting methods presented there are two more which can be useful:

  • setProdGroup(groupString)
    • Where the groupString is a simple description of the production. examples are 'MC09', 'FEST'... This allows simple searching for production group on the production monitoring page.
  • setProdPlugin(pluginName)
    • Where the pluginName is a string with the name of the plugin to be used when creating the jobs. For FEST reconstruction of the FULL stream this should be set to 'CCRC_RAW'.
 

Example 1 - From a typical simulation request email to events in the BK

In this case there was a request for a production with 'old' versions of the projects that don't support LHCbApp(). The standard options are printed by the Production.py API and it was sufficient to remove traces of the LHCbApp() module (relying on the defaults from the options). Note that the create() function here is used just to write a script for BK publishing, this allows to check that the processing pass is ok before making it visible (if it is ever to be made visible) and will only generate the workflow XML e.g. this can be tested locally before entering to the production system.

Line: 299 to 320
 
(DIRAC3-production)[paterson@lxplus223 ~]$ cat insertBKPass4614.py 
# Bookkeeping publishing script created on Wed Apr  8 12:34:44 2009 by
Changed:
<
<
# by $Id: ProductionAPI.txt,v 1.4 2009/04/09 15:39:08 StuartPaterson Exp $
>
>
# by $Id: ProductionAPI.txt,v 1.5 2009/04/21 14:26:12 AndrewCSmith Exp $
 from BookkeepingClient import BookkeepingClient bkClient = BookkeepingClient() bkDict = {'Production': 4614,'Steps': {'Step1': {'ApplicationName': 'Boole', 'ApplicationVersion': 'v16r3', 'ExtraPackages': '', 'DDDb': 'head-20081002', 'OptionFiles': 'Boole-2008.py', 'CondDb': 'head-20081002'}, 'Step0': {'ApplicationName': 'Gauss', 'ApplicationVersion': 'v35r1', 'ExtraPackages': 'AppConfig.v1r1', 'DDDb': 'head-20081002', 'OptionFiles': 'Gauss-2008.py;$APPCONFIGROOT/options/Gauss/RICHmirrorMisalignments.py;$DECFILESROOT/options/10000000.opts', 'CondDb': 'head-20081002'}}, 'GroupDescription': 'MC08-v1', 'SimulationConditions': {'BeamEnergy': '7 TeV', 'Generator': 'Pythia', 'Luminosity': 'Fixed 2 10**32', 'MagneticField': '-1', 'BeamCond': 'Collisions', 'DetectorCond': 'VeloClosed', 'SimDescription': 'Beam7TeV-VeloClosed-MagDown'}}
Line: 382 to 403
 
(DIRAC3-production)[paterson@lxplus243 ~]$ cat insertBKPass12345.py
# Bookkeeping publishing script created on Sun Apr  5 17:15:17 2009 by
Changed:
<
<
# by $Id: ProductionAPI.txt,v 1.4 2009/04/09 15:39:08 StuartPaterson Exp $
>
>
# by $Id: ProductionAPI.txt,v 1.5 2009/04/21 14:26:12 AndrewCSmith Exp $
 from BookkeepingClient import BookkeepingClient bkClient = BookkeepingClient() bkDict = {'Production': 12345, 'Steps': {'Step1': {'ApplicationName': 'DaVinci', 'ApplicationVersion': 'v22r1', 'ExtraPackages': 'AppConfig.v2r2', 'DDDb': 'head-20090112', 'OptionFiles': '$APPCONFIGOPTS/DaVinci/DVMonitorDst.py', 'CondDB': 'head-20090112'}, 'Step0': {'ApplicationName': 'Brunel', 'ApplicationVersion': 'v34r2', 'ExtraPackages': 'AppConfig.v2r2', 'DDDb': 'head-20090112', 'OptionFiles': '$APPCONFIGOPTS/Brunel/FEST-200903.py', 'CondDB': 'head-20090112'}}, 'GroupDescription': 'FEST-Reco-v1', 'DataTakingConditions': 'DataTaking6153'}
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback