-- StuartPaterson - 28-Jul-2010

Launching and Testing Productions from the Production Request page

Prerequisites

The prerequisites for launching productions are:

  • Your certificate must be available in your browser
  • To run a test production starting from a request in PPG_OK or Submitted status you must have either lhcb_tech or lhcb_prmgr roles
  • You must have the lhcb_prmgr role active in your session to be able to edit requests in Accepted or Active status
  • Note that in the following examples lhcb_prmgr role will be used exclusively
  • In case of any confusion or problems simply stop and speak to an expert in the offline control room (OCR) or send a mail to the lhcb-production-manager list

Some useful links:

When running production tests it is assumed that the standard DIRAC environment has been set up and the user has a valid proxy.

DIRAC configuration settings for tests at CERN

At the time of writing, the default client installation at CERN does not contain all necessary DIRAC configuration settings. The following can be added to your !$HOME/.dirac.cfg file in order to proceed:

LocalSite
{
  LocalSE = CERN-RAW
  LocalSE += CERN-RDST
  LocalSE += CERN-USER
  LocalSE += CERN_MC_M-DST
  LocalSE += CERN_M-DST
  LocalArea = /tmp/paterson/LocalArea  # Wherever is writeable for you
  Site = LCG.CERN.ch
  SharedArea = /afs/cern.ch/project/gd/apps/lhcb/lib # The grid shared area
  Architecture = slc4_ia32_gcc34  #  Or x86_64-slc5-gcc43-opt if you prefer
}

This can be translated accordingly for other sites as necessary.

Testing simulation productions from from the production request page

This facility is intended to allow applications or production managers to test productions as defined in the production request page, before submission to the Grid. It is hoped that it will allow to catch basic incompatibilities in the requested application software versions or options.

In this example we will use "Active" request 1357 and the lhcb_prmgr role to generate a production API script that can be executed in a DIRAC environment. As a technical expert with lhcb_tech role the procedure is the same. As a first step, go to the request page and click on the request you want to launch, selecting "Edit" from the menu:

test-0

At this point it is good to check the "Comments" section of the page in case anything exotic is requested, e.g. the workflow should run only on a certain system configuration. Proceed by clicking on the "Generate" button:

test-1

Next we enter the "Generate production script" menu where we now select the MC_Simulation_run.py template for simulation productions:

test-2

After pressing "Next" we are presented with the list of parameters that can be changed.

test-3

For the purposes of testing a given production most of these can be safely ignored with the exception of:

test-4

which must be set to boolean True in order to indicate that the production script we will generate is only to run a test job. As an aside in the case of a request with subrequests you select the event type you wish to test:

test-5

Then by clicking on "Script Preview" the production API script is generated from the request parameters and the chosen template:

test-6

from this point select the entire contents as above and copy and paste to a local file say productionScript.py for our purposes.

The last step requires the DIRAC environment is set up and a valid proxy has already been generated. The production API script can now be executed via the following simple recipe:

> python productionScript.py

Since running a test job normally generates a lot of files it is recommended to do this in a temporary directory. By default the production test scripts will run over 5 events.

Launching productions from the production request page

This guide gives a step by step procedure of how to launch productions starting from a production request in "Accepted" status.

Notes on the simulation production template

That there is now a single simulation template to satisfy the vast majority of use-cases. This is called MC_Simulation_run.py and allows to set the following (non-exhaustive) set of parameters:

  • number of events
  • CPU time in seconds
  • final output file type
  • number of jobs to extend the initial MC production
  • resulting WMS priority for each job
  • BK config name and version
  • system configuration
  • output file mask (e.g. to retain intermediate step output)
  • merging priority, plugin and group size
  • whether or not to ban Tier1 sites as destination
  • whether or not a merging production should be created
  • whether or not a transformation should be created
  • whether outputs should be uploaded to CERN only for testing
  • string to append to the production name

Creating a simulation production step-by-step

In the following example a simulation production will be launched from a request. lhcb_prmgr must be set in order to see the "Edit" menu option.

As a first step, go to the request page and click on the request you want to launch, selecting "Edit" from the menu. Check the "Comments" section of the page in case anything exotic is requested:

launch-1

Next we enter the "Generate production script" menu where we now select the MC_Simulation_run.py template:

launch-2

After pressing "Next" we are presented with the list of parameters that can be changed. It is intended that the description of each parameter in the template serves as documentation but if a parameter is not clear or some functionality is missing please talk to an expert in the OCR or email the lhcb-production-manager list.

launch-3

At the conclusion of pressing the Generate button you may see the following (harmless) error:

launch-4

"Unparsable reply from web portal" as described in this Savannah ticket. Otherwise, you will see the standard output from creating the production.

As a final check you can go to the production monitoring page and keep track of the jobs which will subsequently be submitted.

Topic attachments
I Attachment History Action Size Date Who Comment
JPEGjpg editRequest1.jpg r1 manage 454.9 K 2010-07-28 - 16:06 UnknownUser LaunchScreenshot1
JPEGjpg generateButton.jpg r1 manage 213.2 K 2010-07-28 - 16:07 UnknownUser LaunchScreenshot2
JPEGjpg requestParameters.jpg r1 manage 206.8 K 2010-07-28 - 16:07 UnknownUser LaunchScreenshot3
JPEGjpg simulationGenerate.jpg r1 manage 145.6 K 2010-07-28 - 16:08 UnknownUser LaunchScreenshot4
JPEGjpg test0.jpg r1 manage 466.1 K 2010-07-28 - 17:27 UnknownUser TestScreenshot0
JPEGjpg test1.jpg r1 manage 255.9 K 2010-07-28 - 17:27 UnknownUser TestScreenshot1
JPEGjpg test2.jpg r1 manage 211.3 K 2010-07-28 - 17:28 UnknownUser TestScreenshot2
JPEGjpg test3.jpg r1 manage 239.9 K 2010-07-28 - 17:28 UnknownUser TestScreenshot3
JPEGjpg test4.jpg r1 manage 25.3 K 2010-07-28 - 17:28 UnknownUser TestScreenshot4
JPEGjpg test5.jpg r1 manage 175.9 K 2010-07-28 - 17:29 UnknownUser TestScreenshot4
JPEGjpg test6.jpg r1 manage 253.8 K 2010-07-28 - 17:29 UnknownUser TestScreenshot6
Edit | Attach | Watch | Print version | History: r4 < r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r4 - 2010-07-28 - unknown
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LHCb All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback