Difference: WGProductions (1 vs. 19)

Revision 192018-02-19 - DanielJohnson

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
-- AntonioFalabella - 08-Dec-2011
Line: 68 to 68
  At this point you can click on Save and your step will appear in the list of Registered Steps.
Added:
>
>

Testing a new step

See https://twiki.cern.ch/twiki/bin/view/LHCb/ProdReqPreLaunchTest
 

Submit a Production Request

When your step or steps are ready you can submit a new Request. On the LHCbDirac web app click on Production Request to access the production requests manager and click on

Revision 182017-05-02 - DanielJohnson

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
-- AntonioFalabella - 08-Dec-2011
Line: 15 to 15
 The result are then published in the bookkeeping so that everyone can use them. Last but not least, when your are sure that your code works fine the control is in the hands of the prodution team that take care of the technical aspects.

Working Group Software Package

Added:
>
>
COMMENT FROM DAN: Suggest that this section be simplified. In the simplest case, code can simply be released in WG/{yourWG}Config. We should add simple instructions to tag_package WG/{yourWG}Config and then the step can be defined with respect to that "{yourWG}Config.vXrY" release after deployment.
 At the time being LHCbDIRAC can run software that uses a release of the following main projects: Gauss, Boole, Brunel, DaVinci, LHCb and Moore. LHCbDIRAC runs the code with the aid of python options. The python options coded by the working groups must be released in a package that is known by SetupProject. So for example let's suppose you write your_script.py that uses the package extrapackage of the project Project . You will release your code such that the following lines can be issued:
Line: 52 to 54
 Here you have to fill in the entries corresponding to your case:

  • Name: Some string meaningful for you (please put here only alphanumeric characters - better if you also avoid blank spaces e.g. WG_Selection_DVv29r2_CPConfigv1r5)
Changed:
<
<
  • Processing Pass: Something meaningful that remind your input dataset (e.g. RecoXX-StrippingXX)
>
>
  • Processing Pass: Something meaningful that remind your input dataset (e.g. RecoXX-StrippingXX) COMMENT FROM DAN: I was requested to change this to something meaningful like, in my case, "WG B2OC AmAn DDhDalitz Selection2"
 
  • Application: Here you can choose between the list of possible projects and versions.
Changed:
<
<
  • Option Files: This the path of your script.
  • Extra packages: Here you have to issue the package that contain your script. The syntax here is that of SetupProject i.e. Package.vxry.
  • Runtime project: Additional project that can be used. Check the SetupProject page for the list of projects.
>
>
  • Option Files: This the path of your script. (i.e. whatever you would need to put on the command line to gaudirun.py)
  • Extra packages: Here you have to issue the package that contain your script. The syntax here is that of SetupProject i.e. Package.vxry. (in the simplest case, e.g. "B2OCConfig.v1r34")
  • Runtime project: Additional project that can be used. Check the SetupProject page for the list of projects. (can be left blank)
 
  • CondDB: Choose the conditions CondDB tag.
  • DDDB: Choose the conditions DDDB tag.
  • Visible: Visibility flag. If it's Y a folder containing the processing pass of the step will be created in the bookkeeping.
  • Usable: Y if you want to use the step. If you plain to update your step with a new one it's better to set this flag to obsolete so that the step will no more appear when creating request production( to understand why read the next paragraph production request paragraph )
  • Input File Type: In this form you can choose the file name of input files. The file name are registred in bookkeeping so you can choose an existing one or Add a brand new file name. Usually you don't need to create a new filename for the input because you pick up file from bookkeeping and by definition they are already there.
Changed:
<
<
  • Output File Type: As before you can choose an existing filename or create a brand new one that will be registered in the bookkeeping. It's reasonable here that you need to input your filename, so even if there is no particular rule yet it is recommended to follow the typing schema of the other filenames.
>
>
  • Output File Type: As before you can choose an existing filename or create a brand new one that will be registered in the bookkeeping. It's reasonable here that you need to input your filename, so even if there is no particular rule yet it is recommended to follow the typing schema of the other filenames. COMMENT FROM DAN This must match a line in your DaVinci script like "TupleFile = "B2OC_AMAN_B2DDHDALITZSEL.ROOT""
  At this point you can click on Save and your step will appear in the list of Registered Steps.

Revision 172017-03-23 - AntonioFalabella

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
-- AntonioFalabella - 08-Dec-2011
Line: 103 to 103
 Choose how many events you want to run (usually -1 - all events).

Comments

Changed:
<
<
Here you can wrie some comments useful for the production team. So far the most important additional information that the prodution team need to know is the method used for writing the output. For example in the Comments pad you would write:
>
>
Here you can write some comments useful for the production team. So far the most important additional information that the prodution team need to know is the method used for writing the output. For example in the Comments pad you would write:
 
Changed:
<
<
from library import class.method method()=outputfilename.extension
>
>
Request to be run on small fraction of file for validation.
 

test-0

Revision 162017-01-27 - AntonioFalabella

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
-- AntonioFalabella - 08-Dec-2011
Line: 38 to 38
 The example above refers to SetupProject and not to the new environament tools because the compatibility is still not full.

Setting Up a New Step

Changed:
<
<
In the LHCbDIRAC workflow for your script to be executed it needs to be included as a option file in a brand new workflow step. The step can be built by everyone that has lhcb_tech rights access to LHCbDIRAC web portal which can be reached through this link:

>
>
Through the LHCbDirac web app you can define a new step to for your application
 
Added:
>
>
The step can be built by everyone that has lhcb_tech rights access to LHCbDIRAC web portal which can be reached through this link:
 To access this page you need to load in your browser a valid grid certificate. In the example below you can see the step manager page.
Changed:
<
<
test-0
>
>
test-0
 
Changed:
<
<
Click on New Step on the bottom bar to access the new step configuration form
>
>
Click on New on the top bar to access the new step configuration form
 
Changed:
<
<
test-0
>
>
test-0
 Here you have to fill in the entries corresponding to your case:

  • Name: Some string meaningful for you (please put here only alphanumeric characters - better if you also avoid blank spaces e.g. WG_Selection_DVv29r2_CPConfigv1r5)
Line: 66 to 64
 
  • Input File Type: In this form you can choose the file name of input files. The file name are registred in bookkeeping so you can choose an existing one or Add a brand new file name. Usually you don't need to create a new filename for the input because you pick up file from bookkeeping and by definition they are already there.
  • Output File Type: As before you can choose an existing filename or create a brand new one that will be registered in the bookkeeping. It's reasonable here that you need to input your filename, so even if there is no particular rule yet it is recommended to follow the typing schema of the other filenames.
Deleted:
<
<
test-0
 At this point you can click on Save and your step will appear in the list of Registered Steps.

Submit a Production Request

Changed:
<
<
When your step or steps are ready you can submit a new Request. This can be done from the dirac web portal:

test-0

To submit a new request you have to login as lhcb_user

>
>
When your step or steps are ready you can submit a new Request. On the LHCbDirac web app click on Production Request to access the production requests manager and click on
 
test-0
Added:
>
>
To submit a new request you have to login as lhcb_user. This can done from the Settings menu on the bottom left part of the page.
 Doing this a New request button will appear, click on it and choose the production type.
Changed:
<
<
test-0
>
>
test-0
 
Deleted:
<
<
If you need to run your code on stripped DSTs you will choose Stripping.
 After this the New_request form will be opened:
Changed:
<
<
test-0
>
>
test-0
  Request pad
  • Name: Some string meaningful for you (please put here only alphanumeric characters - better if you also avoid blank spaces e.g. Name_of_the_WG_Selection_Additional_infos)
  • Information also: Here you can add some email of people that you think they have to know about your production. By default all people of the production team receive an email when a production request has been submitted.
Changed:
<
<
test-0
>
>
test-0
  Input data pad
Changed:
<
<
  • Select from BK: First select your input data from the bookkeeping. In the example it's been selected the DIMUON stream of Stripping17.
>
>
  • Select from BK: First select your input data from the bookkeeping.
 
  • DQ flag: Here select OK.
  • Production: Here you can select a particular production on which run your code. If you are interested on all the productions select ALL.
  • TCKs: Here you can even choose to run on some particular trigger configurations. Choose 0 if you need to run on all the possible configurations.
Line: 104 to 97
 Processing pass() pad Here you add the step (or steps) you prepared before. If you click on Add step the system list you all the steps with the input type specified in the Input data pad. This will work the same for the others steps. The system let you choose the steps compatible with the output data type of the previous step.
Changed:
<
<
test-0
>
>
test-0
  Event pad_ Choose how many events you want to run (usually -1 - all events).
Line: 116 to 109
 method()=outputfilename.extension
Changed:
<
<
test-0
>
>
test-0
  When you will finish you can "Save without submission" for later changes or "Submit to the production team".
Line: 129 to 122
 
META FILEATTACHMENT attachment="step_manager_page.png" attr="" comment="" date="1323351113" name="step_manager_page.png" path="step_manager_page.png" size="145697" stream="step_manager_page.png" tmpFilename="/usr/tmp/CGItemp58887" user="afalabel" version="1"
META FILEATTACHMENT attachment="New_Step.png" attr="" comment="" date="1323351678" name="New_Step.png" path="New_Step.png" size="49299" stream="New_Step.png" tmpFilename="/usr/tmp/CGItemp62819" user="afalabel" version="1"
META FILEATTACHMENT attachment="Request_manager.png" attr="" comment="" date="1323767578" name="Request_manager.png" path="Request_manager.png" size="192079" stream="Request_manager.png" tmpFilename="/usr/tmp/CGItemp53876" user="afalabel" version="1"
Deleted:
<
<
META FILEATTACHMENT attachment="New_Request.png" attr="" comment="" date="1323767799" name="New_Request.png" path="New_Request.png" size="12675" stream="New_Request.png" tmpFilename="/usr/tmp/CGItemp53810" user="afalabel" version="1"
 
META FILEATTACHMENT attachment="Prod_type.png" attr="" comment="" date="1323768397" name="Prod_type.png" path="Prod_type.png" size="192610" stream="Prod_type.png" tmpFilename="/usr/tmp/CGItemp53983" user="afalabel" version="1"
META FILEATTACHMENT attachment="New_request_form.png" attr="" comment="" date="1323769538" name="New_request_form.png" path="New_request_form.png" size="55330" stream="New_request_form.png" tmpFilename="/usr/tmp/CGItemp53926" user="afalabel" version="1"
META FILEATTACHMENT attachment="New_request_form2.png" attr="" comment="" date="1323774429" name="New_request_form2.png" path="New_request_form2.png" size="62553" stream="New_request_form2.png" tmpFilename="/usr/tmp/CGItemp58596" user="afalabel" version="1"
Line: 137 to 129
 
META FILEATTACHMENT attachment="Step_Manager_FINAL.png" attr="" comment="" date="1326118646" name="Step_Manager_FINAL.png" path="Step_Manager_FINAL.png" size="60684" stream="Step_Manager_FINAL.png" tmpFilename="/usr/tmp/CGItemp63075" user="afalabel" version="2"
META FILEATTACHMENT attachment="request_FINAL.png" attr="" comment="" date="1326119322" name="request_FINAL.png" path="request_FINAL.png" size="103068" stream="request_FINAL.png" tmpFilename="/usr/tmp/CGItemp63126" user="afalabel" version="1"
META FILEATTACHMENT attachment="Tutorial-Training.pdf" attr="" comment="Training tutorial slides" date="1351178340" name="Tutorial-Training.pdf" path="Tutorial-Training.pdf" size="2965355" user="fstagni" version="1"
Added:
>
>
META FILEATTACHMENT attachment="Step_1.png" attr="" comment="" date="1485530778" name="Step_1.png" path="Step_1.png" size="108775" user="afalabel" version="1"
META FILEATTACHMENT attachment="Step_2.png" attr="" comment="" date="1485530925" name="Step_2.png" path="Step_2.png" size="110854" user="afalabel" version="1"
META FILEATTACHMENT attachment="New_Request_type.png" attr="" comment="" date="1485532410" name="New_Request_type.png" path="New_Request_type.png" size="77274" user="afalabel" version="1"
META FILEATTACHMENT attachment="New_Request_submit.png" attr="" comment="" date="1485532410" name="New_Request_submit.png" path="New_Request_submit.png" size="120009" user="afalabel" version="1"
META FILEATTACHMENT attachment="New_Request_add_step.png" attr="" comment="" date="1485532410" name="New_Request_add_step.png" path="New_Request_add_step.png" size="100301" user="afalabel" version="1"
META FILEATTACHMENT attachment="New_Request_bkk.png" attr="" comment="" date="1485532410" name="New_Request_bkk.png" path="New_Request_bkk.png" size="74174" user="afalabel" version="1"
META FILEATTACHMENT attachment="New_Request_2.png" attr="" comment="" date="1485532410" name="New_Request_2.png" path="New_Request_2.png" size="107595" user="afalabel" version="1"
META FILEATTACHMENT attachment="New_Request.png" attr="" comment="" date="1485533115" name="New_Request.png" path="New_Request.png" size="84925" user="afalabel" version="1"

Revision 152017-01-27 - AntonioFalabella

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
-- AntonioFalabella - 08-Dec-2011
Line: 16 to 16
 

Working Group Software Package

At the time being LHCbDIRAC can run software that uses a release of the following main projects: Gauss, Boole, Brunel, DaVinci, LHCb and Moore. LHCbDIRAC runs the code with the aid of python options.
Changed:
<
<
The python options coded by the working groups must be released in a package that can be seen by SetupProject. So for example let's suppose you write your_script.py that uses the package extrapackage of the project Project . You will release your code such that the following lines can be issued:
>
>
The python options coded by the working groups must be released in a package that is known by SetupProject. So for example let's suppose you write your_script.py that uses the package extrapackage of the project Project . You will release your code such that the following lines can be issued:
 
Changed:
<
<
> SetupProject Project vXrY --use="extrapackage vXrY"
>
>
> SetupProject /
 > gaudirun.py path/your_script.py
where path means the path of your script in the lhcb releases tree.
Changed:
<
<
So let's make a working example. We have the python options BuBdBs_DST_selection.py which perfom a particular selection on stripped SDSTs. The options have been released under the DBASE package. The path is DBASE/WG/CPConfig/v1r6/scripts/. LHCbDIRAC can use the code setting the right environment:
>
>
So let's make a working example. We have the python options for the Erasmus package. You can set the right environment by issueing:
 
Changed:
<
<
> SetupProject --use="CPConfig v1r1" DaVinci v29r1

This command is exactly the same that one would run to test the BuBdBs_DST_selection.py options on lxplus. With the correct environment set up LHCbDIRAC can run the application (e.g. DaVinci) with the additional options:

> gaudirun.py   $CPCONFIGROOT/scripts/BuBdBs_DST_selection.py
>
>
> SetupProject --use=ProdConf --use=PIDCalib Erasmus v13r5p2
 

Some notes about software

Added:
>
>
Note the presence of the option --use=ProdConf, this sets up the prodconf package along with your application package. Although this is not strickly needed to run interactly it is used by LHCbDirac to determine some parameters at runtime. Among the others the filenames and filetypes. Anyone who want to run a production should test the code against prodconf in order to simulate the LHCbDirac behaviour.
 
Changed:
<
<
Unfortunately the LHCbDIRAC machinery exploits some limitations such that the setup of a customized production need to be tested carefully and some issues have to be fixed by hand. First of all if your gaudirun python options script includes some code that is not present in the release of the relative project the simplest way to use it is to release it with your script. Let's come back to the previous example of BuBdBs_DST_selection.py. If you look for the code in the releases of lhcb you can see that the script uses the python method DSTSelections present in the library BetaSReaders that is not part of DaVinci

from BetaSReaders import DSTSelections

In this case the BetaSReaders script has been released with the BuBdBs_DST_selection.py one.

Some other issues refers to the input output methods. It's important to not refer to a particular file in your code because this is handled by the production team. What is needed is the method used to write the output. In the BuBdBs_DST_selection.py for example the method used is SelDSTWriter. Usually the writers uses a parameter to define the filename. What is important for the prodution team is to be able to change the output filename. So a good way to proceed should be to comment the lines that write the output before releasing the code. Another option could be to simple remove the line but refer to them while submitting the request (as you will see in the production request paragraph).

>
>
The example above refers to SetupProject and not to the new environament tools because the compatibility is still not full.
 

Setting Up a New Step

In the LHCbDIRAC workflow for your script to be executed it needs to be included as a option file in a brand new workflow step. The step can be built by everyone that has lhcb_tech rights access to LHCbDIRAC web portal which can be

Revision 142014-05-20 - MarcoCattaneo

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
-- AntonioFalabella - 08-Dec-2011
Line: 11 to 11
 A workflow is basically a sequence of atomic processing steps that run specific LHCb software on a certain list of input data. Each step produces its output files so that within the workflow the output of each step feed in the next step. With LHCbDIRAC It is than possible to setup a production request with a customized workflow to meet the needs of working groups.
Changed:
<
<
This possibility can be helpful in situations where some task need to be performed on a well defined dataset in the bookkeping. The advantages for a working group to have it's own centralized productions are not negligible. First of all they can run on all the data present in the bookkeeping basically at run time without keeping track of new files published, because when a production have been started it will gather new file of the same type as they are produced.
>
>
This possibility can be helpful in situations where some task need to be performed on a well defined dataset in the bookkeping. The advantages for a working group to have its own centralized productions are not negligible. First of all they can run on all the data present in the bookkeeping basically at run time without keeping track of new files published, because when a production have been started it will gather new file of the same type as they are produced.
 The result are then published in the bookkeeping so that everyone can use them. Last but not least, when your are sure that your code works fine the control is in the hands of the prodution team that take care of the technical aspects.

Working Group Software Package

Revision 132014-02-27 - MarcoCattaneo

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
-- AntonioFalabella - 08-Dec-2011
Line: 11 to 11
 A workflow is basically a sequence of atomic processing steps that run specific LHCb software on a certain list of input data. Each step produces its output files so that within the workflow the output of each step feed in the next step. With LHCbDIRAC It is than possible to setup a production request with a customized workflow to meet the needs of working groups.
Changed:
<
<
This possibility can be helpful in situations where some task need to be performed on a well defined dataset in the bookkeping. The advantages for a worging group to have it's own centralized productions are not negligible. First of all they can run on all the data present in the bookkeeping basically at run time without keeping track of new files published, because when a production have been started it will gather new file of the same type as they are produced.
>
>
This possibility can be helpful in situations where some task need to be performed on a well defined dataset in the bookkeping. The advantages for a working group to have it's own centralized productions are not negligible. First of all they can run on all the data present in the bookkeeping basically at run time without keeping track of new files published, because when a production have been started it will gather new file of the same type as they are produced.
 The result are then published in the bookkeeping so that everyone can use them. Last but not least, when your are sure that your code works fine the control is in the hands of the prodution team that take care of the technical aspects.
Changed:
<
<

Worging Group Software Package

At the time being LHCbDIRAC can run software that uses a release of the following main projects: Gauss, Boole, Brunel, DaVinci, LHCb and Moore. LHCbDIRAC runs the code with the aid of gaudipython configurables. The gaudipython configurable coded by the working groups must be released in a package that can be seen by SetupProject. So for example let's suppose you write your_script.py that uses the package extrapackage of the project Project . You will release your code such that the following lines can be issued:
>
>

Working Group Software Package

At the time being LHCbDIRAC can run software that uses a release of the following main projects: Gauss, Boole, Brunel, DaVinci, LHCb and Moore. LHCbDIRAC runs the code with the aid of python options. The python options coded by the working groups must be released in a package that can be seen by SetupProject. So for example let's suppose you write your_script.py that uses the package extrapackage of the project Project . You will release your code such that the following lines can be issued:
 
> SetupProject Project vXrY --use="extrapackage vXrY"
Line: 24 to 24
  where path means the path of your script in the lhcb releases tree.
Changed:
<
<
So let's make a working example. We have the gaudipython configurable BuBdBs_DST_selection.py which perfomes a particular selection on stripped SDSTs. The configurable has been released under the DBASE package. It's path is DBASE/WG/CPConfig/v1r6/scripts/. LHCbDIRAC can use the code setting the right environment:
>
>
So let's make a working example. We have the python options BuBdBs_DST_selection.py which perfom a particular selection on stripped SDSTs. The options have been released under the DBASE package. The path is DBASE/WG/CPConfig/v1r6/scripts/. LHCbDIRAC can use the code setting the right environment:
 
> SetupProject --use="CPConfig v1r1"  DaVinci v29r1 
Changed:
<
<
This command is exactly the same that one would run to test the BuBdBs_DST_selection.py configurable on lxplus. With the correct environment set up LHCbDIRAC can run the application (e.g. DaVinci) with the addtional configurables:
>
>
This command is exactly the same that one would run to test the BuBdBs_DST_selection.py options on lxplus. With the correct environment set up LHCbDIRAC can run the application (e.g. DaVinci) with the additional options:
 
> gaudirun.py   $CPCONFIGROOT/scripts/BuBdBs_DST_selection.py
Line: 41 to 41
 

Some notes about software

Changed:
<
<
Unfortunatly the LHCbDIRAC machinery exploits some limitations such that the setup of a customized production need to be tested carefully and some issues have to be fixed by hand. First of all if your gaudirun configurable code include some code that is not present in the release of the relative project the simplest way to use it is to release it with your configurable.
>
>
Unfortunately the LHCbDIRAC machinery exploits some limitations such that the setup of a customized production need to be tested carefully and some issues have to be fixed by hand. First of all if your gaudirun python options script includes some code that is not present in the release of the relative project the simplest way to use it is to release it with your script.
 Let's come back to the previous example of BuBdBs_DST_selection.py. If you look for the code in the releases of lhcb you can see that the script uses the python method DSTSelections present in the library BetaSReaders that is not part of DaVinci

Revision 122012-10-25 - FedericoStagni

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
-- AntonioFalabella - 08-Dec-2011
Line: 142 to 142
  -- AntonioFalabella - 09-Jan-2012
Added:
>
>
 
META FILEATTACHMENT attachment="step_manager_page.png" attr="" comment="" date="1323351113" name="step_manager_page.png" path="step_manager_page.png" size="145697" stream="step_manager_page.png" tmpFilename="/usr/tmp/CGItemp58887" user="afalabel" version="1"
META FILEATTACHMENT attachment="New_Step.png" attr="" comment="" date="1323351678" name="New_Step.png" path="New_Step.png" size="49299" stream="New_Step.png" tmpFilename="/usr/tmp/CGItemp62819" user="afalabel" version="1"
META FILEATTACHMENT attachment="Request_manager.png" attr="" comment="" date="1323767578" name="Request_manager.png" path="Request_manager.png" size="192079" stream="Request_manager.png" tmpFilename="/usr/tmp/CGItemp53876" user="afalabel" version="1"
Line: 152 to 154
 
META FILEATTACHMENT attachment="New_request_form3.png" attr="" comment="" date="1323792504" name="New_request_form3.png" path="New_request_form3.png" size="120907" stream="New_request_form3.png" tmpFilename="/usr/tmp/CGItemp56230" user="afalabel" version="1"
META FILEATTACHMENT attachment="Step_Manager_FINAL.png" attr="" comment="" date="1326118646" name="Step_Manager_FINAL.png" path="Step_Manager_FINAL.png" size="60684" stream="Step_Manager_FINAL.png" tmpFilename="/usr/tmp/CGItemp63075" user="afalabel" version="2"
META FILEATTACHMENT attachment="request_FINAL.png" attr="" comment="" date="1326119322" name="request_FINAL.png" path="request_FINAL.png" size="103068" stream="request_FINAL.png" tmpFilename="/usr/tmp/CGItemp63126" user="afalabel" version="1"
Added:
>
>
META FILEATTACHMENT attachment="Tutorial-Training.pdf" attr="" comment="Training tutorial slides" date="1351178340" name="Tutorial-Training.pdf" path="Tutorial-Training.pdf" size="2965355" user="fstagni" version="1"

Revision 112012-01-09 - AntonioFalabella

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
-- AntonioFalabella - 08-Dec-2011
Line: 31 to 31
 > SetupProject --use="CPConfig v1r1" DaVinci v29r1
Changed:
<
<
This command is exactly the same that one would run to test the BuBdBs_DST_selection.py configurable in lxplus.
>
>
This command is exactly the same that one would run to test the BuBdBs_DST_selection.py configurable on lxplus.
 With the correct environment set up LHCbDIRAC can run the application (e.g. DaVinci) with the addtional configurables:
Line: 41 to 41
 

Some notes about software

Changed:
<
<
Unfortunatly the LHCbDIRAC machinery exploits some limitations such that the setup of a customized production need to be tested carefully and some issues have to be fixed by hand. First of all if your gaudirun configurable code include some code that is not present in the release of the relative project the simplest way to use it is to release it with the your configurable. Let's come back to the previous example of BuBdBs_DST_selection.py. If look for the code in the releases are of lhcb you can see that the script include the library that is not part of DaVinci
>
>
Unfortunatly the LHCbDIRAC machinery exploits some limitations such that the setup of a customized production need to be tested carefully and some issues have to be fixed by hand. First of all if your gaudirun configurable code include some code that is not present in the release of the relative project the simplest way to use it is to release it with your configurable. Let's come back to the previous example of BuBdBs_DST_selection.py. If you look for the code in the releases of lhcb you can see that the script uses the python method DSTSelections present in the library BetaSReaders that is not part of DaVinci
 
from BetaSReaders import DSTSelections
Line: 50 to 50
  In this case the BetaSReaders script has been released with the BuBdBs_DST_selection.py one.
Changed:
<
<
Some other issues refers to the input output. It's important to not refer to a particular file in your code because this is handled by the production team. What is needed is the method used to write the output. In the BuBdBs_DST_selection.py
>
>
Some other issues refers to the input output methods. It's important to not refer to a particular file in your code because this is handled by the production team. What is needed is the method used to write the output. In the BuBdBs_DST_selection.py
 for example the method used is SelDSTWriter. Usually the writers uses a parameter to define the filename. What is important for the prodution team is to be able to change the output filename. So a good way to proceed should be to comment the lines that write the output before releasing the code. Another option could be to simple remove the line but refer to them while submitting the request (as you will see in the production request paragraph).
Line: 72 to 72
 Here you have to fill in the entries corresponding to your case:

  • Name: Somestring meaningful for you (please put here only alphanumeric characters - better if you also avoid blank spaces e.g. WG_Selection_DVv29r2_CPConfigv1r5)
Changed:
<
<
  • Processin Pass: Something meaningful that remind your input dataset (e.g. RecoXX-StrippingXX)
  • Application: Here you can choose in the list of possible projects and versions.
>
>
  • Processing Pass: Something meaningful that remind your input dataset (e.g. RecoXX-StrippingXX)
  • Application: Here you can choose between the list of possible projects and versions.
 
  • Option Files: This the path of your script.
  • Extra packages: Here you have to issue the package that contain your script. The syntax here is that of SetupProject i.e. Package.vxry.
  • Runtime project: Additional project that can be used. Check the SetupProject page for the list of projects.
  • CondDB: Choose the conditions CondDB tag.
  • DDDB: Choose the conditions DDDB tag.
Changed:
<
<
  • Visible: Visibility flag. If it's Y a folder with the output filename of your production will b created in the bookkeeping.
  • Usable: Y if you want to use the step.
  • Input File Type: In this form you can choose the file name of input files. The file name are registred in bookkeeping so you can choose an existing one or Add a brand new file name. Usually you don't need to create a new filename for input because you pick up file from bookkeeping and by definition they are already there.
>
>
  • Visible: Visibility flag. If it's Y a folder containing the processing pass of the step will be created in the bookkeeping.
  • Usable: Y if you want to use the step. If you plain to update your step with a new one it's better to set this flag to obsolete so that the step will no more appear when creating request production( to understand why read the next paragraph production request paragraph )
  • Input File Type: In this form you can choose the file name of input files. The file name are registred in bookkeeping so you can choose an existing one or Add a brand new file name. Usually you don't need to create a new filename for the input because you pick up file from bookkeeping and by definition they are already there.
 
  • Output File Type: As before you can choose an existing filename or create a brand new one that will be registered in the bookkeeping. It's reasonable here that you need to input your filename, so even if there is no particular rule yet it is recommended to follow the typing schema of the other filenames.

test-0
Line: 89 to 89
 At this point you can click on Save and your step will appear in the list of Registered Steps.

Submit a Production Request

Changed:
<
<
When your step or steps are ready you can submit a new Request. This can be done for dirac web portal:
>
>
When your step or steps are ready you can submit a new Request. This can be done from the dirac web portal:
 
test-0
Changed:
<
<
To submit a nw request you have to login as lhcb_user
>
>
To submit a new request you have to login as lhcb_user
 
test-0
Line: 108 to 108
  Request pad
  • Name: Somestring meaningful for you (please put here only alphanumeric characters - better if you also avoid blank spaces e.g. Name_of_the_WG_Selection_Additional_infos)
Changed:
<
<
  • Information also: Here you can add some email of people that you know they have to know about your production. By default all people of production team receive an email when you submit a production request.
>
>
  • Information also: Here you can add some email of people that you think they have to know about your production. By default all people of the production team receive an email when a production request has been submitted.
 
test-0
Line: 124 to 124
 
test-0
Changed:
<
<
Event pad
>
>
Event pad_
 Choose how many events you want to run (usually -1 - all events).

Comments

Changed:
<
<
Here you can wrie some comments useful for production team. So far the most important additional information that the prodution team need to know is the method used for writing the output and from you include that. For example in the Comments pad you would write:
>
>
Here you can wrie some comments useful for the production team. So far the most important additional information that the prodution team need to know is the method used for writing the output. For example in the Comments pad you would write:
 
from library import class.method
method()=outputfilename.extension
Line: 140 to 140
  At this the production team take care of the technical details, make some preliminary tests.
Added:
>
>
-- AntonioFalabella - 09-Jan-2012
 
META FILEATTACHMENT attachment="step_manager_page.png" attr="" comment="" date="1323351113" name="step_manager_page.png" path="step_manager_page.png" size="145697" stream="step_manager_page.png" tmpFilename="/usr/tmp/CGItemp58887" user="afalabel" version="1"
META FILEATTACHMENT attachment="New_Step.png" attr="" comment="" date="1323351678" name="New_Step.png" path="New_Step.png" size="49299" stream="New_Step.png" tmpFilename="/usr/tmp/CGItemp62819" user="afalabel" version="1"

Revision 102012-01-09 - AntonioFalabella

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
-- AntonioFalabella - 08-Dec-2011
Line: 39 to 39
 
Changed:
<
<

Some note about software

>
>

Some notes about software

Unfortunatly the LHCbDIRAC machinery exploits some limitations such that the setup of a customized production need to be tested carefully and some issues have to be fixed by hand. First of all if your gaudirun configurable code include some code that is not present in the release of the relative project the simplest way to use it is to release it with the your configurable. Let's come back to the previous example of BuBdBs_DST_selection.py. If look for the code in the releases are of lhcb you can see that the script include the library that is not part of DaVinci

from BetaSReaders import DSTSelections

In this case the BetaSReaders script has been released with the BuBdBs_DST_selection.py one.

Some other issues refers to the input output. It's important to not refer to a particular file in your code because this is handled by the production team. What is needed is the method used to write the output. In the BuBdBs_DST_selection.py for example the method used is SelDSTWriter. Usually the writers uses a parameter to define the filename. What is important for the prodution team is to be able to change the output filename. So a good way to proceed should be to comment the lines that write the output before releasing the code. Another option could be to simple remove the line but refer to them while submitting the request (as you will see in the production request paragraph).

 
Deleted:
<
<
It's important to not refer to a particular file in your code because this is handled by the production team. What is needed is the method used to write the output. So for example a good way to procede to help the production team should be to comment the line that write the output before releasing the code. Another option could be to simple remove the line but refer to them while submitting the request (as you will see in the production request paragraph)
 

Setting Up a New Step

Changed:
<
<
In the LHCbDIRAC workflow for your script to be executed it needs to be included as a option file in a brand new workflow step. The step can be built by everyone that has lhcb_tech rights access to LHCbDIRAC web portal which can be reached through this link:
>
>
In the LHCbDIRAC workflow for your script to be executed it needs to be included as a option file in a brand new workflow step. The step can be built by everyone that has lhcb_tech rights access to LHCbDIRAC web portal which can be reached through this link:
 
Line: 71 to 84
 
  • Input File Type: In this form you can choose the file name of input files. The file name are registred in bookkeeping so you can choose an existing one or Add a brand new file name. Usually you don't need to create a new filename for input because you pick up file from bookkeeping and by definition they are already there.
  • Output File Type: As before you can choose an existing filename or create a brand new one that will be registered in the bookkeeping. It's reasonable here that you need to input your filename, so even if there is no particular rule yet it is recommended to follow the typing schema of the other filenames.
Added:
>
>
test-0
 At this point you can click on Save and your step will appear in the list of Registered Steps.

Submit a Production Request

Line: 119 to 134
 method()=outputfilename.extension
Changed:
<
<
>
>
test-0
  When you will finish you can "Save without submission" for later changes or "Submit to the production team".
Line: 134 to 149
 
META FILEATTACHMENT attachment="New_request_form.png" attr="" comment="" date="1323769538" name="New_request_form.png" path="New_request_form.png" size="55330" stream="New_request_form.png" tmpFilename="/usr/tmp/CGItemp53926" user="afalabel" version="1"
META FILEATTACHMENT attachment="New_request_form2.png" attr="" comment="" date="1323774429" name="New_request_form2.png" path="New_request_form2.png" size="62553" stream="New_request_form2.png" tmpFilename="/usr/tmp/CGItemp58596" user="afalabel" version="1"
META FILEATTACHMENT attachment="New_request_form3.png" attr="" comment="" date="1323792504" name="New_request_form3.png" path="New_request_form3.png" size="120907" stream="New_request_form3.png" tmpFilename="/usr/tmp/CGItemp56230" user="afalabel" version="1"
Added:
>
>
META FILEATTACHMENT attachment="Step_Manager_FINAL.png" attr="" comment="" date="1326118646" name="Step_Manager_FINAL.png" path="Step_Manager_FINAL.png" size="60684" stream="Step_Manager_FINAL.png" tmpFilename="/usr/tmp/CGItemp63075" user="afalabel" version="2"
META FILEATTACHMENT attachment="request_FINAL.png" attr="" comment="" date="1326119322" name="request_FINAL.png" path="request_FINAL.png" size="103068" stream="request_FINAL.png" tmpFilename="/usr/tmp/CGItemp63126" user="afalabel" version="1"

Revision 92012-01-09 - AntonioFalabella

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
-- AntonioFalabella - 08-Dec-2011
Line: 15 to 15
 The result are then published in the bookkeeping so that everyone can use them. Last but not least, when your are sure that your code works fine the control is in the hands of the prodution team that take care of the technical aspects.

Worging Group Software Package

Changed:
<
<
At the time being LHCbDIRAC can run software that uses a release of the following main projects: Gauss, Boole, Brunel, DaVinci, LHCb and Moore. LHCbDIRAC runs the code with the aid of gaudipython configurables. So for example The scripts coded by the working groups must be released in a package that can be seen by SetupProject. So for example let's suppose you write your_script.py that uses the package extrapackage of the project Project . You will release your code such that the following lines can be issued:
>
>
At the time being LHCbDIRAC can run software that uses a release of the following main projects: Gauss, Boole, Brunel, DaVinci, LHCb and Moore. LHCbDIRAC runs the code with the aid of gaudipython configurables. The gaudipython configurable coded by the working groups must be released in a package that can be seen by SetupProject. So for example let's suppose you write your_script.py that uses the package extrapackage of the project Project . You will release your code such that the following lines can be issued:
 
> SetupProject Project vXrY --use="extrapackage vXrY"
Line: 22 to 22
 > SetupProject Project vXrY --use="extrapackage vXrY"
> gaudirun.py path/your_script.py
Deleted:
<
<
 where path means the path of your script in the lhcb releases tree.
Added:
>
>
So let's make a working example. We have the gaudipython configurable BuBdBs_DST_selection.py which perfomes a particular selection on stripped SDSTs. The configurable has been released under the DBASE package. It's path is DBASE/WG/CPConfig/v1r6/scripts/. LHCbDIRAC can use the code setting the right environment:

> SetupProject --use="CPConfig v1r1"  DaVinci v29r1 

This command is exactly the same that one would run to test the BuBdBs_DST_selection.py configurable in lxplus. With the correct environment set up LHCbDIRAC can run the application (e.g. DaVinci) with the addtional configurables:

> gaudirun.py   $CPCONFIGROOT/scripts/BuBdBs_DST_selection.py
 

Some note about software

It's important to not refer to a particular file in your code because this is handled by the production team. What is needed is the method used to write the output. So for example a good way to procede to help the production team should be to comment the line that write the output before releasing the code. Another option could be to simple remove the line but refer to them while submitting the request (as you will see in the production request paragraph)

Revision 82012-01-09 - AntonioFalabella

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
-- AntonioFalabella - 08-Dec-2011
Line: 14 to 14
 This possibility can be helpful in situations where some task need to be performed on a well defined dataset in the bookkeping. The advantages for a worging group to have it's own centralized productions are not negligible. First of all they can run on all the data present in the bookkeeping basically at run time without keeping track of new files published, because when a production have been started it will gather new file of the same type as they are produced. The result are then published in the bookkeeping so that everyone can use them. Last but not least, when your are sure that your code works fine the control is in the hands of the prodution team that take care of the technical aspects.
Changed:
<
<

Software Prerequisites

At the time being LHCbDIRAC can run software that uses the a release of the following main projects: Gauss, Boole, Brunel, DaVinci, LHCb and Moore. The scripts coded by the working groups must be released in a package that can be seen by SetupProject. So for example let's suppose you write your_script.py that uses the package extrapackage of the project Project . You will release your code such that the following lines can be issued:

>
>

Worging Group Software Package

At the time being LHCbDIRAC can run software that uses a release of the following main projects: Gauss, Boole, Brunel, DaVinci, LHCb and Moore. LHCbDIRAC runs the code with the aid of gaudipython configurables. So for example The scripts coded by the working groups must be released in a package that can be seen by SetupProject. So for example let's suppose you write your_script.py that uses the package extrapackage of the project Project . You will release your code such that the following lines can be issued:
 
> SetupProject Project vXrY --use="extrapackage vXrY"

Revision 72011-12-13 - AntonioFalabella

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
-- AntonioFalabella - 08-Dec-2011
Line: 16 to 16
 

Software Prerequisites

Changed:
<
<
At the time being LHCbDIRAC can run software that uses the a release of the following main projects: Gauss, Boole, Brunel, DaVinci, LHCb and Moore. The scripts coded by the working groups must be released in a package that can be seen by SetupProject. So for example let's suppose you write. You will release your code under a lhcb software package such that the following line can be issued:
>
>
At the time being LHCbDIRAC can run software that uses the a release of the following main projects: Gauss, Boole, Brunel, DaVinci, LHCb and Moore. The scripts coded by the working groups must be released in a package that can be seen by SetupProject. So for example let's suppose you write your_script.py that uses the package extrapackage of the project Project . You will release your code such that the following lines can be issued:
 
> SetupProject Project vXrY --use="extrapackage vXrY"
> gaudirun.py path/your_script.py
Changed:
<
<
where path means the path of you script in the lhcb releases tree.
>
>
where path means the path of your script in the lhcb releases tree.
 

Some note about software

Changed:
<
<
It's important no not refer to a particular file in your code because this is handled on the production side. What is needed is the method used to write the output. So for example a good way to procede to help the production team should to comment the line the write the output before releasing the code. Another option could be to simple remove the line but refer to them while submitting the request (as you will see in the production request paragraph)
>
>
It's important to not refer to a particular file in your code because this is handled by the production team. What is needed is the method used to write the output. So for example a good way to procede to help the production team should be to comment the line that write the output before releasing the code. Another option could be to simple remove the line but refer to them while submitting the request (as you will see in the production request paragraph)
 

Setting Up a New Step

Changed:
<
<
In the LHCbDIRAC workflow for your script to be executed it needs to be included as a option file in brand new workflow step. The step can be built by everyone has lhcb_tech rights access to LHCbDIRAC web portal which can be reached through this link:
>
>
In the LHCbDIRAC workflow for your script to be executed it needs to be included as a option file in a brand new workflow step. The step can be built by everyone that has lhcb_tech rights access to LHCbDIRAC web portal which can be reached through this link:
 
Changed:
<
<
To access this page you need to load in your browser a valid grid certificate. In the example below you can the step manager page.
>
>
To access this page you need to load in your browser a valid grid certificate. In the example below you can see the step manager page.
 
test-0
Line: 42 to 42
 

test-0
Changed:
<
<
Here you have to fill in the entries the with your case:
>
>
Here you have to fill in the entries corresponding to your case:
 
  • Name: Somestring meaningful for you (please put here only alphanumeric characters - better if you also avoid blank spaces e.g. WG_Selection_DVv29r2_CPConfigv1r5)
Changed:
<
<
  • Processin Pass: Something meaningful that refer to your input dataset (e.g. RecoXX-StrippingXX)
>
>
  • Processin Pass: Something meaningful that remind your input dataset (e.g. RecoXX-StrippingXX)
 
  • Application: Here you can choose in the list of possible projects and versions.
  • Option Files: This the path of your script.
  • Extra packages: Here you have to issue the package that contain your script. The syntax here is that of SetupProject i.e. Package.vxry.
Line: 57 to 57
 
  • Input File Type: In this form you can choose the file name of input files. The file name are registred in bookkeeping so you can choose an existing one or Add a brand new file name. Usually you don't need to create a new filename for input because you pick up file from bookkeeping and by definition they are already there.
  • Output File Type: As before you can choose an existing filename or create a brand new one that will be registered in the bookkeeping. It's reasonable here that you need to input your filename, so even if there is no particular rule yet it is recommended to follow the typing schema of the other filenames.
Changed:
<
<
At this point you click can on Save and you step will appear in the list of Registered Steps.
>
>
At this point you can click on Save and your step will appear in the list of Registered Steps.
 

Submit a Production Request

Changed:
<
<
After your step of steps have been submitted you can submit a new Request. This can be done for dirac web portal:
>
>
When your step or steps are ready you can submit a new Request. This can be done for dirac web portal:
 
test-0
Changed:
<
<
To do this you have to be lhcb_user
>
>
To submit a nw request you have to login as lhcb_user
 
test-0
Line: 73 to 73
 
test-0

If you need to run your code on stripped DSTs you will choose Stripping.

Changed:
<
<
After this the New_request for will be opened and you need to fill the relevant fields
>
>
After this the New_request form will be opened:
 
test-0

Request pad

  • Name: Somestring meaningful for you (please put here only alphanumeric characters - better if you also avoid blank spaces e.g. Name_of_the_WG_Selection_Additional_infos)
Changed:
<
<
  • Information also: Here you can add some email of people they you suppose they need to know about your request. By default all people of production team receive a mail when you submit a production request.
>
>
  • Information also: Here you can add some email of people that you know they have to know about your production. By default all people of production team receive an email when you submit a production request.
 
test-0

Input data pad

  • Select from BK: First select your input data from the bookkeeping. In the example it's been selected the DIMUON stream of Stripping17.
  • DQ flag: Here select OK.
Changed:
<
<
  • Production: Here can select a particular production where to run your code. If you are interested on all the productions select ALL.
  • TCKs: Here you can even choose to run on particular trigger configurations. Choose ALL if you need on all the possible configurations.
>
>
  • Production: Here you can select a particular production on which run your code. If you are interested on all the productions select ALL.
  • TCKs: Here you can even choose to run on some particular trigger configurations. Choose 0 if you need to run on all the possible configurations.
 

Added:
>
>
Processing pass() pad Here you add the step (or steps) you prepared before. If you click on Add step the system list you all the steps with the input type specified in the Input data pad. This will work the same for the others steps. The system let you choose the steps compatible with the output data type of the previous step.

test-0

Event pad Choose how many events you want to run (usually -1 - all events).

Comments Here you can wrie some comments useful for production team. So far the most important additional information that the prodution team need to know is the method used for writing the output and from you include that. For example in the Comments pad you would write:

from library import class.method
method()=outputfilename.extension

When you will finish you can "Save without submission" for later changes or "Submit to the production team".

At this the production team take care of the technical details, make some preliminary tests.

 
META FILEATTACHMENT attachment="step_manager_page.png" attr="" comment="" date="1323351113" name="step_manager_page.png" path="step_manager_page.png" size="145697" stream="step_manager_page.png" tmpFilename="/usr/tmp/CGItemp58887" user="afalabel" version="1"
META FILEATTACHMENT attachment="New_Step.png" attr="" comment="" date="1323351678" name="New_Step.png" path="New_Step.png" size="49299" stream="New_Step.png" tmpFilename="/usr/tmp/CGItemp62819" user="afalabel" version="1"
Line: 98 to 119
 
META FILEATTACHMENT attachment="Prod_type.png" attr="" comment="" date="1323768397" name="Prod_type.png" path="Prod_type.png" size="192610" stream="Prod_type.png" tmpFilename="/usr/tmp/CGItemp53983" user="afalabel" version="1"
META FILEATTACHMENT attachment="New_request_form.png" attr="" comment="" date="1323769538" name="New_request_form.png" path="New_request_form.png" size="55330" stream="New_request_form.png" tmpFilename="/usr/tmp/CGItemp53926" user="afalabel" version="1"
META FILEATTACHMENT attachment="New_request_form2.png" attr="" comment="" date="1323774429" name="New_request_form2.png" path="New_request_form2.png" size="62553" stream="New_request_form2.png" tmpFilename="/usr/tmp/CGItemp58596" user="afalabel" version="1"
Added:
>
>
META FILEATTACHMENT attachment="New_request_form3.png" attr="" comment="" date="1323792504" name="New_request_form3.png" path="New_request_form3.png" size="120907" stream="New_request_form3.png" tmpFilename="/usr/tmp/CGItemp56230" user="afalabel" version="1"

Revision 62011-12-13 - AntonioFalabella

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
-- AntonioFalabella - 08-Dec-2011
Line: 64 to 64
 
test-0
Changed:
<
<
To do this you have to be lhcb_user
>
>
To do this you have to be lhcb_user
 
test-0
Line: 73 to 73
 
test-0

If you need to run your code on stripped DSTs you will choose Stripping.

Added:
>
>
After this the New_request for will be opened and you need to fill the relevant fields

test-0

Request pad

  • Name: Somestring meaningful for you (please put here only alphanumeric characters - better if you also avoid blank spaces e.g. Name_of_the_WG_Selection_Additional_infos)
  • Information also: Here you can add some email of people they you suppose they need to know about your request. By default all people of production team receive a mail when you submit a production request.

test-0

Input data pad

  • Select from BK: First select your input data from the bookkeeping. In the example it's been selected the DIMUON stream of Stripping17.
  • DQ flag: Here select OK.
  • Production: Here can select a particular production where to run your code. If you are interested on all the productions select ALL.
  • TCKs: Here you can even choose to run on particular trigger configurations. Choose ALL if you need on all the possible configurations.

 
META FILEATTACHMENT attachment="step_manager_page.png" attr="" comment="" date="1323351113" name="step_manager_page.png" path="step_manager_page.png" size="145697" stream="step_manager_page.png" tmpFilename="/usr/tmp/CGItemp58887" user="afalabel" version="1"
META FILEATTACHMENT attachment="New_Step.png" attr="" comment="" date="1323351678" name="New_Step.png" path="New_Step.png" size="49299" stream="New_Step.png" tmpFilename="/usr/tmp/CGItemp62819" user="afalabel" version="1"
Line: 80 to 97
 
META FILEATTACHMENT attachment="New_Request.png" attr="" comment="" date="1323767799" name="New_Request.png" path="New_Request.png" size="12675" stream="New_Request.png" tmpFilename="/usr/tmp/CGItemp53810" user="afalabel" version="1"
META FILEATTACHMENT attachment="Prod_type.png" attr="" comment="" date="1323768397" name="Prod_type.png" path="Prod_type.png" size="192610" stream="Prod_type.png" tmpFilename="/usr/tmp/CGItemp53983" user="afalabel" version="1"
META FILEATTACHMENT attachment="New_request_form.png" attr="" comment="" date="1323769538" name="New_request_form.png" path="New_request_form.png" size="55330" stream="New_request_form.png" tmpFilename="/usr/tmp/CGItemp53926" user="afalabel" version="1"
Added:
>
>
META FILEATTACHMENT attachment="New_request_form2.png" attr="" comment="" date="1323774429" name="New_request_form2.png" path="New_request_form2.png" size="62553" stream="New_request_form2.png" tmpFilename="/usr/tmp/CGItemp58596" user="afalabel" version="1"

Revision 52011-12-13 - AntonioFalabella

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
-- AntonioFalabella - 08-Dec-2011
Line: 12 to 12
 With LHCbDIRAC It is than possible to setup a production request with a customized workflow to meet the needs of working groups.

This possibility can be helpful in situations where some task need to be performed on a well defined dataset in the bookkeping. The advantages for a worging group to have it's own centralized productions are not negligible. First of all they can run on all the data present in the bookkeeping basically at run time without keeping track of new files published, because when a production have been started it will gather new file of the same type as they are produced.

Changed:
<
<
The result are then published in the bookkeeping so that everyone can use them. Last but not least, when your are sure that your code works fine the control is in the hans of prodution team that take care of the technical aspects.
>
>
The result are then published in the bookkeeping so that everyone can use them. Last but not least, when your are sure that your code works fine the control is in the hands of the prodution team that take care of the technical aspects.
 

Software Prerequisites

Line: 57 to 57
 
  • Input File Type: In this form you can choose the file name of input files. The file name are registred in bookkeeping so you can choose an existing one or Add a brand new file name. Usually you don't need to create a new filename for input because you pick up file from bookkeeping and by definition they are already there.
  • Output File Type: As before you can choose an existing filename or create a brand new one that will be registered in the bookkeeping. It's reasonable here that you need to input your filename, so even if there is no particular rule yet it is recommended to follow the typing schema of the other filenames.
Changed:
<
<
At this point you click on Save and you step will appear in the list of Registered Steps.
>
>
At this point you click can on Save and you step will appear in the list of Registered Steps.
 

Submit a Production Request

Added:
>
>
After your step of steps have been submitted you can submit a new Request. This can be done for dirac web portal:

test-0

To do this you have to be lhcb_user

test-0

Doing this a New request button will appear, click on it and choose the production type.

test-0

If you need to run your code on stripped DSTs you will choose Stripping.

 
META FILEATTACHMENT attachment="step_manager_page.png" attr="" comment="" date="1323351113" name="step_manager_page.png" path="step_manager_page.png" size="145697" stream="step_manager_page.png" tmpFilename="/usr/tmp/CGItemp58887" user="afalabel" version="1"
META FILEATTACHMENT attachment="New_Step.png" attr="" comment="" date="1323351678" name="New_Step.png" path="New_Step.png" size="49299" stream="New_Step.png" tmpFilename="/usr/tmp/CGItemp62819" user="afalabel" version="1"
Added:
>
>
META FILEATTACHMENT attachment="Request_manager.png" attr="" comment="" date="1323767578" name="Request_manager.png" path="Request_manager.png" size="192079" stream="Request_manager.png" tmpFilename="/usr/tmp/CGItemp53876" user="afalabel" version="1"
META FILEATTACHMENT attachment="New_Request.png" attr="" comment="" date="1323767799" name="New_Request.png" path="New_Request.png" size="12675" stream="New_Request.png" tmpFilename="/usr/tmp/CGItemp53810" user="afalabel" version="1"
META FILEATTACHMENT attachment="Prod_type.png" attr="" comment="" date="1323768397" name="Prod_type.png" path="Prod_type.png" size="192610" stream="Prod_type.png" tmpFilename="/usr/tmp/CGItemp53983" user="afalabel" version="1"
META FILEATTACHMENT attachment="New_request_form.png" attr="" comment="" date="1323769538" name="New_request_form.png" path="New_request_form.png" size="55330" stream="New_request_form.png" tmpFilename="/usr/tmp/CGItemp53926" user="afalabel" version="1"

Revision 42011-12-08 - AntonioFalabella

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
-- AntonioFalabella - 08-Dec-2011
Line: 7 to 7
 

Introduction

Changed:
<
<
LHCbDIRAC is used to run centralized processing tasks on selected datasets. A processing task in LHCbDIRAC jargon is referred as a Production. A Production is set up keeping in mind the abstract concept of workflow. A workflow is basically a sequence of atomic processing steps that run specific LHCb software on a certain list of input data. Each step produces its output files so that in the workflow the output of each step feed the next step. With LHCbDIRAC It is than possible to setup a production request with a customized workflow to meet the needs of working groups. This possibility can be helpful in situations where some task need to be performed on a well defined dataset in the bookkeping. The advantages of this centralized
>
>
LHCbDIRAC is used to run centralized processing tasks on selected datasets. A processing task in LHCbDIRAC jargon is referred to as a Production. A Production is set up keeping in mind the abstract concept of workflow. A workflow is basically a sequence of atomic processing steps that run specific LHCb software on a certain list of input data. Each step produces its output files so that within the workflow the output of each step feed in the next step. With LHCbDIRAC It is than possible to setup a production request with a customized workflow to meet the needs of working groups.

This possibility can be helpful in situations where some task need to be performed on a well defined dataset in the bookkeping. The advantages for a worging group to have it's own centralized productions are not negligible. First of all they can run on all the data present in the bookkeeping basically at run time without keeping track of new files published, because when a production have been started it will gather new file of the same type as they are produced. The result are then published in the bookkeeping so that everyone can use them. Last but not least, when your are sure that your code works fine the control is in the hans of prodution team that take care of the technical aspects.

 

Software Prerequisites

Line: 21 to 25
  where path means the path of you script in the lhcb releases tree.

Added:
>
>

Some note about software

It's important no not refer to a particular file in your code because this is handled on the production side. What is needed is the method used to write the output. So for example a good way to procede to help the production team should to comment the line the write the output before releasing the code. Another option could be to simple remove the line but refer to them while submitting the request (as you will see in the production request paragraph)

 

Setting Up a New Step

In the LHCbDIRAC workflow for your script to be executed it needs to be included as a option file in brand new workflow step. The step can be built by everyone has lhcb_tech rights access to LHCbDIRAC web portal which can be reached through this link:
Line: 51 to 59
  At this point you click on Save and you step will appear in the list of Registered Steps.
Changed:
<
<

Submit a Production Request

>
>

Submit a Production Request

 
META FILEATTACHMENT attachment="step_manager_page.png" attr="" comment="" date="1323351113" name="step_manager_page.png" path="step_manager_page.png" size="145697" stream="step_manager_page.png" tmpFilename="/usr/tmp/CGItemp58887" user="afalabel" version="1"
META FILEATTACHMENT attachment="New_Step.png" attr="" comment="" date="1323351678" name="New_Step.png" path="New_Step.png" size="49299" stream="New_Step.png" tmpFilename="/usr/tmp/CGItemp62819" user="afalabel" version="1"

Revision 32011-12-08 - AntonioFalabella

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
-- AntonioFalabella - 08-Dec-2011
Line: 7 to 7
 

Introduction

Changed:
<
<
Within LHCbDIRAC It is possible to setup a production request with a customized workflow to meet the needs of working groups. This possibility can be helpful in situations where some task need to be performed on a well defined dataset in the bookkeping.
>
>
LHCbDIRAC is used to run centralized processing tasks on selected datasets. A processing task in LHCbDIRAC jargon is referred as a Production. A Production is set up keeping in mind the abstract concept of workflow. A workflow is basically a sequence of atomic processing steps that run specific LHCb software on a certain list of input data. Each step produces its output files so that in the workflow the output of each step feed the next step. With LHCbDIRAC It is than possible to setup a production request with a customized workflow to meet the needs of working groups. This possibility can be helpful in situations where some task need to be performed on a well defined dataset in the bookkeping. The advantages of this centralized
 

Software Prerequisites

Changed:
<
<
At the time being LHCbDIRAC can run software that uses the a release of the following main projects: Gauss, Boole, Brunel, DaVinci and Moore. The scripts coded by the working groups must be released in a package that can be seen by SetupProject. So for example let's suppose you write. You will release your code under a lhcb software package such that the following line can be issued:
>
>
At the time being LHCbDIRAC can run software that uses the a release of the following main projects: Gauss, Boole, Brunel, DaVinci, LHCb and Moore. The scripts coded by the working groups must be released in a package that can be seen by SetupProject. So for example let's suppose you write. You will release your code under a lhcb software package such that the following line can be issued:
 
> SetupProject Project vXrY --use="extrapackage vXrY"
Line: 21 to 22
 where path means the path of you script in the lhcb releases tree.

Setting Up a New Step

Changed:
<
<
In the LHCbDIRAC workflow the execution your
>
>
In the LHCbDIRAC workflow for your script to be executed it needs to be included as a option file in brand new workflow step. The step can be built by everyone has lhcb_tech rights access to LHCbDIRAC web portal which can be reached through this link:

To access this page you need to load in your browser a valid grid certificate. In the example below you can the step manager page.

test-0

Click on New Step on the bottom bar to access the new step configuration form

test-0
Here you have to fill in the entries the with your case:

  • Name: Somestring meaningful for you (please put here only alphanumeric characters - better if you also avoid blank spaces e.g. WG_Selection_DVv29r2_CPConfigv1r5)
  • Processin Pass: Something meaningful that refer to your input dataset (e.g. RecoXX-StrippingXX)
  • Application: Here you can choose in the list of possible projects and versions.
  • Option Files: This the path of your script.
  • Extra packages: Here you have to issue the package that contain your script. The syntax here is that of SetupProject i.e. Package.vxry.
  • Runtime project: Additional project that can be used. Check the SetupProject page for the list of projects.
  • CondDB: Choose the conditions CondDB tag.
  • DDDB: Choose the conditions DDDB tag.
  • Visible: Visibility flag. If it's Y a folder with the output filename of your production will b created in the bookkeeping.
  • Usable: Y if you want to use the step.
  • Input File Type: In this form you can choose the file name of input files. The file name are registred in bookkeeping so you can choose an existing one or Add a brand new file name. Usually you don't need to create a new filename for input because you pick up file from bookkeeping and by definition they are already there.
  • Output File Type: As before you can choose an existing filename or create a brand new one that will be registered in the bookkeeping. It's reasonable here that you need to input your filename, so even if there is no particular rule yet it is recommended to follow the typing schema of the other filenames.

At this point you click on Save and you step will appear in the list of Registered Steps.

 

Submit a Production Request

Added:
>
>
META FILEATTACHMENT attachment="step_manager_page.png" attr="" comment="" date="1323351113" name="step_manager_page.png" path="step_manager_page.png" size="145697" stream="step_manager_page.png" tmpFilename="/usr/tmp/CGItemp58887" user="afalabel" version="1"
META FILEATTACHMENT attachment="New_Step.png" attr="" comment="" date="1323351678" name="New_Step.png" path="New_Step.png" size="49299" stream="New_Step.png" tmpFilename="/usr/tmp/CGItemp62819" user="afalabel" version="1"

Revision 22011-12-08 - AntonioFalabella

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Deleted:
<
<
 -- AntonioFalabella - 08-Dec-2011

Working Groups Customized Productions

Line: 12 to 10
 Within LHCbDIRAC It is possible to setup a production request with a customized workflow to meet the needs of working groups. This possibility can be helpful in situations where some task need to be performed on a well defined dataset in the bookkeping.

Software Prerequisites

Added:
>
>
At the time being LHCbDIRAC can run software that uses the a release of the following main projects: Gauss, Boole, Brunel, DaVinci and Moore. The scripts coded by the working groups must be released in a package that can be seen by SetupProject. So for example let's suppose you write. You will release your code under a lhcb software package such that the following line can be issued:

> SetupProject Project vXrY --use="extrapackage vXrY"
> gaudirun.py path/your_script.py

where path means the path of you script in the lhcb releases tree.

Setting Up a New Step

In the LHCbDIRAC workflow the execution your

Submit a Production Request

Revision 12011-12-08 - AntonioFalabella

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="LHCbComputing"

-- AntonioFalabella - 08-Dec-2011

Working Groups Customized Productions

Introduction

Within LHCbDIRAC It is possible to setup a production request with a customized workflow to meet the needs of working groups. This possibility can be helpful in situations where some task need to be performed on a well defined dataset in the bookkeping.

Software Prerequisites

 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback