Difference: UsersTaskForceStDenis (1 vs. 22)

Revision 222005-11-30 - RichardStDenis

Line: 1 to 1
 
META TOPICPARENT name="UsersTaskForceWiki"
<!-- This is the default ATLAS template. 
Please modify it in the sections indicated to create your topic!
Line: 17 to 17
 
<!-- Add an introduction here, describing the purpose of this topic. !-->
This topic leads to other topics regarding work related to the Users Task Force
Changed:
<
<

Talks at the UTF meeting

>
>

Talks

 
Changed:
<
<
>
>
 

Plan

<!--Add the main topic here. i.e. create some new headings as follows:!-->
Line: 229 to 229
 
Added:
>
>
 
META FILEATTACHMENT attr="" comment="Version 1 Talk for UTF Meeting 22-Nov05" date="1132419391" name="RStDUTFNov2205v1.ppt" path="RStDUTFNov2205v1.ppt" size="53760" user="stdenis" version="1.1"
Added:
>
>
META FILEATTACHMENT attr="h" comment="Talk at Ganga Meeting Dec 1, 2005" date="1133382069" name="RStDGanga1Dec05.ppt" path="RStDGanga1Dec05.ppt" size="90112" user="stdenis" version="1.1"

Revision 212005-11-28 - RichardStDenis

Line: 1 to 1
 
META TOPICPARENT name="UsersTaskForceWiki"
<!-- This is the default ATLAS template. 
Please modify it in the sections indicated to create your topic!
Line: 179 to 179
 
    • Reconstruction
    • Produce AOD
    • AOD Analysis
Changed:
<
<
    • Ntuple stop.gif Failed: crashed.
>
>
    • Ntuple
 
  • 100 Event Test on the Grid
Added:
>
>
    • Generation
    • Simulation
  • 1000 Event Test on the Grid
    • Generation
    • Simulation
 

Running Ganga

Revision 202005-11-26 - RichardStDenis

Line: 1 to 1
 
META TOPICPARENT name="UsersTaskForceWiki"
<!-- This is the default ATLAS template. 
Please modify it in the sections indicated to create your topic!
Line: 17 to 17
 
<!-- Add an introduction here, describing the purpose of this topic. !-->
This topic leads to other topics regarding work related to the Users Task Force
Added:
>
>

Talks at the UTF meeting

 

Plan

<!--Add the main topic here. i.e. create some new headings as follows:!-->
Line: 56 to 59
 

Problems and Issues

This outlines the problems and issues that are pending and have been solved

Pending

Changed:
<
<
Ganga Issues in going from 4.0.0-beta3 to 4.0.0beta5
>
>
Ganga Issues:
 

  1. It lost track of the jobs I had sitting around and not yet cleaned up. Indeed, the jobs are gone when the initialization was repeated, but a gangadir directory was repeated. How to recover the state?
  2. "Cannot plan: BrokerHelper: no compatible resources" error after submission of Athena AOD analysis to LCG. The error is reported here. The error was reproduced again on Nov 15, 2005 and the sequence of studies of this are reported here.
  3. Submission of Athena Hello World has too many output lines and there is a compile error in stderr. Details here. A second test shows a different error in stderr, and still has too many lines in the stdout.
  4. Unable to kill a job. This is documented in this place.
Added:
>
>
Grid Issues
  • Unable to get the digitisation phase to work on scotgrid.
  • Full generation with 10.0.1 and VBF physics crashes in the 5 event test.
 

Fixed but needs New Release for Me to test

  1. I get an error message from j.application.exe='/bin/echo'. As of Nov 7, 2005 Dietrich has made a fix and this needs to be tested when it is available.
  2. As documented in the instructions on how to run Hello World Different responses depending on how I use args to a job:
Line: 163 to 169
  The RStDWorkBookGridWork details the work here. Successful steps thusfar:
Added:
>
>
  • 5 Event Test on the Grid (storage of each stage output to RAL and Scotgrid)
 
  • Getting an Account
  • Setting up your Account
  • Running HelloWorld
  • Running Athena HelloWorld
Added:
>
>
    • Generation (on Scotgrid specifically)
    • Digitization
    • Reconstruction
    • Produce AOD
    • AOD Analysis
    • Ntuple stop.gif Failed: crashed.
  • 100 Event Test on the Grid
 

Running Ganga

Revision 192005-11-24 - RichardStDenis

Line: 1 to 1
 
META TOPICPARENT name="UsersTaskForceWiki"
<!-- This is the default ATLAS template. 
Please modify it in the sections indicated to create your topic!
Line: 193 to 193
 

Running the Physics Analysis

The AOD Tutorial will help.
Added:
>
>

The physics coding

Considering EventView

InstructionsForEventViewin100x

InstructionsForEventViewIn1001 to build an event view in 10.0.1 and look at missing et.

 
<!-- MAJOR UPDATES
For significant updates to the topic, consider adding your 'signature' (beneath this editing box) !-->

Revision 182005-11-19 - RichardStDenis

Line: 1 to 1
 
META TOPICPARENT name="UsersTaskForceWiki"
<!-- This is the default ATLAS template. 
Please modify it in the sections indicated to create your topic!
Line: 203 to 203
 
Added:
>
>
 
Added:
>
>
META FILEATTACHMENT attr="" comment="Version 1 Talk for UTF Meeting 22-Nov05" date="1132419391" name="RStDUTFNov2205v1.ppt" path="RStDUTFNov2205v1.ppt" size="53760" user="stdenis" version="1.1"

Revision 172005-11-17 - RichardStDenis

Line: 1 to 1
 
META TOPICPARENT name="UsersTaskForceWiki"
<!-- This is the default ATLAS template. 
Please modify it in the sections indicated to create your topic!
Line: 61 to 61
 
  1. It lost track of the jobs I had sitting around and not yet cleaned up. Indeed, the jobs are gone when the initialization was repeated, but a gangadir directory was repeated. How to recover the state?
  2. "Cannot plan: BrokerHelper: no compatible resources" error after submission of Athena AOD analysis to LCG. The error is reported here. The error was reproduced again on Nov 15, 2005 and the sequence of studies of this are reported here.
Changed:
<
<
  1. Submission of Athena Hello World has too many output lines and there is a compile error in stderr. Details here. A second test shows a different error in stderr, and still has
too many lines in the stdout.
>
>
  1. Submission of Athena Hello World has too many output lines and there is a compile error in stderr. Details here. A second test shows a different error in stderr, and still has too many lines in the stdout.
  2. Unable to kill a job. This is documented in this place.
 

Fixed but needs New Release for Me to test

  1. I get an error message from j.application.exe='/bin/echo'. As of Nov 7, 2005 Dietrich has made a fix and this needs to be tested when it is available.
  2. As documented in the instructions on how to run Hello World Different responses depending on how I use args to a job:
Line: 153 to 153
 
  • Ntuple

Running at Glasgow

Changed:
<
<
Done on cdfg.ph.gla.ac.uk -- details here.
>
>
Done on cdfg.ph.gla.ac.uk -- details here. Working on scotgrid as well.
 

Running the Full analysis Chain with Grid

Work to do this both with the Workbook instructions and within the Ganga framework is

Revision 162005-11-17 - RichardStDenis

Line: 1 to 1
 
META TOPICPARENT name="UsersTaskForceWiki"
<!-- This is the default ATLAS template. 
Please modify it in the sections indicated to create your topic!
Line: 61 to 61
 
  1. It lost track of the jobs I had sitting around and not yet cleaned up. Indeed, the jobs are gone when the initialization was repeated, but a gangadir directory was repeated. How to recover the state?
  2. "Cannot plan: BrokerHelper: no compatible resources" error after submission of Athena AOD analysis to LCG. The error is reported here. The error was reproduced again on Nov 15, 2005 and the sequence of studies of this are reported here.
Changed:
<
<
  1. Submission of Athena Hello World has too many output lines and there is a compile error in stderr. Details here.
>
>
  1. Submission of Athena Hello World has too many output lines and there is a compile error in stderr. Details here. A second test shows a different error in stderr, and still has
too many lines in the stdout.
 

Fixed but needs New Release for Me to test

  1. I get an error message from j.application.exe='/bin/echo'. As of Nov 7, 2005 Dietrich has made a fix and this needs to be tested when it is available.
  2. As documented in the instructions on how to run Hello World Different responses depending on how I use args to a job:

Revision 152005-11-16 - RichardStDenis

Line: 1 to 1
 
META TOPICPARENT name="UsersTaskForceWiki"
<!-- This is the default ATLAS template. 
Please modify it in the sections indicated to create your topic!
Line: 60 to 60
 

  1. It lost track of the jobs I had sitting around and not yet cleaned up. Indeed, the jobs are gone when the initialization was repeated, but a gangadir directory was repeated. How to recover the state?
Deleted:
<
<
  1. Cannot select the grid site as in page 7 of the Ganga Overview talk in the Ganga Tutorial. See this screen dump of the attempt. As of 15-Nov 2005, there are two choices for syntax. One of these fails on submission. The other at least submits but need to wait for job to actually succeed.
 
  1. "Cannot plan: BrokerHelper: no compatible resources" error after submission of Athena AOD analysis to LCG. The error is reported here. The error was reproduced again on Nov 15, 2005 and the sequence of studies of this are reported here.
Added:
>
>
  1. Submission of Athena Hello World has too many output lines and there is a compile error in stderr. Details here.
 

Fixed but needs New Release for Me to test

  1. I get an error message from j.application.exe='/bin/echo'. As of Nov 7, 2005 Dietrich has made a fix and this needs to be tested when it is available.
  2. As documented in the instructions on how to run Hello World Different responses depending on how I use args to a job:
Line: 81 to 81
 
    • Solution: Do the ganga initialization again.
  1. The path changes -- i assume I got it right now.
    • Solution: Yes, the new path works. This is in my Ganga Guide.
Added:
>
>
  1. Cannot select the grid site as in page 7 of the Ganga Overview talk in the Ganga Tutorial. See this screen dump of the attempt. As of 15-Nov 2005, there are two choices for syntax. One of these fails on submission and the other works. We have decided to provide only one of these methods in the documentation -- it is in fact the more straightfoward method.
 

Wish List

  • Time stamp when state notification like

Line: 88 to 89
  occurs.

Questions

Added:
>
>

Pending

Answered

 
  • How do I remove some jobs from my repository but keep the ones I want to use to copy and modify?
Added:
>
>
use j=jobs[126] and then delete j.
 
  • How do I clean up specific jobs and not delete all of them in a cleanup?
Added:
>
>
Same ans for the first answer. The cleanup only removes the input and output areas (like those in
inputdir = '/afs/cern.ch/user/s/stdenis/gangadir/workspace/Local/128/input/' ,
outputdir = '/afs/cern.ch/user/s/stdenis/gangadir/workspace/Local/128/output/' ,
It does not remove tar files. You may have other jobs that use them. So you need to decide that or write a script that looks at the job objects, pulls the tar file name from the object and sees if it is the last one that uses a specific tar file for the object you are deleting.

Warnings

  • If you say j=job1.copy then you get a funny error and it really copies the pointer. You must use j=job1.copy() so it makes a copy of the object. Nonetheless, you can say j.submit although you should say j.sumbit(). This is because IPython is less strict. I am not sure this is a good thing. But then I don't like the way ROOT forgives pointers vs objects.
  • Problems with "Job proxy expired" means that the job was around two long.

Two workarounds:

    • Find a site that is less busy
    • Use proxy renewal
You can add a keyword to the configuration MyProxyServer = myproxy.cern.ch

If you upoload a proxy to the myproxy server, the WMS will renew your proxy (and your job can wait even much longer in the queue)

I have not tried it recently, the command to upload a proxy is caled myproxy-init. It should be discussed in the LCG User Guide.

 

Obtaining Grid credentials

Line: 117 to 141
  Unable to run the simulation as a batch job. This was traced (9-Nov-05) to a skipevent 10 card. A log of the work to get back to running is here.
Added:
>
>
Progress:
  • Athena Hello World DONE
  • Pythia DONE
  • Simulation DONE
  • Digitization
  • Reconstruction
  • Produce AOD
  • AOD Analysis
  • Ntuple
 

Running at Glasgow

Done on cdfg.ph.gla.ac.uk -- details here.

Revision 142005-11-16 - RichardStDenis

Line: 1 to 1
 
META TOPICPARENT name="UsersTaskForceWiki"
<!-- This is the default ATLAS template. 
Please modify it in the sections indicated to create your topic!
Line: 61 to 61
 
  1. It lost track of the jobs I had sitting around and not yet cleaned up. Indeed, the jobs are gone when the initialization was repeated, but a gangadir directory was repeated. How to recover the state?
  2. Cannot select the grid site as in page 7 of the Ganga Overview talk in the Ganga Tutorial. See this screen dump of the attempt. As of 15-Nov 2005, there are two choices for syntax. One of these fails on submission. The other at least submits but need to wait for job to actually succeed.
Changed:
<
<
  1. "Cannot plan: BrokerHelper: no compatible resources" error after submission of Athena AOD analysis to LCG. The error is reported here.
>
>
  1. "Cannot plan: BrokerHelper: no compatible resources" error after submission of Athena AOD analysis to LCG. The error is reported here. The error was reproduced again on Nov 15, 2005 and the sequence of studies of this are reported here.
 

Fixed but needs New Release for Me to test

  1. I get an error message from j.application.exe='/bin/echo'. As of Nov 7, 2005 Dietrich has made a fix and this needs to be tested when it is available.
  2. As documented in the instructions on how to run Hello World Different responses depending on how I use args to a job:

Revision 132005-11-15 - RichardStDenis

Line: 1 to 1
 
META TOPICPARENT name="UsersTaskForceWiki"
<!-- This is the default ATLAS template. 
Please modify it in the sections indicated to create your topic!
Line: 81 to 81
 
    • Solution: Do the ganga initialization again.
  1. The path changes -- i assume I got it right now.
    • Solution: Yes, the new path works. This is in my Ganga Guide.
Added:
>
>

Wish List

  • Time stamp when state notification like
Ganga.Lib.LCG							 : INFO	  job 126 has changed status to Scheduled
occurs.

Questions

  • How do I remove some jobs from my repository but keep the ones I want to use to copy and modify?
  • How do I clean up specific jobs and not delete all of them in a cleanup?
 

Obtaining Grid credentials

This turned out to be harder than expected, and took up about 3 weeks of time (plus 2 weeks I was away on holiday). I was thwarted from getting

Revision 122005-11-15 - RichardStDenis

Line: 1 to 1
 
META TOPICPARENT name="UsersTaskForceWiki"
<!-- This is the default ATLAS template. 
Please modify it in the sections indicated to create your topic!
Line: 59 to 59
 Ganga Issues in going from 4.0.0-beta3 to 4.0.0beta5

Deleted:
<
<
 
  1. It lost track of the jobs I had sitting around and not yet cleaned up. Indeed, the jobs are gone when the initialization was repeated, but a gangadir directory was repeated. How to recover the state?
Changed:
<
<
  1. I get an error message from j.application.exe='/bin/echo'.
  2. As documented in the instructions on how to run Hello World Different responses depending on how I use args to a job:
>
>
  1. Cannot select the grid site as in page 7 of the Ganga Overview talk in the Ganga Tutorial. See this screen dump of the attempt. As of 15-Nov 2005, there are two choices for syntax. One of these fails on submission. The other at least submits but need to wait for job to actually succeed.
  2. "Cannot plan: BrokerHelper: no compatible resources" error after submission of Athena AOD analysis to LCG. The error is reported here.

Fixed but needs New Release for Me to test

  1. I get an error message from j.application.exe='/bin/echo'. As of Nov 7, 2005 Dietrich has made a fix and this needs to be tested when it is available.
  2. As documented in the instructions on how to run Hello World Different responses depending on how I use args to a job:
 
    • Using the following for a Grid Hello World job gives only Hello as the response:(See
      • j.application.env={'MESSAGE':'Hello World'}
      • j.application.args=['$MESSAGE']
Line: 72 to 74
 
    • Using the following gives the full "Hello World" back as a response.
      • j4.application.args=['Hello World']
    • For this case, see screen dumps of the job object and the output.
Changed:
<
<
  1. Cannot select the grid site as in page 7 of the Ganga Overview talk in the Ganga Tutorial. See this screen dump of the attempt.
  2. Cannot find dataset when preparing to run Athena AOD analysis on LSF. The error is reported here.
  3. "Cannot plan: BrokerHelper: no compatible resources" error after submission of Athena AOD analysis to LCG. The error is reported here.
>
>
    • As of Nov 7, 2005 Dietrich has made a fix and this needs to be tested on a new release.
  1. Cannot find dataset when preparing to run Athena AOD analysis on LSF. The error is reported here. As of Nov 7, 2005 Dietrich has made a fix but need to get release and test.
 

Solved

  1. Ganga 4.0.0-beta5 requires a .gangrc file -- tried to soft link to .ganga4 and while I can get things going, there is a complaint in the startup. See the screen shot of startup.
    • Solution: Do the ganga initialization again.
Line: 138 to 139
 
  • Running Athena HelloWorld
    • Locally
    • On LSF
Added:
>
>
    • On Grid
  stop.gif The next steps would be
Changed:
<
<
  • To run Athena HelloWorld on the Grid. Upgraded to version 4.0.0-beta5 and submitted. Job has been running a couple of hours. Seems long.
  • To run Athena AOD analysis on LSF. This fails (see report above) because the dataset cannot be located in the preparation of the job.
>
>
  • To run Athena AOD analysis on LSF. This fails (see report above) because the dataset cannot be located in the preparation of the job. As of 15-Nov-05, Dietrich has made a fix. What to do to make this work is a query to him.
 
  • To run Athena AOD analysis on LCG. This returns an error from the resource broker as reported above.
-- RichardStDenis - 20 Oct 2005 2:49 BST

Revision 112005-11-15 - RichardStDenis

Line: 1 to 1
 
META TOPICPARENT name="UsersTaskForceWiki"
<!-- This is the default ATLAS template. 
Please modify it in the sections indicated to create your topic!
Line: 25 to 25
 
  • hello world
  • athena hello world
  • following the workbook but with
Changed:
<
<
gaudi as well as the Grid
>
>
ganga as well as the Grid
  instructions imbedded in the workbook for MC generation

Test analysis:

Revision 102005-11-14 - RichardStDenis

Line: 1 to 1
 
META TOPICPARENT name="UsersTaskForceWiki"
<!-- This is the default ATLAS template. 
Please modify it in the sections indicated to create your topic!
Line: 104 to 104
 

Running the Full analysis chain without Grid

Running on lxplus

Changed:
<
<
Unable to run the simulation as a batch job.
>
>
Unable to run the simulation as a batch job. This was traced (9-Nov-05) to a skipevent 10 card. A log of the work to get back to running is here.
 

Running at Glasgow

Done on cdfg.ph.gla.ac.uk -- details here.

Revision 92005-10-27 - RichardStDenis

Line: 1 to 1
 
META TOPICPARENT name="UsersTaskForceWiki"
<!-- This is the default ATLAS template. 
Please modify it in the sections indicated to create your topic!
Line: 145 to 145
 -- RichardStDenis - 20 Oct 2005 2:49 BST

Running the Physics Analysis

Changed:
<
<
The [[http://atlas.web.cern.ch/Atlas/SUB_DETECTORS/TILE/Tutorials/AOD-10.0.1/tutorial.html][AOD Tutorial] will help.
>
>
The AOD Tutorial will help.
 
<!-- MAJOR UPDATES
For significant updates to the topic, consider adding your 'signature' (beneath this editing box) !-->

Revision 82005-10-27 - RichardStDenis

Line: 1 to 1
 
META TOPICPARENT name="UsersTaskForceWiki"
<!-- This is the default ATLAS template. 
Please modify it in the sections indicated to create your topic!
Line: 144 to 144
 
  • To run Athena AOD analysis on LCG. This returns an error from the resource broker as reported above.
-- RichardStDenis - 20 Oct 2005 2:49 BST
Added:
>
>

Running the Physics Analysis

The [[http://atlas.web.cern.ch/Atlas/SUB_DETECTORS/TILE/Tutorials/AOD-10.0.1/tutorial.html][AOD Tutorial] will help.
 
<!-- MAJOR UPDATES
For significant updates to the topic, consider adding your 'signature' (beneath this editing box) !-->

Revision 72005-10-21 - RichardStDenis

Line: 1 to 1
 
META TOPICPARENT name="UsersTaskForceWiki"
<!-- This is the default ATLAS template. 
Please modify it in the sections indicated to create your topic!
Line: 107 to 107
 Unable to run the simulation as a batch job.

Running at Glasgow

Changed:
<
<
Done on cdfg.ph.gla.ac.uk -- details need to be added here
>
>
Done on cdfg.ph.gla.ac.uk -- details here.
 

Running the Full analysis Chain with Grid

Work to do this both with the Workbook instructions and within the Ganga framework is

Revision 62005-10-20 - RichardStDenis

Line: 1 to 1
 
META TOPICPARENT name="UsersTaskForceWiki"
<!-- This is the default ATLAS template. 
Please modify it in the sections indicated to create your topic!
Line: 73 to 73
 
      • j4.application.args=['Hello World']
    • For this case, see screen dumps of the job object and the output.
  1. Cannot select the grid site as in page 7 of the Ganga Overview talk in the Ganga Tutorial. See this screen dump of the attempt.
Added:
>
>
  1. Cannot find dataset when preparing to run Athena AOD analysis on LSF. The error is reported here.
  2. "Cannot plan: BrokerHelper: no compatible resources" error after submission of Athena AOD analysis to LCG. The error is reported here.
 

Solved

  1. Ganga 4.0.0-beta5 requires a .gangrc file -- tried to soft link to .ganga4 and while I can get things going, there is a complaint in the startup. See the screen shot of startup.
    • Solution: Do the ganga initialization again.
Line: 136 to 138
 
    • Locally
    • On LSF
Changed:
<
<
stop.gif The next step would be to run Athena HelloWorld on the Grid. This failed with 4.0.0beta3 because it is not possible to specify input as None. Tried to use an input file anyway (even if not needed) and job failed with expired proxy. Upgraded to 4.0.0beta5 and able to submit. Job had not run. Seems to be Athena gets stuck on grid: the hello world /bin/echo job ran. The fact that moving to beta5 was not clean because the config files were not quite done right may be the reason things stick. Dietrich suggests reconfiguring for beta5 and then going on again. -- RichardStDenis - 19 Oct 2005 3:11 BST
>
>
stop.gif The next steps would be
  • To run Athena HelloWorld on the Grid. Upgraded to version 4.0.0-beta5 and submitted. Job has been running a couple of hours. Seems long.
  • To run Athena AOD analysis on LSF. This fails (see report above) because the dataset cannot be located in the preparation of the job.
  • To run Athena AOD analysis on LCG. This returns an error from the resource broker as reported above.
-- RichardStDenis - 20 Oct 2005 2:49 BST
 

<!-- MAJOR UPDATES

Revision 52005-10-20 - RichardStDenis

Line: 1 to 1
 
META TOPICPARENT name="UsersTaskForceWiki"
<!-- This is the default ATLAS template. 
Please modify it in the sections indicated to create your topic!
Line: 59 to 59
 Ganga Issues in going from 4.0.0-beta3 to 4.0.0beta5

Deleted:
<
<
  1. It requires a .gangrc file -- tried to soft link to .ganga4 and while I can get things going, there is a complaint in the startup. See the screen shot of startup.
  2. The path changes -- i assume I got it right now.
  3. It lost track of the jobs I had sitting around and not yet cleaned up. (presumably related to (1))
  4. I get an error message as follows:
In [5]: j.application.exe='/bin/echo'
/afs/cern.ch/sw/ganga/install/slc3_gcc323/4.0.0-beta5/python/Ganga/GPIDev/Ba
se/Proxy.py:232:
FutureWarning: hex()/oct() of negative int will return a signed string in
Python 2.4 and up
  return '<'+repr(self._impl)+' PROXY at '+hex(id(self))+'>'
 
Added:
>
>
  1. It lost track of the jobs I had sitting around and not yet cleaned up. Indeed, the jobs are gone when the initialization was repeated, but a gangadir directory was repeated. How to recover the state?
  2. I get an error message from j.application.exe='/bin/echo'.
  3. As documented in the instructions on how to run Hello World Different responses depending on how I use args to a job:
    • Using the following for a Grid Hello World job gives only Hello as the response:(See
      • j.application.env={'MESSAGE':'Hello World'}
      • j.application.args=['$MESSAGE']
    • For this case, see screen dumps of:
    • Using the following gives the full "Hello World" back as a response.
      • j4.application.args=['Hello World']
    • For this case, see screen dumps of the job object and the output.
  4. Cannot select the grid site as in page 7 of the Ganga Overview talk in the Ganga Tutorial. See this screen dump of the attempt.
 

Solved

Added:
>
>
  1. Ganga 4.0.0-beta5 requires a .gangrc file -- tried to soft link to .ganga4 and while I can get things going, there is a complaint in the startup. See the screen shot of startup.
    • Solution: Do the ganga initialization again.
  2. The path changes -- i assume I got it right now.
    • Solution: Yes, the new path works. This is in my Ganga Guide.
 

Obtaining Grid credentials

This turned out to be harder than expected, and took up about 3 weeks of time (plus 2 weeks I was away on holiday). I was thwarted from getting my GridPP credentials when, after getting them, I was told by my browser to use the master password,

Revision 42005-10-19 - RichardStDenis

Line: 1 to 1
 
META TOPICPARENT name="UsersTaskForceWiki"
<!-- This is the default ATLAS template. 
Please modify it in the sections indicated to create your topic!
Line: 118 to 118
 

Running Ganga

The

<!--RStDGaudiGridWork -->
Changed:
<
<
RStDGangaGridWork details the work here. Successful steps thusfar:
>
>
RStDGangaGridWork details the work here. RStDGangaGuide describes how to use Ganga. Successful steps thusfar:
 
  • Getting an Account
  • Setting up your Account

Revision 32005-10-19 - RichardStDenis

Line: 1 to 1
 
META TOPICPARENT name="UsersTaskForceWiki"
<!-- This is the default ATLAS template. 
Please modify it in the sections indicated to create your topic!
Line: 53 to 53
  executed.

Execution of the Plan

Added:
>
>

Problems and Issues

This outlines the problems and issues that are pending and have been solved

Pending

Ganga Issues in going from 4.0.0-beta3 to 4.0.0beta5

  1. It requires a .gangrc file -- tried to soft link to .ganga4 and while I can get things going, there is a complaint in the startup. See the screen shot of startup.
  2. The path changes -- i assume I got it right now.
  3. It lost track of the jobs I had sitting around and not yet cleaned up. (presumably related to (1))
  4. I get an error message as follows:
In [5]: j.application.exe='/bin/echo'
/afs/cern.ch/sw/ganga/install/slc3_gcc323/4.0.0-beta5/python/Ganga/GPIDev/Ba
se/Proxy.py:232:
FutureWarning: hex()/oct() of negative int will return a signed string in
Python 2.4 and up
  return '<'+repr(self._impl)+' PROXY at '+hex(id(self))+'>'

Solved

 

Obtaining Grid credentials

This turned out to be harder than expected, and took up about 3 weeks of time (plus 2 weeks I was away on holiday). I was thwarted from getting my GridPP credentials when, after getting them, I was told by my browser to use the master password,
Line: 111 to 131
 
    • On LSF

stop.gif The next step would be to run Athena HelloWorld on the Grid. This failed with 4.0.0beta3 because

Changed:
<
<
it is not possible to specify
>
>
it is not possible to specify input as None. Tried to use an input file anyway (even if not needed) and job failed with expired proxy. Upgraded to 4.0.0beta5 and able to submit. Job had not run. Seems to be Athena gets stuck on grid: the hello world /bin/echo job ran. The fact that moving to beta5 was not clean because the config files were not quite done right may be the reason things stick. Dietrich suggests reconfiguring for beta5 and then going on again. -- RichardStDenis - 19 Oct 2005 3:11 BST
 

<!-- MAJOR UPDATES

Revision 22005-10-19 - RichardStDenis

Line: 1 to 1
 
META TOPICPARENT name="UsersTaskForceWiki"
<!-- This is the default ATLAS template. 
Please modify it in the sections indicated to create your topic!
Line: 17 to 17
 
<!-- Add an introduction here, describing the purpose of this topic. !-->
This topic leads to other topics regarding work related to the Users Task Force
Changed:
<
<

Main heading

>
>

Plan

 
<!--Add the main topic here. i.e. create some new headings as follows:!-->
Added:
>
>

Overall Goals:

  • hello world
  • athena hello world
  • following the workbook but with gaudi as well as the Grid instructions imbedded in the workbook for MC generation

Test analysis:

To use the target anlaysis of VBF H with MH=160 for say 1000 events and see how to get this stored and cataloged. (Need to see what the resource impact is).

Also pursue this on the Glasgow computers in parallel to just get events made and in a place where data handling issues are not so severe.

Data handing is a big problem. We clearly have to submit jobs that generate small numbers of events and hence make small files. These then need concatenation. This is hard to do. We cannot store the intermediate files on tape and it seems that this job handling is not well integrated with the data handling. For analysis it needs to be. We also need a tutorial on the data handling issues and the state of the art there.

This as the beginning of a process and we should try to at least get some simulated files to a disk. This may mean having a large disk repository that our output goes to and we don't catalog. Later this should be a remote repository where it is cataloged and the full chain can be executed.

Execution of the Plan

Obtaining Grid credentials

This turned out to be harder than expected, and took up about 3 weeks of time (plus 2 weeks I was away on holiday). I was thwarted from getting my GridPP credentials when, after getting them, I was told by my browser to use the master password, something I had set when I got my laptop and never used since. This would mean revoking the certificate and since our CA had had this problem with the last 3 users he feared losing credibility. I therefore tried to use the Fermilab kerberos certificates. This does not work since the openssl versions had a conflict in the identity of the USERID field and when I registered with the VO, I had a mangled version of the field whereas when i used the credentials to get a proxy, I had an unmangled version.

This was not resolved and it remains the case that Fermilab kerberos credentials cannot be used to access LCG resouces with the ATLAS VO. This problem was not obviously on anyone's goals to solve, so I set this aside and went for registration through GridPP. This certificate was revoked and many many passwords later, I do have a certificate that works.

The experience, problems and attempts are documented here.

hand.gif It seems that at this early stage, it is unwise to frighten CA administrators to the extent that errors made by users cause them to put off users getting certificates.

Running the Full analysis chain without Grid

Running on lxplus

Unable to run the simulation as a batch job.

Running at Glasgow

Done on cdfg.ph.gla.ac.uk -- details need to be added here

Running the Full analysis Chain with Grid

Work to do this both with the Workbook instructions and within the Ganga framework is considered here.

Running Grid from the Workbook instructions

The RStDWorkBookGridWork details the work here. Successful steps thusfar:

  • Getting an Account
  • Setting up your Account
  • Running HelloWorld
  • Running Athena HelloWorld

Running Ganga

The

<!--RStDGaudiGridWork -->
RStDGangaGridWork details the work here. Successful steps thusfar:

  • Getting an Account
  • Setting up your Account
  • Running HelloWorld:
    • Locally
    • On LSF
    • On Grid
  • Running Athena HelloWorld
    • Locally
    • On LSF

stop.gif The next step would be to run Athena HelloWorld on the Grid. This failed with 4.0.0beta3 because it is not possible to specify

 
<!-- MAJOR UPDATES
For significant updates to the topic, consider adding your 'signature' (beneath this editing box) !-->

Revision 12005-10-18 - RichardStDenis

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="UsersTaskForceWiki"
<!-- This is the default ATLAS template. 
Please modify it in the sections indicated to create your topic!
If you have any comments/complaints about this template, then please email me:
edward.moyse at cern.ch
-->

<!-- By default the title is the WikiWord used to create this topic !-->
<!-- if you want to modify it to something more meaningful, just replace UsersTaskForceStDenis below with i.e "My Topic"!-->

UsersTaskForceStDenis

Introduction

<!-- Add an introduction here, describing the purpose of this topic. !-->
This topic leads to other topics regarding work related to the Users Task Force

Main heading

<!--Add the main topic here. i.e. create some new headings as follows:!-->

<!-- MAJOR UPDATES
For significant updates to the topic, consider adding your 'signature' (beneath this editing box) !-->

Major updates:
<!--Future editors should add their signatures beneath yours!-->
-- RichardStDenis - 18 Oct 2005

 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback