Introduction

We collect here the information to evaluate the support needs of the activities of the CERN Theory unit. This is a working document, not a report. Please keep it updated.

Application patterns

  • Peter SKANDS:

I do two major tasks:

1. Tuning and development: bursts of 100 short (order one hour) jobs that ideally need to have low (preferably zero) waiting periods so that several iterations can be done in one day, to check tails of distributions and optimize model parameters. I do these kinds of runs pretty regularly, either in the context of tuning, or of model development.

2. Production: these are many more jobs (thousands, or even tens of thousands) which are much longer (order one day each), to generate, say hundreds of millions of events in total for several models. These jobs do not need to start immediately and if they finish today or tomorrow is not so essential. I do this kind of production only infrequently, with long periods (months) of only light development-work inbetween.

  • Fulvio PICCININI:

So far used lxplus5. Submission of 500-1000 jobs, lasting about 1 week each, once every 1-2 months

  • Mauro MORETTI:

have been running recently a continuous 300-500 jobs on lxplus for two months. individual job duration 2-7 days (depending on cpu, e.g. on 1nw). This was a peak usage, otherwise 1000-1500 jobs of similar type, but submitted once every 30-45 days.

  • Stefano FRIXIONE:

100-1000 jobs, with duration from few hrs to 1-2 days. Each job is an element of a single computation, so the result is avilable only after they have all completed.

  • GRAEME WATT:

I tend to optimise the number of jobs I submit to a batch queue to the built-in limit of the maximum number of jobs per user allowed to run simultaneously. From past experience in the UCL HEP group and at IPPP Durham, this limit was somewhere between 60 and 100, which proved to be adequate. The length of my jobs can vary from about an hour to several days, but I can usually restart the longer jobs if they are killed prematurely.

  • Jeppe ANDERSEN

When I submit to the batch-queue, I typically submit 100 jobs, which each run for ~1 hour (wall-clock) (depending on the number of matching terms included) and each create about ~200kb of output at the end of the job. So the CPU utilisation is 100% during run time, no wait for disk I/O. I therefore typically submit to the batch queue called '1nd80'. In principle I could have used the 8 'hour' queue (8nh80) since it has a higher priority, but for some reason (maybe just fluctuation) I noticed that my jobs once in a while get stuck in that queue and never advance to run status.

With the recent advance in disk space, I might want to generate more output and store events that can be analysed by people who cannot yet run the code themselves. But disk I/O still will not be a problem for the batch jobs.

=

I have attached a tar ball of the program I would like tom run on the grid. Start by initialising the necessary software environment (basically to enable compiling/linking with ROOT,fastjet,CLHEP,GSL): Note this is valid on lxplus5

. /afs/cern.ch/sw/lcg/hepsoft/0.2/x86_64-slc5-gcc44-opt/setup.sh
Then unpack the tar-ball, 'cd ToGrid', 'make WJetGen'. You should now edit line 17 of 'submittoCERNbatch.pl' in order to reflect the directory where the file resides. If you now run 'submittoCERNbatch.pl 1', a file called 'batch1.q' will be created, and submitted to the batch queue. It is this procedure I would like adapted to the grid.

  • Stefano POZZORINI

    • These numbers refer to present/past activities = mostly R&D.
    • Numbers in parenthesis correspond to my estimated needs for say next year.

A) Monte Carlo jobs (Fortran) - few times per month

    • number of simultaneously running jobs: 10-20 (100)
    • typical job duration: 1-2 days
    • needed disk space: 1-2 GB (10 GB)
    • the relatively large size of the executable (of order 100MB) might cause troubles when running multiple jobs on a batch system

B) Computer Algebra jobs (Mathematica) - few times per month

    • number of simultaneously running jobs: 1-5 (20)
    • typical job duration: 0.1-5 days
    • needed disk space: 1-2 GB (10-20 GB)

  • Roberto PITTAU:

50-60 jobs, 24-36 hrs each, 2-3 times/month

-- MassimoLamanna - 19-Apr-2010

Edit | Attach | Watch | Print version | History: r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r3 - 2010-05-10 - MassimoLamanna
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback