PanDA share distribution and monitoring

Work in progress, under construction WORK IN PROGRESS

What is the idea behind?

Currently PanDA job assignment is done according to priorities: higher priority jobs are the first in line to be sent to a site. At occasions, priorities are not sufficient to regulate the type of jobs and shares. There are occasions where ATLAS Management wants to prioritise a certain campaign and assign it most of the available grid resources. It is also easier for the Ops team to regulate and control the resources through shares than priorities.

The request from ADC Ops is to have a multi-level/nested share system that allows changing the fraction of resources assigned to the different activities. One example:

  • 50% of resources go to Analysis and 50% to Production.
  • Out of the Production share, the operator wants to define that 50% goes to MC Production and 50 % to Group Production, and so on.
  • If MC Production is not consuming their 50% share, then Group Production can overflow and fill it up - but not Analysis... Analysis can only overflow if Production is not filling the Production share.

Share_system.png

Implementation design

- We want to implement an independent and asynchronous service - Web interface to monitor and manage shares - DB schema is flat - Agent:
  1. Figures out how many slots are available to ATLAS (e.g. 150.000)
  2. Figures out the capabilities of the slots
  3. Splits the slots according to the shares and capabilities
- PanDA server brokerage:
  1. Queries the system
  2. Brokers
  3. Updates the slots assigned
Edit | Attach | Watch | Print version | History: r3 < r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r1 - 2015-04-03 - FernandoHaraldBarreiroMegino
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    PanDA All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback