Different nature of non-pledged resources

Non-pledged resources could be of different nature

• Commecial clouds
• HPC
• BOINC
• So called mid-SLA
• ...

Required actions and known issues

In order to integrate usage of such resources into APEL, there are few important matters to be addressed:

• Describe these resources from the topology point of view
• Define how accounting report to APEL is generated. There are several alternatives :
  • Enable resource provider to generate a report
  • Since experiments do account usage of all available resources via their work load management systems, inject information from the experiment accounting system into APEL (smry records)
• Benchmarking . While accounting of such resources in terms of wallclock time is normally available at least via experiment work load management systems, benchmarking of such resources if experiments did not perform this task can be problematic

Issues to be solved:

• How do we tag those resources in order to indicate that these are non-pledged resources, even though they can be provided by T1 or T2
• How do we make sure that we avoid double counting. Supposing non-pledged resources are reported from the site A, while for site B this reporting is not enabled. If we enable import of the accounting information of non-pledged resources into APEL from the experiment-specific system, need to make sure that site A non-pledged usage is accounted only once
• Agree on the policy against which site these resources are accounted. For example, it could be either site which provides HW where jobs are running or site which sets up a queue to which these resources are attached. Ensure that we implement proper mapping of usage vs site in our data flow according to the agreed policy. The most evident solution is to resolve it on the queue level and correspondingly do aggregation of the accounting information on the queue level. However in some cases can be misleading.

Experiment input

Experiments were asked to provide input for the following questions:

• Whether your experiment would be interested that opportunistic resources are accounted in APEL? (we alreday collected this input in the beginning of the task force activities, just to confirm that nothing changed)
• If yes, what are possible scenarios?
• Whether these opportunistic resources are already accounted in the experiment-specific systems?
• How/whether benchmarking of such resources performed?
• How these resources are described regarding topology?
• Would it be possible to retrieve accounting data for the opportunistic resources from the experiment-specific systems via APIs?

Answers are summarized in the table below

-- JuliaAndreeva - 2017-03-08