Requirements for a CE service

A "CE" could be generally defined as a secure abstraction of resources at a site hiding the implementation details from the user communities. We believe that for the foreseeable future we will have multiple CE technologies in place (hence, a VO may need to be able to use all of them).

Experiments use a variety of ways to submit to CE's, through either the WMS (partly CMS, ATLAS typically for software installations only [currently migrating to glideins], LHCb for sites not supporting direct submission, for SAM tests currently by all experiments), or via direct submission, with Condor-G and CREAM API's interfaces being common requirements.

From a site point of view, a requirement is that CE's support the batch system installed at a site (for example, PBS/Torque, LSF, SGE, Condor, possibly SLURM). It was pointed out by some sites that scaling issues or support shortcomings may arise with some batch systems (see also the report from the Operations and Tools TEG), and that some sites may need or want to support more than one batch system at a time. At the site level, it should also be possible to migrate to another common batch system and not being prevented from doing so by lack of support by the CE. However, a site cannot expect to pick any new batch system it likes and assume a third party will develop CE support for it.

For what regards SLURM support in particular, there was a request to CREAM developers to support it. EGI made a survey in early 2011 about the currently deployed LRMS and those considered to be deployed in the future. SLURM did not score much interest at the time (5-10 sites out of 170 answers), but given further requests EGI made another survey. As a result of that, a requirement to EMI to introduce SLURM support in CREAM was issued [1].

Another requirement is that it should be possible to define an alias to a CE name, with a set of individual CE's hidden behind this alias; functionally, in this case it should be possible to add or remove individual CE's on the fly. A single entry point might then be defined into an information system. Partial fulfillment of this requirement is already available in the CREAM version released with EMI-1 [2], with a full solution planned for the CREAM version released with EMI-3.

As input to technology providers, possible future work could involve the identification of a set of logical API's that each CE should provide to be usable by WLCG VO's.

Notes

[1] EGI ticket 3262: SLURM support in CREAM, https://rt.egi.eu/guest/Ticket/Display.html?id=3262

[2] Define a DNS alias to refer to set of CREAM CEs, https://wiki.italiangrid.it/twiki/bin/view/CREAM/SystemAdministratorGuideForEMI1#1_2_2_Define_a_DNS_alias_to_refe

-- DavideSalomoni - 03-Feb-2012