Status of Batch System support on CREAM

Warning: this page is outdated

This page tracks the status of batch system support on the CREAM CE.

Acceptance criteria

The SA1 strategy for the transition from LCG-CE to CREAM CE lists a number of acceptance criteria that will have to be met before the transition can be started. Some of these criteria relate to batch system support. These are:

• Default integration of the following batch systems:
  • Torque/Maui
  • LSF
  • Sun Grid Engine
  • Condor
• The process for integration other batch systems should be fully documented
• Performance criteria: see https://twiki.cern.ch/twiki//bin/view/EGEE/CEStressTesting

See also This table by Nick Thackray.

Summarily, the current state could be described as follows:

• the integration of batch systems for CREAM seems to be well underway
• there are no conclusive performance figures available yet.

Batch system integration

(Main page: BatchSystems).

Integration of a batch system with CREAM involves mainly the BLAH scripts (for submit, cancel, status, hold, resume of jobs). According to the Installation and Configuration guide, the production version of CREAM/BLAH only supports Torque, LSF and Condor.

The blah CVS has incorporated the BLAH scripts for Torque, LSF, Condor and SGE, so a future release will likely have support for all of these systems.

Documentation for integrating other batch systems

This documentation is currently in the works, part of task #8921, to appear in this Wiki as CreamLRMSCookBook. There is a (postponed) task #8494 assigned to INFN for general support; the BLAH guide proves to be very useful.

Certification

Torque

• INFN is doing pre-certification and developed functional tests (for which batch systems? FIXME)
• GRNET (see CertTestBedAtGRNET) is doing certification for Torque https://savannah.cern.ch/patch/?func=detailitem&item_id=2667

Condor

Condor integration is done by PIC, and certification by Nikhef. Condor_utils is in no longer in PreProduction ... There should be a new task ASAP.

-- DVanDok - 23 Apr 2009

SGE

Integration work for SGE is underway, by CESGA. See savannah task #9126. Planned to be done in about a month. Who is doing certification?

Performance testing

• Russian partners in SA3 have a test infrastructure to do performance tests.
• There is a postponed task for benchmarking CREAM assigned to GRNET. task #8302

There is no task assigned to this at the moment.

Some preliminary findings indicate that CREAM is using a lot of CPU power under load. https://savannah.cern.ch/patch/?2669#comment28

See also https://twiki.cern.ch/twiki//bin/view/EGEE/CEStressTesting.

-- DVanDok - 05 Feb 2009