--++++ Title AliEn Extreme Brokering

Abstract

The AliEn workload management system is based on a central job queue wich holds all tasks that have to be executed. The job brokering model itself is based on pilot jobs: the system submits generic pilots to the compuiting centres batch gateways, and the assignment of a real job is done only when the pilot wakes up on the worker node. The model facilitates a flexible fair share user job distribution. This job model has proven stable and reliable over the past years and has surviced well the first two years of LHC operation with very little changes. Nonetheless there are several areas where the model can be pushed to the next level: most notably in the area of 'just in time' job and data assignment, where the decisions will be based on data closeness (relaxed locality) and the data which already has been processed. This methods will have significant efficiency enhancement effect for end user analysis tasks.

This contribution will describe the new mechanism implemented in AliEn for the job splitting and matchmaking. It will also show how the users benefit from such a model.

Summary

No

Author

Pablo Saiz

Co-authors

ZHU, Jianlin (Central China Normal University (CN)) SCHREINER, Steffen (Technische Universitaet Darmstadt (DE)) GRIGORAS, Costin (CERN) GRIGORAS, Alina Gabriela (CERN) BETEV, Latchezar (CERN) CARMINATI, Federico (CERN) PORTER, Jeff (Lawrence Berkeley National Lab. (US)) MONTIEL GONZALEZ, Almudena Del Rocio (GSI - Gesellschaft fuer Schwerionen forschung (GSI)) BAGNASCO, Stefano (I.N.F.N. TORINO) GOYAL, Dushyant (LNM Institute of Information Technology (IN)) ABRAMYAN, Armenuhi (Yerevan Physics Institute) BANERJEE, Subho Sankar (LNM Institute of Information Technology) MANUKYAN, Narine (A.I. Alikhanyan National Scientific Laboratory (AM))

Track

Distributed Processing and Analysis on Grids and Clouds

Presentation type

Parallel

Comments

No

-- PabloSaiz - 28-Oct-2011