Creating a simplified global unique file catalogue

The AliEn File Catalogue is a global and unique namespace that provides the mapping between logical file names and physical file names. The File Catalogue is the heart of the AliEn GRID system. It presents UNIX-like interface to the end user, hiding all the complexity of the GRID. On top of that, the AliEn File Catalogue provides mechanisms to store hierarchical metadata information, and powerful tools to search it.

The File Catalogue is the heart of the AliEn GRID system. All the other components depend on it:

  • All the data (raw, reconstructed, simulated, user files) is registered in the catalogue.
  • Jobs' input and output are pointers to entries in the catalogue
  • The jobs are allocated to the different sites based, among other things, on the file location.
  • All the packages, and the instructions of how to install them and use them, are also registered in the catalogue.

In the case of ALICE, the AliEn File catalogue currently contains around 700 million entries. Those files usually have several replicas, distributed among the more than 80 sites that participate in the experiment. Combining replica records in the AliEn File Catalogue with topological and functional status of the respective storage elements the closest location is indicated to clients that need to read and write data to the Grid.

For all these reasons, a lot of effort has been invested into ensuring that the AliEn File Catalogue can cope with the load. More over, different approaches to enhance the performance have been investigated. The latest improvements have gone in the direction of simplifying the schema and functionality, in particular removing the GUID layer, and handling sets of files at once. This contribution describes all the latest changes that have been done to the system.

  • Track: Distributed Processing and Data Handling
  • Co-Authors from IT-ES: M. Litmaath, P. Saiz
  • Presentation Type: Parallel
  • Author list:: M. Martinez, A. Abramyan, S. Bagnasco, L. Betev, F. Carminati, A. Grigoras, C. Grigoras, M. Litmaath, N. Manukyan, P. Saiz
Edit | Attach | Watch | Print version | History: r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r1 - 2013-03-14 - PabloSaiz
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback