The ATLAS LFC merge

This talk describes the peculiarities of the big initiative that led the ATLAS experiment to merge together its worldwide instances of LFC into an unique instance hosted at CERN. The ATLAS LFC merge process constituted a massive exercise, made more challenging by its size, aiming at collecting hundreds of millions of file metadata entries, to serve them through a robust, redundant setup of LFC.

Moreover, we also describe the main reasons why the merging process had to be done with a specialized multithreaded application, which was developed in order to accommodate the peculiarities that make this process much more challenging than a mere movement of data between SQL database instances. The process, in fact, has to take into account a number of situations where the metadata records clash, or contain errors that have to be fixed on the fly, still guaranteeing a high level of performance.

• Track: Distributed Processing and Analysis on Grids and Clouds
• Primary Author: Fabrizio Furano (IT-GT)
• Co-Authors from IT-ES: Simone Campana
• Presentation Type: parallel

-- SimoneCampana - 06-Oct-2011