The ATLAS LFC merge

This talk describes the peculiarities of the big initiative that led the ATLAS experiment to merge together its worldwide instances of LFC into an unique instance hosted at CERN. The ATLAS LFC merge process constituted a massive exercise, made more challenging by its size, aiming at collecting hundreds of millions of file metadata entries, to serve them through a robust, redundant setup of LFC.

Moreover, we also describe the main reasons why the merging process had to be done with a specialized multithreaded application, which was developed in order to accommodate the peculiarities that make this process much more challenging than a mere movement of data between SQL database instances. The process, in fact, has to take into account a number of situations where the metadata records clash, or contain errors that have to be fixed on the fly, still guaranteeing a high level of performance.

  • Track: Distributed Processing and Analysis on Grids and Clouds
  • Primary Author: Fabrizio Furano (IT-GT)
  • Co-Authors from IT-ES: Simone Campana
  • Presentation Type: parallel

-- SimoneCampana - 06-Oct-2011

Edit | Attach | Watch | Print version | History: r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r2 - 2011-10-12 - SimoneCampana
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback