XRootDMonitoring < LCG

LCG Web>WLCGCommonComputingReadinessChallenges>WLCGOperationsWeb>ItEsHome>CHEP2013Abstracts>XRootDMonitoring (2013-03-14, JuliaAndreeva) (raw view)

EditAttachPDF

---++  Monitoring of large-scale federated data storage: XRootD and beyond.

  The computing models of the LHC experiments are gradually moving from hierarchical data models with centrally managed data pre-placement
towards federated storage which provides seamless access to data files independently of their location and dramatically improved recovery due
to fail-over mechanisms. Enabling loosely coupled data clusters to act as a single storage resource should increase opportunities for data
analysis and should enable more effective use of computational resources at sites with limited storage capacities. Construction of the data
federations and understanding the impact of the new approach to data management on user analysis requires complete and detailed monitoring.
Monitoring functionality should cover the status of all components of the federated storage, measuring data traffic and data access
performance, as well as being able to detect any kind of inefficiencies and to provide hints for resource optimization and effective data
distribution policy. Data mining of the collected monitoring data provides a deep insight into new patterns of usage of the storage resources,
beyond that provided by other monitoring strategies.

  In the WLCG context, there are several federations currently based on the XRootD technology. The talk will focus on monitoring for the ATLAS
and CMS XRootD federations (Federated Atlas XRootD (FAX) and Any Data, Any Time, Anywhere (AAA)) implemented in the Experiment Dashboard
monitoring framework. Both federations consist of many dozens of sites accessed by many hundreds of clients and they continue to grow in
size. Handling of the monitoring flow generated by these systems has to be well optimized in order to achieve the required performance.

  The talk will demonstrate that though FAX and AAA Dashboards are being developed for XRootD federations, the implementation is generic and can
be easily adapted for other technologies, such as HTTP/WebDAV federations.



   * Track: Distributed Processing and Data Handling
   * Co-Authors from IT-ES: A. Beche, J. Andreeva, D. Giordano, P. Saiz, D. Tuckett
   * Presentation Type: Parallel
   * Author list:: A. Beche, J. Andreeva, D. Giordano, P. Saiz, D. Tuckett , D. Diéguez Arias, S. Belov, M. Tadel, I. Vukotic, D. Oleynik, A. Petrosyan

Topic revision: r1 - 2013-03-14 - JuliaAndreeva

LCG Wikis

LCG Service
Coordination

LCG Grid
Deployment

LCG
Apps Area

Public webs

Welcome Guest

- Cern Search
- TWiki Search
- Google Search
LCG All webs

Copyright &© 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback