ContentDeliveryCaching

Introduction

Contact and agenda

  • Mailing list : wlcg-doma-access AT cern.ch

Mandate

  • Study Data Access by applications including content delivery services and networks, smart clients, caching solutions for LHC experiments and sciences with similar challenges.
  • Provide a forum to share experience gained on improving remote and local data access by the Experimentsí current and future workloads, taking into account developments such as machine learning and compact analysis data formats.
  • Gather measurements and metrics on the different data access studies and compile quantitative information with the primary goal to be used by the WLCG DOMA activities.
  • Based on the shared experience and the needs of WLCG-DOMA this WG will identify, by open discourse, areas where further R&D is required and prioritise these topics
  • This process is intended to stimulate collaboration between different parties and foster increased commonalities between experiments, storage solutions and site infrastructures
  • This WG will track and regularly report on the progress of the identified topics
  • This WG will maintain close links to relevant the DOMA project WGs and activities: TPC/Protocols, Networking, QoS, Authorisation and Authentication, etc.

Current Activities and Projects

  • Generic developments
    • Rucio
    • The XDC Project
    • Compute / performance measurements
  • Study performances
    • CERN UP Team Data Access related Activities
    • Estimating Cache hit rates based on Data Popularity data (CERN UP Team Data Access related Activities)
    • Measurement of the impact of an xrootd based cache on throughput of the experimentsí standard workloads as they have been provided for the HSF/WLCG Performance and Cost Modeling Working group (CERN UP Team Data Access related Activities)
    • Measuring the sensitivity of arbitrary workloads on latency and bandwidth limitations (CERN UP Team Data Access related Activities)
    • French initiative: evaluation by French computing community of performances and cost of remote access and future distributed storage services
  • Network
    • SENSE: SDN for End-to-end Networked Science at the Exascale
    • SANDIE: SDN Assisted Named Data Networking (NDN) for Data Intensive Experiments *The SANDIE System
  • Data pattern access
    • R&D on data access patterns @ CMS
    • R&D on data access patterns @ ATLAS
  • Deploying cache mechanism
    • Xrootd
      • Production Xrootd Cache across Southern California (UCSD/Caltech):
      • Production StashCache (Xrootd & CVMFS combo) open for all of science in OSG:
      • Production XRootD Proxy Cache in Edinburgh for ATLAS
    • Dpm
      • R&D in the context Belle II -HTTP Data Federation eco-system with caching functionality using DPM Volatile Pool + Dynafed - (added by S. Pardi, D. Michelino, B. Spisso)
    • Dcache
      • Distributed dCache deployment with caching enabled
      • ATLAS with xcache
    • EOS
      • EOS Smart Caching - XrdPss + XCache prototype (part of the XDC project)
    • Independent of Storage technology
      • Italian XCache Deployment for CMS:
      • National geo distributed federation and automated setup for Dynamic showcases (INFN)
      • Prototype and R&D on Coordinating Caches for Opportunistic Data Locality (KIT)
      • US ATLAS Activities
      • ARC Cache

Detailed descriptions of the the current list of projects and activities is available in this googledoc (link). If you had a new project, try to associate to the right group.

XCache initiatives:

Participants

Coordination : F. Wuerthwein (U. California), I Vukotic (U. Chicago), S. Jezequel (LAPP) with support from M. Schultz (CERN-IT) and X. Espinal (CERN-IT)

-- XavierEspinal - 2018-09-18

Edit | Attach | Watch | Print version | History: r8 < r7 < r6 < r5 < r4 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r8 - 2019-06-12 - XavierEspinal
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback