Moving clients off SRM

Introduction

With the end of support for DPM's legacy stack (which includes the SRM implementation), sites are being encouraged to do the following:

  • Activate the "Dome" subsystem
  • Retire the SRM service
  • Enable "Gridftp redirection"

This has two main impacts on clients

  • Clients using the SRM protocol will have to use alternative protocols
  • Older gridftp clients will suffer performance problems

How to adapt clients

You will have to update clients if either of the following apply:

  • They use SRM
  • They don't support the GridFTP redirection system used in DPM. Problematic clients include lcg-utils (inc. lcg-cp) and uberftp.

All SRM functionality made available via DPM is now available through other protocols. In general, we recommend using HTTPS for file access and GridFTP for file transfer. Most DPM deployments also offer the xrootd protocol for high-performance data access.

Use gfal2

We recommend using the multi-protocol grid file access library gfal2 (and associated gfal-util command line tools) as a replacement for any older clients. It supports all the protocols mentioned in this article (including SRM).

Information on how to obtain gfal2 is available - http://dmc.web.cern.ch/

More detailed documentation on the use of the library can be found at https://dmc-docs.web.cern.ch/dmc-docs/

For those coming from lcg-util, the gfal2 project maintains a table indicating how to migrate:

https://dmc-docs.web.cern.ch/dmc-docs/lcgutil-equivalence.html

Library bindings

The Gfal2 library has Python and C bindings. The API is protocol independent. One could easily reimplement existing workflows, retaining SRM, and then subsequently switch to a different protocol.

Storage resource reporting

Space reporting is now possible through a json file which the DPM can publish. Site administrators will be able to provide the URL of this file.

Note on GridFTP

A couple of technical details on GridFTP are worth mentioning.

  • The redirection system used in DPM (inherited from the Globus gridftp server) is "Delayed passive". This allows the head node to arrange for the data stream to go directly to the correct disk server
  • GridFTP should not be used directly on DPM systems which have not enabled redirection. This will cause data to be tunnelled through the head node. This refers to using "gsiftp://" style URLs for data transfers. The traditional use of GridFTP "under the hood" with SRM is fine in this case.

-- OliverKeeble - 2019-07-31

Edit | Attach | Watch | Print version | History: r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r2 - 2019-08-01 - OliverKeeble
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    DPM All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback