The libarcdata2 library and the corresponding arc* data clients provides a uniform way to move data from point to point supporting different data transfer protocols. This is a successor of the previous ng* data clients which are still widely used and will be maintained.

The libarcdata2 library has a modular structure to support different data transfer protocols. The core libarcdata2 library does not introduce any additional external dependencies. The plugins (DMCs) for specific data access protocols can however have various external dependencies. This separation of external dependencies from the core library helps reduce the minimum set of requirements for ARC while allowing the support for additional access protocols requiring special dependencies to be installed by those who need it.

Most of these components and the clients are also available on different platforms (Linux, Windows, Mac, Solaris), and the libarcdata2 is also available from Python, and to some extent from Java.

The ARC data clients which the team is responsible for are the following:

  • ngls/arcls is a simple utility that allows to list contents and view some attributes of objects of a specified (by a URL) remote directory.
  • ngcp/arccp is a powerful tool to copy files from a data point to on other data point where both can use different protocols.
  • ngrm/arcrm is a command allows users to erase files at any location specified by a valid URL.

Currently the following transfer protocols and metadata servers are supported:

  • ftp: ordinary File Transfer Protocol (FTP)
  • gsiftp: GridFTP, the Globus-enhanced FTP protocol with security, encryption, etc.
  • http: ordinary Hyper-Text Transfer Protocol (HTTP) with PUT and GET methods using multiple streams
  • https: HTTP with SSL v3
  • httpg: HTTP with Globus GSI
  • ldap: ordinary Lightweight Data Access Protocol (LDAP)
  • rls: Globus Replica Location Service (RLS)
  • lfc: LFC catalog and indexing service of EGEE gLite
  • srm: Storage Resource Manager (SRM) service
  • file: local to the host file name with a full path
  • arc: for the Chelonia storage service

EMI-1 release notes

What's new:

  • New plugins-based data library, libarcdata2, replaces previous data library.
  • Data access is now managed through DMCs, protocol-specific plugins.
  • New data clients (arcls, arccp, arcrm, arcsrmping) replace previous data clients (ngls, ngcp, ngrm).
  • Improved default output for arcls, now closer to UNIX ls. Options are provided to specify non-default output.
  • Improved error messages for arcls and arccp.
  • Improved calculation of file checksums.
  • Fixed a bug where the check for readability (arcls -c) failed incorrectly in some cases.
  • Fixed a bug where metadata from gridftp objects were incorrectly reported.

Deployment notes:

  • Previous data library and clients are now available from nordugrid-arc-compat package.
  • The data clients are available from the nordugrid-arc-client package.
  • The data library is available from the nordugrid-arc package.
  • The DMCs are available from nordugrid-arc-plugins-needed and nordugrid-arc-plugins-globus (the latter requires globus).
  • Some options have been changed or removed between ng* and arc* commands, and options have been added to the arc* commands, see documentation for details.
  • No configuration changes are needed from previous release.

Known issues:


ARC Data library and clients components are released under Apache v2.0 license.


  • NIIFI: Zsombor Nagy
  • UiO: Jon Kerr Nilsen (PT leader)
  • UU: Salman Toor (?), Mattias Ellert (?)

-- JonKerrNilsen - 21-Sep-2010

Edit | Attach | Watch | Print version | History: r4 < r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r4 - 2011-02-21 - JonKerrNilsen
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    EMI All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback