Task Force Data Clients Consolidation: Evaluation of Existing Clients

Authors

Introduction

This document is an evaluation of the existing clients for ARC and gLite in the data area.

Middleware Clients

The middlewares mentioned in the following sections are ordered alphabetically, the order does not express any preferences.

ARC

Classes in ARC

The ARC middleware builds on several sets of classes both utilized by clients and services. These sets cover common, data handling, message handling/communication, module loading, security, info system classes among others. The data, message and security handler classes are general classes, which is extended by specific modules, e.g. for data handling SRM, gsiftp, http, etc., for message handling tls, http, gsi, etc., and for security handling x509, usernametoken etc. These modules are pluggable and can be used on the server as well as on the client side.

ng* data clients

These clients will be phased out in the near future.

arc* data clients and libarcdata2

The libarcdata2 library and the corresponding arc* data clients provides a uniform way to move data from point to point supporting different data transfer protocols. This is a successor of the previous ng* data clients which are still widely used and will be maintained.

The libarcdata2 library has a modular structure to support different data transfer protocols. The core libarcdata2 library does not introduce any additional external dependencies. The plugins (DMCs) for specific data access protocols can however have various external dependencies. This separation of external dependencies from the core library helps reduce the minimum set of requirements for ARC while allowing the support for additional access protocols requiring special dependencies to be installed by those who need it.

Most of these components and the clients are also available on different platforms (Linux, Windows, Mac, Solaris), and the libarcdata2 is also available from Python, and to some extent from Java.

Currently the following transfer protocols and metadata servers are supported: [http://www.nordugrid.org/documents/arc-ui.pdf]

• ftp: ordinary File Transfer Protocol (FTP)
• gsiftp: GridFTP, the Globus-enhanced FTP protocol with security, encryption, etc.
• http: ordinary Hyper-Text Transfer Protocol (HTTP) with PUT and GET methods using multiple streams
• https: HTTP with SSL v3
• httpg: HTTP with Globus GSI
• ldap: ordinary Lightweight Data Access Protocol (LDAP)
• rls: Globus Replica Location Service (RLS)
• lfc: LFC catalog and indexing service of EGEE gLite
• srm: Storage Resource Manager (SRM) service
• file: local to the host file name with a full path
• arc: for the Chelonia storage service

The arc* data clients are the following:

• arcls is a simple utility that allows to list contents and view some attributes of objects of a specified (by a URL) remote directory.
• arccp is a powerful tool to copy files from a data point to on other data point where both can use different protocols.
• arcrm is a command allows users to erase files at any location specified by a valid URL.

Advantages of using the ARC Data libraries

• It can be used on different platforms (Linux, Windows, Mac Solaris).
• It can be used from different languages (C++, python, java).
• It has a pluggable infrastructure which allows for a very limited set of dependencies from other libraries.
• Plugins exist for several widely used data transfer and management protocols.
• It is light-weight and user-friendly, with easy client-side configuration (the default config would work in most cases).

gLite

Summary

-- JonKerrNilsen - 03-Nov-2010