JRA1.3 - Definition and Implementation of the Data Area Work Plan (EMI 2/3)

Common Data Area Working Groups, as described in the DJRA1.2.2

This is a summary page and switchboard of the objectives of EMI 2/3 in the Data arena, according to deliverable DJRA-1.2.2 and the higher level deliverable DNA-1.3.2, which both have been accepted by the EU Commission. For you convencience, please find two tables at the end of the page, summarizing the Data specific and cross area objectives.

Meetings

What	When	Details
Activity Leader Phone Conference	Wednesday Aug 24, 11:00	Minutes

Legend

The red numbers in brackets refer to the paragraph with the same number in DJRA 1.2.2 plus the due date for the objective. The blue numbers refer to that objective in the deliverable DNA 1.3.2 and point to the corresponding savannah ticket in the EMI Development Tracker (Tasks).

GLUE 2.0 migration.

Details on the GLUE 2 Migration

• [4.1,M12] All Storage Elements must publish initial GLUE 2.0 storage information.
• [4.2,M16] Enabling storage clients comsuming GLUE 2.0 information published by storage elements. DNA1.3.2-D1
• [4.3,M22] Enabling all storage elements to publishing the full set of GLUE 2.0 storage information and EMI data clients to be capable consuming that, including publishing coherent GLUE 2 based version information. DNA 1.3.2-D6 DNA 1.3.2-X1

UNICORE integration

Details on the UNICORE Integration

• [4.4,M18] Providing UNICORE access to the gLite file catalogue. DNA 1.3.2-D4
• [4.5,M22] Providing UNICORE access to GRID storage elements via the SRM protocol. DNA1.3.2-D7
• [4.6,M28] Providing UNICORE access to the AMI AMGA meta data service. DNA1.3.2-D13

Posix file system access for EMI storage elements and intergration into SRM supported protocols.

More on the POSIX file system access

• [4.7,M12] All EMI storage elements must offer at least a prototype-level support for the file:// access protocol.
• [4.8,M18] All SRM-capable EMI data clients and services must add file:// to the already supported file access protocols. DNA 1.3.2-D3
• [4.9,M24] All EMI storage elements must offer a production level support for the file:// protocol. [DNA 1.3.2-D?]

SRM over http(s).

More on SRM over https

• [4.10,M12] Using https instead of httpg for the SRM protocol as a prototype implementation in one storage element and a client.
• [4.11,M25] Using https instead of httpg for the SRM protocol as a production implementation in all the storage elements and clients, utilizing the EMI delegation. DNA 1.3.2-D12 DNA 1.3.2-X12

Http(s) and WebDAV for data access and catalogue operations.

• [4.12,M18] All storage elements must offer support for the http(s) protocol for reading files. DNA 1.3.2-D8
• [4.13,M25] All storage elements must offer support for the WebDAV protocol. DNA 1.3.2-D11
• [4.14,M16] (Details) Investigating the possibility to support http or WebDAV for accessing the LFC, in order to provide a standard user-friendly access method to the EMI catalogue. DNA 1.3.2-D2

The EMI common data library

More in the EMI Data client library consolidation

• [4.15,M22] Providing a common set of data access libraries at least between gLite and ARC (EMI-DATALIB). DNA 1.3.2-D5
• [4.16,M32] Completing the migration to the common EMI-Datalib data access libraries. DNA 1.3.2-D15

Storage Element and Catalogue synchronization

More on the SE and Catalogue synchronization

• [4.17,M25] Solving the sychronization problem of the Storage Elements and the EMI file catalogue and providing an interface to allow the LHC experiment file catalogues to be intergrated. DNA 1.3.2-D10

The next generation FTS (FTS-3)

More in the next generation FTS

• [4.18,M30] Designing and implementing the next generation FTS: a distributed next generation file transfer service that, amoung others, utilizes the common messaging system. DNA 1.3.2-D14

EMI Storage accounting record design and implementation

More on the EMI Storage accounting record

• [4.19,M12] Achieving an agreement over a common storage accouting record, including the refinement, definition and adoption (IF/WHEN applicable) of relevant standards.
• [4.20,M32] Adding support for storage space usage accouting to SEs, based on the agreed record. DNA 1.3.2-D16

ARGUS integration into SE's and catalogues.

• [4.21,M18] Integrating ARGUS into the EMI data components. DNA 1.3.2-X9

Investigating into EMI Data Clouds based on recommendations of the EMI Cloud Strategry group.

• [4.22,M32] Implementing the EMI cloud strategy within the Data Area. DNA 1.3.2-D17

Persistent data ID's

• [4.23,M24] Evaluating solutions to work with EMI data services in the context of persistent data ID's DNA 1.3.2-D9

Improve user satisfaction : Improving usability, maintainability, portability and monitoring of EMI Data products.

• [4.24,M18] Adhering operating system standards for service operation and control, regarding configuration, log, termporary file location and service start/status/stop. DNA 1.3.2-X6
• [4.25,M18] Providing and supporting monitoring probes for EMI services (E.g. NAGIOUS(R)). DNA 1.3.2-X4
• [4.26,M22] Improving usability of client tools, based on customer feedback by ensuring
  • better, more informative, less contradictory error messages
  • coherency of command line parameters. DNA 1.3.2-X8
• [4.27,M22] Porting, releasing and supporting EMI comonents on identified platforms (full distribution on SL6 and Debian 6, UI on SL5/32 and the latest UBUNTU). DNA 1.3.2-X7
• [4.28,M28] Introducing minimal denial of service protection for EMI services via configurable resource limits. DNA 1.3.2-X10
• [4.29,M30] Providing optimized semi-automated configuration of service backends (e.g. databases) for standard deployments. DNA 1.3.2-X13

Summary of data-only related tasks

ID	Task Description	Details
DNA 1.3.2-D1	GLUE 2.0 support in data clients	More
DNA 1.3.2-D2	HTTP/!WebDAV for the gLite LFC	Not started yet
DNA 1.3.2-D3	Add the file:// protocol to the SRM clients and services	More
DNA 1.3.2-D4	LFC support within UNICORE	More
DNA 1.3.2-D5	Design and implementation of EMI_datalib	More
DNA 1.3.2-D6	GLUE 2.0 support in the storage elements	More
DNA 1.3.2-D7	Add SRM interface to UNICORE	More
DNA 1.3.2-D8	Add https protocol to EMI SEs	PT private
DNA 1.3.2-D9	Persistent data ID's	Not started yet
DNA 1.3.2-D10	Synchronization of LFC and SEs services	More
DNA 1.3.2-D11	Add WebDAV protocol to EMI SEs	Pending
DNA 1.3.2-D12	Implement https and EMI delegation-based SRM transport	More
DNA 1.3.2-D13	AMGA support within UNICORE	More
DNA 1.3.2-D14	Next generation FTS	PT private
DNA 1.3.2-D15	Migration to the EMI_datalib	More
DNA 1.3.2-D16	EMI accounting record implementation by storage services	More
DNA 1.3.2-D17	Cloud strategy implementation within the data area	No started yet

Summary of cross area related tasks

ID	Task Description	Details
DNA 1.3.2-X1	GLUE must publish component version information	Not started yet
DNA 1.3.2-X4	Nagios probles for EMI services	Not started yet
DNA 1.3.2-X6	Standard service operation and control	Not started yet
DNA 1.3.2-X7	Porting products to additional platforms	Not started yet
DNA 1.3.2-X8	Improving usability of clients	Not started yet
DNA 1.3.2-X9	ARGUS and SE's integration	Not started yet
DNA 1.3.2-X10	Minimal DOS protection	Not started yet
DNA 1.3.2-X12	Phasing out legasy GSI	Not started yet
DNA 1.3.2-X13	Semi-automated service backend configuration	Not started yet

-- PatrickFuhrmann - 01-Aug-2011