JRA1.3 - Definition and Implementation of the Data Area Work Plan (EMI 2/3)

Common Data Area Working Groups, as described in the DJRA1.2.2

emi-two.png This is a summary page and switchboard of the objectives of EMI 2/3 in the Data arena, according to deliverable DJRA-1.2.2 and the higher level deliverable DNA-1.3.2, which both have been accepted by the EU Commission. For you convencience, please find two tables at the end of the page, summarizing the Data specific and cross area objectives.

What When Details
Activity Leader Phone Conference Wednesday Aug 24, 11:00 Minutes

The red numbers in brackets refer to the paragraph with the same number in DJRA 1.2.2 plus the due date for the objective. The blue numbers refer to that objective in the deliverable DNA 1.3.2 and point to the corresponding savannah ticket in the EMI Development Tracker (Tasks).

GLUE 2.0 migration.

Details on the GLUE 2 Migration
  • [4.1,M12] All Storage Elements must publish initial GLUE 2.0 storage information.
  • [4.2,M16] Enabling storage clients comsuming GLUE 2.0 information published by storage elements. DNA1.3.2-D1
  • [4.3,M22] Enabling all storage elements to publishing the full set of GLUE 2.0 storage information and EMI data clients to be capable consuming that, including publishing coherent GLUE 2 based version information. DNA 1.3.2-D6 DNA 1.3.2-X1

UNICORE integration

Details on the UNICORE Integration
  • [4.4,M18] Providing UNICORE access to the gLite file catalogue. DNA 1.3.2-D4
  • [4.5,M22] Providing UNICORE access to GRID storage elements via the SRM protocol. DNA1.3.2-D7
  • [4.6,M28] Providing UNICORE access to the AMI AMGA meta data service. DNA1.3.2-D13

Posix file system access for EMI storage elements and intergration into SRM supported protocols.

More on the POSIX file system access
  • [4.7,M12] All EMI storage elements must offer at least a prototype-level support for the file:// access protocol.
  • [4.8,M18] All SRM-capable EMI data clients and services must add file:// to the already supported file access protocols. DNA 1.3.2-D3
  • [4.9,M24] All EMI storage elements must offer a production level support for the file:// protocol. [DNA 1.3.2-D?]

SRM over http(s).

More on SRM over https
  • [4.10,M12] Using https instead of httpg for the SRM protocol as a prototype implementation in one storage element and a client.
  • [4.11,M25] Using https instead of httpg for the SRM protocol as a production implementation in all the storage elements and clients, utilizing the EMI delegation. DNA 1.3.2-D12 DNA 1.3.2-X12

Http(s) and WebDAV for data access and catalogue operations.

  • [4.12,M18] All storage elements must offer support for the http(s) protocol for reading files. DNA 1.3.2-D8
  • [4.13,M25] All storage elements must offer support for the WebDAV protocol. DNA 1.3.2-D11
  • [4.14,M16] (Details) Investigating the possibility to support http or WebDAV for accessing the LFC, in order to provide a standard user-friendly access method to the EMI catalogue. DNA 1.3.2-D2

The EMI common data library

More in the EMI Data client library consolidation
  • [4.15,M22] Providing a common set of data access libraries at least between gLite and ARC (EMI-DATALIB). DNA 1.3.2-D5
  • [4.16,M32] Completing the migration to the common EMI-Datalib data access libraries. DNA 1.3.2-D15

Storage Element and Catalogue synchronization

More on the SE and Catalogue synchronization
  • [4.17,M25] Solving the sychronization problem of the Storage Elements and the EMI file catalogue and providing an interface to allow the LHC experiment file catalogues to be intergrated. DNA 1.3.2-D10

The next generation FTS (FTS-3)

More in the next generation FTS
  • [4.18,M30] Designing and implementing the next generation FTS: a distributed next generation file transfer service that, amoung others, utilizes the common messaging system. DNA 1.3.2-D14

EMI Storage accounting record design and implementation

More on the EMI Storage accounting record
  • [4.19,M12] Achieving an agreement over a common storage accouting record, including the refinement, definition and adoption (IF/WHEN applicable) of relevant standards.
  • [4.20,M32] Adding support for storage space usage accouting to SEs, based on the agreed record. DNA 1.3.2-D16

ARGUS integration into SE's and catalogues.

  • [4.21,M18] Integrating ARGUS into the EMI data components. DNA 1.3.2-X9

Investigating into EMI Data Clouds based on recommendations of the EMI Cloud Strategry group.

  • [4.22,M32] Implementing the EMI cloud strategy within the Data Area. DNA 1.3.2-D17

Persistent data ID's

  • [4.23,M24] Evaluating solutions to work with EMI data services in the context of persistent data ID's DNA 1.3.2-D9

Improve user satisfaction : Improving usability, maintainability, portability and monitoring of EMI Data products.

  • [4.24,M18] Adhering operating system standards for service operation and control, regarding configuration, log, termporary file location and service start/status/stop. DNA 1.3.2-X6
  • [4.25,M18] Providing and supporting monitoring probes for EMI services (E.g. NAGIOUS(R)). DNA 1.3.2-X4
  • [4.26,M22] Improving usability of client tools, based on customer feedback by ensuring
    • better, more informative, less contradictory error messages
    • coherency of command line parameters. DNA 1.3.2-X8
  • [4.27,M22] Porting, releasing and supporting EMI comonents on identified platforms (full distribution on SL6 and Debian 6, UI on SL5/32 and the latest UBUNTU). DNA 1.3.2-X7
  • [4.28,M28] Introducing minimal denial of service protection for EMI services via configurable resource limits. DNA 1.3.2-X10
  • [4.29,M30] Providing optimized semi-automated configuration of service backends (e.g. databases) for standard deployments. DNA 1.3.2-X13

Summary of data-only related tasks

ID Task Description Details
DNA 1.3.2-D1 GLUE 2.0 support in data clients More
DNA 1.3.2-D2 HTTP/!WebDAV for the gLite LFC Not started yet
DNA 1.3.2-D3 Add the file:// protocol to the SRM clients and services More
DNA 1.3.2-D4 LFC support within UNICORE More
DNA 1.3.2-D5 Design and implementation of EMI_datalib More
DNA 1.3.2-D6 GLUE 2.0 support in the storage elements More
DNA 1.3.2-D7 Add SRM interface to UNICORE More
DNA 1.3.2-D8 Add https protocol to EMI SEs PT private
DNA 1.3.2-D9 Persistent data ID's Not started yet
DNA 1.3.2-D10 Synchronization of LFC and SEs services More
DNA 1.3.2-D11 Add WebDAV protocol to EMI SEs Pending
DNA 1.3.2-D12 Implement https and EMI delegation-based SRM transport More
DNA 1.3.2-D13 AMGA support within UNICORE More
DNA 1.3.2-D14 Next generation FTS PT private
DNA 1.3.2-D15 Migration to the EMI_datalib More
DNA 1.3.2-D16 EMI accounting record implementation by storage services More
DNA 1.3.2-D17 Cloud strategy implementation within the data area No started yet

Summary of cross area related tasks

ID Task Description Details
DNA 1.3.2-X1 GLUE must publish component version information Not started yet
DNA 1.3.2-X4 Nagios probles for EMI services Not started yet
DNA 1.3.2-X6 Standard service operation and control Not started yet
DNA 1.3.2-X7 Porting products to additional platforms Not started yet
DNA 1.3.2-X8 Improving usability of clients Not started yet
DNA 1.3.2-X9 ARGUS and SE's integration Not started yet
DNA 1.3.2-X10 Minimal DOS protection Not started yet
DNA 1.3.2-X12 Phasing out legasy GSI Not started yet
DNA 1.3.2-X13 Semi-automated service backend configuration Not started yet

-- PatrickFuhrmann - 01-Aug-2011

Edit | Attach | Watch | Print version | History: r9 < r8 < r7 < r6 < r5 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r9 - 2011-08-31 - PatrickFuhrmann
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    EMI All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2023 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback