Task Table

1. SRM specification consolidation and integration into UNICORE
   1. Start discussion with the OGF SRM community group [PM 3]: A discussion on how to progress with the idea of a remote storage resource manager protocol beyond SRM 2.2 has already been taken place at OGF in May in Chicago. The responsible group within OGF agreed with the goals of EMI to stabilize and consolidate the currently available specification based on feedback from the available implementations.
   2. Collection of community feedback [PM 9]: As the SRM has been adopted by groups outside of the scope of EMI, EMI data will collect user feedback to find the flaws in the current specification and problems with the client and server implementations.
   3. Initial document [PM 12]: A document to supplement the SRM v2.2 specification will be provided to OGF, clarifying all issues found so far. Moreover an “SRM 2.2 developers guide” is envisioned describing common use cases and known pitfalls.
   4. SRM integration into UNICORE [PM 24]: Based on the previous work, UNICORE data will either integrate an SRM 2.2 client by implementing the specification or by adopting existing solutions. SRM plus 4.3.3 (POSIX data access) enables UNICORE to seamlessly interoperate with EMI storage elements.
   5. Review and final document. [PM 24]: To goal is to have OGF accepting the EMI document as an input to the SRM community efforts and its specification process.
2. Replacing httpg with SSL/X509 (https) for the SRM
   1. Creating one prototype SE and one client [PM 12]: One EMI storage element and one client implementation will be chosen to provide a prototype SRM using SSL/X509 instead of the Globus httpg security protocol. Other SRM software providers are contacted and a joint plan is made to migrate between PM 24 to PM 36.
   2. All EMI Storage Elements plus EMI clients [PM 24]: Based on the experience with the above prototype, all EMI SE’s and clients should support https alternatively to httpg. In parallel, the development progress of non-EMI storage elements and clients is tracked.
   3. Migration scenario design for infrastructures [PM 36]: The migration plan is implemented.
3. POSIX data access (native or NFS 4.1)
   1. Setting up a system for performance and reliability testing [PM 5]: Reasonably large test Compute and Storage Elements are setup to evaluate reliability and performance of the native POSIX and NFS 4.1 implementations.
   2. Report of first performance results at CHEP [PM 10]: Reports on the results of 4.3.3.1 will be reported at the CHEP’10 conference.
   3. Production POSIX access for StoRM and dCache. [PM 12]: As StoRM is based on POSIX file systems (GFPS, Lustre), this milestone it fulfilled for StoRM with day zero. DCache will provide a production NFS 4.1 implementation and DPM is planning for a prototype with PM 12.
   4. All EMI storage elements provide POSIX access [PM 24]: All EMI storage elements will provide genuine POSIX file system through mounted file systems.
4. Web access, http(s) and WebDav
   1. Storage Elements provide access via http(s) for downloading data. [PM 12]: dCache and DPM will provide read access to data via http(s). StoRM will make this decision dependent on user requirements.
   2. Storage Elements provide partial or full access to data via WebDav [PM 24]: dCache will provide protected WebDav access to data. StoRM and DPM will make this decision dependent on user requirements.
5. Catalogue synchronization
   1. Agreed design [PM 8]: A decision on how to tackle the catalogue name-space synchronization problem will be made and agreed on between the EMI catalogue and Storage Element providers.
   2. Prototype as ‘proof of concept’ [PM 12]: A prototype of the above agreement will be implemented.
   3. LFC plus one storage element provide full synchronization functionality [PM 18]: Dependent on the results of the above prototype, the LFC and at least one SE will provide the synchronization feature in production.
   4. All EMI SE’s provide synchronization functionality [PM 24]: The name space of the LFC and all EMI SE’s will be able to stay synchronized.
6. File catalogue access for UNICORE
   1. Implementation of file catalogue integration [PM 12]: A prototype of the file catalogue access will be implemented.
   2. Evaluation of file catalogue integration [PM 15]: The prototype will be evaluated and tested and the final version will be published.
7. GLUE 2.0
   1. Common agreement on the interpretation of the GLUE 2.0 schema. [PM 6]: Although GLUE 2.0 is already well defined, all partners, which will have to implement the standard, will have to agree on a common interpretation of the specification, and possibly extend it aligning with the OGF PGI group and their extensions if necessary.
   2. Publishing GLUE 1.3 data as with GLUE 2.0 schema [PM 12]: The currently available information in the various components will be published, using the GLUE 2.0 schema in addition to GLUE 1.3. Addition information required by the GLUE 2.0 specification might still be missing. Clients are required to cope with the partially available information.
   3. EMI data components fully GLUE 2.0 compatible [PM 24]: All required information is published using GLUE 2.0.
8. Data client library consolidation
   1. Development of a consolidation between the ARC and gLite data clients. [PM 24]: A design will be presented on how to merge the ARC and gLite data access libraries and a plan on how to migrate.
   2. Migration of the new data client component. [PM 36]: Full migration to the merged data access libraries will be finalized.
9. Integration of ARGUS
   1. ARGUS Blacklist integration of at least one EMI Storage Element. [PM 12]: At least one EMI storage element will have integrated the EMI ARGUS blacklisting mechanism.
   2. ARGUS Blacklist integration of all EMI Storage Elements. [PM 24]: All EMI storage elements will have the EMI ARGUS blacklisting mechanism integrated.
   3. Integration of further ARGUS functionality into EMI SE’s, if required. [PM 36]: Each storage element implementation may integrate more features, the ARGUS software is offering. Replacing internal file system or catalogue authentication with the ARGUS authentication is not envisioned.
10. Storage Accounting
    1. Definition of a storage accounting record. [PM 8]: A storage accounting record is defined (or a standardized schema like the OGF URs might be extended), reflecting practical, financial and legal requirements of storage location, usage and space and data flow.
    2. Add support of storage accounting in FTS and EMI storage elements. [PM 36]: FTS and the EMI storage elements will provide information according to the agreed record in 4.3.10.1.
11. Monitoring
    1. Collaborating with the EMI infrastructure group to define an interface for EMI data monitoring. [PM 12]: EMI data will support the EMI infrastructure group to define an interface for commonly monitor availability, problems and performance of EMI data components.
    2. Implementing server side sensors accordingly. [PM 24]: The agreed interface will be implemented in EMI storage elements as well as in FTS and the LFC.
12. Manageability
    1. CLI and Web Interfaces, where required. [PM 24]: Up to now, the development of most of the EMI data components has been focused on reliability and performance. Where necessary, the maintainers of the components, in collaboration with their users will evaluate the necessity of improved command line interfaces (CLIs) or Web based interfaces. The progress is tracked by EMI data but not enforced.
13. Evaluation: Message passing in EMI-data
    1. Collecting possible use cases for message passing in EMI data. [PM 7]:

EMI infrastructure will provide a common mechanism for message passing for all EMI components. EMI-data will provide input to the infrastructure group to help selecting an appropriate mechanism and convenient interfaces.

-- PatrickFuhrmann - 16-Aug-2010