JRA1.1 - EMI-1 Development and Test Plans

Responsible
Riccardo Zappi
Last Update
2011-02-24
(Morris Riedel)

EMI-1 Component StoRM

StoRM is a service compliant with the open standard Storage Resource Manager (SRM) v2.2 being able to manage any POSIX file systems supporting the Access Control List (ACL) mechanism, including high performance storage systems based on cluster file system (such as GPFS from IBM and Lustre from Oracle/Sun Microsystems).

  • EMI File Access Solution
  • Managed by data area
  • Part of Product Team StoRM [ Details ]
  • Component in ETICS : emi.storm [ Details ]
  • Supported Platforms : SL5 - 64 Bit
  • EMI 1 Release
  • SA2 Testbed
    • 2011-02-22 : Production Version deployed [ More Details ]
    • 2011-02-22 : Riccardo : New features more or less installed, team works on that, planned to be ready on 2011-02-24 (Thursday)

Overall Component Status

2011-02-22_Storm.PNG

Mandatory Features for EMI 1 - Addressed Technical Objectives

  • EMI technical objective Data Area 1 : All storage elements publishing initial GLUE 2.0 storage information and one storage client (library) is capable consuming that. (M12)
    • Addressed by developers : [In Testing and Certification]
  • EMI technical objective Data Area 3 : All storage elements offering support for the http(s) protocol (M12)
    • Addressed by developers : [In Testing and Certification]
  • EMI technical objective Data Area 4 : All storage elements offering at least a prototype-level support for the "file://" access protocol. (M12)
    • Addressed by developers : [In Testing and Certification]

Mandatory Features for EMI 1 (ready)

The following features are implemented and considered to be part of the EMI 1 release 'Kebnekaise'.

GLUE 2.0 : The Information Provider in StoRM will publish data using the GLUE 2.0 and GLUE 1.3 schema in parallel. However, initially the content will be the same. Over time the information published with GLUE 2.0 will become richer.

  • Status: Developed; now in testing and certification phase
  • Verification : Testing Strategy 1 - GLUE2 Information [ More Details ]
    • 2011-02-22 : Riccardo : Not in the verification and validation plan yet
  • Milestone 1: 12/2010 - Development in beta quality present [Achieved]
  • Milestone 2: 01/2011 - Tests in place for continous internal JRA1 testings during 02/2011 (JRA1.7, JRA1.8) [Pending]
    • 2011-02-22 : Riccardo : No tests yet - but from Oliver and Laurence we can take tests to test this (Morris)
    • Warning : No tests are still available for this
  • Feature Deadline : 02/2011 [Achieved]
  • Addressing EMI technical objective Data Area 1 : All storage elements publishing initial GLUE 2.0 storage information and one storage client (library) is capable consuming that. (M12)
  • Development Task Tracker : [ Details ]

Web access, http(s) and WebDav : For EMI-1 StoRM will provide http(s).

  • Status: Developed; now in testing and certification phase
    • WebDAV is scheduled for M24 (TBD : Morris checks with Patrick)
  • Description : Developed a new sub-component of StoRM, named "StoRM-GridHTTPs". It can be deployed optionally.
  • Verification : Testing Strategy 2 - SEs Access Protocols [ More Details ]
  • Milestone 1: 12/2010 - Development in beta quality present [Achieved]
  • Milestone 2: 01/2011 - Tests in place for continous internal JRA1 testings during 02/2011 (JRA1.7, JRA1.8) [Achieved]
  • Feature Deadline : 02/2011 [Achieved]
  • Addressing EMI technical objective Data Area 3 : All storage elements offering support for the http(s) protocol (M12)
  • Development Task Tracker : [ Details ]

HSM support : Using GPFS as StoRM storage backend, migrating and retrieving data to and from the TSM Storage System will be supported.

  • Status: Developed; now in testing and certification phase
  • Description : 2011-02-22 Riccardo : The implementation is already in production since version 1.5.x of StoRM. In EMI 1 there will be StoRM 1.7.0, but the support for Tape will be not included.
    • Trying to make optional the support for GPFS in the StoRM BackEnd since the nodes of ETICS still do not have its libraries available
    • (license issue? See: https://savannah.cern.ch/task/?18584, https://savannah.cern.ch/task/?18436, https://savannah.cern.ch/task/?18387).
    • Excluding the support for GPFS means losing the Tape support. This lack doesn't represent a real problem for EMI, but it means that we must make a parallel release of StoRM (only the BE component) in order to satisfy our major users, the Italian Tier-1.
    • This represent an extra-work for StoRM team, and this might not be considered a good note for the management of INFN.
    • TBD : Follow-up License issue with GPFS and EMI (Morris, Riccardo, already in EMT tracker )
  • Verification : Testing Strategy 2 - SEs Access Protocols [ More Details ]
  • Milestone 1: 12/2010 - Development in beta quality present [Achieved]
  • Milestone 2: 01/2011 - Tests in place for continous internal JRA1 testings during 02/2011 (JRA1.7, JRA1.8) [Achieved]
  • Feature Deadline : 02/2011 [Achieved]
  • Addressing EMI technical objective Data Area 4 : All storage elements offering at least a prototype-level support for the "file://" access protocol. (M12)
  • Development Task Tracker : [ Details ]

Optional Features for EMI 1 (ready)

  • none

Feature Summary for EMI 1

http(s) WebDav GLUE 2.0
Limited
SRM/SSL GPFS
StoRM P - P - P

P = Production Quality
experimental = Pre-Production Quality

Optional Features towards EMI 2 (planned and/or in development)

High availability : StoRM will provide a cluster solution for high availability.

  • Status: In development
  • Description : 2011-02-22 Riccardo : Almost all components are configurable in cluster mode, except for the most important, the BackEnd. We plan to provides this feature in the next release (EMI-1.x, x>0).
  • Addressing EMI component improvements

Log analyzer : An administrative tool to interrogate StoRM log files.

  • Status: In development
  • Description : 2011-02-22 Ricardo : Will be not finalized for EMI 1
  • Addressing EMI component improvements

ARGUS blacklisting : StoRM will try to integrate the ARGUS blacklisting functionality.

  • Status: Developments stopped
  • Description : 2011-02-22 Riccardo : We stopped this developments, but idea and plan exists
  • Addressing EMI component improvements

Tracking

...
2011-02-22 Updated with information send from Riccardo on status of technical objectives and features (2011-02-18)
2011-02-24 Added TOC
Topic attachments
I Attachment History Action Size Date Who Comment
PNGpng 2011-02-22_Storm.PNG r1 manage 23.0 K 2011-02-22 - 19:43 MorrisRiedelExCern 2011-02-22 : Status STORM
Edit | Attach | Watch | Print version | History: r10 < r9 < r8 < r7 < r6 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r10 - 2011-02-24 - MorrisRiedelExCern
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    EMI All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback