NOTE: this twiki is for internal use only (atm) by Daniele and Wahid.

Storage Management TEG: Daniele's draft twiki on CMS answers

First of all, a general CMS statement on this process of interaction with the Storage Management TEG and the Data Management TEG (referred to as S-TEG and D-TEG in the following, respectively). We perceive the line between Storage Management and Data Management as somehow blurry, and there are overlaps, also in the questions, so we tried to be inclusive and consistent in a unique set of answers but we are also aware there might be items on which a specific TEG might want to ask us further: if so, we are available to provide deeper and further details as long as we will be asked by both TEGs. Said that, answers (A) to the questions (Q) are drafted below.

Q: " In your view, what are the 3 main current issues in Storage Management (SM)? "

From the CMS perspective, some of the main areas (open issues) we identified are:

  • Disk management in a hierarchical storage
    • i.e. be able to manage the disk in a way we can be sure data is on disk where CMS wants it to be

  • Access to data from disk with increasing volumes and fewer spindles
    • as we get larger and larger disks we have fewer spindles, more apps and more cores trying to read, the IO of the application and the sw performances get even hotter topics

  • WAN access to storage
    • i.e. at the moment, refer to the xrootd work by Brian et al, and also to the Federated Storage ws in Lyon (end November 2011)

  • Reliability
    • an area that's easy to overlook, but it's crucial. In some sense the previous item (i.e. WAN access) stands as a sort of acknowledgement we need a fallback when the storage was not working, but it is worth to list it as a separate item. Think of it as 1) "we wants less errors from the storage systems" as well as 2) "if any, errors should be reported from the storage systems to the CMS application layer that intercepts them in a not-misleading, complete, clear manner".

  • Storage monitoring (shared with the Operations TEG)
    • We need a better monitoring of what's actually on disk and tape.

Also, but read the caveat in the comments, we would add:

  • intra-VO authorization (shared with the Security TEG)
    • i.e. one needs to have the right privileges in writing to different storage areas, especially copying data from other sites. E.g. jobs running at the T1s with the /t1access VOMS role trying to write at T2 storage where the priority user role is required to write to the group area. Actually, this can be identified as a problem that we definitely have, but not a particular priority. At least it's not urgent, in the sense that once we open the T1s to a more intense use by Analysis we will have this problem as a priority. Currently, while we agree it would be nice to have a better solution, we also recognize that we have an operational solution in which we solve this issues by granting the needed privileges to users who actually need it.

Q: " What is the greatest future challenge which would greatly impact the SM sector? "

Large data volumes and need for access.

Q: " What is your site/experiment/middleware currently working on in SM? "

Disk and Tape Management at the Tier-1s.

-- DanieleBonacorsi - Nov-2011

Edit | Attach | Watch | Print version | History: r4 < r3 < r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r1 - 2011-12-01 - unknown
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback