Storage Management TEG: Questionnaire Level 1 - Jens Jensen

This twiki is to collect the input of Jens Jensen. Please answer the questions below. For more information, please refer to the Storage TEG main twiki.

Question 1

- In your view, what are the 3 main current issues in Storage Management (SM)?

In no particular order.

Interoperation - ensuring that things can talk to each other, because this tends to help problem solving. Mainly to avoid developing and supporting bespoke "solutions" for individual communities.

Performance and overhead and robustness. Possibly cheating by naming more than one thing, the infrastructure needs to perform well and be robust to be truly useful. Need to be able to identify the bottlenecks.

Roadmaps - ensuring sufficient consultation on storage, again to help ensure we don't paint ourselves into a corner by solving problems limited to individual communities.

Put here your answer

Question 2

- What is the greatest future challenge which would greatly impact the SM sector?

Addressing scalability - how to scale to bigger files, exabyte stores, more files.

The size of a data service is an issue: e.g. individual data repositories, departmental, institutional, national. Recommendations swing from one to the other. Possibly a mix has to be supported.

Put here your answer

Question 3

- What is your site/experiment/middleware currently working on in SM?

• Cloud services, and related security issues.
• Federated access to repositories.
• Data services on a European scale: services for ESFRI projects and similar, ie multi-community. Also specialised services for neutron and synchrotron sources (again on a European level). Data management for space science (including ESA).
• iRODS.
• Scaling to exabyte.
• Management of storage services: virtualisation, deployment, configuration, monitoring.
• Increasing the accuracy of accounting. Storage accounting records.

Put here your answer

Question 4

- What are the big developments that you would like to see from your site/experiment/storage system in the next 5 years?

• Use of NFS4, particularly for local (CE-to-SE) file access.
• Support roadmap - what is the strategic view?

Put here your answer

Question 5

- In your experience and area of competence, what are the (up to) 3 main successes in SM so far?

• WLCG is quite a success. Life sciences and climate science complain that they can't cope with the data volumes, yet HEP has "solved" these, to some extent.
• EMI middleware. It may not be perfect but it is sufficiently modular to allow mixing and matching components, and it has served us fairly well over the past years, and is being used by other projects.
• Building a community of people who deploy and support the infrastructure with experience and skill.

Put here your answer

Question 6

- In your experience and area of competence, what are the (up to) 3 main failures or things you would like to see changed in SM so far?

• Complexity of middleware - although the problem is complex and one should be wary of the simple solution solving only 95% of the problem, the middleware stack (both our own and the components it depends on) is now quite complex.
• Related to that, the lack of robustness - again a tremendously difficult problem, but there are bits of the data management stack that just do not fail over in a reasonable way, leading to the development of yet higher layer components to cope with failures.

Put here your answer

That's it!

Thanks! Feel free to edit again at any time, until the date of the kick-off meeting.

-- DanieleBonacorsi - November 2011