StorageSpaceAccounting < LCG

LCG Web>AccountingTaskForce>StorageSpaceAccounting (2021-03-17, HorstSeverini) (raw view)

%TOC%
---+++ Storage Space Accounting introduction

The goal of the WLCG Storage Space accounting project is to enable the high level overview of the total and available space provided by the WLCG infrastructure. What is required is a possibility to account total used and total free space for all distinct storage areas (equivalent to the space quotas in SRM) available to the experiments.

Work on storage space accounting is going in several directions:
   * Enable description of the storage topology and all storage areas which have to be accounted separately
   * Enable possibility to query storage space accounting information for all kind of storage implementation
   * Enable the data flow of the accounting information from the information source to the data repository and setting up user interface and APIs for data retrieval

The storage space accounting work is done in close collaboration with the WLCG Data Management Steering group. The WLCG Data Management Steering group will coordinate with the storage providers in order to enable storage resource reporting. The WLCG accounting is just one of the use cases where storage resource reporting is required.
The latest version of the storage resource reporting proposal can be found [[https://docs.google.com/document/d/1yzCvKpxsbcQC5K9MyvXc-vBF1HGPBk4vhjw3MEXoXf8/edit?usp=sharing][here]].

WSSA service is available [[https://monit-grafana.cern.ch/d/mHqFLAbik/wlcg-storage-space-accounting?orgId=20][here]].

More details:
   * [[https://indico.cern.ch/event/696513/contributions/2856798/attachments/1589030/2513865/WLCG_Storage_Space_Accounting_Status_Update.pdf][Jan 25, 2018 status update]]
   * [[https://indico.cern.ch/event/739874/contributions/3278159/attachments/1779732/2894840/2019.01.16_WLCG_Storage_Space_Accounting_Now_In_Production.pdf][Jan 16, 2019 GDB presentation]]

Validation of data provided by WSSA
   * [[%ATTACHURL%/log_grafana_disktape_ddm_010119_310119.xlsx][ January 2019 comparison WSSA vs ATLAS DDM Dashboard ]]
   * [[%ATTACHURL%/log_grafana_egi_010119_310119(1).xlsx][ January 2019 comparison WSSA vs EGI accounting prototype]]

---+++ Description of the storage topology and all storage areas which have to be accounted separately

Storage systems should provide total used and total free space for all distinct space quotas available to the experiments. Therefore, description of all those distinct space quotas or storage areas  which have to be accounted separately is requred. In difference with the accounting information, this is pretty static information. The new WLCG topology and configuration system CRIC is foreseen as a place where this information will be stored and exposed via UI/API.
The final format of the storage topology description and how this information will be provided by the sites hosting the storage service is still under discussion and will be followed up in the collaboration with the Data Management Steering Group. More details can be found in the [[https://docs.google.com/document/d/1yzCvKpxsbcQC5K9MyvXc-vBF1HGPBk4vhjw3MEXoXf8/edit?usp=sharing][document]].



*The latest version of the json format specification can be found [[%ATTACHURL%/SRR.v6.pdf][here]]*

The previous version in the
[[https://docs.google.com/document/d/1sA_Y7D1OD4GTn1WLPyY8AJFun_Lcqp2ZcvQcsfOwFHc/edit][googledoc]]

---+++ Query storage space accounting information for all kind of storage implementation

The minimum requirement is to have two numbers free and used space for every space quota/storage area. The accessibility of these numbers depends upon the storage system type, the protocol, and configuration decisions relating spaces quotas with the namespace. In reality, there are three possibilities, gridFTP, HTTP and xrootd. A storage system should implement resource reporting in at least one protocol. The relevant numbers will be made available through the gfal2 interface.

Alternative solution for providing accounting information is to provide it in the json file, similar to the one which describes storage topology , but extended with the accounting data.

More details can be found in the latest version of the [[https://docs.google.com/document/d/1yzCvKpxsbcQC5K9MyvXc-vBF1HGPBk4vhjw3MEXoXf8/edit][document]].

---+++ Implementation of the data flow for storage space accounting information

The central WLCG Storage Space Accounting (WSSA) service  every 30min/1hour does the following:
   * Queries the topology source, currently experiment-specific sources are used. Cric will be used in the future
   * Queries distributed storage instances in order to retrieve accounting info per every space quota/storage area
   * An alternative way would be to retrieve both topology and accounting data from the json file described below
This information is then published to the central repository ( ES backend) using standard MONIT infrastructure. Grafana is used for the UI implementation.

Implementation of this data flow has been already prototypes using available information sources. For ATLAS and LHCb - SRM queries, for ALICE - ALICE ldapsearch queries. In their turn ATLAS and LHCb are using SRM for their internal storage space accounting, while ALICE uses xrootd queries. THere are couple of known issues with xrootd space queries.
For some storage configuration they can return too high numbers, double counting, which is in most cases fixed on the ALICE side . That is why it was decided to use ALISE ldap instead of raw xrootd queries. Another issye is with dcache storage for which currently xrootd space queries do not work. The problem is being followed up by the dCache developers.

---+++ SRR implementation by the storage middleware providers

SRR implementation is being followed up with all storage middleware providers. Mailing list for people involved in this activity is srr-implementation@cern.ch

---+++++ SRR implementation status

| *Storage type* | *Implementation status* | *Deployment status* | *Storage contact* |
| DPM | [[DPM/DpmSetupManualInstallation#Publishing_space_usage][Implemented starting from version 1.10.3]]. Requires DPM re-configuration with DOME enabled | [[DPMupgrade][DPM upgrade Task Force]] | Fabrizio Furano & Oliver Keeble |
| dCache | First prototype is [[https://www.dcache.org/manuals/Book-5.0/srr.shtml][ready]] for testing and validation. Path for storage shares is missing, requires a bit more work | [[DCacheUpgrade][dCache upgrade Task Force]] | Paul Millar |
| EOS | Enabled at CERN. Some issues to be fixed. Not yet ready for deployment to other sites | CERN only | Andreas Peters |
| STORM | First prototype at CNAF is ready for testing and validation, [[https://issues.infn.it/jira/browse/STOR-1139][STOR-1139]], [[http://italiangrid.github.io/storm/documentation/how-to/how-to-publish-json-report/][documentation]] | - | Andrea Ceccanti |
| xRootd | First prototype provided beginning of November | - | Wei Yang |

---++++++ Examples of implementation
   * [[%ATTACHURL%/praguelcg2.json][ DPM example for Prague ]]
   * [[%ATTACHURL%/CERN-PROD.json][ EOS example for CERN ]]
   * [[%ATTACHURL%/StoRM.json][ STORM example for CNAF ]]
   * [[%ATTACHURL%/dCache.json][ dCache example for Desy ]]
   * [[%ATTACHURL%/xrootd.srr.slac.json][Xrootd example/template for SLAC]]
   * [[%ATTACHURL%/OU-SRR-example.txt][Xrootd example/template for OU]]


---++++++ Meetings with storage middleware providers

   * [[https://indico.cern.ch/event/759910/][Meeting on storage topology description 24.09.2018]]
   * [[https://indico.cern.ch/event/770062/][Meeting SRR specification 06.11.2018]]

-- Main.JuliaAndreeva - 2018-10-04

   * [[%ATTACHURL%/xrootd.srr.slac.json][xrootd.srr.slac.json]]: Xrootd SRR json at SLAC</verbatim>

   * [[%ATTACHURL%/OU-SRR-example.txt][OU-SRR-example.txt]]: OU-SRR-example.txt</verbatim>

   * [[%ATTACHURL%/srr-json.py.txt][srr-json.py.txt]]: srr-json.py.txt</verbatim>

   * [[%ATTACHURL%/quota_file_srr][quota_file_srr]]: quota_file_srr</verbatim>

Attachments

Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
json	CERN-PROD.json	r1	manage	31.9 K	2018-10-04 - 16:25	JuliaAndreeva
txt	OU-SRR-example.txt	r2 r1	manage	1.2 K	2021-03-17 - 23:28	HorstSeverini
pdf	SRR.v6.pdf	r1	manage	67.0 K	2020-05-20 - 11:50	JuliaAndreeva
json	StoRM.json	r1	manage	7.0 K	2018-11-06 - 17:03	JuliaAndreeva
json	dCache.json	r1	manage	6.7 K	2018-11-06 - 17:02	JuliaAndreeva
xlsx	log_grafana_disktape_ddm_010119_310119.xlsx	r1	manage	8.7 K	2019-03-13 - 11:37	JuliaAndreeva	Comparison of January data ATLAS DDM vs WSSA
xlsx	log_grafana_egi_010119_310119(1).xlsx	r1	manage	8.2 K	2019-03-13 - 11:35	JuliaAndreeva	WSSA vs EGI
json	praguelcg2.json	r1	manage	7.4 K	2018-10-04 - 16:25	JuliaAndreeva
ext	quota_file_srr	r1	manage	0.2 K	2021-03-17 - 22:58	HorstSeverini
txt	srr-json.py.txt	r1	manage	11.8 K	2021-03-17 - 22:58	HorstSeverini
json	xrootd.srr.slac.json	r1	manage	3.2 K	2019-12-02 - 22:46	WeiYang	Xrootd SRR json at SLAC

Topic revision: r17 - 2021-03-17 - HorstSeverini

LCG Wikis

LCG Service
Coordination

LCG Grid
Deployment

LCG
Apps Area

Public webs

Welcome Guest

- Cern Search
- TWiki Search
- Google Search
LCG All webs

Copyright &© 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback