SLS Tape Metrics Review - OBSOLETE (newer version here)

Introduction

This page contains documentation related to the WLCG SLS tape questionnaire and review presented at the WCLG Management Board meeting in November 2010. The goal is to review existing SLS tape metrics and to ensure they are suited for reporting how well and how efficient a site is performing; that they are providing appropriate information from the user (experiment) perspective for improving operations and to allow meaningful comparisons between tape MSS sites.

SLS tape metrics questionnaire

Link Description
MB-112010 Initial presentation to the WCLG MB, November 2010
t0-t1-tape-monitoring-questionnaire.txt Questionnaire sent out to WLCG T0/T1 sites and WCLG VO's
MB-022011 WLCG presentation on questionnaire results, February 2011
MB-112011 WLCG MB presentation on final specification and implementation status, November 2011

Questionnaire Results

From the 17 sites/VO's contacted, 16 have returned a filled-in questionnaire. There is a handful of conclusions which can be derived from the received answers:

  1. All T0/T1 sites do have their own internal tape monitoring in place which is used for daily operations. The SLS tape metrics are rather used for "exporting" availability / efficiency metrics, than for internal use; for several sites, it is not clear who the target audience is supposed to be.
  2. Most sites look periodically at the SLS plots; sites tend to only look at their own SLS graphs - this is mostly done to check that the SLS export is working correctly (see previous point).
  3. From the 3 VO's responding to the questionnaire, the interest in the SLS tape metrics is currently moderate, with one VO not looking into them and the other two only occasionally.
  4. There are differences between the metrics considered relevant by the VO's and by the sites:
    • From the core metrics, VO's are more interested in availability, average file size and overall data transfer rate. Sites pay attention to tape repeat mounting and volume per mount which are not considered to be important by the VO's.
    • From other existing metrics, VO's are particularly interested in total volume read/written and tape access (queueing) time.
    • Looking at what new metrics could be added, the VO wish-list is the file queue length, the disk cache failure rate, and tapes containing inactive (not recently accessed) data. Drive transfer efficiency and drives used for housekeeping are only seen as interesting by the sites.

Initial proposal

From the questionnaire, it seems that the SLS tape views can be made more attractive by clearly setting the target audience to the VO's. Metrics could be readjusted to fit what the VO's want to see. This would allow to significantly simplify the current metric set, dropping many metrics which are not considered relevant and internal to the sites, and to concentrate on the half a dozen metrics the VO's are really interested in following.

The core metrics which can be proposed to be retained are :-

  • Availability (definition to be reviewed)
  • Average file size
  • Average data transfer rate
  • Average access queueing time
  • Fraction of inactive data
  • Total data stored

The proposed metric set is not tape-specific and could be used as well for non-tape based archival storage (e.g. cloud-based storage). Frequency, and exact specifications of each metric can be worked on once the metric set has been agreed on.

Excel summary sheet

Link Description
questionnaire-evaluation.xlsx summary of questionnaire answers

Per-site / VO questionnaire responses

  • VO's

Link VO From
ALICE.eml ALICE Latchezar Betev
(none) ATLAS Kors Bos
CMS.eml CMS Ian Fisk
LHCb.eml LHCb Roberto Santinelli

  • T0/T1 sites
Link Site From
ASGC.eml ASGC Jhen-Wei Huang
BNL.eml BNL Michael Ernst
CC-IN2P3.eml CC-IN2P3 Pierre-Emmanuel Brinette
CERN.eml CERN Vladimir Bahyl
CNAF.eml CNAF Luca dell'Agnello
KIT.eml KIT Jos van Wezel
FNAL.eml FNAL Jon Bakken
LHCb.eml LHCb Roberto Santinelli
NDGF.eml NDGF Mattias Wadenstein
NL-T1.eml NL-T1 Mark van de Sanden
PIC.eml PIC Gonzalo Merino
RAL.eml RAL Matthew Viljoen
TRIUMF.eml TRIUMF Simon Liu

Proposal specification

A specification for the above proposal can be found here: TapeMetricsSpec

Proposal implementation status

See TapeMetricsImplementation for the current implementation status for each T0/T1 site.

More information

Link Description
SLS-Tape T0/T1 tape metrics as reported in SLS
MssEfficiency Old definition of core tape metrics set

-- GermanCancio

Topic attachments
I Attachment History Action Size Date Who Comment
Microsoft Outlook e-mail fileeml ALICE.eml r1 manage 8.7 K 2011-02-01 - 17:39 GermanCancio  
Microsoft Outlook e-mail fileeml ASGC.eml r1 manage 39.1 K 2011-02-01 - 17:39 GermanCancio  
Microsoft Outlook e-mail fileeml BNL.eml r1 manage 65.3 K 2011-02-01 - 17:40 GermanCancio  
Microsoft Outlook e-mail fileeml CC-IN2P3.eml r1 manage 11.7 K 2011-02-01 - 17:40 GermanCancio  
Microsoft Outlook e-mail fileeml CERN.eml r1 manage 5.1 K 2011-02-01 - 17:41 GermanCancio  
Microsoft Outlook e-mail fileeml CMS.eml r1 manage 41.8 K 2011-02-01 - 17:41 GermanCancio  
Microsoft Outlook e-mail fileeml CNAF.eml r1 manage 15.8 K 2011-02-01 - 17:41 GermanCancio  
Microsoft Outlook e-mail fileeml FNAL.eml r1 manage 59.2 K 2011-02-18 - 19:43 GermanCancio  
Microsoft Outlook e-mail fileeml KIT.eml r1 manage 15.8 K 2011-02-01 - 17:42 GermanCancio  
Microsoft Outlook e-mail fileeml LHCb.eml r1 manage 33.7 K 2011-02-01 - 17:42 GermanCancio  
Microsoft Outlook e-mail fileeml NDGF.eml r1 manage 8.5 K 2011-02-01 - 17:43 GermanCancio  
Microsoft Outlook e-mail fileeml NL-T1.eml r1 manage 62.1 K 2011-02-01 - 17:43 GermanCancio  
Microsoft Outlook e-mail fileeml PIC.eml r1 manage 9.2 K 2011-02-01 - 17:43 GermanCancio  
Microsoft Outlook e-mail fileeml RAL.eml r1 manage 54.4 K 2011-02-01 - 17:44 GermanCancio  
Microsoft Outlook e-mail fileeml TRIUMF.eml r1 manage 9.8 K 2011-02-01 - 17:44 GermanCancio  
Unknown file formatxlsx questionnaire-evaluation.xlsx r4 r3 r2 r1 manage 36.3 K 2011-02-18 - 20:07 GermanCancio  
Texttxt t0-t1-tape-monitoring-questionnaire.txt r1 manage 4.4 K 2011-02-01 - 18:33 GermanCancio  
Edit | Attach | Watch | Print version | History: r18 < r17 < r16 < r15 < r14 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r18 - 2018-01-19 - VladimirBahyl
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    HEPTape All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback