WLCG Transfers Dashboard Latency Monitoring

This page documents the development of latency monitoring, which is a component of WLCGTransfersDashboard.

Requirements

TBD

See also WLCGTransfersDashboard#Email_30_05_2012_from_Alessandro

Development

Environment:

Cycle:
  • Develop on dev SVN branch.
  • Deploy to development environment.
  • On delivery of a functional milestone:
    • review functionality with stakeholders
    • review code
    • merge to SVN trunk
    • deploy/test in integration
    • deploy/test in production

Milestones

  1. Application setup in the testbed : https://dashb-wlcg-transfers-dev-latency.cern.ch

Delivered

  1. Application presented to the ADC monitoring meeting, 22/08/2012
  2. First requirements reached. PRESENTED 10/07/2012
    • Link to the presentation: TransfersLatencyUpdate.pdf
    • Include a large set of chart breakdown (vo, server, channel, source and destination country, site)
    • Time bins can be customize (from 5 minutes to 1 hour) as well their number (from 1 to 10)
    • Include a prototype for channel filtering (working but subject to further discussion)
  3. First proposal presentation. PRESENTED 20/06/2012
  4. Create development branch of code in SVN. DELIVERED 06/06/2012
    • Create development branch of code in SVN.
    • Create skeleton view object for latency tab. * Deliverable: * Development environment section of twiki updated with SVN links.

Completed

  • DB and collector:
    • Idea of a single table containing only active transfers have been dropped (We want to keep history)
    • 3 new tables have been added:
      • t_transfers: snapshot of the active transfers
      • t_transfers_stats: 10 minutes aggregation data from t_transfers
      • t_transfers_stats_a: daily aggregation data from t_transfers
  • User Interface:
    • Latency, latency history and ranking plots tabs have been added
    • Filter by Channel
    • Detailled page with running transfers

In progress

  • Xbrowse plugin for plots and table

Future (draft)

  • User Interface:
    • Investigation on the plot wanted by CMS (channel occupancy, stream / throughput, SRM overhead)
  • Add new metrics: * major requests: statistics per channel/host * config: N files in parallel -> drilldown per file * change the number of streams by the number of files

Links

-- DavidTuckett - 08-Jun-2012
Topic attachments
I Attachment History Action Size Date Who Comment
PDFpdf ADC-latency-monitoring.pdf r1 manage 689.8 K 2012-08-23 - 13:57 AlexandreBeche  
PDFpdf LatencyMonitoring-August.pdf r1 manage 475.4 K 2012-08-23 - 13:59 AlexandreBeche  
PDFpdf TransfersLatency-FirstProposal.pdf r1 manage 788.2 K 2012-06-20 - 12:15 AlexandreBeche  
PDFpdf TransfersLatencyUpdate.pdf r1 manage 400.4 K 2012-07-11 - 10:38 AlexandreBeche  
Edit | Attach | Watch | Print version | History: r5 < r4 < r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r5 - 2012-08-23 - AlexandreBeche
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback