WLCG Transfers Dashboard Latency Monitoring
This page documents the development of latency monitoring, which is a component of
WLCGTransfersDashboard.
Requirements
TBD
See also
WLCGTransfersDashboard#Email_30_05_2012_from_Alessandro
Development
Environment:
- Development branch of code in SVN
- Development database account.
- Use developer db account initially.
- Development VM.
- Use developer machine initially.
- NOTES:
Cycle:
- Develop on dev SVN branch.
- Deploy to development environment.
- On delivery of a functional milestone:
- review functionality with stakeholders
- review code
- merge to SVN trunk
- deploy/test in integration
- deploy/test in production
Milestones
- Application setup in the testbed : https://dashb-wlcg-transfers-dev-latency.cern.ch
Delivered
- Application presented to the ADC monitoring meeting, 22/08/2012
- First requirements reached. PRESENTED 10/07/2012
- Link to the presentation: TransfersLatencyUpdate.pdf
- Include a large set of chart breakdown (vo, server, channel, source and destination country, site)
- Time bins can be customize (from 5 minutes to 1 hour) as well their number (from 1 to 10)
- Include a prototype for channel filtering (working but subject to further discussion)
- First proposal presentation. PRESENTED 20/06/2012
- Create development branch of code in SVN. DELIVERED 06/06/2012
- Create development branch of code in SVN.
- Create skeleton view object for latency tab. * Deliverable: * Development environment section of twiki updated with SVN links.
Completed
- DB and collector:
- Idea of a single table containing only active transfers have been dropped (We want to keep history)
- 3 new tables have been added:
- t_transfers: snapshot of the active transfers
- t_transfers_stats: 10 minutes aggregation data from t_transfers
- t_transfers_stats_a: daily aggregation data from t_transfers
- User Interface:
- Latency, latency history and ranking plots tabs have been added
- Filter by Channel
- Detailled page with running transfers
In progress
- Xbrowse plugin for plots and table
Future (draft)
- User Interface:
- Investigation on the plot wanted by CMS (channel occupancy, stream / throughput, SRM overhead)
- Add new metrics: * major requests: statistics per channel/host * config: N files in parallel -> drilldown per file * change the number of streams by the number of files
Links
--
DavidTuckett - 08-Jun-2012