WLCG Transfers Dashboard FTS Queue Status Monitoring

QUEUE STATUS MONITORING HAS NOW BEEN INCORPORATED INTO THE TRUNK CODE AND IS DEPLOYED TO THE INTEGRATION SERVER. SEE SupportWLCGTransferDashboard#Integration_Server. THE REMAINDER OF THIS PAGE IS NOW OBSOLETE.

This page documents the development of FTS queue status monitoring, which is a component of WLCGTransfersDashboard.

Requirements

Key features:

  • Consume FTS queue status messages from Dashboard MSG brokers (see WLCGTransferMonitoring#FTS_message_structure).
  • Insert FTS queue status messages into Dashboard Oracle database.
  • Expose current queue status data via web API (JSON).
  • Visualise current queue status within Dashboard UI (see http://dashb-wlcg-transfers.cern.ch/ui/).
  • Expose historical queue status data via web API (JSON).
  • Visualise queue status history within Dashboard UI.

Possible extensions:

  • Correlation between queue status and transfer statistics.

Requirements will be further detailed in consultation with the VOs, see WLCGTransfersDashboard#VO_requests.

Development

Estimate:

  • The development is predicted to take 2-3 FTE months.

Environment:

Cycle:

  • Develop on dev SVN branch.
  • Deploy to dev VM and DB account.
  • On delivery of a functional milestone:
    • review functionality with stakeholders
    • review code
    • merge to SVN trunk
    • deploy/test in integration
    • deploy/test in production

Milestones

Delivered

  1. Create development environment (CERN). DELIVERED 06/06/2012
    • Create development branch of code in SVN.
    • Request development database account.
    • Set-up VM with WLCG Transfers Dashboard (from SVN branch).
    • Deliverable:
      • Access to SVN, DB and VM provided to Dubna team

In progress

Future (draft)

  1. Consume FTS queue status messages to DB table.
    • The consumer should re-use the existing collector code.
    • The table should by partitioned by created_time.
    • Deliverable:
      • Demonstration that messages are moved from the queue to DB.
  2. Propose visualisation of current queue status
    • The visualisation should be designed to integrate into the current UI.
    • It may include an expandable table, plots, etc.
    • Deliverable:
      • Document / presentation outlining the proposed solution.
  3. Expose current queue status via web API
    • This should be done within the Dashboard framework
    • Deliverable:
      • Demonstration of web API returning JSON.
      • API documented on twiki.
  4. Visualise current queue status
    • This is the implementation of the above proposal.
    • It should use the above web API.
    • Deliverable:
      • Demonstration of web UI integrated into existing UI.
  5. Propose visualisation of queue status history
    • The visualisation should be designed to integrate into the current UI.
    • It may include an expandable table, plots, etc.
    • Deliverable:
      • Document / presentation outlining the proposed solution.
  6. Expose queue status history via web API
    • This should be done within the Dashboard framework
    • Deliverable:
      • Demonstration of web API returning JSON.
      • API documented on twiki.
  7. Visualise queue status history
    • This is the implementation of the above proposal.
    • It should use the above web API.
    • Deliverable:
      • Demonstration of web UI integrated into existing UI.

Links

-- DavidTuckett - 25-Apr-2012

Edit | Attach | Watch | Print version | History: r6 < r5 < r4 < r3 < r2 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r6 - 2013-04-10 - DavidTuckett
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback