FTS Dashboard

Views and APIs

Infrustructure

  • PROD
    • machines : dashb-ai-578.cern.ch / dashb-ai-579.cern.ch
    • alias : dashb-fts-transfers.cern.ch
    • broker : dashb-mb.cern.ch
    • broker topics : transfer.fts_monitoring_start, transfer.fts_monitoring_complete, transfer.fts_monitoring_state
    • broker queues : Consumer.dashb-fts.
    • broker auth : dboard certificate (/home/dboard/.globus/*)
    • database : LCGR (DSN = lcg_dashboard_tfr)
    • database account : lcg_dashboard_tfr<_r,_w>
    • database password : Stored in tbag (dashboard_dao_fts_prod_owner/dashboard_dao_fts_prod_reader/dashboard_dao_fts_prod_writer)
    • puppet hostgroup : dashboard/web_server/fts/production

  • QA
    • machines : dashb-ai-552.cern.ch
    • alias : -
    • broker : dashb-test-mb.cern.ch
    • broker topics : transfer.fts_monitoring_start, transfer.fts_monitoring_complete, transfer.fts_monitoring_state
    • broker queues : Consumer.dashb-fts.
    • broker auth : dboard certificate (/home/dboard/.globus/*)
    • database : INT6R
    • database account : lcg_transfers_test<_r,_w>
    • database password : Stored in tbag (dashboard_dao_fts_int_owner/dashboard_dao_fts_int_reader/dashboard_dao_fts_int_writer)
    • puppet hostgroup : dashboard/web_server/fts/integration

  • DEV (UI only/ES eval.)
    • machines : dashb-ai-657.cern.ch/dashb-ai-613.cern.ch.
    • alias : dashb-fts-transfers-dev.cern.ch
    • database : INT6R
    • database account : lcg_transfers_test<_r,_w>
    • database password : Stored in tbag (dashboard_dao_fts_int_owner/dashboard_dao_fts_int_reader/dashboard_dao_fts_int_writer)
    • puppet hostgroup : dashboard/web_server/fts/integration

Support

Service Ops

  • UI
    • start/stop:
      [root@dashb-ai-552 ~]# service httpd restart
    • log: /opt/dashboard/var/log/dashboard
  • Collectors
    • start/stop:
      [dboard@dashb-ai-552 ~]$ /opt/dashboard/bin/dashb-agent-restart fts.collectors
    • log: /opt/dashboard/var/log/dashb-fts.collectors.log
  • Services simplevisor (collectors)
    • start/stop:
      [dboard@dashb-ai-552 ~]$simplevisor --conf /opt/dashboard/etc/dashboard-simplevisor/service-simplevisor.cfg --daemon stop
      [dboard@dashb-ai-552 ~]$simplevisor --conf /opt/dashboard/etc/dashboard-simplevisor/service-simplevisor.cfg --daemon start 
    • log: /opt/dashboard/var/log/services-simplevisor.log
    • config.: /opt/dashboard/etc/dashboard-simplevisor/services-simplevisor.cfg
  • Consumer simplevisor (stompclt)
    • start/stop:
      [dboard@dashb-ai-552 ~]$simplevisor --conf /opt/dashboard/etc/dashboard-simplevisor/consumer-simplevisor.cfg --daemon stop
      [dboard@dashb-ai-552 ~]$simplevisor --conf /opt/dashboard/etc/dashboard-simplevisor/consumer-simplevisor.cfg --daemon start 
    • log: /opt/dashboard/var/log/consumer-simplevisor.log
    • config.: /opt/dashboard/etc/dashboard-simplevisor/consumer-simplevisor.cfg
  • Database access
    • config.: /opt/dashboard/etc/dashboard-dao/dashboard-dao.cfg

Database

  • RAW tables:
    • t_transfer_start/t_transfer_complete/t_fts_state
  • STATS tables:
    • t_error_summaries
      • View: Errors categorization
    • t_file_stats_detailled
      • View: FTS 3 jobs plots
    • t_tfrs_stats
      • Views: Matrix, Transfer plots, Ranking plots
    • t_transfers
      • View: Latency plots

Topology

  • t_topology table
    • The SE in the following structure is the hostname of the storage as it is sent by FTS while SITE is the GOCDB name.
describe t_topology
Name Null     Type          
---- -------- ------------- 
SE   NOT NULL VARCHAR2(256) 
SITE          VARCHAR2(256) 

  • Flow
    1. The hostname of the storage is sent by FTS as src_hostname/dst_hostname and stored in the RAW tables
    2. The computation jobs run over the RAW tables and store the src_hostname/dst_hostname as they are sent by FTS
    3. At each invocation of the FTS Dasbhoard APIs pointed out above:
      • Lookups.1 is performed and then depending on the requested grouping:
        1. By country: + Lookups.3
        2. By site: + Lookups.2
        3. By host: no further lookup is needed

  • Metadata:
    1. GOCDB name
    2. VO site name
    3. Country

Modules & RPMs

-- HassenRiahi - 2016-02-08

Edit | Attach | Watch | Print version | History: r10 < r9 < r8 < r7 < r6 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r10 - 2016-05-20 - HassenRiahi
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback