FTS Dashboard

Views and APIs

• Matrix
  • URL: http://dashb-fts-transfers.cern.ch/ui/#p.grouping=server
  • API: http://dashb-fts-transfers.cern.ch/dashboard/request.py/transfer-matrix
• Error samples:
  • URL: http://dashb-fts-transfers.cern.ch/ui/#m.content=%28efficiency,errors%29&p.grouping=server
  • API http://dashb-fts-transfers.cern.ch/dashboard/request.py/error-samples
• Transfers monitoring:
  • URL: http://dashb-fts-transfers.cern.ch/ui/#p.grouping=server&tab=transfer_plots
  • API http://dashb-fts-transfers.cern.ch/dashboard/request.py/transfer-bins
• History of transfers:
  • URL: http://dashb-fts-transfers.cern.ch/ui/#p.grouping=server&tab=history_plots
  • API: http://dashb-fts-transfers.cern.ch/dashboard/request.py/transfer-history
• Files status monitoring:
  • URL: http://dashb-fts-transfers.cern.ch/ui/#p.grouping=server&tab=fts_jobs_plots
  • API: http://dashb-fts-transfers.cern.ch/dashboard/request.py/fts-job-bins
• Active transfers:
  • URL: http://dashb-fts-transfers.cern.ch/ui/#p.grouping=server&tab=latency_plots
  • API: http://dashb-fts-transfers.cern.ch/dashboard/request.py/active-transfers
• Transfers ranking:
  • URL: http://dashb-fts-transfers.cern.ch/ui/#p.grouping=server&tab=ranking_plots
  • API: http://dashb-fts-transfers.cern.ch/dashboard/request.py/transfer-ranking
• Transfers map:
  • URL: http://dashb-fts-transfers.cern.ch/ui/#p.grouping=server&tab=map
  • API: http://dashb-fts-transfers.cern.ch/dashboard/request.py/transfer-map

Infrustructure

• PROD
  • machines : dashb-ai-578.cern.ch / dashb-ai-579.cern.ch
  • alias : dashb-fts-transfers.cern.ch
  • broker : dashb-mb.cern.ch
  • broker topics : transfer.fts_monitoring_start, transfer.fts_monitoring_complete, transfer.fts_monitoring_state
  • broker queues : Consumer.dashb-fts.
  • broker auth : dboard certificate (/home/dboard/.globus/*)
  • database : LCGR (DSN = lcg_dashboard_tfr)
  • database account : lcg_dashboard_tfr<_r,_w>
  • database password : Stored in tbag (dashboard_dao_fts_prod_owner/dashboard_dao_fts_prod_reader/dashboard_dao_fts_prod_writer)
  • puppet hostgroup : dashboard/web_server/fts/production

• QA
  • machines : dashb-ai-552.cern.ch
  • alias : -
  • broker : dashb-test-mb.cern.ch
  • broker topics : transfer.fts_monitoring_start, transfer.fts_monitoring_complete, transfer.fts_monitoring_state
  • broker queues : Consumer.dashb-fts.
  • broker auth : dboard certificate (/home/dboard/.globus/*)
  • database : INT6R
  • database account : lcg_transfers_test<_r,_w>
  • database password : Stored in tbag (dashboard_dao_fts_int_owner/dashboard_dao_fts_int_reader/dashboard_dao_fts_int_writer)
  • puppet hostgroup : dashboard/web_server/fts/integration

• DEV (UI only/ES eval.)
  • machines : dashb-ai-657.cern.ch/dashb-ai-613.cern.ch.
  • alias : dashb-fts-transfers-dev.cern.ch
  • database : INT6R
  • database account : lcg_transfers_test<_r,_w>
  • database password : Stored in tbag (dashboard_dao_fts_int_owner/dashboard_dao_fts_int_reader/dashboard_dao_fts_int_writer)
  • puppet hostgroup : dashboard/web_server/fts/integration

• Config.
  • Puppet manifest:
    • https://gitlab.cern.ch/ai/it-puppet-hostgroup-dashboard/blob/master/code/manifests/init.pp
    • https://gitlab.cern.ch/ai/it-puppet-hostgroup-dashboard/blob/qa/code/manifests/init.pp

• • Foreman params. and modules manifests:
    • dashboard_consumer:
      • https://gitlab.cern.ch/ai/it-puppet-hostgroup-dashboard/blob/master/code/manifests/modules/create_simplevisor_consumer_config.pp
      • https://gitlab.cern.ch/ai/it-puppet-hostgroup-dashboard/blob/qa/code/manifests/modules/create_simplevisor_consumer_config.pp
    • dashboard_dao:
      • https://gitlab.cern.ch/ai/it-puppet-hostgroup-dashboard/blob/master/code/manifests/modules/database.pp
      • https://gitlab.cern.ch/ai/it-puppet-hostgroup-dashboard/blob/qa/code/manifests/modules/database.pp
    • dashboard_web_load:
      • https://gitlab.cern.ch/ai/it-puppet-hostgroup-dashboard/blob/master/code/manifests/modules/load.pp
      • https://gitlab.cern.ch/ai/it-puppet-hostgroup-dashboard/blob/qa/code/manifests/modules/load.pp

Support

• FTS Dashboard issues: dashboard-support@cernNOSPAMPLEASE.ch, SNOW FE: WLCG Data Management Monitoring.
• FTS issues (service admins and devels.): fts3-steering@cernNOSPAMPLEASE.ch, SNOW FE: FTS Development.

Service Ops

• UI
  • start/stop:

    [root@dashb-ai-552 ~]# service httpd restart

  • log: /opt/dashboard/var/log/dashboard
• Collectors
  • start/stop:

    [dboard@dashb-ai-552 ~]$ /opt/dashboard/bin/dashb-agent-restart fts.collectors

  • log: /opt/dashboard/var/log/dashb-fts.collectors.log
• Services simplevisor (collectors)
  • start/stop:

    [dboard@dashb-ai-552 ~]$simplevisor --conf /opt/dashboard/etc/dashboard-simplevisor/service-simplevisor.cfg --daemon stop
    [dboard@dashb-ai-552 ~]$simplevisor --conf /opt/dashboard/etc/dashboard-simplevisor/service-simplevisor.cfg --daemon start 

  • log: /opt/dashboard/var/log/services-simplevisor.log
  • config.: /opt/dashboard/etc/dashboard-simplevisor/services-simplevisor.cfg
• Consumer simplevisor (stompclt)
  • start/stop:

    [dboard@dashb-ai-552 ~]$simplevisor --conf /opt/dashboard/etc/dashboard-simplevisor/consumer-simplevisor.cfg --daemon stop
    [dboard@dashb-ai-552 ~]$simplevisor --conf /opt/dashboard/etc/dashboard-simplevisor/consumer-simplevisor.cfg --daemon start 

  • log: /opt/dashboard/var/log/consumer-simplevisor.log
  • config.: /opt/dashboard/etc/dashboard-simplevisor/consumer-simplevisor.cfg
• Database access
  • config.: /opt/dashboard/etc/dashboard-dao/dashboard-dao.cfg

Database

• RAW tables:
  • t_transfer_start/t_transfer_complete/t_fts_state
• STATS tables:
  • t_error_summaries
    • View: Errors categorization
  • t_file_stats_detailled
    • View: FTS 3 jobs plots
  • t_tfrs_stats
    • Views: Matrix, Transfer plots, Ranking plots
  • t_transfers
    • View: Latency plots