FTS Dashboard
Views and APIs
- Matrix
- Error samples:
- Transfers monitoring:
- History of transfers:
- Files status monitoring:
- Active transfers:
- Transfers ranking:
- Transfers map:
Infrustructure
- PROD
- machines : dashb-ai-578.cern.ch / dashb-ai-579.cern.ch
- alias : dashb-fts-transfers.cern.ch
- broker : dashb-mb.cern.ch
- broker topics : transfer.fts_monitoring_start, transfer.fts_monitoring_complete, transfer.fts_monitoring_state
- broker queues : Consumer.dashb-fts.
- broker auth : dboard certificate (/home/dboard/.globus/*)
- database : LCGR (DSN = lcg_dashboard_tfr)
- database account : lcg_dashboard_tfr<_r,_w>
- database password : Stored in tbag (dashboard_dao_fts_prod_owner/dashboard_dao_fts_prod_reader/dashboard_dao_fts_prod_writer)
- puppet hostgroup : dashboard/web_server/fts/production
- QA
- machines : dashb-ai-552.cern.ch
- alias : -
- broker : dashb-test-mb.cern.ch
- broker topics : transfer.fts_monitoring_start, transfer.fts_monitoring_complete, transfer.fts_monitoring_state
- broker queues : Consumer.dashb-fts.
- broker auth : dboard certificate (/home/dboard/.globus/*)
- database : INT6R
- database account : lcg_transfers_test<_r,_w>
- database password : Stored in tbag (dashboard_dao_fts_int_owner/dashboard_dao_fts_int_reader/dashboard_dao_fts_int_writer)
- puppet hostgroup : dashboard/web_server/fts/integration
- DEV (UI only/ES eval.)
- machines : dashb-ai-657.cern.ch/dashb-ai-613.cern.ch.
- alias : dashb-fts-transfers-dev.cern.ch
- database : INT6R
- database account : lcg_transfers_test<_r,_w>
- database password : Stored in tbag (dashboard_dao_fts_int_owner/dashboard_dao_fts_int_reader/dashboard_dao_fts_int_writer)
- puppet hostgroup : dashboard/web_server/fts/integration
-
- Foreman params. and modules manifests:
- dashboard_consumer:
- dashboard_dao:
- dashboard_web_load:
Support
Service Ops
- UI
- Collectors
- Services simplevisor (collectors)
- Consumer simplevisor (stompclt)
- Database access
- config.: /opt/dashboard/etc/dashboard-dao/dashboard-dao.cfg
Database
- RAW tables:
- t_transfer_start/t_transfer_complete/t_fts_state
- STATS tables:
- t_error_summaries
- View: Errors categorization
- t_file_stats_detailled
- t_tfrs_stats
- Views: Matrix, Transfer plots, Ranking plots
- t_transfers
Topology
- Lookups:
- hostname (of the storage) ---> GOCDB name: statically using the t_topology table
- GOCDB name ---> vo site name: using the cached file of the following IS url for each vo
- GOCDB name ---> country: https://wlcg-rebus.cern.ch/apps/topology/all/json
- t_topology table
- The SE in the following structure is the hostname of the storage as it is sent by FTS while SITE is the GOCDB name.
describe t_topology
Name Null Type
---- -------- -------------
SE NOT NULL VARCHAR2(256)
SITE VARCHAR2(256)
- Flow
- The hostname of the storage is sent by FTS as src_hostname/dst_hostname and stored in the RAW tables
- The computation jobs run over the RAW tables and store the src_hostname/dst_hostname as they are sent by FTS
- At each invocation of the FTS Dasbhoard APIs pointed out above:
- Lookups.1 is performed and then depending on the requested grouping:
- By country: + Lookups.3
- By site: + Lookups.2
- By host: no further lookup is needed
- Metadata:
- GOCDB name
- VO site name
- Country
Modules & RPMs
- dashboard-transfers-collector: It includes the config and the code of the FTS collector (https://gitlab.cern.ch/exp_dashboard/cosmic/tree/master/dashboard.transfers-collector
)
- dashboard-transfers: APIs, template of simplevisors config., FTS DAOs, FTS specific UI, MVC actions config. for FTS (https://gitlab.cern.ch/exp_dashboard/cosmic/tree/master/dashboard.transfers
)
- dashboard-transfers-topology: toplogy releated stuff - topology update cron, static topology files and code for the topology resolution (https://gitlab.cern.ch/exp_dashboard/cosmic/tree/master/dashboard.transfers-topology
)
- The other RPMs from the dashboard framework that are needed and not specific to FTS: dashboard-common, dashboard-xbrowse, dashboard-util-url, dashboard-dao-oracle, dashboard-service-config, dashboard-cli, dashboard-dao, dashboard-web)
--
HassenRiahi - 2016-02-08