Quattor installation of Gridview

Gridview Related Links:

  1. Gridview Twiki Page
  2. Gridview Service Dash Board Notes
  3. Gridview Deployment at CERN
  4. Gridview Admin Guide
  5. Quattor Installation of Gridview
  6. Gridview Software Release Status
  7. Installation and Configuration of Gridview Publisher
  8. Installation and Configuration of Gridview Web Service Clients
  9. Using Gridview XML Interface
  10. URL for Excel Report generation
  11. Downloads
  12. New Gridview Interface
  13. Gridview Monthly Availabilities and Reports

Contents

Gridview can be installed and configured on production systems using Quattor. For this the following are available:

  1. Set of quattor templates for gridview software
  2. Gridview NCM component for configuring gridview
  3. Sindes script for retrieving and delivering sensitive information to gridview machines

Quattor templates

The list of quattor templates for gridview are:
  • pro_declaration_component_gridview.tpl : Declarations for Gridview specific CDB data structures
  • pro_software_component_gridview.tpl : Template containing configurations, common to all gridview instances. This template alone should be included in a machine's template after activating the required gridview modules. This template will in turn include other required templates.
  • pro_software_component_gridview_frontend.tpl : Template configuring Gridview frontend instances.
  • pro_software_component_gridview_summarizer.tpl : Template configuring Gridview summarizer instances.
  • pro_software_component_gridview_synchronizer.tpl : Template configuring Gridview synchronizer instances.
  • pro_software_component_gridview_archiver.tpl : Template to be configuring Gridview archiver instances.
  • pro_component_httpd_defaults_gridview.tpl : Apache specific settings for gridview frontend
  • pro_component_tomcat_defaults_gridview.tpl : Tomcat specific settings for gridview archivers.
  • pro_component_php_defaults_gridview.tpl: PHP settings for gridview.

Individual templates and their usage

pro_software_component_gridview

This template is the master template to be included in the gridview machine's template. It contains

  • Addition of common software packages such as Oracle instantclient and logrotate.
  • Configuration of logrotate service for logs in /var/log/gridview directory
  • Configuration of Sindes service for retrieving secure information
  • Conditional inclusion of other gridview templates based on which module is to be installed in the system

pro_software_component_gridview_frontend

pro_software_component_gridview_frontend.tpl.jpg

This template should be included in those machines which are to be configured as Gridview frontends. It contains

  • Addition of necessary packages (httpd and php) and their configurations
  • Configuration of logrotate service for httpd logs
  • Gridview frontend specific configurations

Paths to be set in the machine template to activate gridview frontend:

"/software/components/gridview/frontend/active" = true;

Pre dependencies

  • spma
  • sindes
  • php
  • httpd
Post dependencies
  • altlogrotate

Files modified/created:

  • /etc/httpd/conf/httpd.conf
  • /etc/httpd/conf.d/ncm-ssl.conf
  • /etc/httpd/conf.d/ncm-gridview.conf
  • /etc/php.ini
  • /etc/logrotate.d/httpd.ncm-altlogrotate
  • /opt/gridview/etc/gridview-common.conf

pro_software_component_gridview_summarizer

pro_software_component_gridview_summarizer.tpl.jpg

This template should be included in the template of the machines which are to be configured as a gridview summarizer. The template contains

  • Inclusion of necessary packages (php and cron)
  • Configuration of PHP
  • Addition of specific CRON jobs depending on which of the different summarizers are to be activated on this machine.

Paths to be set in the machine template to activate individual gridview summarizers:

"/software/components/gridview/summarizer/sum_dt/active" = true;         # For activating data transfer summarizer
"/software/components/gridview/summarizer/sum_js/active" = true;         # For activating job summarizer
"/software/components/gridview/summarizer/sum_sa/active" = true;        # For activating service availability summarizer
One or more of the three Gridview summarizers can be activated on a summarizer machine. It should be noted that the activation statements should be entered before including the template.

Pre-dependencies

  • spma
  • sindes
  • php
Post-dependencies
  • cron
  • altlogrotate

Files modified/created:

  • /etc/php.ini
  • /etc/cron.d/ncm-gridview-summarizer*
  • /etc/logrotate.d/gridview.ncm-altlogrotate

pro_software_component_gridview_synchronizer

pro_software_component_gridview_synchronizer.tpl.jpg

This template should be included in the template of the machine which are to be configured as gridview synchronizers. The template contains

  • Inclusion of necessary packages (php and cron)
  • Configuration of PHP
  • Addition of specific CRON jobs depending on which of the different summarizers are to be activated on this machine.
  • Synchonizer specific configurations

Paths to be set in the machine template to activate gridview synchronizers:

"/software/components/gridview/synchronizer/gocdb/active" = true;         # For activating GOCDB synchronizer
"/software/components/gridview/synchronizer/cicdb/active" = true;         # For activating CICDB synchronizer
One or more of the two Gridview synchronizers can be activated on a synchronizers machine. It should be noted that the activation statements should be entered before including the template.

Pre dependencies

  • spma
  • sindes
  • php
Post dependencies
  • cron
  • altlogrotate

Files modified/created:

  • /etc/php.ini
  • /etc/cron.d/ncm-gridview-synchronizer*
  • /etc/logrotate.d/gridview.ncm-altlogrotate
  • /opt/gridview/etc/gridview-common.conf

pro_software_component_gridview_archiver

pro_software_component_gridview_archiver.tpl.jpg

This template should be included in the template of those machines on which Gridview archivers are to be configured. The template contains

  • inclusion of necessary packages (java, tomcat, rgmaclient) and their configurations
  • Configuration of grid_service_certificates component to copy certificates to /opt/gridview/certificates
  • Archiver specific configurations
  • Enabling rgma-archiver services through init.d subsystem if rgma archiver is enabled
  • Enabling tomcat service if web service archiver is enabled

Paths to be set in the machine template to activate gridview archivers:

"/software/components/gridview/archiver/rgma/arch_dt/active" = true;    # For activating rgma data transfer archiver
"/software/components/gridview/archiver/rgma/arch_js/active" = true;    # For activating rgma job archiver
"/software/components/gridview/archiver/wsarch/arch_dt/active" = true; # For activating web service data transfer archiver
"/software/components/gridview/archiver/wsarch/arch_js/active" = true; # For activating web service job archiver

One or more of the different Gridview archivers can be activated on a archiver machine. It should be noted that the activation statements should be entered before including the template.

Pre-dependencies

  • spma
  • sindes
  • rgmaclient # For R-GMA archivers
  • grid_service_certificates # For R-GMA archivers
  • tomcat # For web service archivers
Post dependencies
  • altlogrotate
  • chkconfig

Files modified/created:

  • /etc/tomcat5/tomcat5.conf
  • /etc/tomcat5/server.xml
  • /etc/tomcat5/tomcat-users.xml
  • /opt/gridview/gridview-common.conf
  • /opt/gridview/rgma-authentication.conf
  • /opt/gridview/rgmaarch.conf
  • /opt/gridview/wsarch.conf
  • /etc/logrotate.d/gridview.ncm-altlogrotate

Secure Information Delivery through Sindes and the Sindes Gridview script

The gridview application needs to be configured with secure information which cannot be embedded inside rpms, CVS or quattor templates. This information includes:
  • Gridview Oracle DB usernames and passwords
  • GOCDB Oracle DB usernames and passwords
  • CIC Oracle DB usernames and passwords
This information is kept in the CERN Sindes service and delivered to gridview machines. The secure information file in the 'gridview' item in SINDES is copied into /opt/gridview/etc/sconfig file by Sindes and the SINDES-script-gridview script. This file is further used by Gridview NCM Component to generate the application's configuration files.

Contents of the SINDES gridview component

File

A file that specifies DB connection parameters for the Gridview DB and the external GOCDB and CIC DBs. The syntax of this file is as below:
@COMMON_DB_READER_HOSTSPEC@=
@COMMON_DB_READER_USERNAME@=
@COMMON_DB_READER_PASSWORD@=
@COMMON_DB_READER_DATABASE@=
@COMMON_DB_READER_OPTIONS@=
@COMMON_DB_READER_PORT@=
@COMMON_DB_READER_PROTOCOL@=

@COMMON_DB_WRITER_HOSTSPEC@=
@COMMON_DB_WRITER_USERNAME@=
@COMMON_DB_WRITER_PASSWORD@=
@COMMON_DB_WRITER_DATABASE@=
@COMMON_DB_WRITER_OPTIONS@=
@COMMON_DB_WRITER_PORT@=
@COMMON_DB_WRITER_PROTOCOL@=

@SYNCHRONIZER_GOCDB_DB_HOSTSPEC@=
@SYNCHRONIZER_GOCDB_DB_USERNAME@=
@SYNCHRONIZER_GOCDB_DB_PASSWORD@=
@SYNCHRONIZER_GOCDB_DB_DATABASE@=
@SYNCHRONIZER_GOCDB_DB_OPTIONS@=
@SYCHRONIZER_GOCDB_DB_PORT@=
@SYNCHRONIZER_GOCDB_DB_PROTOCOL@=

@SYNCHRONIZER_CICDB_DB_HOSTSPEC@=
@SYNCHRONIZER_CICDB_DB_USERNAME@=
@SYNCHRONIZER_CICDB_DB_PASSWORD@=
@SYNCHRONIZER_CICDB_DB_DATABASE@=
@SYNCHRONIZER_CICDB_DB_OPTIONS@=
@SYCHRONIZER_CICDB_DB_PORT@=
@SYNCHRONIZER_CICDB_DB_PROTOCOL@=

Script

The SINDES-script-gridview script copies the above file to /opt/gridview/etc/sconfig

Gridview NCM Component

The Gridview NCM component goes about the task of configuring the Gridview application on the machine. Depending on which gridview module is active on the machine, the component performs different tasks.

The component has pre-dependencies on

   * spma
   * sindes
   * httpd (on frontend system)
   * php  (on frontend, summarizer and synchronizer)
   * tomcat (on archiver)
   * rgma (on archiver)
   * grid_service_certificates (on archiver)
The component has post dependencies on
   * cron (on summarizer and synchronizer)
   * altlogrotate
   * chkconfig (on archiver)

The component configures different modules of gridview as follows:

Common configuration

The gridview-common module will be installed on all gridview machines. The Gridview NCM component creates the following file.

/opt/gridview/etc/gridview-common.conf

Archiver configuration

The NCM Component will create following configuration files for the archiver module
  • /opt/gridview/etc/rgma-authentication.conf
  • /opt/gridview/etc/rgmaarch.conf
  • /opt/gridview/etc/wsarch.conf

The Gridview NCM Component should be run anytime a configuration path related to Gridview changes.

Quattorized Deployment of Gridview:

The gridview application is deployed through quattor on four machines that are part of the gridview cluster. The machines and the modules running on them are:

Machine name Gridview Module installed
gridview101 Frontend
gridview102 Frontend
gridview103 Summarizer, Synchronizer
gridview104 Archiver

Lemon Monitoring and Alarms

The following existing standard alarms should be enabled on Gridview machines

Machine Gridview module Sensor Metric Metric_ID Exception Exception_ID
gridview101, gridview102 gridview-frontend Linux httpd 4019 httpd_wrong 30040
gridview103 gridview-summarizer, gridview-synchronizer Linux crond 43 crond_wrong 30021
gridview104 gridview-archiver Linux tomcat 40 tomcat_wrong 30055

Here are two New metrics and exceptions for gridview104 only:

Machine name Sensor Metric Class Metric_ID Exception Exception_ID
gridview104 Linux gridview-archiver-rgma-dt system.numberOfprocesses 24 gridview-archiver-rgma-dt_wrong 30136
gridview104 Linux gridview-archiver-rgma-js system.numberOfProcesses 25 gridview-archiver-rgma-js_wrong 30137
Edit | Attach | Watch | Print version | History: r11 < r10 < r9 < r8 < r7 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r11 - 2008-01-02 - RajeshKalmady
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback