Savannah Statistics Documentation

Description


The Savannah Statistics are a set of scripts, built in PHP, which are used to extract statistics information about Patches, Bugs and other parameters useful to have a global view of the progress in the gLite release process.

Those scripts read information from a local dump of Savannah DB and they create a lot of statistics graphs.

The bugs/patches states, on which those scripts work, will be analysed in the relevant section of this documentation.

The purpose of this document is to provide an installation method of those scripts, and to provide information about the scripts-system so it will be possible to modify or to improve the script.

Server currently in Production *lxbra1902*

How to install / setup


The Savannah Statistics are PHP scripts that gather data information from a local mysql copy of the savannah DB, created in a computer host. The information gathered are used, from the Statistics scripts, to show graphs. The local mysql DB is updated, from a dump of the Savannah DB, each hour (the updating can be scheduled).

The procedure to configure properly a computer host to run those scripts is the following:

Step 1 *Packages Needed:*

yum install mysql mysql-server php cvs php-pear php-mysql php-gd rpm-build

Jpgraph Tool in production right now 1.26:

cd $HOME
wget http://hem.bredband.net/jpgraph/jpgraph-1.26.tar.gz
tar zxvf jpgraph-1.26.tar.gz -C /var/www/

Here the jpgraph repository updated at time of writing:

  • For php version 4: Here
  • For php version 5 (>=5.1.x): Here

Step 2 Download CVS and Set Folders

#cd $HOME

#export CVSROOT=:ext:<username>@glite.cvs.cern.ch:/cvs/jra1mw/

#export CVS_RSH=ssh

#mkdir /home/cvs/

#cd /home/cvs/

#cvs checkout org.glite.savannah.viewer

#mkdir /cronjob/

#cp org.glite.savannah.viewer/cron-utils/cert/* /cronjob/

#mkdir -p /storage/gLite/Savannah/web/Statistics/

#cp org.glite.savannah.viewer/web/Statistics/* /storage/gLite/Savannah/web/Statistics/

Step 3 PHP and HTTPD Configuration
Edit php.ini and change the following values:

  • max_execution_time = 360
  • max_input_time = 360
  • memory_limit = 200M

Edit httpd.conf and change the following values:

DocumentRoot to "/storage"

ServerName <your_server_hostname>

<Directory "/storage">
... Default Values are ok
</Directory>

Add this block after the first one..

<Directory "/storage/gLite/Savannah">
Options Indexes
AllowOverride None
Order allow,deny
Allow from all
</Directory>

Step 4 Setup MySQL DB
Create MySQL DB:
#service mysqld start

#mysqladmin -u root password "password"
#Setup a root password
#mysql -u root -p

#mysql>CREATE DATABASE savannah;

#mysql>GRANT ALL PRIVILEGES ON savannah.* TO 'savannah-writer'@'localhost' IDENTIFIED BY 'check the password for this in the file /cronjob/upload-savannah-db.sh';

#mysql>GRANT SELECT ON savannah.* TO 'savannah'@'localhost' IDENTIFIED BY 'check the password for this in the !WatchDog php file that is in the /cronjob folder';

#mysql>quit
Fill the DB manually for the first time by running:
#./cronjob/upload-savannah-db.sh
#Set the execute permission if needed

In this way you can test if everything is properly configured.

Step 5 Setup Cron Job
The cron jobs are used to schedule the start of a script.

The main cronjob configuration file is in: /etc/crontab

It's necessary setup the following script:

  • uplodad-savannah-db.sh #Used to update the local Savannah DB

The line to be added in the crontab file:

  • 01 * * * * root /cronjob/upload-savannah-db.sh

The script is scheduled the first minute of each hour. Remember to set the correct execute permission for the file.

Step 6 Install fonts for jpgraph
Install the tool Microsoft Fonts, needed by jpgraph:

Option 1: Fast way

  • Use yum or download the rpm from the web for msttcorefonts (and all its dependencies)

Option 2: Slow way

Option 3: Another way

  • Build the cabextract rpm:
  • rpmbuild -bb /usr/src/redhat/SPECS/cabextract.spec

  • If everything went well just do:
  • rpm -ivh /usr/src/redhat/RPMS/i386/cabextract-1.2-1.i386.rpm

  • Build the msttcorefonts rpm:
  • rpmbuild -bb msttcorefonts-2.0-1.spec

  • and:
  • rpm -ivh /usr/src/redhat/RPMS/noarch/msttcorefonts-2.0-1.noarch.rpm
  • mkdir -p /usr/X11R6/lib/X11/fonts/truetype/
  • cp /usr/share/fonts/msttcorefonts/* /usr/X11R6/lib/X11/fonts/truetype/

  • Test it: http://<your server>/gLite/Savannah/web/Statistics/statIndex.php

Info Start Stop DB and HTTPD
To start/stop MySQL daemon:

  • /etc/init.d/mysqld
  • service mysqld start #start the service
  • service mysqld stop #stop the service

To start/stop httpd daemon:

  • /etc/init.d/httpd
  • service httpd start # start the service
  • service httpd stop #stop the service

If you need other information, please contact: lorenzo_DOT_sbolgi[AT]cern_DOT_ch

This guide How to Setup a Patch Monitor contains the old installation process, for the statistics scripts of savannah.

Bugs/Patches States


The Savannah statistics scripts work using the following states:

Patch States
Value Description
100 None
113 With Provider
114 Ready for Integration
109 In configuration
111 Configured
102 Ready for certification
103 In certification
108 Certified
112 PPS pre-deployment
104 PPS pre-production
105 In production
107 Rejected
106 Obsolete
Bug States
Value Description
100 None
1 Fixed
2 Invalid
3 Wont fix
5 Remind
6 Unreproducible
7 Duplicate
121 Closed
114 Accepted
116 Ready for Review
117 Ready for Test
118 Ready for Integration
120 In progress
122 Integration Candidate
123 Integrated

Scripts Structure


The main HTML scripts page is:
  • statIndex.php
this contains all the link on production of the PHP scripts.

The header and the footer of this page (used also from a lot of other scripts) are:

  • statHeader.php
  • statFooter.php

All the scripts created in this structure follow the general templates:

  • templatePage.php
  • templateGraph.php

If your goal is to create a new script, follow the previous templates file. The first script creates an HTML page and load the statistics graph as an image from the second script. Those two file are full of comments and they are useful to understand how to build a graph in few minutes.

If your goal is to modify or improve an existing graph:

  • search how many and which scripts are involved in your future fix
  • try to understand how the scripts extract the data
  • apply your modification

Details


Some scripts file use a DEFINE for the directory where are stored the Microsoft Fonts:
  • DEFINE ("TTF_DIR","/usr/share/fonts/TrueType/" );
If you change this directory fix manually all the scripts involved in your modification.

Link


  • Savannah Statistics Web link: Here

-- LorenzoSbolgi - 08 Apr 2009

Edit | Attach | Watch | Print version | History: r7 < r6 < r5 < r4 < r3 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r7 - 2010-06-24 - unknown
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    EGEE All webs login

This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright & by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Ask a support question or Send feedback