Savannah Statistics Documentation

Description

---

The Savannah Statistics are a set of scripts, built in PHP, which are used to extract statistics information about Patches, Bugs and other parameters useful to have a global view of the progress in the gLite release process.

Those scripts read information from a local dump of Savannah DB and they create a lot of statistics graphs.

The bugs/patches states, on which those scripts work, will be analysed in the relevant section of this documentation.

The purpose of this document is to provide an installation method of those scripts, and to provide information about the scripts-system so it will be possible to modify or to improve the script.

Server currently in Production	*lxbra1902*

How to install / setup

---

The Savannah Statistics are PHP scripts that gather data information from a local mysql copy of the savannah DB, created in a computer host. The information gathered are used, from the Statistics scripts, to show graphs. The local mysql DB is updated, from a dump of the Savannah DB, each hour (the updating can be scheduled).

The procedure to configure properly a computer host to run those scripts is the following:

Step 1	*Packages Needed:*

• mysql
• mysql-server
• php
• cvs
• php-pear
• php-mysql
• php-gd
• rpm-build
• jpgraph
• Savannah cron-utils: Savannah cron-utils repository
• Savannah Statistics scripts: Savannah Stastistics Repository

yum install mysql mysql-server php cvs php-pear php-mysql php-gd rpm-build

Jpgraph Tool in production right now 1.26:

cd $HOME
wget http://hem.bredband.net/jpgraph/jpgraph-1.26.tar.gz
tar zxvf jpgraph-1.26.tar.gz -C /var/www/

Here the jpgraph repository updated at time of writing:

• For php version 4: Here
• For php version 5 (>=5.1.x): Here

Step 2	Download CVS and Set Folders

#cd $HOME

#export CVSROOT=:ext:<username>@glite.cvs.cern.ch:/cvs/jra1mw/

#export CVS_RSH=ssh

#mkdir /home/cvs/

#cd /home/cvs/

#cvs checkout org.glite.savannah.viewer

#mkdir /cronjob/

#cp org.glite.savannah.viewer/cron-utils/cert/* /cronjob/

#mkdir -p /storage/gLite/Savannah/web/Statistics/

#cp org.glite.savannah.viewer/web/Statistics/* /storage/gLite/Savannah/web/Statistics/

Step 3	PHP and HTTPD Configuration

Edit php.ini and change the following values:

• max_execution_time = 360
• max_input_time = 360
• memory_limit = 200M

Edit httpd.conf and change the following values:

DocumentRoot to "/storage"

ServerName <your_server_hostname>

<Directory "/storage">
... Default Values are ok
</Directory>

Add this block after the first one..

<Directory "/storage/gLite/Savannah">
Options Indexes
AllowOverride None
Order allow,deny
Allow from all
</Directory>

Step 4	Setup MySQL DB

Create MySQL DB:

#service mysqld start

#mysqladmin -u root password "password"

#Setup a root password

#mysql -u root -p

#mysql>CREATE DATABASE savannah;

#mysql>GRANT ALL PRIVILEGES ON savannah.* TO 'savannah-writer'@'localhost' IDENTIFIED BY 'check the password for this in the file /cronjob/upload-savannah-db.sh';

#mysql>GRANT SELECT ON savannah.* TO 'savannah'@'localhost' IDENTIFIED BY 'check the password for this in the !WatchDog php file that is in the /cronjob folder';

#mysql>quit

Fill the DB manually for the first time by running:

#./cronjob/upload-savannah-db.sh

#Set the execute permission if needed

In this way you can test if everything is properly configured.

Step 5	Setup Cron Job

The cron jobs are used to schedule the start of a script.

The main cronjob configuration file is in: /etc/crontab

It's necessary setup the following script:

• uplodad-savannah-db.sh #Used to update the local Savannah DB

The line to be added in the crontab file:

• 01 * * * * root /cronjob/upload-savannah-db.sh

The script is scheduled the first minute of each hour. Remember to set the correct execute permission for the file.

Step 6	Install fonts for jpgraph

Install the tool Microsoft Fonts, needed by jpgraph:

Option 1: Fast way

• Use yum or download the rpm from the web for msttcorefonts (and all its dependencies)

Option 2: Slow way

• Follow this guide -> http://corefonts.sourceforge.net/ If you had the same luck as me you need to compile cabextract from source.

Option 3: Another way

• wget http://corefonts.sourceforge.net/msttcorefonts-2.0-1.spec
• wget http://www.cabextract.org.uk/cabextract-1.2-1.src.rpm #RPM-based Linux distributions (Red Hat, SuSE, etc.)

• Build the cabextract rpm:
• rpmbuild -bb /usr/src/redhat/SPECS/cabextract.spec

• If everything went well just do:
• rpm -ivh /usr/src/redhat/RPMS/i386/cabextract-1.2-1.i386.rpm

• Build the msttcorefonts rpm:
• rpmbuild -bb msttcorefonts-2.0-1.spec

• and:
• rpm -ivh /usr/src/redhat/RPMS/noarch/msttcorefonts-2.0-1.noarch.rpm
• mkdir -p /usr/X11R6/lib/X11/fonts/truetype/
• cp /usr/share/fonts/msttcorefonts/* /usr/X11R6/lib/X11/fonts/truetype/

• Test it: http://<your server>/gLite/Savannah/web/Statistics/statIndex.php

Info	Start Stop DB and HTTPD

To start/stop MySQL daemon:

• /etc/init.d/mysqld
• service mysqld start #start the service
• service mysqld stop #stop the service

To start/stop httpd daemon:

• /etc/init.d/httpd
• service httpd start # start the service
• service httpd stop #stop the service

If you need other information, please contact: lorenzo_DOT_sbolgi[AT]cern_DOT_ch

This guide How to Setup a Patch Monitor contains the old installation process, for the statistics scripts of savannah.

Bugs/Patches States

---

The Savannah statistics scripts work using the following states:

Patch States Value Description 100 None 113 With Provider 114 Ready for Integration 109 In configuration 111 Configured 102 Ready for certification 103 In certification 108 Certified 112 PPS pre-deployment 104 PPS pre-production 105 In production 107 Rejected 106 Obsolete

Bug States Value Description 100 None 1 Fixed 2 Invalid 3 Wont fix 5 Remind 6 Unreproducible 7 Duplicate 121 Closed 114 Accepted 116 Ready for Review 117 Ready for Test 118 Ready for Integration 120 In progress 122 Integration Candidate 123 Integrated

Scripts Structure

---

The main HTML scripts page is:

• statIndex.php

this contains all the link on production of the PHP scripts.

The header and the footer of this page (used also from a lot of other scripts) are:

• statHeader.php
• statFooter.php

All the scripts created in this structure follow the general templates:

• templatePage.php
• templateGraph.php

If your goal is to create a new script, follow the previous templates file. The first script creates an HTML page and load the statistics graph as an image from the second script. Those two file are full of comments and they are useful to understand how to build a graph in few minutes.

If your goal is to modify or improve an existing graph:

• search how many and which scripts are involved in your future fix
• try to understand how the scripts extract the data
• apply your modification

Details

---

Some scripts file use a DEFINE for the directory where are stored the Microsoft Fonts:

• DEFINE ("TTF_DIR","/usr/share/fonts/TrueType/" );

If you change this directory fix manually all the scripts involved in your modification.

Link

---

• Savannah Statistics Web link: Here

• CVS link:
  • Savannah cron-utils: Here
  • Savannah Statistics Project: Here
  • Savannah Statistics Code Root in CVSweb: http://jra1mw.cvs.cern.ch/cgi-bin/jra1mw.cgi/org.glite.savannah.viewer

-- LorenzoSbolgi - 08 Apr 2009