Easysamportal of the certification testbed
Installation
Installing the publisher
The rpm installed is =same-query-x.x.x.
Creating the database schema:
export CVSROOT=:pserver:anonymous@glite.cvs.cern.ch:/cvs/glite
cvs co same
cd same/server/db/scripts
sqlplus username/password@database < create_same_standalone.sql
Database information
The Oracle database is on CERN central database server.
username: lcg_cert_sam
password: ask Gergo
database: INT3R
Installing the web portal
Install easysam rpm from:
/afs/cern.ch/project/gd/www/gergo/rpms/lcg-sam-server-easyportal-1.0.0-4.noarch.rpm
The database
You can use Valentin's script (
same/server/db/cron/bdii2oracle.py
) to populate the database using the
BDII information.
In order to work properly first you have to fill out the
REGION
,
SITE
and
VO
tables. Connect to the database and launch the following commands:
insert into region (regionid,regionname,REGIONDESCRIPTION) values(1,'CERTTESTBED','Certification testbed');
insert into site values(1,1,1,1,'SL3CERTTB1A','1A CERT SITE','Certified','Production','N');
insert into site values(2,1,1,1,'SL3CERTTB1B','1B CERT SITE','Certified','Production','N');
insert into site values(3,1,1,1,'SL3CERTTB2','2 CERT SITE','Certified','Production','N');
insert into vo values(3,'dteam','N');
Due to a bug, please create and fill the following table, as well:
create table regions(REGIONID NUMBER(38),NAME VARCHAR2(50), REGIONDESCRIPTION VARCHAR2(255));
insert into regions values(1,'CERTTESTBED','Certification testbed');
In order the script to work you have to do the followings:
- Install
python-ldap
package:
wget wget http://belnet.dl.sourceforge.net/sourceforge/python-ldap/python-ldap-2.0.7.tar.gz
tar -xzvf python-ldap-2.0.7.tar.gz
cd python-ldap-2.0.7
Edit the setup.cfg
file, in order to resolve dependency problems, Change
requires = python openldap2-client openssl cyrus-sasl2
to
requires = python openldap-clients openssl cyrus-sasl
Build and install the rpm:
python setup.py bdist_rpm
rpm -i ./build/bdist.linux-i686/rpm/RPMS/i386/python-ldap-2.0.7-1.i386.rpm
- Modify the script at some point. Since we are not using GOC DB, FCR, and other thing, some of the tables should not be filled, so one should modify the
bdii2oracle.py
script. Here is the ldiff output of the original and the modified script:
< SELECT NLS_LOWER(sitename), siteid FROM sites WHERE isdeleted='N'"""
---
> SELECT NLS_LOWER(sitename), siteid FROM site WHERE isdeleted='N'"""
622,623c622,623
< set_action("Getting site BDIIs")
< services += get_site_bdiis(cursor)
---
> # set_action("Getting site BDIIs")
> # services += get_site_bdiis(cursor)
636,638c636,638
< for res_name in ["CE", "SE"]:
< set_action("Updating %s resources" % res_name)
< update_resources(cursor, services, res_name.lower())
---
> # for res_name in ["CE", "SE"]:
> # set_action("Updating %s resources" % res_name)
> # update_resources(cursor, services, res_name.lower())
646,648c646,648
< set_action("Updating list of core nodes")
< run_query(cursor, clear_core_flag_sql, {})
< run_query(cursor, set_core_nodes_sql, {})
---
> # set_action("Updating list of core nodes")
> # run_query(cursor, clear_core_flag_sql, {})
> # run_query(cursor, set_core_nodes_sql, {})
Run the script:
./bdii2oracle.py -H ldap://lxb2017:2170 -d LCG_CTB/samectb1@devdb10 -vv
In case of error try with more
-vvvvv
. If the script doesn't find a node for some reason, you can still insert a node manualy into the DB:
insert into node values(1,1,12,'N','lxb2018.cern.ch');
Further tables has to be set up:
create table NodeMaintenance(nodeid NUMBER(38), inMaintenance CHAR(1));
insert into NodeMaintenance select nodeid, 'N' from node;
Setting all node as not beeing under maintenance, ensures that during test submission they will pass the filter applied on the list of available nodes.
Configuring the client part
Check out sam from CVS (see above). Edit
./same/client/etc/same.conf
to be something like:
# Default configuration for SAME
[DEFAULT]
# Settings for locations
workdir=%(home)s/.same
logdir=%(same_home)s/var/log
resdir=%(same_home)s/var/results
secresdir=%(same_home)s/var/results-secure
webdir=%(same_home)s/web
cachedir=%(same_home)s/var/cache
# Logging levels:
# CRITICAL, ERROR, WARNING, INFO, DEBUG, NOTSET
# Logging level for the log file
loglevel=INFO
# Logging level for console messages
verbosity=CRITICAL
[sensors]
common_attrs="sitename nodename inmaintenance"
common_filter="type=Production status=Certified"
CE_filter="serviceabbr=CE ismonitored=Y"
gCE_filter="serviceabbr=gCE"
FTS_filter="serviceabbr=FTS"
FTS_attrs="sitename nodename inmaintenance tier"
SE_filter="serviceabbr=SE"
SRM_filter="serviceabbr=SRM"
LFC_filter="serviceabbr=LFC voname=dteam"
host-cert_filter=""
[statuscode]
ok=10
info=20
notice=30
warning=40
error=50
critical=60
maintenance=100
[submission]
vo=dteam
test_timeout=240
[scheduler]
max_processes=10
default_timeout=300
shell=/bin/sh
[webservices]
publisher_wsdl=http://lxb0714.cern.ch:8080/gridview/services/WebArchiver?wsdl
query_wsdl=http://lxb0714.cern.ch:8080/same-query/services/Database?wsdl
# edit the line below if you want to publish to old-style SFT webservice
# sft_publisher_url=http://lxb0714.cern.ch:8083/sft/publishTuple
If you add a new test/sensor do not forget to add a variable called
<mytest_name>_filter=""
to this config file !
Testing the installation
- Here are some sqlplus command which could be used to test the installation:
- List all object:
select object_name from user_objects;
- List test result:
select * from testdata;
- List/check the services:
select * from serviceinstances;
Test submission
Submit a simple test:
cd same/client/bin
./same-exec SE
the normal output should be something like:
operation: submit
Sensor: SE
Node attrs: ['sitename', 'nodename', 'inmaintenance']
Node filter: {'status': ['Certified'], 'serviceabbr': ['SE'], 'type': ['Production']}
Sensor args:
Matching nodes: 2
Executing prepare script
----------------------------------------
prepare-FTS script started.
got nodes: 2 nodes.map
publishing test definitions... DONE
----------------------------------------
Prepare script finished successfully
Executing check script for all nodes
----------------------------------------
lxb1921.cern.ch: started
lxb1737.cern.ch: started
lxb1921.cern.ch: publishing
lxb1737.cern.ch: publishing
lxb1921.cern.ch: finished
lxb1737.cern.ch: finished
----------------------------------------
Executing check scripts finished
Check the result on the portal.
Collection of tests, CVS
- The sensors are be collected in the gLite CVS
Problems
Here are the current problem we are facing:
- When the SAM portal displays the result it displays only the nodes which are supposed to run the type of service which was tested by the sensor, even if the sensor was running on other hosts, as well. To be changed, corrected in the future.
Links, tutorials, How-Tos
- See the attachment
- SAM Metric calculation < removed invalid link >
- SAM publishing method < removed invalid link >
- SAM submission framework < removed invalid link >
- ... and of course you can always go to Valentin, Piotr, Judit.....
- Description of SAM tests SAM Tests Twiki page
-- Main.gdebrecz - 10 Jan 2007