RDF notification module for the CERN dashboard

Introduction

RDF frameworks for Python

Comparison of different frameworks for Python (slightly outdated)
http://www4.wiwiss.fu-berlin.de/bizer/toolkits/#python

Identified rdflib as the framework to use because of its SPARQL support
http://rdflib.net/

rdflib requires python2.5 to work the RPM-packages for slc4 i386 can be fetched from:
http://ddm-build.cern.ch/ddm/apt/RPMS.external/python25-2.5.2-1.slc4.i386.rpm
http://ddm-build.cern.ch/ddm/apt/RPMS.external/python2.5-2.5.2-1pydotorg.i386.rpm
http://ddm-build.cern.ch/ddm/apt/RPMS.external/python2.5-devel-2.5.2-1pydotorg.i386.rpm

python-curl for python2.5
to build python-curl from source curl-devel is required:
/afs/cern.ch/atlas/offline/external/GRID/EXTERNALS/curl-7.16.4/el4-32/curl-devel-7.16.4-1.i386.rpm
download and build the python-curl library (version 7.16.4 is compatible with the curl package):
http://pycurl.sourceforge.net/download/pycurl-7.16.4.tar.gz
python2.5 setup.py install

python-mysql for python2.5
to build python-mysql from source mysql-devel is required:
ftp://mirror.switch.ch/pool/3/mirror/centos/4.6/updates/i386/RPMS/mysql-devel-4.1.20-3.RHEL4.1.el4_6.1.i386.rpm
Download the MySQL-python source code
wget/curl http://downloads.sourceforge.net/mysql-python/MySQL-python-1.2.2.tar.gz?modtime=1172959928&big_mirror=0
Uncompress the tar.gz and open the source folder, compile and install python-mysql with the following commands:
python2.5 setup.py build && python2.5 setup.py install

pysqlite2 for python2.5
to build pysqlite2 from source sqlite-devel is required:
http://rpm.pbone.net/index.php3/stat/4/idpl/5497017/com/sqlite-devel-3.3.6-0.3.el4.i386.rpm.html
Download the pysqlite2 source code
wget/curl http://oss.itsystementwicklung.de/download/pysqlite/2.5/2.5.0/pysqlite-2.5.0.tar.gz
Uncompress the tar.gz and open the source folder, compile and install pysqlite2 with the following commands:
python2.5 setup.py build && python2.5 setup.py install

pyxml for python2.5
pyxml.sourceforge.net

Installation of rdflib

the installation is straightforward
wget http://rdflib.net/rdflib-2.4.0.tar.gz
tar xvvzf rdflib-2.4.0.tar.gz
/usr/bin/python2.5 setup.py install

Useful links for the development

Sample RDF schema
http://www.schemaweb.info/

Good introduction to RDF
http://www.rdfabout.com/intro/

A simple example on how to use rdflib
http://www.xml.com/pub/a/2003/02/12/rdflib.html

Example on SPARQL queries (that works with v2.4.0)
http://www.science.uva.nl/research/air/wiki/RDFlib

W3C RDF validator
http://www.w3.org/RDF/Validator/

RDFS

<!DOCTYPE rdf:RDF [<!ENTITY xsd "http://www.w3.org/2001/XMLSchema#">]>
<!DOCTYPE rdf:RDF [<!ENTITY xsd "http://www.w3.org/2001/XMLSchema#">]>
<rdf:RDF xmlns="http://dashboard.cern.ch/0/"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-0#"
    xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
>

<!-- 
RDF schema to be used for the CERN dashboard - RDF alert application.
markus.huber@cern.ch
-->


<!-- Classes are listed first: Site, Cloud -->
    <rdfs:Class rdf:about="http://dashboard.cern.ch/0#Cloud">
	<rdfs:label xml:lang="en">Cloud</rdfs:label>
        <dc:description xml:lang="en">A computing cloud within the CERN Atlas project</dc:description>
    </rdfs:Class>
    <rdfs:Class rdf:about="http://dashboard.cern.ch/0#Site">
	<rdfs:label xml:lang="en">Site</rdfs:label>
        <dc:description>A computing site within the CERN Atlas project</dc:description>
    </rdfs:Class>
<!-- Properties: ispartof, hasactivity -->
    <rdf:Property rdf:about="http://dashboard.cern.ch/0#ispartof">
	<rdfs:label xml:lang="en">is part of</rdfs:label>
        <dc:description xml:lang="en">Predicate to state a certain Site is part of a Cloud</dc:description>
        <rdfs:domain rdf:resource="http://dashboard.cern.ch/0#Site"/>
        <rdfs:range rdf:resource="http://dashboard.cern.ch/0#Cloud"/>
    </rdf:Property>
    <rdf:Property rdf:about="http://dashboard.cern.ch/0#hasactivity">
	<rdfs:label xml:lang="en">has activity</rdfs:label>
        <dc:description xml:lang="en">Predicate to state specific activies of a Service</dc:description>
        <rdfs:domain rdf:resource="http://dashboard.cern.ch/0#Site"/>
        <rdfs:range rdf:resource="http://dashboard.cern.ch/0#ddm"/>
        <rdfs:range rdf:resource="http://dashboard.cern.ch/0#production"/>
    </rdf:Property>
 <!-- Properties for DDM -->
  <!-- Blank Node for DDM values -->
    <rdf:Property rdf:about="http://dashboard.cern.ch/0#ddm">
	<rdfs:label xml:lang="en">DDM Blank node</rdfs:label>
        <dc:description xml:lang="en">Predicate to refer to specific values/properties of a DDM service</dc:description>
        <rdfs:domain rdf:resource="http://dashboard.cern.ch/0#ddm"/>
        <rdfs:range rdf:resource="http://dashboard.cern.ch/0#period"/>
        <rdfs:range rdf:resource="http://dashboard.cern.ch/0#transferrate"/>
        <rdfs:range rdf:resource="http://dashboard.cern.ch/0#transfererror"/>
        <rdfs:range rdf:resource="http://dashboard.cern.ch/0#filesdone"/>
        <rdfs:range rdf:resource="http://dashboard.cern.ch/0#datasetsdone"/>
    </rdf:Property>
  <!-- DDM Properties -->
    <rdf:Property rdf:about="http://dashboard.cern.ch/0#period">
	<rdfs:label xml:lang="en">period</rdfs:label>
        <dc:description xml:lang="en">Period the data refers to in seconds</dc:description>
	<rdfs:rdfs:subPropertyOf rdf:resource="http://dashboard.cern.ch/0#ddm"/>
        <rdfs:domain rdf:resource="http://dashboard.cern.ch/0#Site"/>
        <rdfs:range rdf:resource="&xsd;integer"/>
    </rdf:Property>
    <rdf:Property rdf:about="http://dashboard.cern.ch/0#transferrate">
	<rdfs:label xml:lang="en">transferrate</rdfs:label>
        <dc:description xml:lang="en">Transferrate of a certain DDM service</dc:description>
	<rdfs:rdfs:subPropertyOf rdf:resource="http://dashboard.cern.ch/0#ddm"/>
        <rdfs:domain rdf:resource="http://dashboard.cern.ch/0#Site"/>
        <rdfs:range rdf:resource="&xsd;integer"/>
    </rdf:Property>
    <rdf:Property rdf:about="http://dashboard.cern.ch/0#transfererror">
	<rdfs:label xml:lang="en">transfererror</rdfs:label>
        <dc:description xml:lang="en">Number of transfer-errors</dc:description>
	<rdfs:rdfs:subPropertyOf rdf:resource="http://dashboard.cern.ch/0#ddm"/>
        <rdfs:domain rdf:resource="http://dashboard.cern.ch/0#Site"/>
        <rdfs:range rdf:resource="&xsd;integer"/>
    </rdf:Property>
    <rdf:Property rdf:about="http://dashboard.cern.ch/0#filesdone">
	<rdfs:label xml:lang="en">filesdone</rdfs:label>
        <dc:description xml:lang="en">Number of completed files</dc:description>
	<rdfs:rdfs:subPropertyOf rdf:resource="http://dashboard.cern.ch/0#ddm"/>
        <rdfs:domain rdf:resource="http://dashboard.cern.ch/0#Site"/>
        <rdfs:range rdf:resource="&xsd;integer"/>
    </rdf:Property>
    <rdf:Property rdf:about="http://dashboard.cern.ch/0#datasetsdone">
	<rdfs:label xml:lang="en">datasetsdone</rdfs:label>
        <dc:description xml:lang="en">Number of completed datasets</dc:description>
	<rdfs:rdfs:subPropertyOf rdf:resource="http://dashboard.cern.ch/0#ddm"/>
        <rdfs:domain rdf:resource="http://dashboard.cern.ch/0#Site"/>
        <rdfs:range rdf:resource="&xsd;integer"/>
    </rdf:Property>
</rdf:RDF>

Sample RDF-XML (Site)

<?xml version="1.0" encoding="utf-8"?>
<rdf:RDF
  xmlns:atlas='http://dashboard.cern.ch/0/'
  xmlns:rdf='http://www.w3.org/1999/02/22-rdf-syntax-ns#'
>
  <atlas:Site rdf:about="http://dashboard.cern.ch/0/siteA">
    <atlas:hasactivity>
      <atlas:ddm>
        <atlas:filesdone rdf:datatype="http://www.w3.org/2001/XMLSchema#int">38</atlas:filesdone>
        <atlas:transfererror rdf:datatype="http://www.w3.org/2001/XMLSchema#int">1</atlas:transfererror>
        <atlas:period rdf:datatype="http://www.w3.org/2001/XMLSchema#int">3600</atlas:period>
        <atlas:transferrate rdf:datatype="http://www.w3.org/2001/XMLSchema#int">23</atlas:transferrate>
        <atlas:datasetsdone rdf:datatype="http://www.w3.org/2001/XMLSchema#int">21</atlas:datasetsdone>
      </atlas:ddm>
    </atlas:hasactivity>
  </atlas:Site>
</rdf:RDF>

Sample Knowledgebase N3 ("AGIS")

@prefix atlas: <http://dashboard.cern.ch/0/>.
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>.

 atlas:SiteA atlas:ispartof atlas:Cloud1,
         atlas:Cloud2.

 atlas:SiteB atlas:ispartof atlas:Cloud2.

 atlas:Cloud1 a atlas:Cloud.

 atlas:Cloud2 a atlas:Cloud.

RDFStore performance

-- MarkusHuber - last update: 19 Sept 2008