ETICS Submission Webservice

The ETICS Submission Webservice

The ETICS Submission Webservice [hereafter ESW] is an architecture developed to enable ETICS to submit its tests and builds to different grid based middleware infrastructures. The logic of the submission management is hidden inside the architecture so that the consumer (the ETICS configuration web service) has only to submit it's build or test and wait for the response email.

Architecture

The ESW provides a simple web service interface by which the ETICS Configuration Web Service can perform the tree basic operation concerning the submission management: the submission of a build or test of a configuration, the query of the current status of a submission and the deletion of a submission. The interface is designed so that details of the actual execution of the job are not exposed.

The ESW is intended to be a framework on which to plug in existing submission management middleware. Once a plug in implementation is provided for a particular submitter ETICS can be able to use that Submitter for it's submissions. Different Submitter implementations are not mutually exclusive for a single ESW installation. Instead it is possible to run the ESW with more than one Submitter available, letting to choose a different one for each submission request. This feature opens the ESW to any priority - load balancing - affinity model implementation.The architecture of the ESW has been designed to be keep a clear division between the basic grid job operations so that the logic of each middleware can be easily merged with in the framework. The user interface and plugin API have been designed to reflect this choice.

The ETICS Submission Webservice architecture is composed of several service that resides either in the main execution thread of the web servlet or in dedicated threads with periodical execution. In the first class of services there are a Unique Id Generator service, an Id Mapping service and a Status Mapper service, in the latter a Job Status Fetcher service and a Result Uploader service.

Interface

Need a description of the interface, and link to internal documents and ETICS packages containing the WSDL.

Unique Id Generator service

The Unique Id Generator service provides to the framework a way to obtain a unique id (in the form af a UUID) that can be considered to be unique for each invocation also between different runs of the ESW. Those ids are used by the ESW to uniquely identify each submission request that it's receives giving them an internaly generated that hides the specific way in wich each submitter identifies it's job. In this way the user can threat each submission without worry about the submitter that has thaken it in charge. He has simply to rely on the UUID that the ESW has returned in the submission invocation. This so called Global Id in case of a submission request on more than one platform is generated only once for all the job submissions so that the user can rely on only a single identifier whereas this identifier internally is multiplexed on as much job submission as mutch how many platforms have been passed.

Id Mapping service

The mapping beetween the Global Id and the job id generated by the submitters is managed by the Id Mapping service. This service provides any facility to get informations on submission statuses and everything is required to perform application crash recovery. The informations managed by the Id Mapping service are mantained on a persistent database to witch the ESW is interfaced by the Hibernate framework. The Id Mappind is the bridge that ties the ESW and it's Submitter plugins. Every time that a request invlolving a submitted job arrives the ESW queryes the Id Mapping service to obtain the Submitter Id that corresponds to the givend Global Id and use it to identify the job on the proper Submitter implementation.

Status Fetcher service

The Status Fetcher service is responsible to retrieve the output of each submission once the submitted job has completed it's execution. For this purpose the Status Fetcher periodically queries the status of each job that has not yet completed it's execution. This status query is performed by interacting with the Submitter Implementation on witch the job has been submitted. After the invocation succedes the Status Fetcher has to be able to understand if the status string returned by a Submitter Implementation has to be considered representing a complete status, a running status, a failure status and so on. In order to accomplish to this task the Status Fetcher has top be able to understand the meaning of each status string of each Submitter Implementation. This is done by the Status Mapper service.

Status Mapper service

Status Mapper service translates the Submitter specific status strings to a common internal rappresentation of the statuses. Thanks to his service all the requests for the execution status of a job submission can be processed in a common way without requesting any effort to manage the difference between different Submitters, symply once the status string is available the ESW ask to the Status Mapper service to translate it in it's format.

Result Uploader service

The Result Uploader service is responsible to upload the packages and the reports genereted by the build/test to the ETICS repository and to send the submission result email to the user that has submitted the build/test. The last two service executes the more time consuming operations in the execution flow of a job in the ESW because they involves the transfer of the submission resources on the internet. To avoid that this became a bottlenek for the architecture they are run in separate threads that cooperates each other and with the ESW by means of simple service-like operations.

Plugins

Available Plugins

How to plug-in another submission engine

How to plug in the ESWs? - Instruction on how to implement a new submitter and plugs the implementation in the ESWs architecture

Installation Instruction

The Etics Submission Webservice is distributed with one or more submitter implementations. In order to install the ESW you have to follow the common set of instruction concerning the Webservice architecture and proceed with the specific instruction of the submitter/s included in the distribution.

Submission Webservice Installation Instruction

For the Etics Sumbission Webservice installation you have to follow the following steps:

  • Install java JDK version >= 1.5.0_12

  • Install the etics repository client and configure it to use your personal certificate in P12 format by editing the configuration file INSTALLATION_DIRECTORY/etc/eticsRepositoryClientJavaConfiguration.xml and setting properly the parameter remote.keyStore and remote.keyStorePassword. (the etics repository client must be installed in your ETICS_HOME)

  • Install mysql
  • Create a database for the Etics Sumbission Webservice

  • Copy the component's etc folder content in $ETICS_HOME/etc folder
  • Copy the component's share/webapps folder content in $ETICS_HOME/share/webapps folder

The ESM is distributed as a Web Apllication and needs to be run inside Tomcat. For this you need to Install tomcat version 5.0.28.
Because we need SSL support for the ESW web application you need to own a server certificate for the machine where the service will be deployed. To configure Tomcat for SSL support you can refere to the proper sectio of the tomcat Installation Guide at http://tomcat.apache.org/tomcat-5.5-doc/ssl-howto.html

  • Edit the configuration file of the ESW submitter.properties and specify the following information:
    the location where the ESW will put the generated temporary files
    the location where the ESW will put the output of the submissions
    the amount of disk space that can be used by the Webservice for temporary and output files
    the ETICS Server host to be used
    the ETICS Repository host to be used
    the host that where you have installed mysql
    the name of the mysql created database
    the user owner of the database to be used to access it
    the password of this user (none if no password is required)
    the SMTP server to be used to send emails
    the mail addres to be used to send emails

Here is a sample of the configuration file:

  
    submitter.name = submitterA, submitterB 
    implementation.className = some.package.submitterA , some.other.package.submitterB
    implementation.default = submitterA
    workDirectory.absolutePath = /somewhere/work
    outputDirectory.absolutePath = /someOtherPlace/out
    eticsDirectory.size = 500
    server.host = etics.cern.ch
    repository.host = etics-repository.cern.ch
    database.host = localhost
    database.name = myDatabase
    database.userName = myUsername
    database.userPassword = myPassword
    smtp.host = mySmtp.com
    smtp.mailAddress = myAddres@mySmtp.com

  • Edit the authorized users file for the Submission Webservice users.xml located at ETICS_HOME/etc
    adding a new user for each certificate authorized to use the Submission Webservice.
    A user represent the distinguish name of the certificate owner, the mapping must be done as follows:
    create a <user> tag for the user and specify a friendly name for him in a "name" property
    inside the <user> tag for each not null element of the DN insert one of the following tags:
    for the "EMAILADDRESS" a <email> tag
    for each "DC" a <domainComponent> tag (be carefull, order matters)
    for each "CN" a <name> tag
    for each "OU" a <organizationalUnit> tag
    for the "L" a <location> tag
    for the "ST" a <place> tag
    for the "C" a <country> tag

Here is a sample of the users files with a single user Sergio Rossi with DN
"EMAILADDRESS=SergioRossi@something.com, DC=com, DC=something, CN=Sergio Rossi, CN=srossi, OU=Users, OU=Employes, L=Bologna, ST=Emilia, C=IT" :

    <user name="Sergio Rossi">
       <email>SergioRossi@something.com</email>
        <domainComponent>com</domainComponent>
        <domainComponent>something</domainComponent>
        <name>Sergio Rossi</name>
        <name>srossi</name>
        <organizationalUnit>Users</organizationalUnit>
        <organizationalUnit>Employes</organizationalUnit>
        <organization>Company</organization>
        <location>Bologna</location>
        <place>Emilia</place>
        <country>IT</country>  
    </user>
    

  • Edit the context file located in component's etc folder by substituting the string @@etics_home@@ with your ETICS_HOME
  • Put this context file in tomcat conf/Catalina/localhost
  • Start tomcat

Glite Plugin Installation Instruction

For the installation of the gLite submitter plugin you have to follow the following steps:

  • Install the gLite UI

  • Edit the configuration file of the gLite plugin submitter.properties located at ETICS_HOME/etc/glite
    and specify the following information:
    the validity of the proxy that will generated from your certificate
    the minimum lifetime remaining to a proxy before to regenerate it (mandatory)
    the location where the proxy is stored (the complete file path)
    the name of the virtual organization your certificate is registered in (mandatory)
    the role for which your certificate is registered in the virtual organization
    the list of the CE where jobs have to be submitted with complete queue name
    the location of the folder containing vomses informations
    the pem certificate location
    the pem key location
    the certificate password (mandatory)
    the minimum lifetime remaining to a delegation (mandatory)
    the id to be used to delegate your generated proxy to the wms (mandatory)
    the endpoint of the wms to be used for your submission (mandatory)
    the validity of the myproxy registration of the generated proxy
    the host of the myproxy server
    the port of the myproxy server
    the password of the myproxy server
    the folder where temporary files will be stored

Here is a sample of the configuration file mandatory fields:

   proxy.vo.name = eticsproject.eu
   certificate.password = myPassword
   proxy.minTimeLeft = 1800
   delegation.minTimeLeft = 1800
   delegation.id = eticsproject.eu
   wms.endpoint = https://egee-wms-01.cnaf.infn.it:7443/glite_wms_wmproxy_server
   
  • Edit the configuration file of the Submission Webservice submitter.properties located at ETICS_HOME/etc
    and specify the following information:
    implementation.className = eu.eticsproject.submission.implementation.glite.GliteImplementation


Metronome Plugin Installation Instruction

For the installation of the Nmi Metronome submitter plugin you have to follow the following steps:

  • Install the Nmi Metronome Framework

To install Metronome first of all you need to have Condor and MySQL installed on your machine.

The Metronome site contains the steps to install the sofware, the url is http://nmi.cs.wisc.edu/. At the moment Etics uses nmi-2.5.2b

For Condor the reference manual is located at http://www.cs.wisc.edu/condor/manual/.

For MySQL you can refer to: http://dev.mysql.com/doc/refman/5.0/en/index.html

Procedure

The sources and binaries for condor can be found at:

http://www.cs.wisc.edu/condor/downloads-v2/download.pl

For installation instructions follow the manual. In particular my advice is to download the last condor version (the dynamic version) from the condor site (they'll request you to join the condor mailing list before downloading the software).

You have also to export the CONDOR_CONFIG variable after the installation:

CONDOR_CONFIG=/opt/condor-x.x.x/etc/condor_config

When you have installed Condor on your machine you have to fill the configuration files condor_config and condor_config.local (as described into the instructions) and you can check if all has gone right executing

condor_status -l

This command gives you the information about which kind of platforms exist (note that the name for a platform in Metronome is not the same as in Etics).

You must activate all the condor daemons (as root):

  • condor_master
  • condor_startd
  • condor_procd
  • condor_schedd
  • condor_collector
  • condor_negotiator

In particular if you don't activate condor_startd and condor_schedd you're not able to submit jobs and Metronome will not work. To start up the Condor daemons, execute:

/sbin/condor_master

The main deamon is the Condor master, whose only job in life is to make sure the other Condor daemons are running. The master keeps track of the daemons, restarts them if they crash, and periodically checks to see if you have installed new binaries (and if so, restarts the affected daemons).

The framework nmi needs a particular version of MySQL server and client that you have to install because each nmi version needs its own. For nmi-2.5.2b you need

  • MySQL-client-standard-5.0.27-0.rhel4
  • MySQL-shared-compat-5.0.27-0.rhel4
  • MySQL-server-community-5.0.51a-0.rhel4

Once you have installed both MySQL client and server with rpm -i, try to run the commands:

/usr/sbin/mysqld

/usr/share/mysql/mysql.server start

and execute also

/etc/init.d/httpd start

The first thing to do is to remove the safe option from the MySQL configuration file that avoid tables modification by the user. In this way it becomes possible for example to insert and remove users from the user table. In order to remove this setting go into /etc/my.cnf configuration file and comment this line (the default is uncommented):

[mysql]
no-auto-rehash
# Remove the next comment character if you are not familiar with SQL
#safe-updates

Next you have to download Metronome from this Link:

http://www.cs.wisc.edu/condor/nmi/nmi-releases/

Then install with rpm -i and follow administrator instructions.

Preparing System Environment

If you are using Linux for your submit node, you must install the mysql-dev and Perl DBI modules.

perl -MCPAN -e "install DBI"
perl -MCPAN -e "install DBD::mysql"

check that at the end you should have perl-DBD-MySQL-2.9004-3.1 for nmi-2.5.2b

Preparing the Database

After installing MySQL you will need to create a database (the default name is nmi_history) and install the default schema.

mysqladmin create nmi_history
mysql -u root -p nmi_history < /opt/nmi-x.y.z/framework/database/schema.mysql

The second command fills the nmi_history database with the schema ones. Now as a privileged user, create the following accounts. The first account shown below is the DB_WRITER_USER, we needs it to be able to insert and update records in the database. The second account, DB_READER_USER, is used by the web interface and other framework utilities that only need read-only access to the database.

NOTE: Be sure to execute FLUSH PRIVILEGES; to make sure these accounts are add appropriately. You may also need to create an additional ‘localhost’ record for each account if the database is running on the same host as the submit node.

# DB_WRITER ACCOUNT
GRANT SELECT,INSERT,UPDATE,DELETE ON nmi_history.* \
TO 'DB_WRITER_USER'@'localhost' IDENTIFIED BY 'DB_WRITER_PASS';

# DB_READER ACCOUNT
GRANT SELECT,CREATE TEMPORARY TABLES ON nmi_history.* \
TO 'DB_READER_USER'@'localhost' IDENTIFIED BY 'DB_READER_PASS';
GRANT UPDATE (notes) ON nmi_history.Run \
TO 'DB_READER_USER'@'localhost' IDENTIFIED BY 'DB_READER_PASS';

Note that as of Metronome 2.2.8, the DB_READER_ACCOUNT also needs privileges on the ‘sessions’ table:
GRANT SELECT,INSERT,UPDATE,DELETE ON nmi_history.sessions \
TO 'DB_READER_USER'@'localhost' IDENTIFIED BY 'DB_READER_PASS';

GRANT command creates instantly the user if it doesn't exists, alternatively you can use INSERT INTO and modify the user table into the sql database (See MySQL reference manual for details). Once you have created these users you can check if everything is ok executing:

mysql -e "select host,db,user from db" mysql

then execute

/usr/share/mysql/mysql.server restart

Installing NMI Framework

Install the NMI software under your chosen prefix:

perl Makefile.PL prefix= (nmi dir location path)
make install

Framework Configuration

Copy nmi-x.y.z/framework/nmi.conf.sample to prefix/etc/nmi.conf and edit as required. Please make sure that all non-trivial configuration parameters are customized for your local site (see Site Configuration Parameters for more information).

mkdir /etc
cp nmi-x.y.z/framework/nmi.conf.sample /etc/nmi.conf
edit /etc/nmi.conf

Edit nmi.conf setting your own properties as condor_config file path (to condor installation directory path), DB_HOST (to localhost), ecc..

nmi.conf.doc

This is an example of nmi configuration file for the localhost.

Build and Test

Preparations

First of all create a non-root user account (Metronome doesn't work as root for securety reasons). On the terminal window you will need to perform a few setup steps, if you want you can cut and paste directly from these pages.

bash$ export NMI_BIN=/nmi/bin

bash$ export NMI_LIB=/nmi/lib

bash$ source $NMI_BIN/config.[sh/csh]

bash$ which nmi_submit
/nmi/bin/nmi_submit

To receive informations about nmi_submit command execute:

bash$ nmi_submit —help

Finally it's mandatory to add a few lines at the end of the condor_config file

# EDIT_ME: In the next line, is a directory
# path where you keep your Hawkeye modules, if any. For example, it
# could be /home/condor/hawkeye_modules.
MODULES = "path to modules"
STARTD_CRON_NAME = NMIPOOL
# Uncomment the following line if NMIPOOL_JOBS has not been defined yet.
# NMIPOOL_JOBS =
# JOB: Report the list of software installed on the system.
NMIPOOL_JOBS = $(NMIPOOL_JOBS) prereq:has_:$(MODULES)/prereq:10m:kill

to enable hawkeye modules.

Hello World Using CVS Input

Download these two files:

http://nmi.cs.wisc.edu/files/perlHelloWorld.submit_3.txt

http://nmi.cs.wisc.edu/files/perlHelloWorld.cvs_.txt

Procedure:

1. Make sure you have downloaded the attachment files here into a working directory on the submit machine (es: /home/michele.carpene/nmi-work/provacvs).

2. Open the submit file perlHelloWorld.submit with your favorite editor and add a notify command. This will tell the build and test system how to notify you when a run is completed. Use the following format:

notify = < your @ email.address > Start the submission by running nmi_submit.

3. Change the platform name with x86_slc_4 or what's your platform operating system architecture

4. execute the command: nmi_submit perlHelloWorld.submit

5. the output will be:

Global ID: michele.carpene_etics-04.cnaf.infn.it_1214324794_20234
Run Directory: /opt/nmi-x.y.z/rundir/michele.carpene/2008/06/michele.carpene_etics-04.cnaf.infn.it_1214324794_20234
Run ID: 4

The output provided by nmi_submit shows some important information about your run. The first line gives you a unique global identifier for the job. The second line points to the directory path where the submission results are stored. The third line shows the condor run id (4) which is used by the Build & Test Overview page.

6. If you want to see that your job has been submitted you can visit the NMI Build System portal at:

http://nmi-web.cs.wisc.edu/nmi/index.php?page=results/overview

7. Check your email. You will receive a message when the run is completed.

8. Into the folder:

/opt/nmi-x.y.z/rundir/michele.carpene/2008/06/michele.carpene_etics-04.cnaf.infn.it_1214324794_20234/userdir/x86_slc_4/ you will find the files .out and .err with the output of the script execution.

Notes: in some cases is possible that nmi does a mistake in reading the option file using as invocation arguments the name of an executable named condor_exec, in that situation it's better to try the execution removing the input arguments from the script.

Submitter:

Once the nmi framework has been installed the remaining steps can be completed

  • Check metronome submitter configuration files have been copied into the ETICS_HOME/etc/nmi directory and also the etics-nmi-scripts.tar.gz and the copier.sh files (be careful to use only the etics-nmi-scripts.tar.gz file downloaded from the etics repository with the submitter, don't use the official etics one).
The configuration files are:

NmiPlatformsConfiguration.xml
NmiSubmitterConfiguration.conf
NmiSubmitterConfiguration.xml

These files are created from nmi/templates .vm files at the first invocation of the service and then they can be customized if the user wants to do it, anyway the service is able to work once it is installed without other settings.

  • Edit the configuration file of the Submission Webservice submitter.properties located at ETICS_HOME/etc
    and specify the following information:
    implementation.className = eu.eticsproject.submission.implementation.nmi.NmiImplementation

  • Edit the authorized users file for the Submission Webservice users.xml located at ETICS_HOME/etc
    adding a new user for each certificate authorized to use the Submission Webservice.
    A user represent the distinguish name of the certificate owner, the mapping must be done as follows:
    create a <user> tag for the user and specify a friendly name for him in a "name" property
    inside the <user> tag for each not null element of the DN insert one of the following tags:
    for the "EMAILADDRESS" a <email> tag
    for each "DC" a <domainComponent> tag (be carefull, order matters)
    for each "CN" a <name> tag
    for each "OU" a <organizationalUnit> tag
    for the "L" a <location> tag
    for the "ST" a <place> tag
    for the "C" a <country> tag

Here is a sample of the users files with a single user Sergio Rossi with DN
"EMAILADDRESS=SergioRossi@something.com, DC=com, DC=something, CN=Sergio Rossi, CN=srossi, OU=Users, OU=Employes, L=Bologna, ST=Emilia, C=IT" :

    <user name="Sergio Rossi">
       <email>SergioRossi@something.com</email>
        <domainComponent>com</domainComponent>
        <domainComponent>something</domainComponent>
        <name>Sergio Rossi</name>
        <name>srossi</name>
        <organizationalUnit>Users</organizationalUnit>
        <organizationalUnit>Employes</organizationalUnit>
        <organization>Company</organization>
        <location>Bologna</location>
        <place>Emilia</place>
        <country>IT</country>  
    </user>
    

  • Start Tomcat and deploy the ESW war file

-- MicheleDibenedetto - 04 Jun 2009

Edit | Attach | Watch | Print version | History: r20 < r19 < r18 < r17 < r16 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r20 - 2010-02-08 - unknown
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    ETICS All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback