Installation

Introduction

The EMI Hydra service is the key fragment storage system. Therefore at least one Hydra service needs to be installed. This Hydra service needs at least as many Web Service instances as the maximum number of key fragments expected. This guide explains how to install the Hydra service.

Installation

For a Fedora-compatible operating system, the EPEL package repository needs to be enabled. Instructions on doing this may be found here.

Install the EMI release package.

yum install yum-priorities yum-protectbase 

Then, for SL5:

rpm -Uvh http://emisoft.web.cern.ch/emisoft/dist/EMI/2/sl5/x86_64/base/emi-release-2.0.0-1.sl5.noarch.rpm

or for SL6:

rpm -Uvh http://emisoft.web.cern.ch/emisoft/dist/EMI/2/sl6/x86_64/base/emi-release-2.0.0-1.sl6.noarch.rpm

Afterwards run:
yum -y makecache

The EGI Trust anchors .repo file is needed. You need to go here.

yum -y install ca-policy-egi-core

You also need to install the fetch-crl

yum -y install fetch-crl

The periodic fetch-crl cron job should be enabled:

/sbin/chkconfig fetch-crl-cron on
/sbin/service fetch-crl-cron start

Then use the commands:

yum -y install emi-version
yum -y install glite-info-provider-service
yum -y install emi-hydra-service
yum -y install glite-yaim-hydra

or

yum -y install emi-version glite-info-provider-service emi-hydra-service glite-yaim-hydra

* Debian: not applicable yet.

Configuration

Host certificate

A host certificate and key is needed on each machine that will run a Hydra instance. It goes without saying that the CA certificates and certificate revocation lists must also be installed. This must be done before the configuration stages otherwise YAIM will fail.

The presence of a valid host certificate may be tested by issuing the following command:
openssl x509 -in /etc/grid-security/hostcert.pem -noout -text
There should not be any error messages.

To check if the correct CA certificates have been installed the following series of commands may help.
openssl x509 -in /etc/grid-security/hostcert.pem -noout -text | grep Issuer:
This will give you the issuing CA for the host certificate, see below.
ca_list=`ls -1 /etc/grid-security/certificates/*.0`
for ca_file in $ca_list; do openssl x509 -in $ca_file -noout -subject; done | grep [your issuer string]

VOMS Directory

If VOMS ACs are to be checked i.e. you are accepting VOMS credentials, there must be a VOMS directory. This directory typically at /etc/grid-security/vomsdir/ contains the hosts certificate(s) of the VOMS service(s) the Hydra service will be contacting. This directory may be empty but must exist or the YAIM configuration step will fail. If the VOMS directory is at some non-standard location this may be communicated to YAIM by changing the environment variable VOMSDIR.

Configuration variables

Some environment variables must be set or verified before the configuration.

So, for example:

export GLITE_LOCATION=/opt/glite/
export GLITE_LOCATION_ETC=/etc
export CONFIGDIR=/etc/hydra-service/
export VOMSDIR=/etc/grid-security/vomsdir

Also make sure:
PATH=$PATH:${GLITE_LOCATION}/yaim/bin

Main configuration File

Hydra is configured by YAIM. Therefore the configuration file does not take any specific name but must follow the syntax of YAIM and is used as input to the yaim command. Here is an example of a Hydra service configuration file.

The "HYDRA_CREATE_n " configuration variable contains the user DN or VOMS FQAN that has permissions to register/unregister keys etc. e.g. HYDRA_CREATE_n="/testers.eu-emi.eu/" allows members of the testers.eu-emi.eu to register/unregister keys. An FQAN of "/testers.eu-emi.eu/test/" would fail.
HYDRA_CREATE_n="" will allow any valid DN/FQAN to register/unregister keys. Dangerous.
This variable does NOT take multiple DNs or FQANs as a list.

The configuration file is given to the yaim command as follows:

yaim -c -s [name of your configuration file] -n HYDRA

BDII Configuration file

The Hydra service needs some work in order to publish the end-points to BDII. The general BDII instructions are to be found here. The resource level BDII instructions are found here. Apart from these, the following methods are used to get the Hydra service to publish to the resource BDII.

The template for the BDII configuration may be found below as glite-info-service-hydra.conf.template. There are also three example configuration files for a Hydra service running three endpoints. The template file contains the following variables:

Variable	Typical value	Comment
init	echo
service_type	echo org.glite.KeyStore	Defined in the GLUE type list
get_version	cat /usr/share/doc/hydra-service/VERSION | grep INTERFACE_VERSION | cut -d = -f 2
get_endpoint	echo https://vtb-generic-13.cern.ch:8443/1/glite-data-hydra-service/services/Hydra	The actual endpoint
get_status	glite-info-service-test HYDRA
WSDL_URL	https://vtb-generic-13.cern.ch:8443/1/glite-data-hydra-service/services/Hydra?wsdl	Location of the service WSDL
semantics_URL	https://twiki.cern.ch/twiki/bin/view/EMI/EMIHydraDocumentation#Hydra_Server	Where to find the documentation
get_starttime	perl -e '@st=stat("/var/run/tomcat5.pid");print "@st[10]\n";'
get_owner	echo -n DN= && openssl x509 -in /etc/grid-security/hostcert.pem -noout -subject | cut -d = -f 2-	DN of credential running the service.
get_implementor	echo
get_acbr	echo
get_data	echo -n DN= && openssl x509 -in /etc/grid-security/hostcert.pem -noout -subject | cut -d = -f 2-
get_services	echo -e "EndPoint1\nEndPoint2"	These are the other endpoints on the resource
get_implementationname	echo Hydra
get_implementationversion	cat /usr/share/doc/hydra-service/VERSION | grep VERSION | grep -v INTERFACE | grep -v SCHEMA | cut -d = -f 2

There must be a one-to-one correspondence between the Hydra endpoints and configuration files.

Therefore, make "n" copies of /etc/glite/info/service/glite-info-service-hydra.conf to reflect the number of endpoints. An example of such a BDII configuration file may be found here.

How to determine the number of endpoints?

1. The Hydra configuration file in the YAIM step gives how many endpoints were created.

2. Or check /var/lib/tomcat5/webapps/ for the number of directories n#glite-data-hydra-service

Each file /etc/glite/info/service/glite-info-service-hydra-n.conf should contain, apart from the variables above, the following information:

For /etc/glite/info/service/glite-info-service-hydra-1.conf

get_endpoint = echo https://[machine name]:8443/1/glite-data-hydra-service/services/Hydra
WSDL_URL = https://vtb-generic-13.cern.ch:8443/1/glite-data-hydra-service/services/Hydra?wsdl
get_services = echo -e "EndPoint2\nEndPoint3"

For /etc/glite/info/service/glite-info-service-hydra-2.conf

get_endpoint = echo https://[machine name]:8443/2/glite-data-hydra-service/services/Hydra
WSDL_URL = https://vtb-generic-13.cern.ch:8443/2/glite-data-hydra-service/services/Hydra?wsdl
get_services = echo -e "EndPoint1\nEndPoint3"

For /etc/glite/info/service/glite-info-service-hydra-3.conf

get_endpoint = echo https://[machine name]:8443/3/glite-data-hydra-service/services/Hydra
WSDL_URL = https://vtb-generic-13.cern.ch:8443/3/glite-data-hydra-service/services/Hydra?wsdl
get_services = echo -e "EndPoint1\nEndPoint2"

These configuration files need to be written carefully. Each Hydra endpoint configuration file MUST reference the other endpoints so that BDII will return the correct results to the client.

The Hydra GLUE information is published into BDII by /var/lib/bdii/gip/provider/glite-info-provider-service-hydra.

Publishing in GLUE-1.x

glite-info-service [configuration file] [GlueSiteUniqueID] [UniqueID]

Publishing in GLUE-2

GLUE-2 provides a method to publish multiple endpoints for a service.

glite-info-glue2-multi [configuration file 1],[configuration file 2],[configuration file n] [GlueSiteUniqueID]

Hydra info provider file

The /var/lib/bdii/gip/provider/glite-info-provider-service-hydra file should look like:

#!/bin/sh
glite-info-service /etc/glite/info/service/glite-info-service-hydra-1.conf UniqueHydraSite EndPoint1
glite-info-service /etc/glite/info/service/glite-info-service-hydra-2.conf UniqueHydraSite EndPoint2
glite-info-service /etc/glite/info/service/glite-info-service-hydra-3.conf UniqueHydraSite EndPoint3
glite-info-glue2-multi /etc/glite/info/service/glite-info-service-hydra-1.conf,/etc/glite/info/service/glite-info-service-hydra-2.conf,/etc/glite/info/service/glite-info-service-hydra-3.conf UniqueHydraSite

Where "UniqueHydraSite" defines a unique name for the Hydra service site e.g. `hostname -f`. "EndPoint'n'" is a unique key store name within the Hydra service site. In this case the Hydra service site contains three key stores.

In order to get this information into BDII, the service must be restarted (or started). This process will run the /var/lib/bdii/gip/provider/glite-info-provider-service-hydra script.

service bdii restart

Check whether the GLUE-1 publishing is working.

ldapsearch -x -h [your machine name] -p 2170 -b mds-vo-name=resource,o=grid

Check whether the GLUE-2 publishing is working.

ldapsearch -x -h [your machine name] -p 2170 -b o=glue

You should be able to 'grep' for something like the WSDL entry.

Publishing to top-tevel BDII

The above procedure will get the Hydra service endpoint(s) information onto your local BDII service. In order to get your information to a top-level BDII then you must contact the owner of such a service for you to be included.

Publishing to EMIR

If you want to publish your Hydra service to EMIR follow the instructions https://twiki.cern.ch/twiki/bin/view/EMI/SERP

Logging Configuration

The Hydra service uses the standard tomcat-style configuration files.

The tomcat service has the logging configuration file: /var/lib/tomcat5/common/classes/log4j.properties

The trustmanager has the logging configuration file: /etc/tomcat5/log4j-trustmanager.properties

An example template file for the endpoint logging configuration file can be found at:
/etc/hydra-service/log4j.template.properties

Each endpoint "n" within the service has a configuration file which is written during the YAIM configuration step: /etc/hydra-service/log4j.n.properties

Any of the above configuration files may be edited by the system administrator. A restart of syslog is needed for the logging changes to take effect. It is NOT necessary to restart the Hydra service.

Other changes in configuring the Hydra service to log to syslog needs some changes to both configuration files and the syslog service itself. Of course, the authoritative source for such changes are found here.

Other Configuration options.

CRL renewal frequency

In the file /etc/tomcat5/server.xml the variable crlUpdateInterval may be set e.g.

crlUpdateInterval="2h"

The Hydra service will need to be restarted for the new frequency to take effect.

service tomcat5 restart

Un-configuring the Hydra service.

DANGER. The following procedure will wipe out your databases and schema!
If needed, the Hydra service may be unconfigured by executing:
$CONFIGDIR/unconfigure

Other notes

During the YAIM configuration stage it is possible to see the following error message:

/usr/bin/build-jar-repository: error: JVM_LIBDIR /usr/lib/jvm-exports/java does not exist or is not a directory 

mkdir /usr/lib/jvm-exports/java 

Operation

Starting & Stopping the Service

The Hydra service is controlled through the standard Tomcat scripts.
service tomcat5 [stop|start|restart|status]
There are no other controls necessary. All the standard Tomcat options may be used and references found in the standard locations.

Service Information

Service Endpoints

A Hydra service presents a number of service endpoints. A Hydra service configured to have three key stores on a machine called "example01.net" will expose:
https://example01.net:8443/1/glite-data-hydra-service/services/Hydra
https://example01.net:8443/2/glite-data-hydra-service/services/Hydra
https://example01.net:8443/3/glite-data-hydra-service/services/Hydra

Logging and Logs

A Hydra service logs information to several places.

The tomcat-specific logging is sent to the standard log files:
/var/log/tomcat5/catalina.out
and
/var/log/tomcat5/tomcat.log

The keystore-specfic logging is sent to:
/var/log/tomcat5/glite-data-hydra-service-n
Where "n" is the keystore instance.

The trustmanager Authentication logging is sent to:
/var/log/tomcat5/trustmanager.log

Logging levels

The logging levels may be set for the tomcat logging in the standard manner. This is not part of Hydra and is not discussed here.

The keystore-specific logging levels are set in the file:
/etc/hydra-service/log4j.n.properties
Where "n" is the keystore instance. A restart of the overall Hydra service is needed to effect the changes.

The trustmanager authentication-specific logging levels are set in the file:
/etc/tomcat5/log4j-trustmanager.properties
A restart of the overall Hydra service is needed to effect the changes.

The logging syntax follows the standard log4j style.

-- JohnWhite

• hydra_bdii_example.conf: hydra_bdii_example.conf