WARNING Preliminary documentation, use at own

FTS 2.2 Documentation

A review of the FTS documentation.

Release status

Changes from previous version

Check the document describing changes from FTS version 2.1 to 2.2 for the list of new features, changes in service configuration and fixed bugs.

Service deployment model

The FTS service deployment model document describes the impact of outages of service components on the overall service quality, and suggests possible deployment scenarios.

Installation

Hardware requirements

You should install the FTS and FTA servers on reasonable Scientific Linux 4 machines, ideally of mid-range server-class.

It is recommended to have a least 512M memory, and a modern processor.

Large amounts of disk space are not critical, so a disk-server class machine is not needed.

Prerequisites

• There should be a valid host certificate and host key in /etc/grid-security/.
• Your system should have Java installed on it, this is available from http://java.sun.com. This is not provided by gLite for redistribution licensing restrictions. FTS uses other components from the JPackage repository. You can find more details about Java and JPackage installation at https://twiki.cern.ch/twiki/bin/view/EGEE/GLite31JPackage
• Your system should have Oracle InstantClient installed on it. This is available from Oracle. The recommended version is currently 10.2.0.3. You need at least oracle-instantclient-basic and oracle-instantclient-sqlplus for a successful installation.
• For the host running the FTS web-service, the firewall should be open for incoming on port tcp/8443 for the web-service. The connection from the client to the web-service is secured with SSL/TLS.
• For the host running the FTA agent daemons, the outgoing firewall (if present) should allow access to your MyProxy server, the database, all source SRM/gridFTP clusters and all destination SRM/gridFTP clusters.

Before you start

Read the Generic Installation and Configuration Guide

Read the generic installation guides:

• Generic Installation and Configuration Guide for gLite 3.1
• YAIM 4 guide
• Use of JPackage for gLite 3.1 Release

Please note that YAIM is only used to configure the system. You have to install the service components either by YUM (via gLite metapackages) or by your local package management system.

For FTS-specific YAIM details, see the Configuration section.

Choose deployment scenario

Examine the possible deployment scenarios from the deployment model section and choose the one that best fits your needs.

The miniumum suggested configuration is:

• two machine for the FTS web-service (for high availability)
• one machine for all the FTA agent daemons (both the VO agents and the channel agents)

Both the web-service instances (if there is more than one) and the agent dameons can be (and should be) spread across machines for scalability.

Grid dependencies

FTS requires:

• a MyProxy server
  • For legacy mode, the MyProxy server must be configured to allow the FTS host certificate to be an "authorized_retriever" (it should allow access from all machines running FTA agents).
  • For delegation renewal mode, FTS can use the same MyProxy as the WMS. Due to MyProxy restrictions, the server cannot be the same used by the resource broker to renew user credentials.
• the production storage endpoints to be running SRM (currently SRM v2.2).
  • SRM services should publish themselves into the EGEE BDII information system. SRM v1.1 is still supported.

Prepare the site-info.def file

The site-wide YAIM configuration file, site-info.def, should be prepared in advance. It must contain the configuration for the web-service and agents machines.

Two template files are available:

• /opt/glite/yaim/examples/siteinfo/site-info.def for site configuration.
• /opt/glite/yaim/examples/fta-info.def for FTS/FTA.

The FTS/FTA example is provided separately as it contains many parameters. If you want to use the example, append it to the end of the example site-info.def file first. The structure of the configuration file and detailed configuration instructions are available in the #FtsYaimValues section.

Identify the agents you need

This will depend on what VO transfers your site is required to serve, and this depends on whether you are the tier-0 or a tier-1. Please read this document which describes the suggested channel deployment model. IS THIS DOCUMENT UP-TO-DATE??? Once you have identified the channels you need and the VOs that you want to serve, you will need to define:

• An agent daemon for every VO you want to serve
• An agent daemon for every channel you want to serve

New installation

Package Installation

In general you have to add the DAG and JPackage repositories besides the node specific gLite repository.

Using YUM:

cd /etc/yum.repos.d
wget http://grid-deployment.web.cern.ch/grid-deployment/glite/repos/lcg-CA.repo
wget http://grid-deployment.web.cern.ch/grid-deployment/glite/repos/jpackage.repo
wget http://grid-deployment.web.cern.ch/grid-deployment/glite/repos/dag.repo

There are two distinct server types to install, FTS (the web-service dameons) and FTA (the agent dameons). From the deployment model you have chosen, identify which machines will be running FTS web-service and which machines will be running FTA agents.

Installing FTS web-service

At the time of writing this document (2008-08-01) there are problems with the dependencies of ecj 1:3.3.0-2.jpp5, so it it recommended to install an older version by hand first: cd /etc/yum.repos.d wget http://grid-deployment.web.cern.ch/grid-deployment/glite/repos/glite-FTS.repo yum install ecj-3.3.1.1-3.jpp5

yum install glite-FTS_oracle

Installing FTA agents

At the time of writing this document (2008-08-01) FTA uses an older version of the log4cpp package than the one in the DAG repository (v1.0), so you have to install the older version by hand: cd /etc/yum.repos.d wget http://grid-deployment.web.cern.ch/grid-deployment/glite/repos/glite-FTA.repo yum install log4cpp-0.3.4b-1.slc4

yum install glite-FTS_oracle

Configuring and Starting services

Configuring FTS web-service

/opt/glite/yaim/bin/yaim -c -s my-sitecfg.h -n  FTS2 

Database schema does not appear to be loaded. Please load it using the command:
sqlplus fts_xxxx/xxxxxxxxx@(DESCRIPTION=(LOAD_BALANCE=no)(ADDRESS=(PROTOCOL=TCP)(HOST=grid8.cern.ch)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=fts-pilot.cern.ch))) @/opt/glite/etc/glite-data-transfer-fts/schema/oracle/oracle-schema.sql

Depending on your connection string, you may have to put quotes around it, as:

sqlplus fts_xxxx/xxxxxxxxx@"(DESCRIPTION=(LOAD_BALANCE=no)(ADDRESS=(PROTOCOL=TCP)(HOST=grid8.cern.ch)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=fts-pilot.cern.ch)))" @/opt/glite/etc/glite-data-transfer-fts/schema/oracle/oracle-schema.sql

export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/lib/oracle/10.2.0.3/client/lib 

Starting the FTS web-service

service tomcat5 start 

Configuring the Initial Channels

For every channel agent you have defined, create the channel using the command:

glite-transfer-channel-add SOURCE-DEST SITE1 SITE2 

Channel parameters can be configured with the glite-transfer-channel-set command line tool.

Configuring FTA agents

For the agent nodes, run (note that the name is FTA2 rather than FTA):

/opt/glite/yaim/bin/yaim -c -s site-info.def -n FTA2 

Starting FTA agents

To start the FTA agent and information system caching daemons:

service transfer-agents start
service glite-sd2cache start

Every agent daemon that you have chosen to configure on the box will be started in sequence.

The second command will start a daemon that populates the services.xml information system cache file as it is described in FtsServicesXmlGliteSd2Cache. You may want to populate it cache initially by running

sh /etc/cron.daily/glite-sd2cache-cron 

To start a single agent daemon:

service transfer-agents start --instance glite-transfer-channel-agent-srmcopy-CERN-FNAL 

You can get a list of the names of the agents defined on the machine with the command

service transfer-agents status

Installing the history package

Setup your site BDII to publish the FTS information

The FTS web-service node is installed with a BDII GRIS where it publishes the FTS endpoints and the channels you have defined. Publish this contents of this resource BDII via your site BDII. This will then be picked up by the top-level BDII servers.

Register the FTS web-service node in the GOC database

Go to the GOCDB and register the FTS web-service node (or its DNS alias) in the Grid Operations Center database. Register with node type FTS. This will automatically add your node to the LCG SAME monitoring.

Upgrade from a previous version

Client Installation

Please install the client as part of the gLite 3.1 UI. See Generic install guide 3.1.

   glite-transfer-channel-list -s https://HOST.cern.ch:8443/TEST/glite-data-transfer-fts/services/ChannelManagement

 export GLITE_SD_PLUGIN=bdii
 export GLITE_SD_SITE=CERN-SC
 export LCG_GFAL_INFOSYS=lcg-bdii.cern.ch:2170

export GLITE_LOCATION=${GLITE_LOCATION:-/opt/glite}
if [ -z "$PATH" ]; then
    export PATH=$GLITE_LOCATION/bin
else
    export PATH=$PATH:$GLITE_LOCATION/bin
fi
if [ -z "$LD_LIBRARY_PATH" ]; then
    export LD_LIBRARY_PATH=$GLITE_LOCATION/lib
else
    export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$GLITE_LOCATION/lib
fi

   grid-proxy-init

   glite-transfer-channel-list -v

   glite-transfer-list -v Pending

Configuration

Administration

Command Line Interface

Command Line Interface

Notes

Links

Contacts

fts-support

configuration issues, software problems...

ftscern.support

issues related to CERN FTS services.

fts-users

open list. Announcements are also sent to this list.

Tests

See tests for FTS version 2.2.

Maintainer: PaoloTedesco