Show Children Hide Children FtsYaimValues21

FTS Server Installation Guide 2.1

This is the installation guide for FTS server 2.1 FtsRelease21. It covers both the agent servers (FTA) and the web-service (FTS) components. It refers to FTS version 2.1 PATCH:1594 and PATCH:1760 from gLite release 3.1.

• If you are upgrading from 1.5 please read the above changes and then follow the instructions in FtsServerUpgrade15to20 first!
• Administration of the service is described in FtsServerAdmin20 (not changed since FTS 2.0).
• Service procedures specific to 2.1 can be found in FtsProcedures20 (not changed since FTS 2.0).

Requirements

1. You should install the FTS and FTA servers on reasonable Scientific Linux 4 machines, ideally of mid-range server-class. It is recommended to have a least 512M memory, and a modern processor. Large amounts of disk space is not critical, so a disk-server class machine is not needed.
2. There should be a valid hostcert and hostkey in /etc/grid-security/.
3. Your system should have Java installed on it, this is available from http://java.sun.com. This is not provided by gLite for redistribution licensing restrictions. FTS uses other components from the JPackage repository. You can find more details about Java and JPackage installation at GLite31JPackage
4. Your system should have Oracle InstantClient installed on it. This is available from Oracle. The recommended version is currently 10.2.0.3. You need at least oracle-instantclient-basic and oracle-instantclient-sqlplus for a successful installation.
5. For the host running the FTS web-service, the firewall should be open for incoming on port tcp/8443 for the web-service. The connection from the client to the web-service is secured with SSL/TLS.
6. For the host running the FTA agent daemons, the outgoing firewall (if present) should allow access to your MyProxy server, the database, all source SRM/gridFTP clusters and all destination SRM/gridFTP clusters.

Read the Generic Installation and Configuration Guide

Please read the generic installation guides at GenericInstallGuide310, YaimGuide400 and GLite31JPackage. Remark: glite-UI installation is not required, so if you have troubles with it, just go on with FTS install below.

Please read to this to understand how YAIM works. Particular FTS details are available: FtsYaimValues21.

Please note that YAIM is only used to configure the system. You have to install the service components either by YUM (via gLite metapackages) or by your local package management system.

Choose deployment scenario

Examine the possible deployment scenarios from FtsServiceDeploymentModel (not version specific) and decide which you want.

The miniumum suggested configuration is:

• two machine for the FTS web-service (for high availability)
• one machine for all the FTA agent daemons (both the VO agents and the channel agents)

Both the web-service instances (if there is more than one) and the agent dameons can be (and should be) spread across machines for scalability.

See FtsServiceReview20 for deployment suggestions on how to improve the availability of the service.

Make sure you know the Grid dependencies

In order to work, the overall FTS service requires a few grid components to be set up and running. It requires:

• a MyProxy server to be set up. For legacy mode, this must be configured to allow the FTS host certificate to be an "authorized_retriever" (it should allow access from all machines which are running FTA agents). For delegation renewal mode, FTS 2.1 can use the same MyProxy as the WMS. Due to restrictions of MyProxy, the server cannot be the same one that the resource broker uses to renew user credentials.

• It requires the production storage endpoints to be running SRM (currently SRM v2.2). The SRM should publish themselves into the EGEE.BDII information system. SRM v1.1 is still supported.

Prepare the site-info.def file

This the site wide config file for YAIM. It should be prepared in advance. The file will contain the configuration for all of the FTA agents (VO and channel) and the web-service(s).

We describe the configuration of the agent daemons first, and give an example. Since the FTS/FTA contains many parameters, the YAIM example is separated into a separate file:

/opt/glite/yaim/examples/fta-info.def

If you want to use the example, cat it onto the end of the example site-info.def file first.

Add ORACLE_LOCATION

You should add the InstantClient ORACLE_LOCATION variable to site-info.def if it is not already there.

For example, for the recommended 10.2.0.3 InstantClient:

ORACLE_LOCATION=/usr/lib/oracle/10.2.0.3

FTA agents

The YAIM variables for configuring the FTA agents are all prefixed with FTA_ in the site-info.def file. Any variable with prefix FTA_ that the configuration script does not understand will cause the script to fail.

What agents do I need?

You should identify what agents you need.

This will depend on what VO transfers your site is required to serve, and this depends on whether you are the tier-0 or a tier-1. Please read this document which describes the suggested channel deployment model. SC4FTSsetupplan.doc

Once you have identified the channels you need and the VOs that you want to serve, you will need to define:

• An agent daemon for every VO you want to serve
• An agent daemon for every channel you want to serve

If you are unsure of what agents you need and how to define them, there is an example at FtsServerDeployExampleTier1.

Agent naming

In order to be consistent and to ensure that the FTA channel discovery mechanism works, please observe the following rules when naming channels and VO agents:

• The agent names should all be upper case, e.g. DTEAM or CERN-RAL
• It is preferable if VO agent names do not contain a dash
• The channel agent names should specify source and destination separated with a single dash. The source and destination should be simple site identifiers (they do not need to be GOC DB names) and should contain only alphanumeric characters. The name should match the pattern *[A-Z]-*[A-Z].

If you are unsure, look at the example for a small tier-1 site FtsServerDeployExampleTier1.

What agents go on which machines

Identify:

• the hostname of the machines you want to be running the FTA agents
• which agents will run on which machine.

In the example, we assume two VO agents (for dteam and atlas) and two channel agents (CERN-CERN and CERN-BNL). We choose to run:

• the dteam VO agent and CERN-CERN channel agent together on the host fts001.test.ch
• the atlas VO agent and CERN-BNL channel agent together on host fts002.test.ch

In the YAIM variable FTA_MACHINES, specify a tag for each node that will be installed (typically ONE, TWO, THREE). In the example:

FTA_MACHINES="ONE TWO" For each machine, specify its hostname, and which agents will run on it. Construct the YAIM variable name using the tag specified above. The VO agents name should be upper-case. The channel agent names should be upper case, with a single dash - between the names.

FTA_AGENTS_ONE_HOSTNAME="fts001.test.ch"
FTA_AGENTS_ONE="DTEAM CERN-CERN"

FTA_AGENTS_TWO_HOSTNAME="fts002.test.ch"
FTA_AGENTS_TWO="ATLAS CERN-BNL"

For each agent listed, specify its type. The type is either:

• URLCOPY for a 3rd party copy channel agent.
• SRMCOPY for an SRM copy channel agent.
• VOAGENT for a VO agent.

Construct the YAIM variable to use as FTA_[agent-name]. Since the underlying shell of YAIM is bash, agent names with a dash '-' should substitute an underscore instead, '_'. For the example:

FTA_CERN_CERN="URLCOPY"
FTA_CERN_BNL="URLCOPY"
FTA_DTEAM="VOAGENT"
FTA_ATLAS="VOAGENT"

This fully specifies which agent of what type will run on which nodes.

Global variables

There are a few global variables that MUST be set in order to make the FTA agents work (mostly the DB connection parameters). Global variables can be identified with the prefix FTA_GLOBAL_. They are applied to all agents.

You must specify the DB type as ORACLE (MYSQL is not supported):

FTA_GLOBAL_DBTYPE=ORACLE

You must specify the database connection string (as an InstantClient OCI connection string), the DB username and the DB password. Ask your database administrator for these. For example:

FTA_GLOBAL_DB_CONNECTSTRING="(DESCRIPTION=(LOAD_BALANCE=no)(ADDRESS=(PROTOCOL=TCP)(HOST=lcgtestdb1.cern.ch)(PORT=1521))(ADDRESS=(PROTOCOL=TCP)(HOST=lcgtestdb2.cern.ch)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=lcg_fts.cern.ch)))"
FTA_GLOBAL_DB_USER=lcg_fts_prod_w
FTA_GLOBAL_DB_PASSWORD=xxxxxxxx

The default logging level is INFO, you may wish to change this?

FTA_GLOBAL_LOG_PRIORITY=DEBUG

It is not recommended to set any other FTA_GLOBAL_ variables.

Agent parameters

The non-global FTA agent configuration parameters can be controlled in two ways.

• TYPEDEFAULT. The parameter in question is applied to all agent instances of that type, e.g. all URLCOPY agents.
• Instance specific. The parameter in question is applied only to the named instance. Most of these parameters are now located inside the database and the YAIM configuration will load them (see FtsChangesFrom20To21).

The prefix of the parameter controls whether it is applied as a type default or to a specific instance. For example, the parameter GUC_TRANSFERTIMEOUT (which controls the globus-url-copy global timeout):

FTA_TYPEDEFAULT_URLCOPY_GUC_TRANSFERTIMEOUT=1800

will apply the value to all URLCOPY agents. Whereas:

FTA_CERN_RAL_GUC_TRANSFERTIMEOUT=3600

will apply the value only to the CERN-RAL agent.

The instance parameters override the TYPEDEFAULT parameters, i.e. if the same parameter is set in the TYPEDEFAULT and is also set in the instance, then the instance value will be used.

If neither the TYPEDEFAULT nor the instance specifies a parameter, then the agent default value will be used. There are a large number of configuration parameters that could be set for the agents - for the majority of them the default value is fine, so you do not need to explicitly specify them.

The FtsYaimValues21 lists all the configuration parameters that may be set.

Type specific parameters

For each agent type specified ("VOAGENT", "URLCOPY", or "SRMCOPY"), you may specify variables which are to be applied to all agents of that type. If you do not specify any value here, the agent internal default value will be used. Very few of the configuration parameters need to be varied from the default.

Type specific parameters are recognized by the prefix "FTA_TYPEDEFAULT_[agent type]", e.g. "FTA_TYPEDEFAULT_URLCOPY_".

The YAIM variables is constructed as "=FTA_TYPEDEFAULT_[agent type]_[parameter_name]=".

Channel agent types

For a basic setup, it is not recommended to vary any other defaults.

VO agent type

For the VO agents, some configuration parameters can be specified but the default (i.e. not setting them) is appropriate.

FTA_TYPEDEFAULT_VOAGENT_PYTHONPATH="${GLITE_LOCATION}/lib/python2.3/site-packages:${GLITE_LOCATION}/lib/python/glite/fts/strategies/"
FTA_TYPEDEFAULT_VOAGENT_ACTIONS_RETRYMODULE=smarter_retry
FTA_TYPEDEFAULT_VOAGENT_ACTIONS_RETRYPARAMS="MaxFailures = 3 ; HoldEnabled = false ; OverwriteFailedFiles = true ; OverwriteExistingFiles = false ; DefaultRetryDelay = 300 ; RetryDelayForTimeoutOnGet = 1800 ; RetryDelayForDestFileExists = 300 ;"

This tells them which retry plugin to use and defines the necessary variables to be passed to the plugin.

For an initial install, it is not recommended to change any other defaults.

Please note that the FTA_TYPEDEFAULT_VOAGENT_PYTHONPATH should be changed on 64bit platforms to point to the lib64 directory instead of lib!

Instance specific parameters

Instance specific configuration parameters are applied only to the named FTA agent daemon.

Instance specific variables are prefixed with FTA_[instance-name]_. Since the underlying shell of YAIM is bash, agent names with a dash '-' should substitute an underscore instead, '_'. The YAIM varibale is constructed as FTA_[instance_name]_[parameter_name]. As an example, for the =CERN-BNL agent:

FTA_CERN_BNL_LOG_PRIORITY=DEBUG

would set the agent logging level to DEBUG for that specific agent, overriding the global default set from FTA_GLOBAL_LOG_PRIORITY above.

For an initial install, it is not recommended to change any instance specific defaults.

FTS web-service

The FTS web-service has far less configuration parameters than the agent. Additionally, the DB parameters (if not specified) will be inherit the value used by the FTA agents.

FTS web-service YAIM variables are all prefixed with =FTS_. Any variable with this prefix that the script does not understand will cause it to fail (to catch spelling mis-types).

There is only one parameter which MUST be set:

FTS_DBURL

for example:

FTS_DBURL="jdbc:oracle:thin:@(DESCRIPTION=DESCRIPTION=(LOAD_BALANCE=no)(ADDRESS=(PROTOCOL=TCP)(HOST=lcgtestdb1.cern.ch)(PORT=1521))(ADDRESS=(PROTOCOL=TCP)(HOST=lcgtestdb2.cern.ch)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=lcg_fts.cern.ch)))"

which sets the Oracle JDBC connection string. This is usually similar to the FTA agent FTA_GLOBAL_DB_CONNECTSTRING parameter, but not in every case, so it must be specified explicitly. It can be provided by your database administrator.

The following parameters may be overriden, if desired (for example if the web-service is using a different DB role from the agents):

Param	Default, if not specified
FTS_DB_TYPE	The value from FTA_GLOBAL_DBTYPE
FTS_DB_USER	The value from FTA_GLOBAL_DB_USER
FTS_DB_PASSWORD	The value from FTA_GLOBAL_DB_PASSWORD

By default, the web-service will publish its endpoint in BDII using its hostname. If instead, you wish to publish this using a DNS alias, set the parameter FTS_HOST_ALIAS, for example:

FTS_HOST_ALIAS=fts-prod.test.ch

The recommendation for an initial install is to set only the required parameter FTS_DBURL and optionally the host alias FTS_HOST_ALIAS.

Package Installation

The package installation is managed by YUM

See GenericInstallGuide310 and GLite31JPackage guides for the details!

In general you have to add the DAG and JPackage repositories besides the node specific gLite repository. Check if your there is NO /etc/yum.repos.d/glite-dm.repo present, if it is there, remove it!

For example:

cd /etc/yum.repos.d
wget http://grid-deployment.web.cern.ch/grid-deployment/glite/repos/3.1/lcg-CA.repo
wget http://grid-deployment.web.cern.ch/grid-deployment/glite/repos/3.1/jpackage.repo
wget http://grid-deployment.web.cern.ch/grid-deployment/glite/repos/3.1/dag.repo

There are two distinct server types to install, FTS (the web-service dameons) and FTA (the agent dameons). From the deployment model you have chosen, identify which machines will be running FTS web-service and which machines will be running FTA agents.

Installing FTS web-service

You need to add the FTS gLite repository and install the glite-FTS_oracle metapackage.

At the time of writing this document (2008-08-01) there are problems with the dependencies of ecj 1:3.3.0-2.jpp5, so it it recommended to install an older version by hand first:

cd /etc/yum.repos.d
wget http://grid-deployment.web.cern.ch/grid-deployment/glite/repos/3.1/glite-FTS.repo
yum install ecj-3.3.1.1-3.jpp5
yum install glite-FTS_oracle

Installing FTA agents

You need to add the FTA gLite repository and install the glite-FTA_oracle metapackage.

At the time of writing this document (2008-08-01) FTA uses an older version of the log4cpp package than the one in the DAG repository (v1.0), so you have to install the older version by hand:

cd /etc/yum.repos.d
wget http://grid-deployment.web.cern.ch/grid-deployment/glite/repos/3.1/glite-FTA.repo
yum install log4cpp-0.3.4b-2.slc4
yum install glite-FTA_oracle

Configuration and Starting services

On each node, run the YAIM configuration script, dependent on the node type.

Configuring FTS web-service

For the web-service nodes, run (note the name FTS2 rather than FTS):

/opt/glite/yaim/bin/yaim -c -s my-sitecfg.h -n  FTS2

If you have a fresh database, it will stop with the note similar to:

Database schema does not appear to be loaded. Please load it using the command:

sqlplus fts_xxxx/xxxxxxxxx@(DESCRIPTION=(LOAD_BALANCE=no)(ADDRESS=(PROTOCOL=TCP)(HOST=grid8.cern.ch)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=fts-pilot.cern.ch))) @/opt/glite/etc/glite-data-transfer-fts/schema/oracle/oracle-schema.sql

Run the suggested command to load the schema and then rerun the YAIM configuration.

Depending on your connection string, you may have to put quotes around it, as:

sqlplus fts_xxxx/xxxxxxxxx@"(DESCRIPTION=(LOAD_BALANCE=no)(ADDRESS=(PROTOCOL=TCP)(HOST=grid8.cern.ch)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=fts-pilot.cern.ch)))" @/opt/glite/etc/glite-data-transfer-fts/schema/oracle/oracle-schema.sql

Normally oracle-instantclient adds itself to the dynamic library configuration via the /etc/ld.so.conf.d/oracle-instantclient.conf file. If it did not happen, then you may also have to add the library path first:

export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/lib/oracle/10.2.0.3/client/lib

You have to use lib64 instead of lib on 64 bit platforms.

Starting the FTS web-service

To start the FTS web-service (if YAIM didn't already):

service tomcat5 start

A single daemon will start under the tomcat:tomcat user.

Configuring the Initial Channels

You should start the web-service first and create the channels in the DB using the command glite-transfer-channel-add BEFORE configuring the FTA agent daemons.

In the phase of configuring the FTA agents the configuration script will try to load the channel specific configuration parameters into the database. In order to achieve this goal the corresponding channels have to be already created.

The FTA agent daemons will also fail to start if the corresponding channel has not yet been created.

For every channel agent you have running, define the channel using the command:

glite-transfer-channel-add CHANNEL-CHANNEL SITE1 SITE2

The channel name should match the VO agent name you specified above. It should be upper case.

The source and destination site names should be the GOCDB site names of the source and destination sites. These should be upper case.

If you are unsure, look at the example for a small tier-1 site FtsServerDeployExampleTier1.

Configuring FTA agents

For the agent nodes, run (note the name FTA2 rather than FTA):

/opt/glite/yaim/bin/yaim -c -s site-info.def -n FTA2

Starging FTA agents

To start the FTA agent and information system caching daemons:

service transfer-agents start
service glite-sd2cache start

Every agent daemon that you have chosen to configure on the box will be started in sequence.

The second command will start a daemon that populates the services.xml information system cache file as it is described in FtsServicesXmlGliteSd2Cache. You may want to populate it cache initially by running

sh /etc/cron.daily/glite-sd2cache-cron 

To start a single agent daemon:

service transfer-agents start --instance glite-transfer-channel-agent-srmcopy-CERN-FNAL

where the instance name is described above.

Installing the history package

This moves old jobs away from the active schema so as to avoid performance problems. See FtsAdminTools20 ("History package") for this.

Setup your site BDII to publish the FTS information

The FTS web-service node is installed with a BDII GRIS where it publishes the FTS endpoints and the channels you have defined.

Publish this contents of this resource BDII via your site BDII. This will then be picked up by the top-level BDII servers.

Register the FTS web-service node in the GOC database

Go to the GOCDB and register the FTS web-service node (or its DNS alias) in the Grid Operations Center database. Register with node type FTS. This will automatically add your node to the LCG SAME monitoring.

Further information

• See the FtsServerAdmin20 guide for more details of administering the service.
• See FtsProcedures20 for common service operational procedures.
• Further troubleshooting and FAQ can be found at DMFtsSupport.

---

Last edit: ZsoltMolnar on 2009-04-30 - 12:21

Maintainer: PaoloTedesco, GavinMcCance, AkosFrohner

---