FtsServerInstall15 < LCG

LCG Web>FtsWlcg>FtsRelease15>FtsServerInstall15 (2007-02-02, FlaviaDonno) (raw view)
---+ FTS Server Installation Guide 1.5

This is the installation guide for FTS server 1.5. It covers both the agent servers (FTA) and the web-service (FTS) components.

It refers to FTS version 1.5 from gLite release 3.0.

Changes from release 1.4 are described in FtsChangesFrom14To15.

If you are upgrading from 1.4 please read the above changes and then follow the instructions in FtsServerUpgrade14to15 instead.

Administration of the service is described in FtsServerAdmin15.

Service procedures specific to 1.5 can be found in FtsProcedures15.

---++ Requirements

   1. You should install the FTS and FTA servers on reasonable Scientific Linux 3 machines, ideally of mid-range server-class. It is recommended to have a least 512M memory, and a modern processor. Large amounts of disk space is not critical, so a disk-server class machine is not needed.
   2. There should be a valid hostcert and hostkey in =/etc/grid-security/=. The installation and configuration procedure should download the CA hash files and CRL lists specified in the current release at http://www.eugridpma.org/distribution/. If these already exist on the system, they will will be upgraded if necessary.
   3. The configuration makes changes to the password file (it creates 2 users). Your system (and in particular the file =/etc/security/limits.conf=) should allow these users to obtain a shell, although there is no requirement for remote login to these accounts.
   4. Your system should have Java installed on it, this is available from http://java.sun.com. This is not provided by gLite for redistribution licensing restrictions. The recommended default is j2re1.4.2_08.
   5. Your system should have Oracle instantclient installed on it. This is available at http://www.oracle.com/technology/software/tech/oci/instantclient/index.html. The recommended version is currently 10.2.0.1.
   6. For the host running the FTS web-service, the firewall should be open for incoming on port tcp/8443 for the web-service. The connection from the client to the web-service is secured with GSI.
   7. For the host running the FTA agent daemons, the outgoing firewall (if present) should allow access to your !MyProxy server, the database, all source SRM/gridFTP clusters and all destination SRM/gridFTP clusters.

---++ Read the Generic Installation and Configuration Guide

The FTS uses the standard [[http://grid-deployment.web.cern.ch/grid-deployment/documentation/LCG2-Manual-Install][Generic Installation and Configuration Guide]] from the [[http://glite.web.cern.ch/glite/documentation/default.asp][gLite documentation page]]. Please read to this to understand how YAIM works.

---++ Choose deployment scenario

Examine the possible deployment scenarios from FtsServerDeploy13 and decide which you want.

The miniumum suggested configuration is:

   * one machine for the FTS web-service 
   * one machine for all the FTA agent daemons (both the VO agents and the channel agents)

Both the web-service instances (if there is more than one) and the agent dameons can be (and should be) spread across machines for scalability.

---++ Make sure you know the Grid dependencies

In order to work, the overall FTS service requires a few grid components to be set up and running. It requires:

   * a !MyProxy server to be set up. This must be configured to allow the FTS host certificiate to be an "authorized_retriever" (it should allow access from all machines which are running FTA agents). Due to restrictions of !MyProxy, the server *cannot* be the same one that the resource broker uses to renew user credentials.

   * It requires the storage endpoints to be running SRM (currently SRM v1.1). The SRM should publish themselves into the !EGEE.BDII information system.

---++ Prepare the =site-info.def= file

This the site wide config file for YAIM. It should be prepared in advance.

The file will contain the configuration for all of the FTA agents (VO and channel) and the web-service(s).

We describe the configuration of the agent daemons first, and give an example.

Since the FTS/FTA contains many parameters, the YAIM example is separated into a separate file:

=/opt/glite/yaim/examples/fta-info.def=

If you want to use the example, cat it onto the end of the example =site-info.def= file first.

---+++ Add ORACLE_LOCATION

You should add the !InstantClient =ORACLE_LOCATION= variable to =site-info.def= if it is not already there.

For exanmple, for the recommended 10.1.0.3 !InstantClient:

=ORACLE_LOCATION=/usr/lib/oracle/10.1.0.3=

---+++ FTA agents

The yaim variables for configuring the FTA agents are all prefixed with =FTA_= in the =site-info.def= file. Any variable with prefix =FTA_= that the configuration script does not understand will cause the script to fail.

---+++++ What agents do I need?

You should identify what agents you need.

This will depend on what VO transfers your site is required to serve, and this dependes on whether you are the tier-0 or a tier-1. Please read the attached document which describes the suggested channel deployment model.

https://uimon.cern.ch/twiki/pub/LCG/FtsServerInstall15/SC4FTSsetupplan.doc

Once you have identified the channels you need and the VOs that you want to serve, you will need to define:

   * An agent daemon for every VO you want to serve
   * An agent daemon for every channel you want to serve

If you are unsure of what agents you need and how to define them, there is an example at FtsServerDeployExampleTier1.

---++++++ Agent naming

In order to be consistent and to ensure that the FTA channel discovery mechanism works, please observe the following rules when naming channels and VO agents:

   * The agent names should all be upper case, e.g. =DTEAM= or =CERN-RAL=
   * It is preferable if VO agent names do not contain a dash
   * The channel agent names should specify source and destination separated with a single dash. The source and destination should be simple site identifiers  (they do not need to be GOC DB names) and should contain only alphanumeric characters. The name should match the pattern =*[A-Z]-*[A-Z]=.

If you are unsure, look at the example for a small tier-1 site FtsServerDeployExampleTier1.

---+++++ What agents go on which machines

Identify:

   * the hostname of the machines you want to be running the FTA agents
   * which agents will run on which machine.

In the example, we assume two VO agents (for =dteam= and =atlas=) and two channel agents (=CERN-CERN= and =CERN-BNL=). We choose to run: 

   * the =dteam= VO agent and =CERN-CERN= channel agent together on the host =fts001.test.ch=
   * the =atlas= VO agent and =CERN-BNL= channel agent together on host =fts002.test.ch=

In the YAIM variable =FTA_MACHINES=, specify a tag for each node that will be installed (typically =ONE=, =TWO=, =THREE=). In the example:

=FTA_MACHINES="ONE TWO"=

For each machine, specify its hostname, and which agents will run on it. Construct the YAIM variable name using the tag specified above.

The VO agents name should be upper-case.

The channel agent names should be upper case, with a single dash =-= between the names.

<verbatim>
FTA_AGENTS_ONE_HOSTNAME="fts001.test.ch"
FTA_AGENTS_ONE="DTEAM CERN-CERN"

FTA_AGENTS_TWO_HOSTNAME="fts002.test.ch"
FTA_AGENTS_TWO="ATLAS CERN-BNL"
</verbatim>

For each agent listed, specify its type. The type is either:

   * =URLCOPY= for a 3rd party copy channel agent.
   * =SRMCOPY= for an SRM copy channel agent.
   * =VOAGENT_PYTHON= for a VO agent.

Construct the YAIM variable to use as =FTA_[agent-name]=. Since the underlying shell of YAIM is bash, agent names with a dash '-' should substitute an underscore instead, '_'. For the example:

<verbatim>
FTA_CERN_CERN="URLCOPY"
FTA_CERN_BNL="URLCOPY"
FTA_DTEAM="VOAGENT_PYTHON"
FTA_ATLAS="VOAGENT_PYTHON"
</verbatim>

This fully specifies which agent of what type will run on which nodes.

---+++++ Global variables

There are a few global variables that MUST be set in order to make the FTA agents work (mostly the DB connection parameters). Global variables can be identified with the prefix =FTA_GLOBAL_=. They are applied to all agents.

You must specify the DB type as =ORACLE= (MYSQL is not yet supported fully by the YAIM configuration):

=FTA_GLOBAL_DBTYPE=ORACLE=

You must specify the database connection string (as an !InstantClient OCI connection string), the DB username and the DB password. Ask your database administrator for these. For example:

<verbatim>
FTA_GLOBAL_DB_CONNECTSTRING="(DESCRIPTION=(LOAD_BALANCE=no)(ADDRESS=(PROTOCOL=TCP)(HOST=lcgtestdb1.cern.ch)(PORT=1521))(ADDRESS=(PROTOCOL=TCP)(HOST=lcgtestdb2.cern.ch)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=lcg_fts.cern.ch)))"
FTA_GLOBAL_DB_USER=lcg_fts_prod_w
FTA_GLOBAL_DB_PASSWORD=xxxxxxxx
</verbatim>

It is recommended to set the logging level of the agent to slightly higher than the default:

=FTA_GLOBAL_LOG_PRIORITY=INFO=

The default will be changed to this in a future release, so specifying it explicitly here will not be needed (see bug [[https://savannah.cern.ch/bugs/index.php?func=detailitem&item_id=16521][16521]]).

It is not recommended to set any other =FTA_GLOBAL_= variables.

---+++++ Agent parameters

The non-global FTA agent configuration parameters can be controlled in two ways.

   * TYPEDEFAULT. The parameter in question is applied to all agent instances of that type, e.g. all =URLCOPY= agents.
   * Instance specific. The parameter in question is applied only to the named instance.

The prefix of the parameter controls whether it is applied as a type default or to a specific instance. For example, the parameter GUC_TRANSFERTIMEOUT (which controls the globus-url-copy global timeout):

=FTA_TYPEDEFAULT_URLCOPY_GUC_TRANSFERTIMEOUT=1800=

will apply the value to all URLCOPY agents. Whereas:

=FTA_CERN_RAL_GUC_TRANSFERTIMEOUT=3600=

will apply the value only to the CERN-RAL agent.

The instance parameters override the TYPEDEFAULT parameters, i.e. if the same parameter is set in the TYPEDEFAULT and is also set in the instance, then the instance value will be used.

If neither the TYPEDEFAULT nor the instance specifies a parameter, then the agent default value will be used. There are a large number of configuration parameters that could be set for the agents - for the majority of them the default value is fine, so you do not need to explicitly specify them.

The FtsServerInstall15YaimReference lists all the configuration parameters that may be set.

---+++++ Type specific parameters

For each agent type specified (="VOAGENT_PYHTON"=, ="URLCOPY"=, or ="SRMCOPY"=), you may specify variables which are to be applied to all agents of that type. If you do not specify any value here, the agent internal default value will be used. Very few of the configuration parameters need to be varied from the default.

Type specific parameters are recognized by the prefix ="FTA_TYPEDEFAULT_[agent type]"=, e.g. ="FTA_TYPEDEFAULT_URLCOPY_"=.

The YAIM variables is constructed as "=FTA_TYPEDEFAULT_[agent type]_[parameter_name]=".

*Channel agent types*

It is recommended to set:

<verbatim>
FTA_TYPEDEFAULT_URLCOPY_GUC_LOGLEVEL=DEBUG
</verbatim>

which varies the default transfer logging level of URLCOPY agents to =DEBUG=. The default will be changed to this in a future release, so specifying it explicitly here will not be needed (see bug [[https://savannah.cern.ch/bugs/index.php?func=detailitem&item_id=16521][16521]]).

For a basic setup, it is not recommended to vary any other defaults.

*VO agent type*

For the VO agents, some configuration parameters MUST be specified (i.e. the default is not appropriate):

<verbatim>
FTA_TYPEDEFAULT_VOAGENT_PYTHON_PYTHON_PYTHONPATH="${GLITE_LOCATION}/lib/python2.2/site-packages:${GLITE_LOCATION}/lib/python/glite/fts/strategies/"
FTA_TYPEDEFAULT_VOAGENT_PYTHON_ACTIONS_RETRYMODULE=smarter_retry
FTA_TYPEDEFAULT_VOAGENT_PYTHON_ACTIONS_RETRYPARAMS="MaxFailures = 3 ; HoldEnabled = false ; OverwriteFailedFiles = true ; OverwriteExistingFiles = false ; DefaultRetryDelay = 300 ; RetryDelayForTimeoutOnGet = 1800 ; RetryDelayForDestFileExists = 300 ;"
</verbatim>

This tells them which retry plugin to use and defines the necessary variables to be passed to the plugin.  The default will be changed to this in a future release, so specifying it explicitly here will not be needed (see bug [[https://savannah.cern.ch/bugs/index.php?func=detailitem&item_id=16520][16520]]).

For an initial install, it is not recommended to change any other defaults.

---+++++ Instance specific parameters

Instance specific configuration parameters are applied only to the named FTA agent daemon.

Instance specific variables are prefixed with =FTA_[instance-name]_=. Since the underlying shell of YAIM is bash, agent names with a dash '-' should substitute an underscore instead, '_'. The YAIM varibale is constructed as =FTA_[instance_name]_[parameter_name]. As an example, for the =CERN-BNL= agent:

<verbatim>
FTA_CERN_BNL_LOG_PRIORITY=DEBUG
</verbatim>

would set the agent logging level to =DEBUG= for that specific agent, overriding the global default set from =FTA_GLOBAL_LOG_PRIORITY= above.

For an initial install, it is not recommeded to change any instance specific defaults.

---+++++ Example configuration for FTA agents

Given the example above, the YAIM variables that need to be specified are summarised in FtsServerInstall15YaimExample.

---+++ FTS web-service

The FTS web-service has far less confiuration parameters than the agent. Additionally, the DB parameters (if not specified) will be inherit the value used by the FTA agents.

FTS web-service YAIM variables are all prefixed with =FTS_. Any variable with this prefix that the script does not understand will cause it to fail (to catch spelling mis-types).

There is only one parameter which *MUST* be set:

=FTS_DBURL=

for example:

<verbatim>
FTS_DBURL="jdbc:oracle:thin:@(DESCRIPTION=DESCRIPTION=(LOAD_BALANCE=no)(ADDRESS=(PROTOCOL=TCP)(HOST=lcgtestdb1.cern.ch)(PORT=1521))(ADDRESS=(PROTOCOL=TCP)(HOST=lcgtestdb2.cern.ch)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=lcg_fts.cern.ch)))"
</verbatim>

which sets the Oracle JDBC connection string. This is usually similar to the FTA agent =FTA_GLOBAL_DB_CONNECTSTRING= parameter, but not in every case, so it must be specified explicitly. It can be provided by your database administrator.

The following parameters may be overriden, if desired (for example if the web-service is using a different DB role from the agents):

|   *Param*   |   *Default, if not specified*   |
|   =FTS_DB_TYPE=   |   The value from =FTA_GLOBAL_DBTYPE=   |
|   =FTS_DB_USER=   |   The value from =FTA_GLOBAL_DB_USER=   |
|   =FTS_DB_PASSWORD=   |   The value from =FTS_GLOBAL_DB_PASSWORD=   |

By default, the web-service will publish its endpoint in EGEE.BDII using its hostname. If instead, you wish to publish this using a DNS alias, set the parameter =FTS_HOST_ALIAS=, for example:

<verbatim>
FTS_HOST_ALIAS=fts-prod.test.ch
</verbatim>

The recommendation for an initial install is to set only the required parameter =FTS_DBURL= and optionally the host alias =FTS_HOST_ALIAS=.

---++ RPM installation

The RPM installation is managed by YAIM.

See the YAIM installation guide for more details on how to install yaim.

http://grid-deployment.web.cern.ch/grid-deployment/documentation/LCG2-Manual-Install/

There are two distinct server types to install, FTS (the web-service dameons) and FTA (the agent dameons). From the deployment model you have chosen, identify which machines will be running FTS web-service and which machines will be running FTA agents.

---+++++ FTS web-service

Run the YAIM install_node script:

=/opt/glite/yaim/scripts/install_node site-info.def glite-FTS=

---+++++ FTA agents

Run the YAIM install_node script:

=/opt/glite/yaim/scripts/install_node site-info.def glite-FTA=

---++ Run the configuration script

On each node, run the yaim configuration script, dependent on the node type.

---+++++ FTS web-service

For the web-service nodes, run:

=/opt/glite/yaim/scripts/configure_node site-info.def FTS=


If you have a fresh database, it will stop with the note similar to:

<verbatim>
Database schema does not appear to be loaded. Please load it using the command:

sqlplus fts_xxxx/xxxxxxxxx@(DESCRIPTION=(LOAD_BALANCE=no)(ADDRESS=(PROTOCOL=TCP)(HOST=grid8.cern.ch)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=fts-pilot.cern.ch))) @/opt/glite/etc/glite-data-transfer-fts/schema/oracle/oracle-schema.sql
</verbatim>

Run the suggested command to load the schema and then rerun the YAIM configuration.

Depending on your connection string, you may have to put quotes around it, as:

<verbatim>
sqlplus fts_xxxx/xxxxxxxxx@"(DESCRIPTION=(LOAD_BALANCE=no)(ADDRESS=(PROTOCOL=TCP)(HOST=grid8.cern.ch)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=fts-pilot.cern.ch)))" @/opt/glite/etc/glite-data-transfer-fts/schema/oracle/oracle-schema.sql
</verbatim>

You may also have to add the library path first:

=export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/lib/oracle/10.1.0.3/client/lib=

---+++++ FTA agents

For the agent nodes, run:

=/opt/glite/yaim/scripts/configure_node site-info.def FTA=

---++ Preparing the services.xml file

To prepare static the information service cache file =services,xml=, please go to FtsServerServicesXml15.

---++ Starting the service

---+++++ Starting the FTS web-service

To start the FTS web-service:

=service tomcat5 start=

A single daemon will start under the =tomcat4:tomcat4= user.

---+++++ Configuring the initial channels

You should start the web-service first and create the channels in the DB using the command =glite-transfer-channel-add= *BEFORE* starting the FTA agent daemons. The FTA agent daemons will fail to start if the corresponding channel has not yet been created.

For every channel agent you have running, define the channel using the command:

=glite-transfer-channel-add CHANNEL-CHANNEL SITE1 SITE2=

The channel name should match the VO agent name you specified above. It should be upper case.

The source and destinaion site names should be the GOCDB site names of the source and destination sites. These should be upper case.

If you are unsure, look at the example for a small tier-1 site FtsServerDeployExampleTier1.

---+++++ FTA agents

To start the FTA agent daemons:

=service transfer-agents start=

Every agent daemon that you have chosen to configure on the box will be started in sequence.

To start a single agent daemon:

=service transfer-agents start --instance glite-transfer-channel-agent-srmcopy-CERN-FNAL=

where the instance name is described above.

---++ Setup your site GIIS to publish the FTS information

The FTS web-service node is installed with a EGEE.BDII GRIS where it publishes the FTS endpoints and the channels you have defined.

Publish this contents of this GRIS vis your site EGEE.BDII. This will then be picked up by the top-level EGEE.BDII servers.

---++ Register the FTS web-service node in the GOC database

Go to http://goc.grid-support.ac.uk/gridsite/gocdb/ and register the FTS web-service node (or its DNS alias) in the Grid Operations Centre database.

Register with node type =FTS=.

This will automatically add your node to the LCG SAME monitoring.

---++ Administering the service

See the FtsServerAdmin15 guide for more details of administering the service.

See FtsProcedures15 for common service operational procedures.

---++ Troubleshooting

---+++++ Config problem on FTA: FTA_TYPEDEFAULT_SRMCOPY_GUC_LOGLEVEL

If the FTA YAIM script stops with:

<verbatim>
ERROR: The variable FTA_TYPEDEFAULT_SRMCOPY_GUC_LOGLEVEL was specified in the configuration file.
This is not used by any of the agents configured in the file.
</verbatim>

this is becasue you have specified a parameter for =SRMCOPY= agents but you have not defined any of the agents to be of type =SRMCOPY=. The script rejects unneeded variables or =FTA_= variables that it does not understand.

You should comment out the line in the =site-info.def= file.

---+++++ FTS/FTA FAQ

Further troubleshooting and FAQ can be found at EGEE.DMFtsSupport.

-----

Last edit: %SEARCH{".*" nosearch="on" regex="on" scope="title" nototal="no" topic="FtsServerInstall15" format="$wikiusername on $date"}%

Maintainer: Main.GavinMcCance

-----
Attachments
Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
doc	SC4FTSsetupplan.doc	r1	manage	42.5 K	2006-05-02 - 11:18	GavinMcCance
Topic revision: r14 - 2007-02-02 - FlaviaDonno
LCG Wikis
LCG Service
Coordination
LCG Grid
Deployment
LCG
Apps Area
Public webs
Welcome Guest
- Cern Search
- TWiki Search
- Google Search
LCG All webs
Copyright &© 2008-2023 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback