Show Children Hide Children

Main FTS Pages
FtsRelease22
Install
Configuration
Administration
Procedures
Operations
Development
Previous FTSes
FtsRelease21
FtsRelease21
All FTS Pages
FtsWikiPages
Last Page Update
PaoloTedesco
2009-01-22

Transfer Agents

Transfer agents run as standalone (C++) daemons, and use Oracle as the database backend.

DAO

The DAO layer is responsible for database interaction.

transfer-agents-dao-class-diagram.png

DAO objects are used by Action objects.

Actions

Classes deriving from Action use the DECLARE_FACTORY_METHOD and REGISTER_FACTORY_METHOD macros to create a subclass of the ActionFactoryMethod class and to register an instance of the subclass to the ActionFactory singleton instance.

actions-class-diagram.png

The SchedulerEntry class calls execute on the Action objects, passing its internal action parameters string.

Channel agents and VO agents read the configuration parameters and add SchedulerEntry objects to an ActionScheduler.

Actions get user's credentials through MyProxy or from the DB according to the configuration (see getProxyForJob function). For each job are stored into the database:

  • the user DN
  • one among:
    • user's MyProxy passphrase (USER_CRED field)
    • delegated credentials ID (CRED_ID field). With this option VOMS roles are supported.

Channel Actions

ChannelAction is the base class for channel actions; overrides execute getting the active transfers cache and then calling the abstract function exec.

Channel agents use a cache for active transfers; when a DAOException is trapped, the active transfers cache must be cleaned (since an error interacting with the database occurred, it is not possible to grant the integrity of the cache).

New transfers are added to the cache by the Fetch action, and removed by Cancel or CheckState.

Cancel (channel action)

Revokes active file transfers marked as canceling on the queue.

CheckState (channel action)

Check the state of active file transfer requests and update the queue with the retrieved information.

Fetch (channel action)

Submit new file transfers to the transfer service.

CleanSDCache (channel action)

Periodically purges the Service Discovery cache.

HeartBeat (channel action)

Sets the lastActive timestamp for the agent.

Active Transfers Cache

A cache of the active transfers for a channel, used by channel action objects. transfer-agents.ActiveTxCache.jpg

VO Actions

VOAction is the base class for VO actions. Overrides execute initializing and finalizing the context and calling the abstract function exec.

The VOAgent has a cache for channel allocation – used above all because of fallback logic for groups/clouds. The channels cache is used only by the Allocate action. The search is performed in two steps: first, a match is searched for the two SEs (A -> B), then if it's not found, the agent determines if this is due to the fact that the match was not in the cache or because it doesn't exist (process is split in 2 steps to keep the cache simple).

Retry (vo action)

Reschedule transfers in waiting state.

The classes BasicRetry and PythonRetry extend the base Retry class.

Allocate (vo action)

Allocate jobs to channels. The channel assignment operation is actually divided in two parts: the first is the LFN->SURL resolution using the catalog; the second is the actual assignment. The parameter “--allocate-in-different-cycle” (which is used only for tests and is not even configurable) allows to split the action in two different cycles.

The VO agent options “enable unknown source/destination” are used to determine if SEs not handled by service discovery are supported. In such a case, the SE is assigned to the “UNKNOWN” site (reserved keyword; sites must not be called UNKNOWN) so that it will be matched by a catch all channel '*'.

CheckReadiness (vo action)

Check if source files are ready to be transferred, i.e. already staged in the S.E. cache.

Currently does nothing (pre-staging is not yet supported). A pre-staging policy will be needed (share per channel), it is not possible to pre-stage everything.

Finalize (vo action)

Finalize a job request, registering when needed new replicas when the transfer is completed.

Has 2 purposes:

  • perform the done->finishing transition
  • call a hook
For a file, the done->finishing transition is done when all the files in the job are finished, otherwise part of the job's files would be handled by the VOAgent, part by the ChannelAgent.

CleanSDCache (vo action)

Periodically purges the Service Discovery cache.

HeartBeat (vo action)

Sets the lastActive timestamp for the agent.


Last edit: PaoloTedesco on 2009-01-22 - 13:32
Number of topics: 1

Maintainer: PaoloTedesco


Topic attachments
I Attachment History Action Size Date Who Comment
PNGpng actions-class-diagram.png r1 manage 100.1 K 2007-10-30 - 09:53 PaoloTedesco transfer agents actions class diagram
PNGpng transfer-agents-dao-class-diagram.png r1 manage 250.9 K 2007-10-30 - 12:25 PaoloTedesco  
JPEGjpg transfer-agents.ActiveTxCache.jpg r1 manage 86.6 K 2007-12-14 - 16:46 PaoloTedesco Active transfers cache class diagram
Edit | Attach | Watch | Print version | History: r7 < r6 < r5 < r4 < r3 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r7 - 2009-01-22 - PaoloTedesco
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    EGEE All webs login

This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright & by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Ask a support question or Send feedback