Transfer Agents
Transfer agents run as standalone (C++) daemons, and use Oracle as the database backend.
DAO
The DAO layer is responsible for database interaction.
DAO objects are used by Action objects.
Actions
Classes deriving from Action use the DECLARE_FACTORY_METHOD and REGISTER_FACTORY_METHOD macros to create a subclass of the ActionFactoryMethod class and to register an instance of the subclass to the ActionFactory singleton instance.
The SchedulerEntry class calls execute on the Action objects, passing its internal action parameters string.
Channel agents and VO agents read the configuration parameters and add SchedulerEntry objects to an ActionScheduler.
Actions get user's credentials through MyProxy or from the DB according to the configuration (see getProxyForJob function). For each job are stored into the database:
- the user DN
- one among:
- user's MyProxy passphrase (USER_CRED field)
- delegated credentials ID (CRED_ID field). With this option VOMS roles are supported.
Channel Actions
ChannelAction is the base class for channel actions; overrides execute getting the active transfers cache and then calling the abstract function exec.
Channel agents use a cache for active transfers; when a DAOException is trapped, the active transfers cache must be cleaned (since an error interacting with the database occurred, it is not possible to grant the integrity of the cache).
New transfers are added to the cache by the Fetch action, and removed by Cancel or CheckState.
Cancel (channel action)
Revokes active file transfers marked as canceling on the queue.
CheckState (channel action)
Check the state of active file transfer requests and update the queue with the retrieved information.
Fetch (channel action)
Submit new file transfers to the transfer service.
CleanSDCache (channel action)
Periodically purges the Service Discovery cache.
HeartBeat (channel action)
Sets the lastActive timestamp for the agent.
Active Transfers Cache
A cache of the active transfers for a channel, used by channel action objects.
VO Actions
VOAction is the base class for VO actions. Overrides execute initializing and finalizing the context and calling the abstract function exec.
The VOAgent has a cache for channel allocation – used above all because of fallback logic for groups/clouds. The channels cache is used only by the Allocate action. The search is performed in two steps: first, a match is searched for the two SEs (A -> B), then if it's not found, the agent determines if this is due to the fact that the match was not in the cache or because it doesn't exist (process is split in 2 steps to keep the cache simple).
Retry (vo action)
Reschedule transfers in waiting state.
The classes BasicRetry and PythonRetry extend the base Retry class.
Allocate (vo action)
Allocate jobs to channels.
The channel assignment operation is actually divided in two parts: the first is the LFN->SURL resolution using the catalog; the second is the actual assignment. The parameter “--allocate-in-different-cycle” (which is used only for tests and is not even configurable) allows to split the action in two different cycles.
The VO agent options “enable unknown source/destination” are used to determine if SEs not handled by service discovery are supported. In such a case, the SE is assigned to the “UNKNOWN” site (reserved keyword; sites must not be called UNKNOWN) so that it will be matched by a catch all channel '*'.
CheckReadiness (vo action)
Check if source files are ready to be transferred, i.e. already staged in the S.E. cache.
Currently does nothing (pre-staging is not yet supported). A pre-staging policy will be needed (share per channel), it is not possible to pre-stage everything.
Finalize (vo action)
Finalize a job request, registering when needed new replicas when the transfer is completed.
Has 2 purposes:
- perform the done->finishing transition
- call a hook
For a file, the done->finishing transition is done when all the files in the job are finished, otherwise part of the job's files would be handled by the VOAgent, part by the ChannelAgent.
CleanSDCache (vo action)
Periodically purges the Service Discovery cache.
HeartBeat (vo action)
Sets the lastActive timestamp for the agent.
Last edit:
PaoloTedesco on 2009-01-22 - 13:32
Number of topics: 1
Maintainer:
PaoloTedesco