FTS Delegation and MyProxy Usage

This text describes the usage of delegation in the FTS as opposed to the current practice of pure MyProxy credential retreival based on a password.

Current (<v1.5) client flow

For current (<v1.5) clients, the flow is as follows:

1. The clients do not check the server version.
2. They will submit a job with MyProxy password specified. They attempt no delegation.
3. The server will obtain the new proxy from MyProxy using the authorized_retriever mechanism and the user's MyProxy password. The MyProxy server will be that configured on the server (we allow this to be overridden in 1.4+ by specifying the MyProxy server name as a job parameter).

Why is the current practice using MyProxy passwords insufficient?

The current objection to the FTS from the security point of view is that it needs the MyProxy password in order to retrieve the user's credential. This means that the authorized_retriver setting on the MyProxy server needs to be set to allow the FTS server certificate to retrieve user certificates. The objection stems from the fact that the password used to retrieve the user credential is permanent (i.e. it has no expiration date).

In addition, just using MyProxy as a credential store does not enable us to use the user's proxy including the VOMS attributes - for this we would need to contact VOMS and re-decorate the proxy with the necessary fields.

In the WMS, MyProxy is not used in authorized retriever mode; it is used in authorized_renewer mode. In this case, a server that wants to renew a user's credential already needs to have a valid credential for that user before they can do this. Typically the server renews a user's credential just before it will expire. The advantage from the security point of view is that the credential used to ask for the renewal is time limited, which reduces the exposure in the case of the credential being compromised. So having delegation in the FTS, i.e. a delegated proxy would allow it to operate in the same mode.

However, this mode also has a lot of issues, as the discussion will show below. For most jobs the FTS will finish within the expiration time and therefore it will not need to ask for renewal at all from MyProxy. We argue that there need to be no dependency of MyProxy at all, and that proxy renewal should not be the worry of the FTS (or even the WMS for that matter).

Using the Delegation Porttype

Since the FTS submission portal is a web-service, delegation can be implemented using the new JRA3 delegation port-type. We describe below how the components would need to be modified to make this work and what needs to be done to ensure backwards compatibility with existing client code. After the JRA3 delegation porttype has been used to delegate the client's credential for usage by the FTS, this certificate has to be made accessible for use to the transfer agents. These are different processes from the FTS web service since the FTS is designed such that it can be scaled up to many channels running for many VOs - potentially with redundant backup servers. It does not make sense to store the user's credentials on any one machine; the best place for the credentials is where to rest of the shared state goes - the database.

From the security point of view the nature of the DB connection has to be checked (is it encrypted) and the DB has to be protected against unauthorized access. In general, since multiple agents potentially on different machines all need access to the proxies, we believe the compromise risk from putting the certificates in the DB is lower than from scattering them over multiple machines. The certificate proxies may be encrypted in the database to prevent network snooping of the proxies, although we note that all the agents that need access to be proxies need to be configured both with the database account and password and also with the decryption key.

Client flow

For the clients, the flow in the delegation case would be as follows:

1. The submitting client should check the interface version and the presence of the delegation port-type. If it is the old server version (<1.5) or the delegation port-type is switched off, proceed with a 1.3/1.4 style job submission: ask the client for the MyProxy password and submit the job as before.
2. If the delegation port-type is available, then first call the delegation method to delegate the client's proxy to the server side. This should be stored in the database using a delegation ID that is the hash of the user's DN and VOMS credential attributes. The primary key will be the DN and delegation ID.
3. The submit method should be called as normal. The server will compute the relevant delegation ID (and DN) and add this information to the job description, so that subsequent agents know which credential to use for this job.
4. The FTS will make no attempt to update the credential on its own.
5. All other web-service methods work as normal.
6. The delegated credential may be renewed by the client by re-delegating the credential, the JRA3 delegation port type interface allows for this explicitly. The delegation ID remains the same, the new credential simply updates the old one in the DB.
7. When serving the job, the relevant agent will use the credential it finds in the database which it gets from the delegation ID specified with the job.

Work to be done to use Delegation

• The schema needs to be changed to add a t_cred table holding the user credential blob, the hashed delegationID, the DN and VOMS attributes. The job table needs to be updated to add the key to this table.
• The web-service should implement the delegation port-type, so that the it can get the client credential - it should then place this is the database for use by the agents or for subsequent renewal. The web-service should be backwards compatible, so as to allow old clients to pass the MyProxy password with the job as before (this behaviour can be deprecated with an option to switch it off when all the clients have been updated to the new version).
• The current agents should be updated to use the credential from the database, rather than retrieve it from MyProxy and use it from disk.
• The CLI submission tool should be updated to attempt a delegation if the server version is >=1.5, otherwise it should replicate the old behaviour.

Issues with using MyProxy in authorized_renewer mode

In order to move to the authorized_renewer mode, the FTS job submission must be accompanied by credential delegation as well, so that work would need to be done anyway. We describe below how the components would need to be modified to make this use case work as well and what would need to be done to ensure backwards compatibility with existing client code. We argue that this mode of operation actually breaks the Service Oriented Architecture design paradigm in gLite and it introduces too much additional complexity.

In the authorized_renewer mode, the proxy update mechanism needs to be changed from the authorized_retriever mode to cope with the fact that the renewal needs to be done while the original proxy is still valid. In the current version of FTS (1.3, 1.4), since we have a permanent-lived credential (the user's MyProxy password) we can pull the credential as and when it is need from MyProxy, regardless of whether the previous one we used has expired or not. In the new mode of operation, we would need a live process that scans the credentials and updates them for users who still have pending jobs in the system.

Client flow

For the clients, the flow in the authorized_renewer case would be as follow:

1. The submitting client should check the interface version and the presence of the delegation port-type. If it is the old server version (<1.4) or the delegation port-type is switched off, proceed with a 1.3/1.4 style job submission: ask the client for the MyProxy password and submit the job as before.
2. If the delegation port-type is available, then first call the delegation method to get the client's proxy on the server side. This should be stored in the database using a delegation ID that is the hash of the user's DN and VOMS credential attributes. The primary key will be the DN and delegation ID.
3. The submit method should be called as normal. Since it will be called without a MyProxy password, also in this case the server will compute the relevant delegation ID (and DN) and add this information to the job description, so that subsequent agents know which credential to use for this job.
4. The MyProxy server may be specified in the transfer job parameters. If it is not specified, the FTS will make no attempt to update the credential.
5. All other web-service methods work as normal.
6. The delegated credential will have to be renewed as necessary by the credential renewing 'process' provided a MyProxy server was specified in the job (and provided that client has uploaded a suitable credential to that MyProxy server). It also will need to be redecorated with all VOMS attributes.
7. When serving the job, the relevant agent will use the credential it finds in the database which it gets from the delegation ID specified with the job.

The relevant points that differ to the delegation case are points 4 and 6. Point 6 explains the additional complexity: there needs to be a 'renewal agent' process running all the time, either a daemon or cron, which scans the database, and renews credentials that are due to expire soon and that belong to users that still have pending or active jobs in the system. This would need to be decoupled from the main transfer and VO agents since it will be necessary to keep it running while maintenance is performed on the other agents.

Downtime

The necessity to renew credentials has issues that are well known with the resource broker. Namely, if the service that is updating the credentials goes down for longer than the update period of the credentials, some of the credentials will expire before they are renewed (making it impossible to renew them subsequently) and consequently the jobs will fail.

There are two classes of downtime: planned and unplanned.

Planned downtime might be handled by providing a mechanism to update all the existing credentials now for a period longer than the planned downtime. This procedure should be run before taking the service down for maintenance. The alternative or complementary solution is to factorize out the code that handles the credential renewal from the rest of the code such that it can remain running separately while maintenance is performed on the rest of the system.

Unplanned downtime is harder to guard against. The best option here would be factorize the credential renewal code from the rest of the running code, such that problems in the main part of the code will not affect the credential renewal part of the service. This factorization could be exploited to set up redundant renewal processes to guard against unplanned downtime.

Duplication of functionality and breaking SOA

This proxy renewal mechanism is already implemented exactly like described here in WMS, i.e. we would need to duplicate the same mechanism in the FTS. The proper Service Oriented Architecture would be to have an external service doing the renewal and re-delegate to WMS and FTS on behalf of the user instead of having to write a 'renewal agent' in each service that may need proxy renewal. The user may decide to use such a renewal service or not. A re-delegation renewal 'service' may also be implemented by application portals if necessary, as many of them already store the user's credentials already.

Also, as a minor point, calling the MyProxy service in order to renew the certificate on the user's behalf automatically in the FTS is second-guessing the user's intentions. If a user does not want to use MyProxy renewal, he needs to submit explicitly an empty MyProxy server field in the job or remove the certificate from the MyProxy server. Having an external service is easier, it will simply stop re-delegating new proxies if the user wishes to do so.

The left-hand side shows how the proxy renewal would need to be done individually by each service, complicating each implementation unnecessarily. On the right-hand side there is a renewal service which can be told to renew a given delegation using a specified myproxy for a certain amount of time.

In both cases first the user has to issue voms-proxy-init (not on the picture).

Flow in the case of MyProxy renewal built into the services as an Agent

1. The user puts a long-lived proxy into myproxy using myproxy-init
2. The user calls the service (job submit) delegating the VOMS proxy.
3. The service has to make sure that the proxy is renewed by calling MyProxy
4. And by redecorating it through VOMS.

Flow in the pure delegation case with an external Renewal Service

1. The user puts a long-lived proxy into myproxy using myproxy-init.
2. The user calls the service (job submit) delegating the VOMS proxy.
3. The user calls the Renewal Service and instructs it to automatically renew the certificate in the desired service through redelegation.
4. The Renewal Service will renew the proxy through MyProxy
5. Redecorate from VOMS
6. Redelegate the renewed certificate to the service.

Advantages of the external Renewal Service as opposed to a service-internal Renewal Agent

• Service reuse. The functionality only needs to be written and maintained in one place.
• The renewal service may be deployed differently from the actual services, even on a user's laptop.
• The client has the freedom to choose to use the proxy renewal service or not, if not, all the interaction reduces to a single step (point B).
• The load on the MyProxy servers is diminished.

Maintainer: GavinMcCance

Last update: MichailSalichos on 2014-06-26 - 16:29