Proxy Delegation problem in FTS 2.2

Problem: In some cases, the delegated proxies got corrupted during FTS run. The origin of the problem was a race condition, covered by Savannah bug #33641. The fixed release did not get to FTS 2.2.

Reproducing the problem was not easy, we did it in the following way:

The test that reproduces the race condition and corrupts the delegation is in org.glite.data.delegation-cli/src/tools/test_33641.c. Compilation produces the executable test-glite-delegation-bug-33641. To create corrupted proxy, we executed the following commands (change the host to the one running the buggy service):

export WEB_SERVICE_HOST=lxbrb1410.cern.ch
export WEB_SERVICE_ENDPOINT=https://$WEB_SERVICE_HOST:8443/glite-data-transfer-fts/services/gridsite-delegation
glite-delegation-destroy -v -s $WEB_SERVICE_ENDPOINT
glite-delegation-init -v -s $WEB_SERVICE_ENDPOINT
glite-delegation-init -v -s $WEB_SERVICE_ENDPOINT
glite-delegation-destroy -v -s $WEB_SERVICE_ENDPOINT
test-glite-delegation-bug-33641 -v -f -s $WEB_SERVICE_ENDPOINT

Then, we logged in to lxbrb1410, and checked directly the proxy. We used the follwing sqlplus command:

sqlplus '__database_user_name__/__database__password__@(DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = lcgr3-v.cern.ch)(PORT = 10121)) (ADDRESS = (PROTOCOL = TCP)(HOST = lcgr2-v.cern.ch)(PORT = 10121)) (ADDRESS = (PROTOCOL = TCP)(HOST = lcgr1-v.cern.ch)(PORT = 10121)) (ADDRESS = (PROTOCOL = TCP)(HOST = lcgr4-v.cern.ch)(PORT = 10121)) (ADDRESS = (PROTOCOL = TCP)(HOST = lcgr5-v.cern.ch)(PORT = 10121)) (ADDRESS = (PROTOCOL = TCP)(HOST = lcgr6-v.cern.ch)(PORT = 10121)) (ADDRESS = (PROTOCOL = TCP)(HOST = lcgr7-v.cern.ch)(PORT = 10121)) (ADDRESS = (PROTOCOL = TCP)(HOST = lcgr8-v.cern.ch)(PORT = 10121)) (ENABLE=BROKEN) (LOAD_BALANCE = yes) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = lcg_fts.cern.ch) (FAILOVER_MODE = (TYPE = SELECT)(METHOD = BASIC)(RETRIES = 200)(DELAY = 15)) ) )'

(__database_user_name__/__database__password__ : certainly, we do not write them here ).

and the queried for the proxy in the following way:

set page 999
set long 10000
select * from t_credential where dlg_id='dbc257a65b77841f151e5b9fed950d1daf77ffbf';

The last SQL command required the delegation ID. That is why we executed the glite-delegation-init command TWICE, because the second one displayed it in the error message...

We copied the displayed proxy into a file (/tmp/a in the example), and executed the following commands:

openssl x509 -in /tmp/a -text -noout

                Modulus (512 bit):
                    00:8e:5e:6f:ac:3c:9e:4e:0a:d4:82:d8:10:44:ed:
                    8b:cb:e4:f1:a6:37:d7:bc:b6:e7:99:e7:13:16:a0:
                    c1:a9:e1:76:31:77:97:eb:a1:8b:c8:c8:99:81:04:
                    2e:a6:7f:70:b9:a3:f4:81:ec:fb:8e:a1:8f:1a:07:
                    67:2b:85:d6:21

openssl rsa -in /tmp/a -text -noout

                modulus:
                    00:99:a6:c0:1d:16:8f:ab:5a:84:42:55:04:03:e6:
                    87:b2:fb:e1:ea:12:d1:d2:73:e1:49:1f:ca:17:d4:
                    76:d9:93:22:5d:e4:b2:29:87:96:ee:57:2b:88:ec:
                    97:04:b9:1e:12:81:85:b6:27:a3:34:60:fb:39:d2:
                    69:0b:30:d1:b5

Then we looked for the "modulus" part in the output of the commands. They were different, the proxy got corrupted -> problem reproduced.

-- ZsoltMolnar - 03-Feb-2010