CMS WebDAV protocol: Installation and Testing

Installation (for admins)

WebDAV is ssuported by different storage systems. Please follow links to get instructions for WebDAV installation according to your storage system:

• XRootD [>= v4.10.0] CMS WebDAV/TPC setup instructions
• dCache [>= v5.2] WLCG DOMA WebDAV/TPC config instructions
• DPM [>= v1.14] WLCG DOMA WebDAV/TPC config information, WLCG DPM upgrade task force
• EOS [>= v4.6.8] WebDAV/TPC config instructions
• StoRM [>= v1.3.1] WebDAV/TPC installation and configuration instructions
• ECHO (and probably other CEPH based storage) [xrootd >= v5.1]

Once installed and configured, you can test by yourself the new endpoint with the following instructions.

Testing (for admins)

In the following we list 7 commands that should be used to progressively test the capabilities of a WebDAV endpoint.

N.B. in all cases --cacert and -E should point to a file with an X509 proxy with cms attribute. Also, please make sure you have read/write permissions on the endpoint and path you are attempting to use

1. Testing that the endpoint supports https access

1.1 Write

curl -v -L --capath /etc/grid-security/certificates/  -H 'X-No-Delegate:true' -H 'Credential: none' --cacert /tmp/x509up_u0 -E /tmp/x509up_u0 -T ./1KB_001 https://stormgf2.pi.infn.it:8443/cms/store/temp/user/ddavila/1KB_001

1.2 Read

curl -v -L --capath /etc/grid-security/certificates/  -H 'X-No-Delegate:true' -H 'Credential: none' --cacert /tmp/x509up_u0 -E /tmp/x509up_u0  https://stormgf2.pi.infn.it:8443/cms/store/temp/user/ddavila/1KB_001 -o 1KB_001.out

1.3 Third Party Copy (TPC)

curl -v -L --capath /etc/grid-security/certificates/ -H 'X-No-Delegate:true' -H 'Credential: none' --cacert /tmp/x509up_u0 -E /tmp/x509up_u0  -H 'Source: https://stormgf2.pi.infn.it:8443/cms/store/temp/user/ddavila/1KB_001' -X 'COPY' https://redirector.t2.ucsd.edu:1094/store/user/ddavila/1KB_001_pisa

2. Testing that the endpoint supports tokens

2.1 Request and decode a Token

curl -L --capath /etc/grid-security/certificates/  -H 'X-No-Delegate:true' -H 'Credential: none' --cacert /tmp/x509up_u0 -E /tmp/x509up_u0 -X 'POST' -H 'Content-Type:application/macaroon-request -d {"caveats":["activity:DOWNLOAD,LIST,UPLOAD"], "validity": "PT30M"}' https://stormgf2.pi.infn.it:8443/cms/store/temp/user/ddavila/1KB_001

Depending on the token format returned you can use one of the following tools to decode the token:

Scitokens are composed of 3 strings separated by dots. To decode paste the token in the “encoded” box in the following link: https://demo.scitokens.org/

Example of a scitoken:

eyJhbGciOiJIzI1NiJ9.eyJhdWQiOiJodHRwczpcL1wvMC4wLjAuMDo4NDQzIiwic3ViIjoiREM9Y2gsREM9Y2VybixPVT1PcmdhbmljIFVuaXRzLE9VPVVzZXJzLENOPWRkYXZpbGEsQ049ODE1MTc3LENOPURpZWdvIERhdmlsYSBGb3lvIiwiaXNzIjoiaHR0cHM6XC9cLzAuMC4wLjA6ODQ0MyIsImV4cCI6MTYzMjI2MTk4OSwiYXV0aG9yaXRpZXMiOlsiRlFBTihcL2Ntc1wvUm9sZTUxMXC9DYXBhYmlsaXR5PU5wpIiwiVk8oY21zKSIsIlNBX1JFQUQoY21zKSIsIlg1MDlfc3ViKERDPWNoLERDPWNlcm4sT1U9T3JnYW5pYyBVbml0cyxPVT1Vc2VycyxDTj1kZGF2aWxhLENOPTgxNTE3NyxDTj1EaWVnbyBEYXZpbGEgRm95bykiLCJTQV9XUklURShjbXMpIl19.cAt-pYbRzYnq1rYju3ytyPL2H0HqrwLM2lhkk670

Macaroons are a single string to decode paste the macaroon in the “Verify - Input macaroon” section of the following link: http://macaroons.io/

MDAxOGxvY2F0aW9uIFQyX1VTX1VDU0QKMDAzNGlkZW50aWZpZXIgNzUyOGE3OGUtZWM1MC00NGU4LThjYjQtNmM3YTZlNzEzZTIxCjAwMTVjaWQgTpkZGF2aWxhCjAwNTJjaWQgYWN0aXZpdHk6UkVBRF9NRVRBREFUQSxVUExPQUQsRE9XTkxPQUQsREVMRVRFLE1BTkFHRSxVUERBVEVfTUVUQURBVEEsTElTVAowMDFmY2lkIGFjdGl2aXR5OkRPV05MT0FELExJU1QKMDAyOWNpZCBwYXG9yZS91c2VyL2RkYXZpbGEvMUtCXzAwMQowMDI0Y2lkIGJlZm9yZToyMDIxLTA5LTIxVDIyOjE0OjAwWgowMDJmc2lnbmF0dXJlIH0-OA7lpvp0vMNcxksfQLhaLlm-9RRHWXRFwgJCPOl3Cg

2.2 Write with a token

curl -v -L --capath /etc/grid-security/certificates/ -H 'X-No-Delegate:true' -H 'Credential: none' -H 'Authorization: Bearer MDAxOGxvY2F0aW9uIFQyX1VTX1VDU0QKNGlkZW50aWZpZXIgZTNiNTAwNjktOGE3MS00OTg4LWJmODItYWI5ZjhiYzk5OTU1CjAwMTVjaWQgbmFtZTpkZGF2aWxhCjAwNTJjaWQgYWN0aXZpdHk6UkVBRF9NRVRBREFUQSxVUExPQUQsRE9XTkxPQUEVMRVRFLE1BTkFHRSxVUERBVEVfTUVUQUEEsTElTVAowMDFkY2lkIGFjdGl2aXR5OlVQTE9BRCxMSVNUCjAwMmRjaWQgcGF0aDovc3RvcmUvdXNlci9kZGF2aWxhLzFLQl8wMDFfMDAyCjAwMjRjaWQgYmVmb3JlOjIwMjEtMDktMjFUMjI6MTc6MzVaCjAwMmZzaWduYXR1cmUgK0qNR1SWC5KfHmiyynHFmTqrodBm1ILagyTQ3YUZXJQK' -T ./1KB_001 https://redirector.t2.ucsd.edu:1094/store/user/ddavila/1KB_001_002

2.3 Read with a token

curl -L --capath /etc/grid-security/certificates/  -H 'X-No-Delegate:true' -H 'Credential: none' -H 'Authorization: Bearer MDAxOGxvY2F0aW9uIFQyX1VTX1VDU0QKMDlkZW50aWZpZXIgZWFjY2I0YTgtZDVjMi00OTdiLTkzZTctNmJjYTJlNDRkNTM1CjAwMTVjaWQgbmFtZTpkZGF2aWxhCjAwNTJjaWQgYWN0aXZpdHk6UkVBRF9NRVRBREFUQSxVUExPQUQsRE9XTkxPQUQsREVMRVRFLE1BTkFHRSxVUERBVVUQURBVEEsTElTVAowMDFmY2lkIGFjdGl2aXR5OkRPV05MT0FELExJU1QKMDAyZGNpZCBwYXRoOi9zdG9yZS91c2VyL2RkYXZpbGEvMUtCXzAwMV8wMDIKMDAyNGNpZCBiZWZvcmU6MjAyMS0wOS0yMVQyMjoxM1oKMDAyZnNpZ25hdHVyZSB41lTs5Gjdo7brYTv48CphR5dOIzGxMpZNvMz6IHAfvwo' https://redirector.t2.ucsd.edu:1094/store/user/ddavila/1KB_001-o ./1KB_001

2.4 Third Party Transfer with a token

curl -v -L --capath /etc/grid-security/certificates/ -H 'X-No-Delegate:true' -H 'Credential: none' -H 'TransferHeaderAuthorization: Bearer MDAxOGxvY2F0aW9uIFQyX1VTX1VDU0QKMDAzNGlkZW50aWZpZXIgZWNhODYwYjEtMDQyOC00NGJjLTkxZWQtODdiYzUxZGFlNzUwCjAwMTVjaWQgbmFtZTpkZaWxhCjAwNTJjaWQgYWN0aXZpdHk6UkVBRF9NRVRBREFUQSxVUExPQUQsRE9XTkxPQUQsREVMRVRFLE1BTkFHRSxVUERBVEVfTUVUQURBVEEsTElTVAowMDI2Y2lkIGFjdGl2aXR5OkRPV\T0FELERFTEVURSxMSVNUCjAwMmRjaWQgcGF0aDovc3RvcmUvdXNlci9kZGF2aWxhLzFLQl8wMDFfMDAyCjAwMjRjaWQgYmVmb3JlOjIwMjEtMDktMjFUMjI6MjA6MTNaCjAwMmZzaWduYXR1cmUgnENJ0hNJ7zGl-m_1MhEmk-clAt1T5SF7X9zsKIScISAK’ -H ‘Authorization: Bearer MDAxOGxvY2F0aW9uIFQyX1VTX1VDU0QKMDAzNGlkZW50aWZpZXIgMGNjNTNjMmYtOWRhOC00OTQ4LThmOWEtYzVmMzhkN2QxNmY4CjAwMTVjaWQgbmFtZTpkaWxhCjAwNTJjaWQgYWN0aXZpdHk6UkVBRF9NRVRBREFUQSxVUExPQUQsRE9XTkxPQUQsREVMRVRFLE1BTkFHRSxVUERBVEVfTUVUQURBVEEsTElTVAowMDI0Y2lkIGFjdGl2aXR5OlVQTE9BRCxERUxFVEUsTElTVAowMDJkY2lkIHBhdGb3JlL3VzZXIvZGRhdmlsYS8xS0JfMDAxXzAwNAowMDI0Y2lkIGJlZm9yZToyMDIxLTA5LTIxVDIyOjIwOjE0WgowMDJmc2lnbmF0dXJlIFb_qR3O7GSx7JY1KzkG1jMyoGcD9oUc_7dnJatdSA64Cg' -H 'Source: https://redirector.t2.ucsd.edu:1094/store/user/ddavila/1KB_001' -X 'COPY' https://redirector.t2.ucsd.edu:1094/store/user/ddavila/1KB_001_004

References

1. Set Rucio from Gitlab https://github.com/dmwm/CMSRucio/blob/master/docker/CMSRucioClient/scripts/setRucioFromGitlab

1. Diego Davila’s fork: https://github.com/ddavila0/CMSRucio/tree/add_webDAV

-- Main.FelipeLeonardoGomezCortes - 2021-11-02