Accessing files on SEs from ROOT

Castor and DPM

A bit of history

Following a mail exchange with Flavia and discussion in the Architect's Forum about being able to use a universal libshift.so that would serve Castor and DPM, I have made a few tests.

Just a bit of history for those who didn't follow the saga: at Mumbai (February 2006!!), it was decided to provide with highest priority a universal libshift that would then steer to Castor or DPM library depending on the flavour of the backend. In August 2006 the WLCG MB agreed the proposed timeline to have it as a prototype in October 2006 and released soon afterwards. In the mean time the EGEE TCG (that is defining the priorities of developments in EGEE) decided to set a very low priority on this point and essentially everything stopped (at least officially).

Now the problem is to be able to set up an application that defines an environment suitable for reading either files on a Castor or a DPM backend. The solution used by CMS (?) and ATLAS on sites supporting DPM is to create a soft link (on the LD_LIBRARY_PATH) called libshift.so.2.1 pointing to libdpm.so. Of course this is cumbersome as the environment setting script has to know a priori whether it will access files through DPM or Castor. LHCb doesn't have (yet) the problem as we don't use site with DPM. However NIKHEF proposes us a disk1tape0 storage at NIKHEF (rather than dCache at SARA) based on DPM. Hence before embarking on a migration we have to test it out...

It was suggested by Flavia to go through the ROOT gfal plugin since gfal is able to steer the appropriate library depending on an environment variable LCG_RFIO_TYPE. She kindly set up an SAPath for LHCb tests on a DPM instance on the CERN PPS. Thanks!

I used this opportunity for testing again all tURLs of the creation against ROOT... I used the file /castor/cern.ch/grid/lhcb/production/DC06/phys-v2-lumi2/00001650/DST/0000/00001650_00000055_5.dst that is on the lhcbdata pool (i.e. srm-durable-lhcb.cern.ch) as well as on the default pool (srm.cern.ch). For LFN tests, I used the file lfn:/grid/lhcb/production/DC06/phys-v3-lumi2/00001857/RDST/0000/00001857_00000024_1.rdst that has its replica defined on srm.cern.ch while the former file is registered in the LFc as srm-durable-lhcb.cern.ch

Results of the tests

Originally for this test, I used an SLC3 LXPLUS node, ROOT 5.14.00e and gfal 1.7.7 as installed in the AA. On SLC4, I used a GFAL plugin kindly built by Gerri on May 23rd for ROOT 5.14.00f and gfal 1.9.0 as it didn't make it to the official release.

On June 22nd, I remade the tests with with ROOT 5.14.00f, gfal 1.9.0, DPM 1.6.5 on both SLC3 and SLC4 platforms. The gfal plugin of ROOT has been added by Bertrand to the released 5.14.00f.

/afs/cern.ch/sw/lcg/external/root/5.14.00f/slc3_ia32_gcc323/root/lib
/afs/cern.ch/sw/lcg/external/Grid/gfal/1.9.0/slc3_ia32_gcc323/lib
/afs/cern.ch/sw/lcg/external/Grid/globus/4.0.3-VDT-1.6.0/slc3_ia32_gcc323/globus/lib
/afs/cern.ch/sw/lcg/external/Grid/DPM/1.6.5/slc3_ia32_gcc323//lib
/afs/cern.ch/sw/lcg/external/Grid/dm-util/1.5.1/slc3_ia32_gcc323/lib
/afs/cern.ch/sw/lcg/external/Grid/voms-api-c/1.7.16/slc3_ia32_gcc323/lib
/afs/cern.ch/sw/lcg/external/Grid/LFC/1.6.5/slc3_ia32_gcc323/lib
/afs/cern.ch/sw/lcg/external/Grid/lcg-dm-common/1.6.5/slc3_ia32_gcc323/lib
/afs/cern.ch/sw/lcg/external/Grid/cgsi-gsoap/1.1.15/slc3_ia32_gcc323/usr/lib

/afs/cern.ch/sw/lcg/external/root/5.14.00f/slc4_ia32_gcc34/root/lib
/afs/cern.ch/sw/lcg/external/Grid/gfal/1.9.0/slc4_ia32_gcc34/lib
/afs/cern.ch/sw/lcg/external/Grid/globus/4.0.3-VDT-1.6.0/slc4_ia32_gcc34/globus/lib
/afs/cern.ch/sw/lcg/external/Grid/DPM/1.6.5/slc4_ia32_gcc34//lib
/afs/cern.ch/sw/lcg/external/Grid/dm-util/1.5.1/slc4_ia32_gcc34/lib
/afs/cern.ch/sw/lcg/external/Grid/voms-api-c/1.7.16/slc4_ia32_gcc34/lib
/afs/cern.ch/sw/lcg/external/Grid/LFC/1.6.5/slc4_ia32_gcc34/lib
/afs/cern.ch/sw/lcg/external/Grid/lcg-dm-common/1.6.5/slc4_ia32_gcc34/lib
/afs/cern.ch/sw/lcg/external/Grid/cgsi-gsoap/1.1.15/slc4_ia32_gcc34/usr/lib

The results are identical on SLC3 and SLC4.

Tests on srm-durable-lhcb

Returns a tURL of the form:

rfio://castorlhcb:9002/?svcClass=lhcbdata&castorVersion=2&path=//castor/cern.ch/grid/lhcb/production/DC06/phys-v2-lumi2/00001650/DST/0000/00001650_00000055_5.dst and similarly for the rootd protocol (rfio: replaced by castor:)

  • Direct call: OK
root [6] f=TFile::Open("rfio://castorlhcb.cern.ch:9002/?svcClass=lhcbdata&castorVersion=2&path=/castor/cern.ch/grid/lhcb/production/DC06/phys-v2-lumi2/00001650/DST/00$
(class TFile*)0x9b0a3d0
  • Through gfal: not OK
root [7] f=TFile::Open("gfal:rfio://castorlhcb.cern.ch:9002/?svcClass=lhcbdata&castorVersion=2&path=/castor/cern.ch/grid/lhcb/production/DC06/phys-v2-lumi2/00001650/D$
SysError in : file rfio://castorlhcb.cern.ch:9002/ can not be opened for reading (No such file or directory)
(class TFile*)0x0
  • Through gfal using the SURL: not OK
root [8] f=TFile::Open("gfal:srm://srm-durable-lhcb.cern.ch:8443/castor/cern.ch/grid/lhcb/production/DC06/phys-v2-lumi2/00001650/DST/0000/00001650_00000055_5.dst")
SysError in : file srm://srm-durable-lhcb.cern.ch:8443/castor/cern.ch/grid/lhcb/production/DC06/phys-v2-lumi2/00001650/DST/0000/00001650_00000055_5.dst can not be opened for reading (No such file or directory)
(class TFile*)0x0

Test of gfal 1.9.2: almost OK
After we were told gfal 1.9.2 was fixing the problems observed above, Oliver installed it on lcg/external and the test was performed again and it works better but not quite yet (gfal: doesn't work:
root [4] f=TFile::Open("gfal:srm://srm-durable-lhcb.cern.ch:8443/castor/cern.ch/grid/lhcb/production/DC06/phys-v2-lumi2/00001650/DST/0000/00001650_00000055_5.dst")
(class TFile*)0x9076dd0
root [5] f=TFile::Open("gfal:rfio://castorlhcb:9002/?svcClass=lhcbdata&castorVersion=2&path=//castor/cern.ch/grid/lhcb/production/DC06/phys-v2-lumi2/00001650/DST/0000/$
SysError in : file rfio://castorlhcb:9002/ can not be opened for reading (Connection reset by peer)
(class TFile*)0x0

Tests on srm.cern.ch

Returns a tURL of the form (note the different number of slashes)
rfio:////castor/cern.ch/grid/lhcb/production/DC06/phys-v2-lumi2/00001650/DST/0000/00001650_00000055_5.dst
castor:/castor/cern.ch/grid/lhcb/production/DC06/phys-v2-lumi2/00001650/DST/0000/00001650_00000055_5.dst

  • Direct call: OK
root [9] f=TFile::Open("rfio:////castor/cern.ch/grid/lhcb/production/DC06/phys-v2-lumi2/00001650/DST/0000/00001650_00000055_5.dst")
(class TFile*)0x9e99a20
root [10] f=TFile::Open("castor:/castor/cern.ch/grid/lhcb/production/DC06/phys-v2-lumi2/00001650/DST/0000/00001650_00000055_5.dst")
Info in : fAuthProto = root, u: castor:///castor/cern.ch/grid/lhcb/production/DC06/phys-v2-lumi2/00001650/DST/0000/00001650_00000055_5.dst
(class TFile*)0x9ec4ec8
Note that ROOT accepts the following number of slashes: 1, 3, 4, 5 (but not 2) for both protocols
  • Through gfal: OK for rfio
    • castor: protocol not supported
    • rfio: OK for 3 or 4 slashes (not 1: invalid argument)
root [15] f=TFile::Open("gfal:rfio:////castor/cern.ch/grid/lhcb/production/DC06/phys-v2-lumi2/00001650/DST/0000/00001650_00000055_5.dst")
(class TFile*)0x9958548
  • Through gfal using the SURL: OK
root [2] f=TFile::Open("gfal:srm://srm.cern.ch:8443/castor/cern.ch/grid/lhcb/production/DC06/phys-v2-lumi2/00001650/DST/0000/00001650_00000055_5.dst")
(class TFile*)0x9dec900
  • Through gfal using the LFN: OK if the SURL is on srm.cern.ch
root [1] TFile::Open("gfal:lfn:/grid/lhcb/production/DC06/phys-v3-lumi2/00001857/RDST/0000/00001857_00000024_1.rdst")
(class TFile*)0x8c8d020
root [2] TFile::Open("gfal:lfn:/grid/lhcb/production/DC06/phys-v2-lumi2/00001650/DST/0000/00001650_00000055_5.dst")
SysError in : file lfn:/grid/lhcb/production/DC06/phys-v2-lumi2/00001650/DST/0000/00001650_00000055_5.dst can not be opened for reading (Invalid argument)
(class TFile*)0x0

Tests on lxdpm102.cern.ch

I copied the file to DPM and got a tURL (using the PPS-BDII). SRM v1.1 and SRM v2.2 give the same tURL

[lxplus209] ~ > lcg-cp -v srm://srm-durable-lhcb.cern.ch:8443/castor/cern.ch/grid/lhcb/production/DC06/phys-v2-lumi2/00001650/DST/0000/00001650_00000055_5.dst srm://lxdpm102.cern.ch:8443/dpm/cern.ch/home/lhcb/00001650_00000055_5.dst
Source URL: srm://srm-durable-lhcb.cern.ch:8443/castor/cern.ch/grid/lhcb/production/DC06/phys-v2-lumi2/00001650/DST/0000/00001650_00000055_5.dst
File size: 188734264
Source URL for copy: gsiftp://lxfsra2803.cern.ch//castor/cern.ch/grid/lhcb/production/DC06/phys-v2-lumi2/00001650/DST/0000/00001650_00000055_5.dst
Destination URL: srm://lxdpm102.cern.ch:8443/dpm/cern.ch/home/lhcb/00001650_00000055_5.dst
# streams: 1
# set timeout to  0 (seconds)
            0 bytes      0.00 KB/sec avg      0.00 KB/sec inst
Transfer took 4830 ms
[lxplus209] ~ > lcg-gt  srm://lxdpm102.cern.ch:8443/dpm/cern.ch/home/lhcb/00001650_00000055_5.dst rfio
rfio://lxdpm102.cern.ch//data02/lhcb/2007-05-22/00001650_00000055_5.dst.162540.0

  • Direct call (after setting libshift.so.2.1 as a link to libdpm.so): OK (was not with previous versions of DPM)
root [2] f=TFile::Open("rfio://lxdpm102.cern.ch//data02/lhcb/2007-05-22/00001650_00000055_5.dst.162540.0")
(class TFile*)0xa871540
  • Through gfal and tURL: OK, either as above or without the libshift link by setting LCG_RFIO_TYPE = dpm
root [1] f=TFile::Open("gfal:rfio://lxdpm102.cern.ch//data02/lhcb/2007-05-22/00001650_00000055_5.dst.162540.0")
(class TFile*)0xaaa9d50
  • Through gfal and SURL: OK as above
root [1] TFile::Open("gfal:srm://lxdpm102.cern.ch:8443/dpm/cern.ch/home/lhcb/00001650_00000055_5.dst")
(class TFile*)0x8cb0ce0

Conclusions

  • Accessing files on Castor from ROOT works fine using the protocols rfio: and castor: with the tURLs obtained by lcg-gt on the SRM endpoints (both old-style and new-style tURLs).
  • Using gfal:rfio: for accessing files also works fine on Castor with the tURLs obtained by lcg-gt on srm.cern.ch (rfio: protocol, not castor: protocol), but not the new-style tURLs obtained from srm-durable-lhcb.cern.ch. Note that gfal supports less syntaxes for the tURL as the native RFIO or ROOTD plugins (only one / is not accepted)
  • Using either a libshift.so.2.1 link to libdpm.so or setting the LCG_RFIO_TYPE variable allows us to read a DPM file (in the first case with both rfio: and gfal:rfio:, in the second case only with gfal:rfio:)

Note that ATLAS are conducting similar tests and their results can be found here

-- Main.phicharp - 22 May 2007

Edit | Attach | Watch | Print version | History: r7 < r6 < r5 < r4 < r3 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r7 - 2007-06-25 - PhilippeCharpentierSecondary
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LHCb All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback