NOTE: For dCache sites using the dCache native xrootd door, we do not have integration with the TFC currently. Instead, for SAM tests, you should create the following file manually:


This file is only read - none of the new directories created need to be writable. All other sites can follow the directions on this page.

TFC Changes necessary for Xrootd

There needs to be a slight TFC change for Xrootd monitoring. In order to properly test if a site is working with the redirector, we need the ability to force requests to a given site; we accomplish this through crafty TFC mappings.

We request the following LFN:


(where $SITENAME is your CMS site name, such as T2_US_Nebraska) map to the same local file as $LFN. For example, at Nebraska, the following two LFNs are equivalent:


Because each site will use a different prefix, we now have a unique namespace we can query at each site - without having to produce a site-specific dataset!

You can find a copy of your TFC in the COMP CVS in SITECONF/$SITENAME/PhEDEx/storage.xml; you should already be using this in your xrootd.cfg in a line like this:

oss.namelib /usr/lib64/ file:/etc/xrootd/storage.xml?protocol=hadoop

Here, /etc/xrootd/storage.xml is the location of the TFC you will be editing and hadoop is the protocol you'll make the change for.

Making the TFC change

The simplest case is if the protocol you use is not chained to another protocol. In this case, you just need to add a more-specific testing line prior to the default one. Here's the "before" picture for Nebraska:

  <lfn-to-pfn protocol="hadoop" destination-match=".*" path-match="/+store/(.*)" result="/user/uscms01/pnfs/$1"/>

And the "after" picture:

  <lfn-to-pfn protocol="hadoop" destination-match=".*" path-match="/+store/test/xrootd/T2_US_Nebraska/store/(.*)" result="/user/uscms01/pnfs/$1"/>
  <lfn-to-pfn protocol="hadoop" destination-match=".*" path-match="/+store/(.*)" result="/user/uscms01/pnfs/$1"/>

A common usage pattern is to chain specific protocols to a generic one called "direct". For example,

  <lfn-to-pfn protocol="direct" path-match="/+store/(.*)" result="/user/uscms01/pnfs/$1"/>
  <lfn-to-pfn protocol="hadoop" chain="direct" path-match="(.*)" result="dcap://$1"/>

In this case, you'll want to add a new, more-specific "direct" rule:

  <lfn-to-pfn protocol="direct" path-match="/+store/test/xrootd/T2_US_Nebraska/store/(.*)" result="/user/uscms01/pnfs/$1"/>
  <lfn-to-pfn protocol="direct" path-match="/+store/(.*)" result="/user/uscms01/pnfs/$1"/>
  <lfn-to-pfn protocol="hadoop" chain="direct" path-match="(.*)" result="dcap://$1"/>

We strongly recommend keeping only one version of your storage.xml at your site; integrate any changes back into the one used for PhEDEx and CMSSW, and commit it to CVS.

Testing the change

Admittedly, TFC manipulation is somewhat of a black art - even for seasoned CMS admins. You'll want to test things out. After performing the change, restart the xrootd and cmsd services. As root:

service xrootd restart
service cmsd restart

Note: after any configuration changes, get in the habit of restarting both xrootd and cmsd! Then, assuming the JobRobot dataset is at your site, you should get the same results from both of the following commands:

xrdcp -f root://$HOSTNAME//store/mc/JobRobot/RelValProdTTbar/GEN-SIM-DIGI-RECO/MC_3XY_V24_JobRobot-v1/0001/56E18353-982C-DF11-B217-00304879FA4A.root /dev/null
xrdcp -f root://$HOSTNAME//store/test/xrootd/$SITENAME/store/mc/JobRobot/RelValProdTTbar/GEN-SIM-DIGI-RECO/MC_3XY_V24_JobRobot-v1/0001/56E18353-982C-DF11-B217-00304879FA4A.root /dev/null

Similarly, the following command should force a transfer to come from your site:

xrdcp -f root://$SITENAME/store/mc/JobRobot/RelValProdTTbar/GEN-SIM-DIGI-RECO/MC_3XY_V24_JobRobot-v1/0001/56E18353-982C-DF11-B217-00304879FA4A.root /dev/null

Replace with as necessary.

This topic: Main > TWikiUsers > BrianBockelman > CmsXrootdArchitecture > XrootdTfcChanges
Topic revision: r3 - 2012-09-14 - BrianBockelman
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback