Redirectors (site managers) subscription overview

AAA - Make accessible Any data from the existing distributed CMS data cache to scientists Anywhere, Anytime. The resulting system - hierarchy of redirectors - improves the scientific productivity of CMS physicists reach the data, as well as effectiveness of the storage infrastructures deployed among the CMS Tier-1/2/3 worldwide. Access is based on XRoodtD protocol.

prod-transit-diagram.png

The hierarchical subscription of XRootD managers (cmsd process) provides resiliency in data access towards clients. Effectively this means if accessed data are not available at given site where job lands client will be resiliently redirected to the closest data available within the hierarchy of redirectors to the storage server where the data exist.

Configuration file(s)

To configure subscription to the XRootD redirector, following file(s) need to be edited in your site redirector or server:
  • the xrootd config file (typically located in /etc/xrootd/ unless site admins locate it elsewhere; it is also generally named xrootd-clustered.cfg but this can depend on your local configuration).
  • storage.xml: site TFC file.

Please note the '+' in the xrootd-clustered.cfg file, it means the IP is known to be multiply resolved, and hence a multiple connection to the redirector will take place. A restart of all the daemons is needed, refer to this section below when your configuration is ready.

Site allow list

Being part of CMS AAA production federation, we do require your site to be whitelisted, otherwise site manager (cmsd process) won't authenticate in the hierarchy of redirectors. If you experience that or you are completely new site, contact us to specify your domain to be added in allow list in the US regional redirector.

US sites subscription

Editing xrootd config

Locate configuration file xrootd-clustered.cfg on your site redirector or server and point it to cmsxrootd.fnal.gov:1213. This requires following change, if you are already subscribed to xrootd.unl.edu replace the line:
   all.manager meta xrootd.unl.edu:1213 

with this line (or add new line if you haven't configured subscription previously):

   all.manager meta all cmsxrootd.fnal.gov+ 1213

Using site local redirector

This configuration assumes site runs local redirector in front of the site's xrootd servers. Local redirector subscription to regional redirector is recommended hierarchical setup of redirectors, where site's xrootd servers should subscribe to local redirector first. Here example:
   # this is part of xrootd-clustered.cfg file of local redirector (site manager).
   all.role manager 
   all.manager <host_name_of_local_redirector>:1213 
   all.manager meta all cmsxrootd.fnal.gov+ 1213    # this subscribes your local redirector to regional US redirector
And all xrootd servers at a site should subscribe to your local redirector as follows:
   # this is part of xrootd-clustered.cfg file of xrootd server
   all.role server
   all.manager <host_name_of_local_redirector>:1213  # this subscribes server to your local redirector

Does your configuration supports TFC (fallback)?

NOT using local redirector

This configuration assumes site not running local redirector thus expects direct subscription of xrootd servers to the regional redirector. Site admin needs to do slightly different subscription method in the xrootd server configuration file:

   # this is part of xrootd-clustered.cfg file of xrootd server
   all.role server
   all.manager cmsxrootd.fnal.gov+ 1213

When ready, continue with TFC (fallback) configuration.

If you think your site needs different subscription method, please, consult settings at hn-cms-wanaccess@cernNOSPAMPLEASE.ch.

EU/Asia sites subscription

Editing xrootd config

You simply need to point your site redirector or servers (depending whether you have a site redirector) to xrootd-cms.infn.it:1213.

If you are already subscribed to the Bari redirector xroot.ba.infn.it, it generally means substituting a line like

all.manager xrootd.ba.infn.it 1213 # if configuring a server

with

all.manager xrootd-cms.infn.it+ 1213  # if configuring a server

OR

all.manager meta xrootd.ba.infn.it 1213 # if configuring a local redir

with

all.manager meta all xrootd-cms.infn.it+ 1213  # if configuring a local

PLEASE NOTE: now we have introduced italian and french redirectors, in order to partition the traffic in a better way. Italian sites should substitute the 1213 with 1313, french sites with 1413. You may want to contact Andrea Sartirana (FR) or Tommaso Boccali (IT).

In any case, the basic action is substituting xrootd.ba.infn.it with xrootd-cms.infn.it. The latter is a RR-DNS entry, which points also to the old server, so the action should not be disruptive. When ready, continue with TFC (fallback) configuration.

Transitional Federation

Transitional Federation isolates all sites not passing production federation criteria plus integrates T3 sites in the separate federation. Such sites are blocked in the production federation. If your site is not (or blocked) in the allowed list of production federation you can still benefit from the data access within transitional federation if you subscribe into it. Primarily, transitional federation serves data among T3 sites and also provides fallback (one-way) access for those in production who request data present only in the transitional federation. Joining transitional federation is easy to do:

Using local redirector

This configuration assumes site runs local redirector in front of the site's xrootd servers. Local redirector subscription to the redirector in higher level of hierarchy of redirectors is recommended. In this sense, site's xrootd servers should subscribe to local redirector first. Here example:
   # this is part of xrootd-clustered.cfg file of local redirector (site manager).
   all.role manager 
   all.manager <host_name_of_local_redirector>:1213 
   all.manager meta all cms-xrd-transit.cern.ch+ 1213    # this subscribes your local redirector to transitional redirector
And all xrootd servers at a site should subscribe to your local redirector as follows:
   # this is part of xrootd-clustered.cfg file of xrootd server
   all.role server
   all.manager <host_name_of_local_redirector>:1213  # this subscribes server to your local redirector
When ready, continue with TFC (fallback) configuration.

NOT using local redirector

This configuration assumes site not running local redirector thus expects direct subscription of xrootd servers to the transitional top level redirector. Each xrootd server should then use following lines in the xrootd configuration file:

   # this is part of xrootd-clustered.cfg file of xrootd server     # this subscribes your xrootd server directly to transitional redirector
   all.role server
   all.manager cms-xrd-transit.cern.ch+ 1213
When ready, continue with TFC (fallback) configuration.

Special configuration for aliased redirector(s)

Some sites (e.g. PIC or KIT), for the High Availability purpose, want to host two or more local redirectors that would act as a single point of subscription for downstream data servers in the topology. While we do not recommend experiment whith optional configuration that may contribute to various discussions about pros and cons of this setup, we do recommend in such case the following configuration for site managers (redirectors):

Considered scenario: Proper way to achieve redirector redundancy is to have them all in a common DNS alias and then configure the servers to subscribe to all the redirectors in the DNS alias. Let's assume site has DNS alias redir-alias.foo.bar that resolves to the two local redirectors site-redir1.foo.bar and site-redir2.foo.bar.

Configuration in site-redir1.foo.bar should have:

# distinguish all.sitename between managers
# 1st redirector
all.sitename Site-Redirector-1
all.role manager
all.manager meta all cms-xrd-global.cern.ch+ 1098
all.manager redir-alias.foo.bar+ 1213

Configuration in site-redir2.foo.bar should have:

# distinguish all.sitename between managers
# 2nd redirector
all.sitename Site-Redirector-2
all.role manager
all.manager meta all cms-xrd-global.cern.ch+ 1098
all.manager redir-alias.foo.bar+ 1213

So the data servers at the site should have this in the xrootd config:

all.role server
all.manager redir-alias.foo.bar:<port>

Fallback: editing site TFC

This section assumes your site has implemented TFC trick - if you are completely new to TFC changes, you should skip it and follow this document.

To apply xrootd changes make sure modify the fallback according to redirector region you're subscribing to. If you previously you had for EU site something like:

<lfn-to-pfn protocol="xrootd" path-match="/+store/(.*)" result="root://xrootd.ba.infn.it//store/$1"/>

You should change with:

<lfn-to-pfn protocol="xrootd" path-match="/+store/(.*)" result="root://xrootd-cms.infn.it//store/$1"/>

PLEASE NOTE: now we have introduced italian and french redirectors, in order to partition the traffic in a better way. Italian sites should substitute the xrootd-cms.infn.it with xrootd-cms.infn.it:1194, french sites with xrootd-cms.infn.it:1294.

or US site:

<lfn-to-pfn protocol="xrootd" path-match="/+store/(.*)" result="root://xrootd.unl.edu//store/$1"/>

You should change with:

<lfn-to-pfn protocol="xrootd" path-match="/+store/(.*)" result="root://cmsxrootd.fnal.gov//store/$1"/>

(So a simple hostname exchange over an already working configuration.)

When ready, restart and test.

Service restart and testing subscription

Assuming you've done subscription, you should restart your service and perform simple test.

Service restart

   $ service xrootd restart
   $ service cmsd restart

Test redirector subscription

You can verify subscription of your site using this command, e.g. from lxplus.cern.ch (have in mind export X509_USER_PROXY env variable belonging to CMS VO):
$ xrdfs xrootd.unl.edu locate -d -m /store/test/xrootd/T2_US_Nebraska/store/mc/SAM/GenericTTbar/GEN-SIM-RECO/CMSSW_5_3_1_START53_V5-v1/0013/CE4D66EB-5AAE-E111-96D6-003048D37524.root
red-gridftp9.unl.edu:1094 Server ReadWrite 
red-gridftp1.unl.edu:1094 Server ReadWrite 
red-gridftp7.unl.edu:1094 Server ReadWrite 
red-gridftp6.unl.edu:1094 Server ReadWrite 
red-gridftp8.unl.edu:1094 Server ReadWrite 
red-gridftp10.unl.edu:1094 Server ReadWrite 
red-gridftp12.unl.edu:1094 Server ReadWrite 
red-gridftp4.unl.edu:1094 Server ReadWrite 
red-gridftp2.unl.edu:1094 Server ReadWrite 
red-gridftp5.unl.edu:1094 Server ReadWrite 
red-gridftp3.unl.edu:1094 Server ReadWrite 
...
Testing specific redirector and site will look like:
$ xrdfs REDIRECTOR locate -d -m /store/test/xrootd/SITE_NAME/store/mc/SAM/GenericTTbar/GEN-SIM-RECO/CMSSW_5_3_1_START53_V5-v1/0013/CE4D66EB-5AAE-E111-96D6-003048D37524.root
...
...

NOTE: REDIRECTOR is the full redirector hostname you should have subscribed into (either US, EU or transitional redirector). SITE_NAME is your particular site according to the CMS site naming convention (e.g. T2_US_Nebraska if you are testing subscription of Nebraska). Success in the process of subscription is defined by 'Location IP' result which will be your site server hosting the SAM test file. If not (e.g. seeing as result No matching files were found ) there is something wrong and you should contact for further assistance: hn-cms-wanaccess@cernNOSPAMPLEASE.ch.

Also, it may take a while things get propagated through the whole system and thus No matching files were found could be expected if you do xrd REDIRECTOR locateall ... right after the change. Reasonable time to wait would be ~30mins. If inpatient, you can still try xrdcp test:

$ xrdcp -d 1 -f -DIReadCacheSize 0 -DIRedirCntTimeout 60 root://REDIRECTOR//store/test/xrootd/SITE_NAME/store/mc/SAM/GenericTTbar/GEN-SIM-RECO/CMSSW_5_3_1_START53_V5-v1/0013/CE4D66EB-5AAE-E111-96D6-003048D37524.root /dev/null

Support

In case of troubles, contact us at:


This topic: Main > TWikiUsers > BrianBockelman > CmsXrootdArchitecture > RedirectorsSubscription
Topic revision: r21 - 2019-07-16 - MarianZvada
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback