Configuring data access via the xroot protocol on DPM for ALICE

This wiki describes version 2.2.0-1, which can be used with DPM >= v1.7.0 and replaces the previous DPM-xrootd series 2.1.x.

Changes

This release of the plugin is built against xrootd-server 20100510.1509.

The plugin is based on xrootd rpms provided by IT-DSS at CERN (see below on where to get these from).

The kXR_locate implementation has been fixed.

Quick overview of installation steps

You need the following packages:

DPM-xrootd

DPM-xrootd is available via the usual channels in gLite (glite.org) as a part of the glite-SE_dpm_mysql and glite-SE_dpm_disk metapackages.

xrootd-server

Get this for SL5/64 from:
http://egee-jra1-data.web.cern.ch/egee-jra1-data/DPM-xrootd_22/sl5/xrootd-server-20100510.1509-1.x86_64.rpm
and for SL4/32 from:
https://project-arda-dev.web.cern.ch/project-arda-dev/xrootd/rpms/slc4/i386/xrootd-server-20100510.1509-1.i386.rpm

xrootd-alicetokenacc & tokenauthz

Currently available only for sl5/64 bit, from http://egee-jra1-data.web.cern.ch/egee-jra1-data/DPM-xrootd_22/sl5

Install, configure and start on every DPM node

Using YAIM

Remove the xrootd RPM (the name of the RPM which previously contained the server) if necessary. Remove xrootd-alice-security if necessary. Install the xrootd-server-20100510.1509-1, xrootd-alicetokenacc-1.1.0-4, tokenauthz-1.1.5-2 (and DPM-xrootd if not already installed).

Define

DPM_XROOTD="yes"

in your site-info.def or services/glite-se_dpm_mysql and services/glite-se_dpm_disk. Remove /opt/lcg/etc/xrd.dpm.cf, /opt/lcg/etc/xrd.dpm.cf, /opt/lcg/etc/xrd.authz.cnf if they exist. Rerun YAIM on all DPM nodes, it will recreate the configuration files as necessary using updated templates and start the xrootd services.

Manually

An example session to install configure and start the DPM xrootd components on a DPM node could look like:

su - root
rpm -e xrootd xrootd-alice-security # if necessary
rpm -Uvh xrootd-server-20100510.1509-1.x86_64.rpm DPM-xrootd-2.2.0-1sec.sl5.x86_64.rpm
rpm -Uvh xrootd-alicetokenacc-1.1.0-4.x86_64.rpm tokenauthz-1.1.5-2.x86_64.rpm
cp /etc/sysconfig/dpm-xrd.templ /etc/sysconfig/dpm-xrd
cp /opt/lcg/etc/xrd.dpm.cf.templ /opt/lcg/etc/xrd.dpm.cf
cp /opt/lcg/etc/xrd.authz.cnf.templ /opt/lcg/etc/xrd.authz.cnf
vi /etc/shift.conf
vi /etc/sysconfig/dpm-xrd
vi /opt/lcg/etc/xrd.dpm.cf
vi /opt/lcg/etc/xrd.authz.cnf
/sbin/service dpm-xrd start
/sbin/service dpm-cms start
/sbin/service dpm-manager-cms start # head node only
/sbin/service dpm-manager-xrd start # head node only

Configuration details

The xrootd-server package contains the standard xroot binaries. The xrootd-alicetokenacc package contains ALICE specific authentication keys and libraries, which themselves require the tokenauthz package.

The DPM-xrootd package contains DPM specific libraries that the standard xrootd binaries will use, four symbolic links for /opt/lcg/bin, some header files, system startup scripts, log rotate script and configuration templates. The configuration for the xroot components is held in three files, one in /etc/sysconfig/ and the other two in /opt/lcg/etc/. In addition a change is needed for the DPM itself in /etc/shift.conf on the head node:

DPM PROTOCOLS rfio gsiftp https xroot

For a default setup (see below) the three xroot configuration files:

The template /etc/sysconfig/dpm-xrd.templ is suitable for use without changes.
The template /opt/lcg/etc/xrd.dpm.cf.templ is suitable for use without changes.
The template /opt/lcg/etc/xrd.authz.cnf.templ needs the line:

EXPORT PATH:/dpm/site.name/home/alice/ VO:*     ACCESS:ALLOW CERT:*

to be changed to reflect the common site SURL leading path for ALICE, e.g.

/dpm/cern.ch/home/alice

The two ALICE key files referenced in xrd.authz.cnf.templ are installed in /opt/lcg/etc/xrootd/ from the xrootd-alicetokenacc RPM. The xroot startup scripts will change the ownership of the two key files to the DPM user (usually dpmmgr). The startup scripts do not edit and copy the configuration files, unlike earlier versions.

Finally the service should be started:

/sbin/service dpm-xrd start
/sbin/service dpm-cms start
/sbin/service dpm-manager-cms start
/sbin/service dpm-manager-xrd start

The RPMs and configuration should be installed on all the DPM machines: the final configuration files should be identical between all the machines. The dpm-manager-cms and dpm-manager-xrd services only need to be started on the DPM head node, starting them on the disk servers will only give an error message.

Notes about upgrading from the previous version

2.1 to 2.2

The 2.2 version of the plugin is based on xrootd rpms provided by IT-DSS at CERN. The plugin will not work with the older server (eg xrootd-20090729). The new xrootd-sever (i.e. xrootd-server-20100510) package only contains the xrootd components, in particular it does not include the components called:

tokenauthz
xrootd-alicetokenacc

which must be installed separately. The new server rpm is also packaged differently and files have been relocated. The older xrootd server package is not obsoleted and it is in fact possible to install the two versions in parallel (not recommended).

The prefixes of the libraries supplied by tokenauthz and xrootd-alicetokenacc have changed (to /usr/lib or /usr/lib64), to be consistent with the new xrootd-server package. The contents of the package xrootd-alice-security which used to be needed are now included in xrootd-alicetokenacc, therefore xrootd-alice-security needs to be removed, if installed.

2.0 to 2.1

Before installing the new version stop the services of the previous version. (i.e. dpm-xrd, dpm-manager-xrd, dpm-olb, dpm-manager-olb). In the new version the service 'olb' is replaced with one called 'cms'.

The set of RPMs associated with the xroot installaiton has been reduced: In particular the RPM containing the xrootd distribution (xrootd-20090729) also includes the components previously available in:

tokenauthz
xrootd-apmon

while the library previously provided by the RPM xrootd-tokenauthzofs is no longer used at all. There is an explicit dependency on libxml2, no dedicated xrootd-libxml2 is provided. Where appropreate the xrootd-XXX RPM should 'obsolete' the relevant RPMs, thus causing them to be removed when xrootd is installed.

Some of the configuration files have changed name and location:

(1) /etc/xrd.dpm.config
(2) /etc/xrd.dpm.config.gsi
(3) /opt/lcg/etc/xrootd/authz.cf

Configuration file (1) was previously used by the service startup scripts to generate a configuration in /opt/lcg/etc/ evey time the service was started. (1) has been effectively moved to

/opt/lcg/etc/xrd.dpm.cf

and it is not automatically rewritten. (2) is removed - the current (and indeed previous) version offers no GSI authentication. (3) is provided by xrootd-alice-security and is still present in an installation but is not used; the configuration file

/opt/lcg/etc/xrd.authz.cnf

is used instead. The public and private ALICE keys are still provided by xrootd-alice-security and installed and used from the previous location.

The confiruation file:

/etc/sysconfig/dpm-xrd

remains in the same location (with a template now provided with '.templ' suffix). However the content has changed. It is recommended that one save a copy of the previous version and then replace it with the new template.

About the default setup and alternative configuration options

By default files are created as owned by root and once xrootd access control is satisfied DPM access is also granted as if the DPM user is root. The storage type for files is P (permanent) by default and by explicit mention in xrd.dpm.cf. There is the possibility of setting an automatic site prefix but this is not enabled by default.

With version 2.1.3 onwards the effective virtual user and groups may be changed, which may be useful for a couple of reasons. i.e.

(1) for access control, to prevent xroot users reading files in the storage element which with any other access method would not be readable by them
(2) to allow proper pool selection when a pool has been restricted to ALICE group(s), which also causes it to be used preferentially to generic pools.
Setting the username and groups requires at least plugin 2.1.3.

The virtual user and groups are set in /opt/lcg/etc/xrd.dpm.cf and should be set the same way on all DPM nodes. The xroot daemons must be restarted after setting the names for the change to take effect. e.g.

dpm.principal alicexroot
dpm.voname alice
dpm.fqan alice
dpm.fqan alice/Role=production

only one value is allowed per line and they do not need to be quoted even if they contain white space. The above sets the DPM virtual username with which all xrootd access is made to 'alicexroot'. The voname ('alice' in the above) should be set to the name of the VO. Two groups would be associated, 'alice' and 'alice/Role=production'. It is usual that one of the groups is the same as the voname. The order of the groups is important, the first is called the primary group and as in unix it is this group to which files created by the user will belong to by default.

The virtual username and groups will be created if they do not already exist. One can use dpns-listusrmap and dpns-listgrpmap to list the existing names. However the username must exist in /opt/lcg/etc/lcgdm-mapfile, so should either be the DN of a user already listed as being a member of the VO, or the user must be added to /opt/lcg/etc/lcgdm-mapfile-local on the DPM head node. e.g. with the above choice

"alicexroot" alice

would need to be entered in /opt/lcg/etc/lcgdm-mapfile-local on the head node, which will cause it to be appended to lcgdm-mapfile the next time the lcgdm-mkgridmap cron job is run.

Care should be taken if changing the username or groups on a system which already has files. Removing groups or changing the username could stop users reading their files or writing to directories to which they previously relied on having access to.

Edit | Attach | Watch | Print version | History: r11 < r10 < r9 < r8 < r7 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r11 - 2011-03-17 - DavidSmith
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback