Configuring data access via the xroot protocol on DPM for ALICE

This wiki describes version 2.1.3-1, which can be used with DPM >= v1.7.0 and replaces the previous DPM-xrootd series 2.0.x. The release of DPM version 1.7.3 replaced the packaged DPM-xrootd v2.0.3-1 with 2.1.0.

Quick overview of installation steps

Get the three RPMs (xrootd, xrootd-alice-security, DPM-xrootd) relevant for your platform here:
http://egee-jra1-data.web.cern.ch/egee-jra1-data/DPM-xrootd_21/

The DPM-xrootd is usually included in a gLite release of the DPM; but without the other two RPMs it is not used. The version described here is 2.1.3-1, which is a later patch version than is included in any current gLite release. (It will become the included version as of the next DPM release)

Install, configure and start on every DPM node

Assuming no configuration tool, such as YAIM, an example session to install configure and start the DPM xrootd components on a DPM node could look like:
su - root
rpm -Uvh xrootd-20090729.0855-1.x86_64.rpm xrootd-alice-security-1.0.2-3.x86_64.rpm DPM-xrootd-2.1.3-1sec.slc4.x86_64.rpm
cp /etc/sysconfig/dpm-xrd.templ /etc/sysconfig/dpm-xrd
cp /opt/lcg/etc/xrd.dpm.cf.templ /opt/lcg/etc/xrd.dpm.cf
cp /opt/lcg/etc/xrd.authz.cnf.templ /opt/lcg/etc/xrd.authz.cnf
vi /etc/shift.conf
vi /etc/sysconfig/dpm-xrd
vi /opt/lcg/etc/xrd.dpm.cf
vi /opt/lcg/etc/xrd.authz.cnf
/sbin/service dpm-xrd start
/sbin/service dpm-cms start
/sbin/service dpm-manager-cms start
/sbin/service dpm-manager-xrd start

Configuration details

The xrootd-XXXXX and xrootd-alice-security RPMs provide standard xroot binaries and ALICE specific keys only. The xrootd-alice-security RPM does not contain binaries despite having an architecture specification in the name.

The DPM-xrootd RPM contains DPM specific libraries that the standard xrootd binaries will use, four symbolic links for /opt/lcg/bin, some header files, system startup scripts, log rotate script and configuration templates. The configuration for the xroot components is held in three files, one in /etc/sysconfig/ and the other two in /opt/lcg/etc/. In addition a change is needed for the DPM itself in /etc/shift.conf on the head node:

/etc/shift.conf needs to be changed on the DPM head node to add or modify the PROTOCOLS clause to include 'xroot', e.g.

DPM PROTOCOLS rfio gsiftp https xroot

For a default setup (see below) the three xroot configuration files:

The template /etc/sysconfig/dpm-xrd.templ is suitable for use without changes.
The template /opt/lcg/etc/xrd.dpm.cf.templ is suitable for use without changes.
The template /opt/lcg/etc/xrd.authz.cnf.templ needs the line:

EXPORT PATH:/dpm/site.name/home/alice/ VO:*     ACCESS:ALLOW CERT:*

to be changed to reflect the common site SURL leading path for ALICE, e.g.

/dpm/cern.ch/home/alice

The two ALICE key files referenced in xrd.authz.cnf.templ are installed in /opt/lcg/etc/xrootd/ from the xrootd-alice-security RPM. The xroot startup scripts will change the ownership of the two key files to the DPM user (usually dpmmgr). The startup scripts do not edit and copy the configuration files, unlike earlier versions.

Finally the service should be started:

/sbin/service dpm-xrd start
/sbin/service dpm-cms start
/sbin/service dpm-manager-cms start
/sbin/service dpm-manager-xrd start

The RPMs and configuration should be installed on all the DPM machines: the final configuration files should be identical between all the machines. The dpm-manager-cms and dpm-manager-xrd services only need to be started on the DPM head node, starting them on the disk servers will only give an error message.

Notes about upgrading from the previous version

Before installing the new version stop the services of the previous version. (i.e. dpm-xrd, dpm-manager-xrd, dpm-olb, dpm-manager-olb). In the new version the service 'olb' is replaced with one called 'cms'.

The set of RPMs associated with the xroot installaiton has been reduced: In particular the RPM containing the xrootd distribution (xrootd-20090729) also includes the components previously available in:

tokenauthz
xrootd-apmon

while the library previously provided by the RPM xrootd-tokenauthzofs is no longer used at all. There is an explicit dependency on libxml2, no dedicated xrootd-libxml2 is provided. Where appropreate the xrootd-XXX RPM should 'obsolete' the relevant RPMs, thus causing them to be removed when xrootd is installed.

Some of the configuration files have changed name and location:

(1) /etc/xrd.dpm.config
(2) /etc/xrd.dpm.config.gsi
(3) /opt/lcg/etc/xrootd/authz.cf

Configuration file (1) was previously used by the service startup scripts to generate a configuration in /opt/lcg/etc/ evey time the service was started. (1) has been effectively moved to

/opt/lcg/etc/xrd.dpm.cf

and it is not automatically rewritten. (2) is removed - the current (and indeed previous) version offers no GSI authentication. (3) is provided by xrootd-alice-security and is still present in an installation but is not used; the configuration file

/opt/lcg/etc/xrd.authz.cnf

is used instead. The public and private ALICE keys are still provided by xrootd-alice-security and installed and used from the previous location.

The confiruation file:

/etc/sysconfig/dpm-xrd

remains in the same location (with a template now provided with '.templ' suffix). However the content has changed. It is recommended that one save a copy of the previous version and then replace it with the new template.

Testing

To add: recommendation for testing.

About the default setup and alternatives

The default configuration is similar to the operation of that of the DPM-xrootd 2.0.3. Files are created as owned by root and once xrootd access control is satisfied DPM access is also granted as if the DPM user is root. However unlike in version 2.0.3 the storage type for files is P (permanent) by default and by explicit mention in xrd.dpm.cf. The new automatic site prefix addition is not enabled.

With version 2.1.3 onwards the effective virtual user and groups may be changed, which may be useful for a couple of reasons. i.e.

(1) for access control, to prevent xroot users reading files in the storage element which with any other access method would not be readable by them
(2) to allow proper pool selection when a pool has been restricted to ALICE group(s), which also causes it to be used preferentially to generic pools.
Setting the username and groups requires at least plugin 2.1.3.

The virtual user and groups are set in /opt/lcg/etc/xrd.dpm.cf and should be set the same way on all DPM nodes. The xroot daemons must be restarted after setting the names for the change to take effect. e.g.

dpm.principal alicexroot
dpm.voname alice
dpm.fqan alice
dpm.fqan alice/Role=production

only one value is allowed per line and they do not need to be quoted even if they contain white space. The above sets the DPM virtual username with which all xrootd access is made to 'alicexroot'. The voname ('alice' in the above) should be set to the name of the VO. Two groups would be associated, 'alice' and 'alice/Role=production'. It is usual that one of the groups is the same as the voname. The order of the groups is important, the first is called the primary group and as in unix it is this group to which files created by the user will belong to by default.

The virtual username and groups will be created if they do not already exist. One can use dpns-listusrmap and dpns-listgrpmap to list the existing names. However the username must exist in /opt/lcg/etc/lcgdm-mapfile, so should either be the DN of a user already listed as being a member of the VO, or the user must be added to /opt/lcg/etc/lcgdm-mapfile-local on the DPM head node. e.g. with the above choice

"alicexroot" alice

would need to be entered in /opt/lcg/etc/lcgdm-mapfile-local on the head node, which will cause it to be appended to lcgdm-mapfile the next time the lcgdm-mkgridmap cron job is run.

Care should be taken if changing the username or groups on a system which already has files. Removing groups or changing the username could stop users reading their files or writing to directories to which they previously relied on having access to.

Components and versions included in xrootd-20090729.0855-2

As well as the xrootd distribution the RPM package also includes:

libtokenauthz v1.1.6
xrdapmon v1.0.1
alicetokenacc v1.1.0
xrdstartscript v1.4.2
xrdshell v1.0.2
ApMon2 v2.2.6
xrdcpapmonplugin v1.1.0
xrdaggregatingN2N v1.0.0

although not all of these components are used in the dpm xrootd setup. In particular the monalisa monitoring components ("apmon"), while present, are not setup to be used.

The xrootd distribution is that of 20090729.0855, plus some patches. The 'release' (-2) of the RPM has RPATH removed from executables and libraries.

-- DavidSmith - 18-Jan-2010

Edit | Attach | Watch | Print version | History: r19 < r18 < r17 < r16 < r15 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r19 - 2010-01-25 - DavidSmith
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback