Installation of EOS FST for federated cloud

Test log

Tested on

CentOS 6.7

-- EygeneRyabinkin

Installation process

Base system and repositories

Standard software repositories:

• RHEL 6.x (CentOS, Scientific Linux, etc)
• EPEL
• CA roots, http://grid-deployment.web.cern.ch/grid-deployment/glite/repos/3.2/lcg-CA.repo

cat << EOF > /etc/yum.repos.d/eos.repo
[eos-aquamarine]
name=EOS aquamarine, modern location
baseurl=https://dss-ci-repo.web.cern.ch/dss-ci-repo/eos/aquamarine/tag/el-$releasever/$basearch/
gpgcheck=0
enabled=1
priority=45

[eos-aquamarine-depends]
name=EOS aquamarine, dependencies
baseurl=https://dss-ci-repo.web.cern.ch/dss-ci-repo/eos/aquamarine-depend/el-$releasever-$basearch/
gpgcheck=0
enabled=1
priority=45
EOF

Scientific Linux and its derivatives have major.minor $releasever, so for these OS variants we should hardcode mainline version into repo files:

cat << EOF > /etc/yum.repos.d/eos.repo
[eos-aquamarine]
name=EOS aquamarine, modern location
baseurl=https://dss-ci-repo.web.cern.ch/dss-ci-repo/eos/aquamarine/tag/el-6/$basearch/
gpgcheck=0
enabled=1
priority=45

[eos-aquamarine-depends]
name=EOS aquamarine, dependencies
baseurl=https://dss-ci-repo.web.cern.ch/dss-ci-repo/eos/aquamarine-depend/el-6-$basearch/
gpgcheck=0
enabled=1
priority=45
EOF

Use yum-priorities plugin and make EOS repository priority higher than EPEL one (default priority is 99, lower number gives more priority).

Packages

Install packages:

yum install -y eos-server eos-client eos-nginx eos-fuse eos-test eos-apmon eos-cleanup jemalloc nscd

Authentication between MGM and FST

Install EOS keytab: to be done by central team. Keytab ownership/mode must be tweaked:

chmod 400 /etc/eos.keytab
chown daemon:daemon /etc/eos.keytab

Firewall

Firewall configuration:

• FST allows incoming connections to the port 1095 from the world: it is the main client port for getting the data

EOS FST configuration

Edit standard SysV -init script configuration:

cat << EOF > /etc/sysconfig/eos
XRD_ROLES="fst"
export EOS_MGM_ALIAS="muon.grid.kiae.ru"
export EOS_MGM_URL="root://${EOS_MGM_ALIAS}:1097"
export EOS_BROKER_URL="root://${EOS_MGM_ALIAS}:1097//eos"
APMON_INSTANCE_NAME=ALICE::KIAE-RUFED::EOS_xrootd
MONALISAHOST=aliendb2.cern.ch
EOF

You can customize EOS_MGM_ALIAS to your taste and project requirements.

Filesystems and registration at MGM

All filesystems where data will be kept are to be mounted under /eosXY, like

/dev/cciss/c0d1 /eos01 xfs defaults 0 0
/dev/cciss/c0d2 /eos02 xfs defaults 0 0
/dev/cciss/c0d3 /eos03 xfs defaults 0 0

One can use any filesystem, but XFS and ext4 are preferred.

The script that will mark your filesystems for EOS:

#!/bin/sh

GROUP=default
OWNER=daemon:daemon

mark_fs() {
[ -z "$1" ] && return
if ! [ -d "$1" ]; then
echo "No such directory '$1'." >&2
exit 1
fi

UUID_FILE="$1"/.eosfsuuid
if [ -e "$UUID_FILE" ]; then
uuid=$(cat "$UUID_FILE")
else
uuid=$(uuidgen)
echo "$uuid" > "$UUID_FILE"
chown "$OWNER" "$UUID_FILE"
fi
echo "eos -b fs add $uuid $(hostname --fqdn):1095 $1 default.${2:-0} rw"
}

group=0
for i in "$@"; do
group=$(($group + 1))
mark_fs "$i" "$group"
done

Save it and run, it will give you the commands to be executed on MGM nodes to register your filesystems,

# /root/mark-eos-fs.sh /eos01 /eos02 /eos03
eos -b fs add 1148556e-5a92-41db-a504-767956d4be7c xr15.grid.kiae.ru:1095 /eos01 default.1 rw
eos -b fs add abe8e289-ab55-44bf-83da-6a338e54dfeb xr15.grid.kiae.ru:1095 /eos02 default.2 rw
eos -b fs add 1bd2d034-0026-4784-b2ea-1e3f67538676 xr15.grid.kiae.ru:1095 /eos03 default.3 rw

After your filesystems will be registered in MGMs, one should restart EOS services:

service eos restart

remarks

• On MGM sever:
  • Change node status

# eos node set $fsthostname on

• • Open tcp ports 1097, 1096 for new fst

• On FST srver
  • check output of command "hostname"
  • Add to file /etc/sysconfig/eos line

export EOS_GEOTAG="DOMAIN NAME"

• • BUG: install libmicrohttpd from eos repo by hand:

yum -y install https://dss-ci-repo.web.cern.ch/dss-ci-repo/eos/aquamarine-depend/el-6-x86_64/libmicrohttpd-0.9.38-eos.slc6.x86_64.rpm

-- EygeneRyabinkin - 2016-03-04