Data Management Testbed

Data Management Testbed web interface allows you to deploy virtual machines easily for any testing purposes.

We also have a set of static test machines. Those machines can be virtual machines not managed by vnode but directly with the standard XEN tools. The building system will be on physical machine exept the Solaris ones.

Resources ready

Vnode is a tool to ask VM on demand trough a really user friendly web interface.

Vnode server :

Documentation on Vnode : Vnode documentation

https://lxb7606.cern.ch/vnode/ Vnode DM serveur

Static machines

WMware based static test machines machines (Not under my responsability contact SteveTraylen in case of problems)

* Problem at boot: "System voltage is out of range", need to press F1 to continue.

Not ready

(Not under my responsability contact SteveTraylen in case of problems)

You can find the database connection strings at /afs/cern.ch/project/egee/jra1/datamgt/accounts.

Build machines

Virtual Machines

One can deploy up to five virtual machines on a physical node. All VM hostnames can be used with any physical machines.

VMs are 64 bits just as physical machines, but you can choose either a 32 bit OS image, or a 64 bit one (except for SLC3 which doesn't support 64 bit architecture).

The web interface also points to a page where you can generate OS images. Although it might be useful for your own purposes, one would need to get that image to the physical node, before being able to use that. So I would recommend not using this feature yet.

Installed OS

The VM will boot with a base operating system installed with the following changes applied:

• Initialy only the owner of the VM have SSH root access to it (no root password).
• SSH access is restricted to the CERN intranet. Access to all other ports (TCP or UDP) is restricted to the machines used by our team and to the other peer virtual machines (i.e. you can set up a UI in one VM and a server in another).
• The LCG CA and VOMS certificates are pre-installed on each VM.

Image upgrades (The GT sysadmin maintain a set of Virtual machines OS images)

When an image is booting, it will automatically apply all available updates. But if the image is to old, this step may be quite long, so it's a good idea to regularly upgrade images with latest updates. Doing is very simple:

• Create a new dynamic VM on dmtb.cern.ch, choosing the OS you want
• Once it is accessible (via SSH), it means that all available updates have been applied
• Shut it down: halt
• Log on the corresponding physical machine (as root), and go to the following directory: '/images'
• Check that the VM is off: xm list
• Mount the VM's root partition to '/images/tmp', and go to that directory
• Remove some files: rm -f etc/grid-security/host*.pem etc/ssh/ssh_host* root/.ssh/authorized_keys
  • These files will be automatically created again when image will boot up
• Create the new image: tar zcf ../slc4_2009-10-09.tar.gz *
• Unmount 'umount /images/tmp'
• Destroy the VM on dmtb.cern.ch

ETICS build

Everything is installed to run a basic ETICS build, however you may want to do some customization in ~/.bashrc:

• set CVSROOT to your username
• copy your certificate to the machine
• configure your certificates

An example session on a virtual machine vmgddb0001.cern.ch:

   # copy your certificate
   scp -r $HOME/.globus vmgddb0001.cern.ch:
   # local settings
   export CVSROOT=":gserver:glite.cvs.cern.ch:/cvs/glite"
   export X509_USER_CERT=$HOME/.globus/usercert.pem
   export X509_USER_KEY=$HOME/.globus/userkey.pem
   # install ETICS client
   mkdir -p workspace
   cd workspace
   etics-workspace-setup
   etics-get-project org.glite
   etics-checkout --project-config glite_branch_3_1_0_dev org.glite.data
   etics-build org.glite.data

You can see more details in the DMReleaseProcedure on how to do the actual builds.

FTS 2.1/SLC4

    rm -f /etc/yum.repos.d/glite-dmtb.repo
    rm -f /etc/yum.repos.d/glite-dm.repo
    yum --enablerepo slc4-cernonly install oracle-instantclient-basic oracle-instantclient-sqlplus java-1.5.0-sun java-1.5.0-sun-devel
    yum install glite-FTS_oracle glite-FTA_oracle
    yum clean packages
    export ORACLE_HOME=$(rpm -ql oracle-instantclient-basic | grep 'client$')
    export ORACLE_LOCATION=$(dirname $ORACLE_HOME)
    export ORACLE_CONNECT='(DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = int11r2-v.cern.ch)(PORT = 10121)) (ADDRESS = (PROTOCOL = TCP)(HOST = int11r1-v.cern.ch)(PORT = 10121)) (LOAD_BALANCE = yes) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = int11r.cern.ch)))'
    # i386/vmdm0013.cern.ch:
        export ORACLE_USER=lcg_fts_dev1
        export ORACLE_PASSWORD=...
    # x86_64/vmdm0014.cern.ch:
        export ORACLE_USER=lcg_fts_dev2
        export ORACLE_PASSWORD=...
    cp /opt/glite/yaim/examples/users.conf /opt/glite/yaim/etc/
    cp /opt/glite/yaim/examples/groups.conf /opt/glite/yaim/etc/
    wget https://twiki.cern.ch/twiki/pub/EGEE/DMTestBed/fts-reference.sql
    wget https://twiki.cern.ch/twiki/pub/EGEE/DMTestBed/fts-site-info.def
    for sql in \
        /opt/glite/etc/glite-data-transfer-fts/schema/oracle/oracle-drop.sql \
        /opt/glite/etc/glite-security-delegation-interface/schema/oracle/oracle-drop.sql \
        /opt/glite/etc/glite-data-transfer-fts/schema/oracle/oracle-schema.sql \
        /opt/glite/etc/glite-security-delegation-interface/schema/oracle/oracle-schema.sql \
        fts-reference.sql;
    do
      if [ -r ${sql} ]; then
        sqlplus "${ORACLE_USER}/${ORACLE_PASSWORD}@${ORACLE_CONNECT}" @${sql}
      fi
    done
    # fixing JAVA_HOME and LD_LIBRARY_PATH in /etc/tomcat5/tomcat5.conf
    # fixing /etc/init.d/tomcat5 to use #!/bin/sh
    /opt/glite/yaim/bin/yaim -c -s fts-site-info.def -n FTS2
    /opt/glite/yaim/bin/yaim -c -s fts-site-info.def -n FTA2
    /etc/init.d/transfer-agents start
    service glite-sd2cache start

FTS 2.2/SLC4

    wget http://glite-dm.web.cern.ch/glite-dm/glite-dm.slc4.repo -O /etc/yum.repos.d/glite-dm.repo
    # add JPackage 5 repository too, if not there yet!
    yum --enablerepo slc4-cernonly install oracle-instantclient-basic oracle-instantclient-sqlplus java-1.5.0-sun java-1.5.0-sun-devel
    yum install glite-FTS_oracle glite-FTA_oracle
    yum clean packages
    export ORACLE_HOME=$(rpm -ql oracle-instantclient-basic | grep 'client$')
    export ORACLE_LOCATION=$(dirname $ORACLE_HOME)
    export ORACLE_CONNECT='(DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = int11r2-v.cern.ch)(PORT = 10121)) (ADDRESS = (PROTOCOL = TCP)(HOST = int11r1-v.cern.ch)(PORT = 10121)) (LOAD_BALANCE = yes) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = int11r.cern.ch)))'
    # i386/vmdm0015.cern.ch:
        export ORACLE_USER=lcg_fts_dev3
        export ORACLE_PASSWORD=...
    # x86_64/vmdm0016.cern.ch:
        export ORACLE_USER=lcg_fts_dev4
        export ORACLE_PASSWORD=...
    cp /opt/glite/yaim/examples/users.conf /opt/glite/yaim/etc/
    cp /opt/glite/yaim/examples/groups.conf /opt/glite/yaim/etc/
    wget https://twiki.cern.ch/twiki/pub/EGEE/DMTestBed/fts-reference.sql
    wget https://twiki.cern.ch/twiki/pub/EGEE/DMTestBed/fts-site-info.def
    for sql in \
        /opt/glite/etc/glite-data-transfer-fts/schema/oracle/oracle-drop.sql \
        /opt/glite/etc/glite-security-delegation-interface/schema/oracle/oracle-drop.sql \
        /opt/glite/etc/glite-data-transfer-fts/schema/oracle/oracle-schema.sql \
        /opt/glite/etc/glite-security-delegation-interface/schema/oracle/oracle-schema.sql \
        /opt/glite/etc/glite-data-transfer-fts/schema/oracle/fts_history_pack.sql \
        /opt/glite/etc/glite-data-transfer-fts/schema/oracle/fts_history_body_pack.sql \
        /opt/glite/etc/glite-data-transfer-fts/schema/oracle/fts_purge_pack.sql \
        /opt/glite/etc/glite-data-transfer-fts/schema/oracle/fts_purge_body_pack.sql \
        $(dirname $0)/fts-reference.sql;
    do
      if [ -r ${sql} ]; then
        sqlplus "${ORACLE_USER}/${ORACLE_PASSWORD}@${ORACLE_CONNECT}" @${sql}
      fi
    done
    # fixing JAVA_HOME and LD_LIBRARY_PATH in /etc/tomcat5/tomcat5.conf
    # fixing /etc/init.d/tomcat5 to use #!/bin/sh
    /opt/glite/yaim/bin/yaim -c -s fts-site-info.def -n FTS2
    source /opt/glite/etc/profile.d/grid-env.sh
    glite-transfer-channel-list
    /opt/glite/yaim/bin/yaim -c -s fts-site-info.def -n FTA2
    /etc/init.d/transfer-agents start
    service glite-sd2cache start

Technology Behind

See GDVirtualization for resources on the technology!

On the base OS the following modifications were applied:

SLC4

    cat /etc/yum.repos.d/lcg-ca.repo
        [CA]
        name=CAs
        baseurl=http://linuxsoft.cern.ch/LCG-CAs/current
    yum install lcg-CA
    cat /etc/yum.repos.d/glite-ui.repo
        [glite-ui]
        name=gLite-UI
        baseurl=http://lxb2042.cern.ch/gLite/APT/R3.1-cert/glite-UI/sl4/i386
    yum install lcg-vomscerts
    $EDITOR /etc/rc.local
         /root/install-hostcert.sh
    useradd -m bob
    cp -a .ssh ~bob/
    chown -R bob.bob ~bob/.ssh
    yum groupinstall 'Development Tools'
    yum groupinstall 'Authoring and Publishing'
    yum install ghostscript transfig
    yum groupinstall 'Editors'
    # preparations for ETICS
    yum install pyOpenSSL PyXML
    yum install vim-enhanced vim-X11
    yum install ccache screen createrepo xorg-x11-xauth
    yum upgrade

SL5

yum groupinstall 'Development Tools'
yum install docbook-utils docbook-utils-pdf psutils docbook-style-xsl ghostscript transfig
yum install vim-enhanced vim-X11 vim-common
yum install ccache screen
yum install pyOpenSSL python-log4py PyXML
yum install xorg-x11-xauth
yum install db4-devel expat-devel
yum upgrade

Solaris

    # Install SunStudio IDE
    pkg install ss-dev
    # Install GCC
    pkg install SUNWgcc

Debian 4

The required packages on a base Debian 4 Etch install:

apt-get install vim vim-runtime vim-scripts
apt-get install gcc make autoconf automake1.9 libtool
apt-get install libterm-readline-gnu-perl libterm-readline-perl-perl
apt-get install cvs subversion dput patchutils wdiff 
apt-get install ccache openssl screen
# for Debian packaging
apt-get install build-essential devscripts dh-make lintian fakeroot bzip2
# for ETICS
apt-get install python-psyco python-pyopenssl python-xml python-pysqlite2
apt-get install python-zsi python-dev python2.4-dev sloccount lsb-release
# for VOMS
apt-get install libssl-dev doxygen docbook-xml docbook-xsl xsltproc

You can create a base image for ChrootEnvs by the following commands:

apt-get install debootstrap
mkdir /var/chroot/debian4
debootstrap --arch i386 etch /var/chroot/debian4 http://ftp.ch.debian.org/debian
chroot /var/chroot/debian4
    apt-get install debian-keyring
    apt-get install openssh-server krb5-user
    apt-get install locales
    dpkg-reconfigure locales # added en_US.UTF-8

• VnodeDemo.swf: How to create a VM using Vnode

• VnodeDemo2.swf: How to destroy a VM using Vnode