Getting CMSSW to work with Ubuntu (9.04, Jaunty)


Most of us who have used CMSSW have done so on Scientific Linux CERN via LXPLUS or some other cluster of machines set up to look a lot like LXPLUS. Most of us are also not the most comfortable using this distribution because it is based on Red Hat Enterprise Linux, a distribution most people do not have readily available for the home machines, and because the most current version of Scientific Linux CERN typically lags 3-5 years behind other operating systems in terms of support hardware and features. This guide is for those people who are comfortable operating in Ubuntu Linux, one of the most popular Linux distributions for the moment, and want to use CMSSW natively without tunnels or running through ssh connections.

Step 1: Kerberos

Required packages

CERN uses Kerberos as its primary authentication tool. While in some instances you are allowed to authenticate yourself with standard ssh passwords, Kerberos is required to use AFS, which we will install in the next section. You will need to begin by installing the following package:
  • krb5-user
You can install it using the Synaptic Package Manager, or by opening a terminal and issuing the following command:
sudo apt-get install krb5-user

Kerberos configuration (/etc/krb5.conf)

You now need to modify the file /etc/krb5.conf to include the CERN.CH realm so that Kerberos will know how to authenticate you. To do this, open up the file /etc/krb5.conf in your favorite editor (like gedit or vim) and make the following modifications:
  • Below the line that reads "[realms]", add the following block of code:
     CERN.CH = {
      default_domain =
      kpasswd_server =
      admin_server =
      kdc =
      kdc =
      kdc =
  • Below the line that reads "[domain_realm]", add the following lines: = CERN.CH = CERN.CH

Step 2: AFS

AFS is a distributed network filesystem that CERN employs heavily. All of the software for funning CMSSW can be found on the AFS file system, which means once we get AFS installed and working properly, we won't have to install any of the actual CMSSW packages locally. In this way, your Ubuntu machine will mimic a CERN LXPLUS machine, using the same executables and libraries that the LXPLUS machines use, but you won't have to worry about maintaining a separate installation.

Required packages

The following packages are required in order to access CERN AFS:
  • openafs-client
  • openafs-krb5
  • openafs-modules-dkms (optional)
Installing these can be done through Synaptic Package Manager or via the terminal with the command:
sudo apt-get install openafs-client openafs-krb5 openafs-modules-dkms
During the configuration phase of the installation, a prompt will appear asking you what AFS cell this workstation belongs to, and the size of the AFS cache in kB. For the AFS cell, enter Leave the AFS cache size at its default value (unless you feel that you do not have that much space available, in which case decrease the size of the cache.)

Step 3: Testing AFS/Kerberos

Before continuing, we should test to see if everything was installed properly. Open up a terminal and type:
This command will authenticate you via Kerberos with one of the CERN servers. Issuing this command will bring up a prompt for your password. On successfully entering your password, it will appear that nothing has happened. However, if you issue the "klist" command, you should see something like the following appear:
$ klist
Ticket cache: FILE:/tmp/krb5cc_1000
Default principal: paste@CERN.CH

Valid starting     Expires            Service principal
09/09/09 10:49:58  09/10/09 11:49:58  krbtgt/CERN.CH@CERN.CH
   renew until 09/14/09 10:49:58

Kerberos 4 ticket cache: /tmp/tkt1000
klist: You have no tickets cached
This means that you have a Kerberos "ticket" that you can use to log in to other machines without having to authenticate yourself again. Note: This assumes your CERN username is the same as the one you have on your personal machine. If this is not the case, replace $USER above with the username you use to log in to the LXPLUS machines. Also, the capital letters in the realm name are important!

Now we can test if AFS is mounted properly. To do this, simply issue the following command:

ls /afs/
A directory listing should come up. If you are on a slow network, this may take a couple of seconds.

The next thing you will want to do is authenticate yourself using your Kerberos ticket so that you have access to your home directory. To authenticate yourself on AFS with your Kerberos ticket, issue one of the following command:

aklog -c -k CERN.CH
When this is done, you should be able to modify files in your home directory. My home directory on AFS is located at /afs/ when I log in to LXPLUS. You can find your home directory by logging into LXPLUS and issuing the command:
To test to see if everything is working, I would run the following command:
touch /afs/
If you can issue commands like these without any permission denied errors, then everything is working correctly.

Step 4: CVS with Kerberos

The standard CVS package that is distributed with Ubuntu does not have Kerberos support built-in. There are CVS clients available that do have Kerberos enabled, however. The one I use is available via the package:
  • cvsnt
You can install this package via Synaptic Package Manager, or with the command: sudo apt-get install cvsnt

Step 5: Environment variables and aliases (~/.bash_profile)

Many of the tools commonly used in CMSSW require certain environment variables to be set. The most important of these is the PATH variable, which tells the operating system where to search for executable files. Assuming you are using bash as your login shell on your Ubuntu machine (which is the default), open (and create, if it doesn't exist) the file ~/.bash_profile in your favorite editor and add the following line:
export PATH=~/bin:${PATH}
We are going to make a directory called ~/bin and fill it with some modified scripts soon. There are a few commands that should be aliased so that you can follow the numerous tutorials for running CMSSW. Add the following lines to ~/.bash_profile as well:
alias cmsenv='eval `scramv1 runtime -sh`'
alias project='export'
The first line is simply a carbon copy of the alias that you find on LXPLUS machines. The second overrides the "project cmssw" command that would normally set your CVSROOT environment variable to something very similar to this one. The difference is that here, the :gserver: server-type is used, while the CMSSW version of the command uses the :kserver: server-type. :kserver: is for Kerberos IV authentication, but this has been deprecated in favor of Kerberos V, which uses :gserver:.

Now you can either open a new terminal window, or in the same window run the command:

source ~/.bash_profile
After this, you should be able to see these changes to your environment and aliases by running the following command:
echo $PATH
alias cmsenv
alias project
The file ~/.bash_profile is read automatically every time you start a new login shell, so you shouldn't have to worry about these environment variables anymore.

Step 6: Fixing scram

Before being able to run scram, we need to fix a problem with it. The problem is a common one--most people who program Linux scripts do not realize that the command /bin/sh does not equate to /bin/bash on all Linux distributions. Ubuntu in particular uses dash instead of bash as the default shell script program. It is much, much faster than bash, but it lacks many of the features of bash that most programmers, like the authors of the scram tool, take for granted. So in order to get scram to run properly, we have to somehow get scram to use /bin/bash instead of /bin/sh, which points to /bin/dash. There are several ways we can do this, but I found the best way is to copy the scripts that have this flaw and modify them so that they work properly.

Duplicating scram

In this method, we make a copy of the scram scripts, but we change some lines in the script to tell Linux to use /bin/bash instead of /bin/sh. To do this, we first need a place to put these scripts so that they are in a directory in the PATH variable. Fortunately, we added ~/bin to the PATH in Step 5. Make this directory if it doesn't already exist with the command:
mkdir -p ~/bin
Next, copy the scripts that we will need from their location on AFS to this directory:
cp /afs/* ~/bin
Finally, open the file ~/bin/scram in your favorite text editor and change the very first line from:
Do the same for the file ~/bin/cmsarch.

You are done! Congratulations! Use scram as you would normally to make an analysis directory and start using CMSSW on your Ubuntu machine!


  • AFS can be slow on some networks. This could slow down CMSSW to an unbearable pace. There is nothing you can do to solve this except move your computer to a faster network or give up and ssh into LXPLUS.
  • Your machine will not be able to access CASTOR, so any data you wish to analyze must be stored locally.
  • CRAB might work, but you will have to install the grid certificates manually. Go to and look at the "CERN Goodies" PPA for more info on how to install the grid certificates in an Ubuntu-friendly manner.

-- PhilKillewald - 2009-09-10

Edit | Attach | Watch | Print version | History: r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r1 - 2009-09-10 - unknown
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Sandbox All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback