Installing CMSSW using cmspkg

Recipe for the impatient:

Use these recipes only if you know what you are doing and you know you have a supported system. In case of problems, please read carefully the full instructions below before reporting an issue.

The latest architectures are :

• Linux :
  • slc6_amd64_gcc630
  • slc7_amd64_gcc630
  • slc6_amd64_gcc700
  • slc6_amd64_gcc530

Recipe for the impatient (CSH):

Initialize/setup the installation area (once per architecure):

setenv VO_CMS_SW_DIR /x/y/z
mkdir -p $VO_CMS_SW_DIR
wget -O $VO_CMS_SW_DIR/bootstrap.sh http://cmsrep.cern.ch/cmssw/repos/bootstrap.sh
setenv SCRAM_ARCH slc6_amd64_gcc530
sh -x $VO_CMS_SW_DIR/bootstrap.sh setup -path $VO_CMS_SW_DIR -arch $SCRAM_ARCH >& $VO_CMS_SW_DIR/bootstrap_$SCRAM_ARCH.log

To install a release (once per release per architecture):

setenv VO_CMS_SW_DIR /x/y/z
setenv SCRAM_ARCH slc6_amd64_gcc530
$VO_CMS_SW_DIR/common/cmspkg -a $SCRAM_ARCH update
$VO_CMS_SW_DIR/common/cmspkg -a $SCRAM_ARCH install cms+cmssw+CMSSW_8_0_20

Recipe for the impatient (SH / BASH / ZSH):

Initialize/setup the installation area (once per architecure):

export VO_CMS_SW_DIR=/x/y/z
mkdir -p $VO_CMS_SW_DIR
wget -O $VO_CMS_SW_DIR/bootstrap.sh http://cmsrep.cern.ch/cmssw/repos/bootstrap.sh
export SCRAM_ARCH=slc6_amd64_gcc530
sh -x $VO_CMS_SW_DIR/bootstrap.sh setup -path $VO_CMS_SW_DIR -arch $SCRAM_ARCH >& $VO_CMS_SW_DIR/bootstrap_$SCRAM_ARCH.log

To install a release (once per release per architecture):

export VO_CMS_SW_DIR=/x/y/z
export SCRAM_ARCH=slc6_amd64_gcc530
$VO_CMS_SW_DIR/common/cmspkg -a $SCRAM_ARCH update
$VO_CMS_SW_DIR/common/cmspkg -a $SCRAM_ARCH install cms+cmssw+CMSSW_8_0_20

Recipes for succesful installation on different Linux distro's

Gentoo

https://twiki.cern.ch/twiki/bin/view/CMS/SWGuideCMSSWOnGentoo

Ubuntu

https://twiki.cern.ch/twiki/bin/view/CMS/SWGuideCMSSWOnUbuntu

F.A.Q.

Q: On my initial install on slc5 the log ends with "There was an error generating the platform seed" exit 1. What happened?

A: It could mean that one or more of the required packages is missing from your SLC5 installation. Refer to the above Prerequisites. Check which ones you are missing and install them.

Example: In the log file, above the "There was an error..." message you see:

-----------------------
+ echo 'Required package zsh is missing.'
-----------------------

so you perform:

sudo yum install zsh

ref: HyperNews forum reference

Q: When I try to access files using castor I get "undefined symbol: Cns_lstat":

A: This is a known bug of some revisions of external+castor+2.1.0 package. Please do the following.

source $VO_CMS_SW_DIR/<arch>/external/apt/<apt-version>/etc/profile.d/init.csh
apt-get update 
apt-get dist-upgrade

Q: apt-get is stuck since hours. What should I do?

First of all you need to kill all the stale rpm processes. Look with:

ps -ef | egrep '(rpm|apt)'

and 'kill -9 (pid)' for each of the rpm and/or apt processes that you see.

After that is done, do one of the following on SLC3:

rm -f $VO_CMS_SW_DIR/$SCRAM_ARCH/var/lib/rpm/__db.*                                # For LCG-style
rpmdb -vv --rebuilddb --dbpath $VO_CMS_SW_DIR/$SCRAM_ARCH/var/lib/rpm

rm -f $OSG_APP/cmssoft/cms/$SCRAM_SRCH/var/lib/rpm/__db.*                         # For OSG-style      
rpmdb --rebuilddb --dbpath $OSG_APP/cmssoft/cms/$SCRAM_ARCH/var/lib/rpm

On SLC4 and SLC5 rpm requires an addition command line option:

rm -f $VO_CMS_SW_DIR/var/lib/rpm/__db.*                                # For LCG-style
rpmdb -vv --define "_rpmlock_path $VO_CMS_SW_DIR/$SCRAM_ARCH/var/lib/rpm/lock" --rebuilddb --dbpath $VO_CMS_SW_DIR/$SCRAM_ARCH/var/lib/rpm

rm -f $OSG_APP/cmssoft/cms/var/lib/rpm/__db.*                         # For OSG-style      
rpmdb -vv --define "_rpmlock_path $OSG_APP/cmssoft/cms/$SCRAM_ARCH/var/lib/rpm/lock" --rebuilddb --dbpath $OSG_APP/cmssoft/cms/$SCRAM_ARCH/var/lib/rpm

If you are doing this at CERN, you may have to replace $VO_CMS_SW_DIR with "$CMS_PATH/sw". In addition you will probably find that rebuild is much quicker if you change the path(s) to point directly at the copy on local disk rather than have it 'stat' its way through afs, just to find a soft link to a local disk. (This is possible for the moment only for SLC3. You must be on lxbuild024 and the local disk path is /build/cmsbuild/rpmDatabase/rpm.)

Q: Something went wrong to my installation and now apt says something about multiple versions of a given package in the system. What should I do?:

There seems to be a bug in apt if the post installation script aborts or it is interrupted. On the subsequent call to apt-get you get a message like the following:

W: There are multiple versions of "xxx+yyy+zzz" in your system.

This package won't be cleanly updated, unless you leave
only one version. To leave multiple versions installed,
you may remove that warning by setting the following
option in your configuration file:

RPM::Allow-Duplicated { "^xxx+yyy+zzz$"; };

To disable these warnings completely set:

RPM::Allow-Duplicated-Warning "false

To solve this you should list the packages that are installaed with the same name:

rpm --dbpath $VO_CMS_SW_DIR/var/lib/rpm -qa | grep xxx+yyy+zzz

and then remove the one you have unsuccessfully installed.

rpm --dbpath $VO_CMS_SW_DIR/var/lib/rpm -e xxx+yyy+zzz-1-jjjj

Q: Apt cannot install software because it say there are missing dependencies, what should I do?

If you get a message similar to:

Running installation of CMSSW_0_8_1
Reading Package Lists...
Building Dependency Tree...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.

Since you only requested a single operation it is extremely likely that
the package is simply not installable and a bug report against
that package should be filed.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
   cms+cmssw+CMSSW_0_8_1: Depends: xxxxxxx but it is not installable
                          Depends: yyyyyyy but it is not installable
Install failed

First of all make sure that rpmbuild is installed on the system.

which rpmbuild

If this is not the case, first of all ask your sysadmin to install it, and then repeat the setup proceedure. If this is still not enought report to hn-cms-sw-develtools@cernNOSPAMPLEASE.ch specifying:

• the distribution you are using
• the result of running

       rpm -qa
       

  on the machine you are working on.
• the result of running

       rpm -q --whatprovides xxxxxx
       rpm -q --whatprovides yyyyyy     
       

  on the machine you are working on.

Notice that the message usually depends on the fact that you don't have one of the following packages installed in your system:

• tk

Q: apt-get fails with error message "E: Sub-process /build/eulisse/100g48/bin/rpm-wrapper returned an error code (some number)"

If apt-get fails with something like the following:

Preparing...                ^@########################################### [100%]
        installing package external+p5-dbi+1.50-1-1009 needs 2MB on the /build filesystem
        installing package external+p5-template-toolkit+2.14-1-1009 needs 12MB on the /build filesystem
E: Sub-process /build/eulisse/100g48/bin/rpm-wrapper returned an error code (_some number_)

It means you ran out of disk space and should free some before continuing. For more information on RPM return codes see here.

Q: I've updated my system installation with some packages and I want to reseed the local rpm database. How do I do it?

$VO_CMS_SW_DIR/bootstrap.sh reseed -path $VO_CMS_SW_DIR -csh

Q: I did apt-cache search cms+cmssw+CMSSW_X_Y_Z and it returned no results, even if I know it is available.

A: apt-cache needs to have the pluses double escaped in order to parse them correctly. The correct command is:

apt-cache search cms\\+cmssw\\+CMSSW_X_Y_Z

Q: I did apt-get install cms+cmssw+CMSSW_X_Y_Z and it wants to install an older version and/or cannot resolve dependencies.

A: Run "apt-cache pkgnames | grep akesystem" and install these packages first.

Q: I've installed a new version of CMSSW / I've done "apt-get dist-upgrade" and now I don't have scramv1 in my path anymore. What should I do?

Due to a problem in the older SCRAMV1 RPMs which would have prevented the usage of scram V1_0_2 in parallel with a new version V1_0_3, soon to be released, the following migration procedure has to be executed before installing any new release of CMSSW in an area where there are old ones as well:

source $VO_CMS_SW_DIR/<arch>/external/apt/<apt-version>/etc/profile.d/init.csh
apt-get update
apt-get dist-upgrade
apt-get reinstall lcg+SCRAMV1+V1_0_2

Please check at this point that you still see the 'scramv1' command in $CMS_PATH/bin. If you do not, rerun the command:

apt-get reinstall lcg+SCRAMV1+V1_0_2

and then verify again that it is in $CMS_PATH/bin.

If 'scramv1' is present, you can then proceed to install CMSSW as usual:

apt-get install cms+cmssw+CMSSW_X_Y_Z

This extra "dist-upgrade" and "reinstall" steps are NOT required if you are installing from scratch.

If you have any problems please report them to the "Software Distribution Tools Discussions" HN forum (hn-cms-softwareDistrib@cernNOSPAMPLEASE.ch).

Q: My software area is NFS mounted, and rpm complains "cannot get shared/exclusive lock".

You need to have the nfslock service running on both nfs client and server. On SL systems this can be achieved with service nfslock start.

Q: How do I remove a software release?

The easiest is to use "apt-get remove --purge cms+cmssw+CMSSW_X_Y_Z". This will remove the CMSSW package, and with the --purge switch removes all dependencies which are not owned by another package.

You can also check out the tool cmsCleanSWArea from http://cmssw.cvs.cern.ch/cgi-bin/cmssw.cgi/COMP/PKGTOOLS/ and run it. Warning This command removed the "rpm" tool in my case, and I had to reinstall, be careful when using the script!

./cmsCleanSWArea -a \$SCRAM_ARCH -e cms+cmssw+CMSSW_CMSSW_X_Y_Z

Q: How do I make cmsRun run "standalone" without network connection?

Usually production jobs access databases via the network. But you can make your laptop provide the subset of conditions from Frontier needed for the cmsRun job(s) you would like to run, for the moment see https://hypernews.cern.ch/HyperNews/CMS/get/recoDevelopment/806/1/1/2.html . See also the section "Personal Squid on a Desktop/Laptop" on SquidForCMS.

Q: I get an error like "cpio: rename failed - Inappropriate ioctl for device"

The apt-get install command reports an error like this:

Executing RPM (-Uvh)...
Preparing...                ##################################################
cms+cmssw+CMSSW_2_2_10      ##################################################
error: unpacking of archive failed on file /opt/storage/exp_software/cms/slc4_ia
32_gcc345/cms/cmssw/CMSSW_2_2_10/config/toolbox/slc4_ia32_gcc345/tools/selected/
ccompiler.xml: cpio: rename failed - Inappropriate ioctl for device

This might have two reasons: The RPM was corrupted during download or there is a filesystem corruption.

First try to flush the cached local of the RPM package, which will trigger a reload on the next apt-get install:

apt-get clean

If this does not help the file system needs to be checked.

Q:The install script fails with the error "cannot open Packages database in /var/lib/rpm"

Harvesting system for locally available software...error: cannot open Packages index using db3 - Permission denied (13)
error: cannot open Packages database in /var/lib/rpm
There was an error generating the platform seed

The install script also requires access to the system rpm database. Check that the rpm database file /var/lib/rpm/Packages is readable from the install account. Normally it will be for standard SL/RHEL installations, but at least one site (FNAL) has made special modifications for security reasons to make it not world readable. If that rpm database file is not readable by the installation account, you will need to do something like:

sudo chmod +r /var/lib/rpm/Packages

or talk to whoever administers the machine to make it visible to the installation user.

Further reference

• APT-RPM homepage
• Debian APT HOWTO
  • Package management
  • Package search

Responsible: ShahzadMalikMuzaffar