Installing CMSSW using cmspkg
Recipe for the impatient:
Use these recipes only if you know what you are doing and you know you have a supported system. In case of problems, please read carefully the full instructions below before reporting an issue.
The latest architectures are :
- Linux :
-
slc6_amd64_gcc630
-
slc7_amd64_gcc630
-
slc6_amd64_gcc700
-
slc6_amd64_gcc530
Recipe for the impatient (CSH):
Initialize/setup the installation area (once per architecure):
setenv VO_CMS_SW_DIR /x/y/z
mkdir -p $VO_CMS_SW_DIR
wget -O $VO_CMS_SW_DIR/bootstrap.sh http://cmsrep.cern.ch/cmssw/repos/bootstrap.sh
setenv SCRAM_ARCH slc6_amd64_gcc530
sh -x $VO_CMS_SW_DIR/bootstrap.sh setup -path $VO_CMS_SW_DIR -arch $SCRAM_ARCH >& $VO_CMS_SW_DIR/bootstrap_$SCRAM_ARCH.log
To install a release (once per release per architecture):
setenv VO_CMS_SW_DIR /x/y/z
setenv SCRAM_ARCH slc6_amd64_gcc530
$VO_CMS_SW_DIR/common/cmspkg -a $SCRAM_ARCH update
$VO_CMS_SW_DIR/common/cmspkg -a $SCRAM_ARCH install cms+cmssw+CMSSW_8_0_20
Recipe for the impatient (SH / BASH / ZSH):
Initialize/setup the installation area (once per architecure):
export VO_CMS_SW_DIR=/x/y/z
mkdir -p $VO_CMS_SW_DIR
wget -O $VO_CMS_SW_DIR/bootstrap.sh http://cmsrep.cern.ch/cmssw/repos/bootstrap.sh
export SCRAM_ARCH=slc6_amd64_gcc530
sh -x $VO_CMS_SW_DIR/bootstrap.sh setup -path $VO_CMS_SW_DIR -arch $SCRAM_ARCH >& $VO_CMS_SW_DIR/bootstrap_$SCRAM_ARCH.log
To install a release (once per release per architecture):
export VO_CMS_SW_DIR=/x/y/z
export SCRAM_ARCH=slc6_amd64_gcc530
$VO_CMS_SW_DIR/common/cmspkg -a $SCRAM_ARCH update
$VO_CMS_SW_DIR/common/cmspkg -a $SCRAM_ARCH install cms+cmssw+CMSSW_8_0_20
Recipes for succesful installation on different Linux distro's
Gentoo
https://twiki.cern.ch/twiki/bin/view/CMS/SWGuideCMSSWOnGentoo
Ubuntu
https://twiki.cern.ch/twiki/bin/view/CMS/SWGuideCMSSWOnUbuntu
F.A.Q.
Q: On my initial install on slc5 the log ends with "There was an error generating the platform seed" exit 1. What happened?
A: It could mean that one or more of the required packages is missing from your SLC5 installation. Refer to the above
Prerequisites. Check which ones you are missing and install them.
Example: In the log file, above the "There was an error..." message you see:
-----------------------
+ echo 'Required package zsh is missing.'
-----------------------
so you perform:
sudo yum install zsh
ref:
HyperNews forum reference
Q: When I try to access files using castor I get "undefined symbol: Cns_lstat":
A: This is a known bug of some revisions of external+castor+2.1.0 package. Please do the following.
source $VO_CMS_SW_DIR/<arch>/external/apt/<apt-version>/etc/profile.d/init.csh
apt-get update
apt-get dist-upgrade
Q: apt-get is stuck since hours. What should I do?
First of all you need to kill all the stale rpm processes. Look with:
ps -ef | egrep '(rpm|apt)'
and 'kill -9 (pid)' for each of the rpm and/or apt processes that you see.
After that is done, do one of the following on SLC3:
rm -f $VO_CMS_SW_DIR/$SCRAM_ARCH/var/lib/rpm/__db.* # For LCG-style
rpmdb -vv --rebuilddb --dbpath $VO_CMS_SW_DIR/$SCRAM_ARCH/var/lib/rpm
rm -f $OSG_APP/cmssoft/cms/$SCRAM_SRCH/var/lib/rpm/__db.* # For OSG-style
rpmdb --rebuilddb --dbpath $OSG_APP/cmssoft/cms/$SCRAM_ARCH/var/lib/rpm
On SLC4 and SLC5 rpm requires an addition command line option:
rm -f $VO_CMS_SW_DIR/var/lib/rpm/__db.* # For LCG-style
rpmdb -vv --define "_rpmlock_path $VO_CMS_SW_DIR/$SCRAM_ARCH/var/lib/rpm/lock" --rebuilddb --dbpath $VO_CMS_SW_DIR/$SCRAM_ARCH/var/lib/rpm
rm -f $OSG_APP/cmssoft/cms/var/lib/rpm/__db.* # For OSG-style
rpmdb -vv --define "_rpmlock_path $OSG_APP/cmssoft/cms/$SCRAM_ARCH/var/lib/rpm/lock" --rebuilddb --dbpath $OSG_APP/cmssoft/cms/$SCRAM_ARCH/var/lib/rpm
If you are doing this at CERN, you may have to replace $VO_CMS_SW_DIR with "$CMS_PATH/sw". In addition you will probably
find that rebuild is much quicker if you change the path(s) to point directly at the copy on local disk rather than have it 'stat' its way through
afs, just to find a soft link to a local disk. (This is possible for the moment only for SLC3. You must be on lxbuild024 and the local disk path is
/build/cmsbuild/rpmDatabase/rpm.)
Q: Something went wrong to my installation and now apt says something about multiple versions of a given package in the system. What should I do?:
There seems to be a bug in apt if the post installation script aborts or it is interrupted. On the subsequent call to apt-get you get a message like the following:
W: There are multiple versions of "xxx+yyy+zzz" in your system.
This package won't be cleanly updated, unless you leave
only one version. To leave multiple versions installed,
you may remove that warning by setting the following
option in your configuration file:
RPM::Allow-Duplicated { "^xxx+yyy+zzz$"; };
To disable these warnings completely set:
RPM::Allow-Duplicated-Warning "false
To solve this you should list the packages that are installaed with the same name:
rpm --dbpath $VO_CMS_SW_DIR/var/lib/rpm -qa | grep xxx+yyy+zzz
and then remove the one you have unsuccessfully installed.
rpm --dbpath $VO_CMS_SW_DIR/var/lib/rpm -e xxx+yyy+zzz-1-jjjj
Q: Apt cannot install software because it say there are missing dependencies, what should I do?
If you get a message similar to:
Running installation of CMSSW_0_8_1
Reading Package Lists...
Building Dependency Tree...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
Since you only requested a single operation it is extremely likely that
the package is simply not installable and a bug report against
that package should be filed.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
cms+cmssw+CMSSW_0_8_1: Depends: xxxxxxx but it is not installable
Depends: yyyyyyy but it is not installable
Install failed
First of all make sure that rpmbuild is installed on the system.
which rpmbuild
If this is not the case, first of all ask your sysadmin to install it, and then repeat the setup proceedure.
If this is still not enought report to
hn-cms-sw-develtools@cernNOSPAMPLEASE.ch specifying:
Notice that the message usually depends on the fact that you don't have one of the following packages installed in your system:
Q: apt-get fails with error message "E: Sub-process /build/eulisse/100g48/bin/rpm-wrapper returned an error code (some number)"
If apt-get fails with something like the following:
Preparing... ^@########################################### [100%]
installing package external+p5-dbi+1.50-1-1009 needs 2MB on the /build filesystem
installing package external+p5-template-toolkit+2.14-1-1009 needs 12MB on the /build filesystem
E: Sub-process /build/eulisse/100g48/bin/rpm-wrapper returned an error code (_some number_)
It means you ran out of disk space and should free some before continuing. For more information on RPM return codes see
here
.
Q: I've updated my system installation with some packages and I want to reseed the local rpm database. How do I do it?
$VO_CMS_SW_DIR/bootstrap.sh reseed -path $VO_CMS_SW_DIR -csh
Q: I did apt-cache search cms+cmssw+CMSSW_X_Y_Z and it returned no results, even if I know it is available.
A: apt-cache needs to have the pluses double escaped in order to parse them correctly. The correct command is:
apt-cache search cms\\+cmssw\\+CMSSW_X_Y_Z
Q: I did apt-get install cms+cmssw+CMSSW_X_Y_Z and it wants to install an older version and/or cannot resolve dependencies.
A: Run "apt-cache pkgnames | grep akesystem" and install these packages first.
Q: I've installed a new version of CMSSW / I've done "apt-get dist-upgrade" and now I don't have scramv1 in my path anymore. What should I do?
Due to a problem in the older SCRAMV1 RPMs which would have prevented
the usage of scram V1_0_2 in parallel with a new version V1_0_3, soon
to be released, the following migration procedure has to be executed
before installing any new release of CMSSW in an area where there are old
ones as well:
source $VO_CMS_SW_DIR/<arch>/external/apt/<apt-version>/etc/profile.d/init.csh
apt-get update
apt-get dist-upgrade
apt-get reinstall lcg+SCRAMV1+V1_0_2
Please check at this point that you still see the 'scramv1' command in
$CMS_PATH/bin. If you do not, rerun the command:
apt-get reinstall lcg+SCRAMV1+V1_0_2
and then verify again that it is in $CMS_PATH/bin.
If 'scramv1' is present, you can then proceed to install CMSSW as usual:
apt-get install cms+cmssw+CMSSW_X_Y_Z
This extra "dist-upgrade" and "reinstall" steps are NOT required if you
are installing from scratch.
If you have any problems please report them to the "Software Distribution
Tools Discussions" HN forum (
hn-cms-softwareDistrib@cernNOSPAMPLEASE.ch).
Q: My software area is NFS mounted, and rpm complains "cannot get shared/exclusive lock".
You need to have the
nfslock
service running on both nfs client and server. On SL systems
this can be achieved with
service nfslock start
.
Q: How do I remove a software release?
The easiest is to use "apt-get remove --purge cms+cmssw+CMSSW_X_Y_Z". This will remove the CMSSW package, and with the --purge switch removes all dependencies which are not owned by another package.
You can also check out the tool cmsCleanSWArea from
http://cmssw.cvs.cern.ch/cgi-bin/cmssw.cgi/COMP/PKGTOOLS/
and run it.
Warning This command removed the "rpm" tool in my case, and I had to reinstall, be careful when using the script!
./cmsCleanSWArea -a \$SCRAM_ARCH -e cms+cmssw+CMSSW_CMSSW_X_Y_Z
Q: How do I make cmsRun run "standalone" without network connection?
Usually production jobs access databases via the network. But you can make your laptop provide the subset of conditions from Frontier needed for the cmsRun
job(s) you would like to run, for the moment see
https://hypernews.cern.ch/HyperNews/CMS/get/recoDevelopment/806/1/1/2.html
.
See also the section "Personal Squid on a Desktop/Laptop" on
SquidForCMS.
Q: I get an error like "cpio: rename failed - Inappropriate ioctl for device"
The apt-get install command reports an error like this:
Executing RPM (-Uvh)...
Preparing... ##################################################
cms+cmssw+CMSSW_2_2_10 ##################################################
error: unpacking of archive failed on file /opt/storage/exp_software/cms/slc4_ia
32_gcc345/cms/cmssw/CMSSW_2_2_10/config/toolbox/slc4_ia32_gcc345/tools/selected/
ccompiler.xml: cpio: rename failed - Inappropriate ioctl for device
This might have two reasons: The RPM was corrupted during download or there is a filesystem corruption.
First try to flush the cached local of the RPM package, which will trigger a reload on the next apt-get install:
apt-get clean
If this does not help the file system needs to be checked.
Q:The install script fails with the error "cannot open Packages database in /var/lib/rpm"
Harvesting system for locally available software...error: cannot open Packages index using db3 - Permission denied (13)
error: cannot open Packages database in /var/lib/rpm
There was an error generating the platform seed
The install script also requires access to the system rpm database. Check that the rpm database file /var/lib/rpm/Packages is readable
from the install account. Normally it will be for standard SL/RHEL installations, but at least one site (FNAL) has made special modifications for
security reasons to make it not world readable. If that rpm database file is not readable by the installation account, you will need to do
something like:
sudo chmod +r /var/lib/rpm/Packages
or talk to whoever administers the machine to make it visible to the installation user.
Further reference
Responsible:
ShahzadMalikMuzaffar