Difference: ProjectRelease (1 vs. 200)

Revision 1992019-09-02 - JoelClosier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

This page is moving to here

Deleted:
<
<

Scope

The following documentation applies to the release of LHCb Physics projects as well as data packages. Ganga is installed on a separate CVMFS volume, and there are specific instructions for LHCbDIrac which can be found here

Warning, important These instructions do not apply to LbScripts. Use LbScriptsRelease instead.

Overview of the procedure

The software is built by the Jenkins continuous integration engine. The configuration of the projects, their dependencies and which stacks should be built is defined in the software configuration database, with the typical procedure including the following steps:

  1. Release manager tags a project (or several) and triggers the build by importing the projects to the software configuration database
  2. Jenkins builds the project(s) and prepares the RPM packages
  3. Release manager checks the outcome of the build and either restarts the build or requests a release in JIRA
  4. Release shifter copies the RPM packages to the official LHCb package repository (YUM format). At this point, anybody can perform local installations using lbinstall
  5. Release shifter marks the RPM as "released" in the software configuration database, and specifies which platforms have effectively been released. This is necessary for the build system to know what is released and isn't when building stacks, but also for the automated release procedures on CVMFS to know which platforms should be installed
  6. Release shifter performs the installation on CVMFS and updates the documentation

Instructions for all the steps are detailed below.

For Release Managers - Preparation of the build

For Projects

  • prepare the project as usual (release notes, tags, etc.), following the instructions at PrepareProjectReleaseGit
  • register the project/version to the software database
    lb-sdb-import <Project> <version>
    
    * Check that the import into the software configuration was correct by running:
     lb-sdb-query listReleaseStacks
    
This should show the projects to release and the platforms for which they should be released. You can select which platforms to release for by using lb-sdb-addplatform --release (they should of course be a subset of the platforms offered by the projects this one depends on).

Within 10-15 min the build should start to appear in the nightlies page https://lhcb-nightlies.cern.ch/release. Note that, for a dependency tree of projects, lb-sdb-import can be called just once for the top level project as it will pull in all the dependencies (unless you need to provide the --sourceurl option, in which case each project has to be added individually, in the dependency order).

  • if there is a problem in the build, it can be re-started via the dedicated button after you must log in (the build will not restart by itself after a retag)
  • once satisfied by the build, take note of the build id (you can use the direct link icon) and make the request via LHCb Deployment Request form

For Data Packages

For Release Shifters - Deployment Prerequisites

See the LHCb shift database to find who is this week's shifter.

Special information

Starting from 13 October 2016, you should set your environment to DEV as : ". /cvmfs/lhcb.cern.ch/lib/LbLoginDev.sh" *Starting from 28 March 2019, you don't need to set your environment to DEV.

Prerequisites for new release shifters

If you are a new release shifter, please make sure you have the prerequisites that are needed at some point of the release procedure.

  • You should be added as a "developer" in the LHCb Deployment Jira project (https://its.cern.ch/jira/browse/LHCBDEP/), to receive and track release requests
  • You must be added to the lhcb-cvmfs-librarians E-group (for access to the IT managed CVMFS installation machine and for communication with other release shifters)
  • You must have a valid grid certificate (For release of DecFiles package)
  • You need to be registered as a software deployment shifter in the shifts database
  • You need to be registered to use Jenkins, to be able to trigger release builds from the web interface (for data packages)
  • Check that the CMTPATH environment variable is not defined in the shell which you are using

As shifter, you can self-assign shifts through the shift database shift database. Usually this is done by block-booking weeks for individual shifters.

Deployment of Projects

Update of the RPM repository

The goal of this step is to make sure that:

  1. The RPMs have been copied to the YUM repository (currently in EOS)
  2. The YUM metadata has been updated (the lb-release-rpm internally uses the "createrepo" yum command).

Actions:

  • Check on the release build summaries page if the build is OK: https://lhcb-nightlies.cern.ch/release
  • If there are problems, ask the project manager to fix them, and restart the build
  • Update the RPMs repository (Warning, important to be done from any host)
    • connect to lxplus.cern.ch
      ssh lxplus7.cern.ch
      bash
      kinit ${USER}@CERN.CH
      
    • publish the produced RPMs (all of them, unless explicitly requested otherwise, see partial install section)
      • Define the build ID to extract RPMS from
        build_id=1234
        
      • Try to copy the RPMs to the repository
        lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2019 /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
        
      • If everything is fine, do the real copy and update the repository info
        lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2019 --copy /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
        
    • Check it's all there
      ls -ltr /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2019/ | tail -10
      
      In case of problems see here

BE CAREFUL: Since 20141010, the release procedure automatically bumps up the release number for packages for which there is already a version in the RPM repo. If your slot contains more projects that the ones requested for release, you need to ONLY copy the needed ones to the RPM repo. In that case the --rpm-regex option of lb-release-rpm should be used.

N.B. If you want to release all rpms EXCEPT some matching a specific string (e.g. gcc62) you can use:

lb-release-rpm -c --rpm-regex '^(?!.*(gcc62)).*'  --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2019 /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms

Update of the Software configuration Database

The goal of this step is to make sure that:

  1. The project is referenced properly in the Software configuration DB
  2. The list of platforms released for the project have been updated

Actions:

  • Verify that the software configuration DB has been updated:
    lb-sdb-query d <Project> <version>
    
    This should show projects in the slot or already released. If some are missing re-run:
    lb-sdb-import <Project> <version>
    

  • Update the software database.
    First try in dry-run mode:
    lb-deployment-updatesdb /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id
    
    If more platforms/projects are being updated than necessary you can limit using the options of the lb-deployment-updatesdb command.If all ok, just run:
    lb-deployment-updatesdb --update /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id
    
    In case of SSO, or certificate problems:
    ssh -fN  -L12345:localhost:7474 lbariadne
    export SDBURL=http://localhost:12345/db/data/
    lb-deployment-updatesdb [...]
    
    You can ignore the warnings such as: WARNING: Error encountered while acquiring CERN SSO cookie for Ariadne. Attempting to connect with no SSO.

  • Check whether the platforms for all projects in the stack being deployed were updated correctly, so for each project do:
    lb-sdb-query listPlatforms <Project> <version>
    
    which should show the complete list of platforms for which your project was released. If it's not the case add the missing ones with
    lb-sdb-add-platforms <Project> <version> <platform1> [<platform2>...]
    
  • Check that the projects are correctly flagged as released: call
    lb-sdb-query listReleases
    
    and check that they are not in the list, otherwise call
    lb-sdb-release -r <Project> <version>
    
    to manually flag them as released.

Software Installation (CVMFS)

Actions:

  • Connect to "cvlhcb" account on host "cvmfs-lhcb"
    ssh yourusername@cvmfs-lhcb
    
  • Check that nobody else is connected to install software
    who -q ; pgrep -fl -u cvlhcb
    
    if someone else is connected and if there are processes running as cvlhcb, perhaps try sending them an email to see what they are doing before proceeding
  • sudo into the all powerful account
    sudo -i -u cvlhcb
    
  • Start a transaction on the server to be able to write to the disk
    cvmfs_transaction
    
    • if the command returns that a transaction was already created, somebody else is installing at the same time. Check with the who command before going any further.
    • Note:
      1. cvmfs_transaction is a wrapper around the CVMFS command that creates the transaction on the repository lhcb.cern.ch, and logs the command and result. It is best to use it instead of using the direct command cvmfs_server transaction lhcb.cern.ch
      2. even if you decide to use cvmfs_server instead of the wrapper, it is best to specify the repository in the command as this server could potentially host several repositories in the future.

  • Install the project (all registered platforms)
    install_software.sh <Project> <version>
    
    • Note:
      1. in case of problems or missing dependencies you can alter the options to lbinstall by setting the OPTS environment variable e.g
         OPTS=--no-strict install_software.sh <Project> <version>
      2. If the binary RPMs do not get deployed (packages with x86_64[...]), check wether the SoftwareConfigurationDB was updated correctly as per the following instructions. The DB is the source of information for the list of platforms to deploy.

  • Check that it all went fine with
    ls -ltr /cvmfs/lhcb.cern.ch/lib/lhcb/<PROJECT>/<PROJECT>_<version>/InstallArea/
    
    the platforms you have installed should all be there.
  • Update the CVMFS catalogs
    cvmfs_publish
    
    • Note:
      1. cvmfs_publish is a wrapper around the CVMFS command that publishes the transaction on the repository lhcb.cern.ch, and logs the command and result. It is best to use it instead of using the direct command cvmfs_server publish lhcb.cern.ch
      2. Even if you decide to use cvmfs_server instead of the wrapper, it is best to specify the repository in the command as this server could potentially host several repositories
  • Remember to log out

Update Software Project Web Page

Note: this is not about Doxygen documentation (automatically generated, with some delay, after the installation)

Actions:

  • Wait for the project to be visible in CVMFS:
    ls -ltr /cvmfs/lhcb.cern.ch/lib/lhcb/<PROJECT>/<PROJECT>_<version>/InstallArea/
    

  • Prepare the web pages links
/eos/project/l/lhcbwebsites/www/projects/scripts/addrel.py

Deployment of Data Packages

Warning, important WARNING The procedure for the release of data packages via RPMs is still under heavy development and the automation is not yet in place.

Data packages are different from standard projects:

  • They do not need to be compiled for several platforms (make is run nonetheless to perform basic actions)
  • They are part of the DBASE or PARAM projects
  • They are not registered in the software configuration Database and the builds may not always be triggered by the release manager.

The release steps are therefore different from normal projects.

Trigger by hand the lhcb-release Jenkins job (if needed)

  • Once the build is completed, check the checkout log
    • go to https://lhcb-nightlies.cern.ch/release/
    • click on the project name (DBASE or PARAM) in the build summary, and look for the package build log (after the line "building data packages in...")

Update of the RPM repository

The goal of this step is to make sure that:

  1. The RPMs have been copied to the YUM repository (currently in EOS)
  2. The YUM metadata has been updated (the lb-release-rpm internally uses the "createrepo" yum command).

Actions:

  • If you didn't trigger the build, check that everything is OK (see trigger data package build section)
  • If there are problems, ask the project manager to fix them, and restart the build
  • Update the RPMs repository (Warning, important *to be done from any host *)
    • connect to lxplus.cern.ch
      ssh lxplus7.cern.ch
      bash
      kinit ${USER}@CERN.CH
      
    • publish the produced RPMs (all of them, unless explicitly requested otherwise, see partial install section)
      • Define the build ID to extract RPMS from
        build_id=1234
        
      • Try to copy the RPMs to the repository
        lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2019 /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
        
        • For old stacks only released as tar files:
          lb-release-oldtar /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/olddist
          
      • If everything is fine, do the real copy and update the repository info
        lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2019 --copy /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
        
        • For old stacks only released as tar files:
          lb-release-oldtar --copy /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/olddist
          
    • Check it's all there
      ls -ltr /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2019/ | tail -10
      
      In case of problems see here

BE CAREFUL: Since 20141010, the release procedure automatically bumps up the release number for packages for which there is already a version in the RPM repo. If your slot contains more projects that the ones requested for release, you need to ONLY copy the needed ones to the RPM repo. In that case the --rpm-regex option of lb-release-rpm should be used.

N.B. If you want to release all rpms EXCEPT some matching a specific string (e.g. gcc62) you can use:

lb-release-rpm -c --rpm-regex '^(?!.*(gcc62)).*'  --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2019 /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms

Software Installation (CVMFS)

Actions:

  • Connect to "cvlhcb" account on host "cvmfs-lhcb"
    ssh yourusername@cvmfs-lhcb
    
  • Check that nobody else is connected to install software
    who -q ; pgrep -fl -u cvlhcb
    
    if someone else is connected and if there are processes running as cvlhcb, perhaps try sending them an email to see what they are doing before proceeding
  • sudo into the all powerful account
    sudo -i -u cvlhcb
    
  • Start a transaction on the server to be able to write to the disk
    cvmfs_transaction
    
    • if the command returns that a transaction was already created, somebody else is installing at the same time. Check with the who command before going any further.
    • Note:
      1. cvmfs_transaction is a wrapper around the CVMFS command that creates the transaction on the repository lhcb.cern.ch, and logs the command and result. It is best to use it instead of using the direct command cvmfs_server transaction lhcb.cern.ch
      2. even if you decide to use cvmfs_server instead of the wrapper, it is best to specify the repository in the command as this server could potentially host several repositories in the future.

  • Install the data package
    cvmfslbinstall install <PROJECT>_<Hat>_<Package>_<version>
    
  • Check that it all went fine with
    ls -ltr /cvmfs/lhcb.cern.ch/lib/lhcb/<PROJECT>/<Hat>/<Package>/<version>/
    
  • Update the CVMFS catalogs
    cvmfs_publish
    
    • Note:
      1. cvmfs_publish is a wrapper around the CVMFS command that publishes the transaction on the repository lhcb.cern.ch, and logs the command and result. It is best to use it instead of using the direct command cvmfs_server publish lhcb.cern.ch
      2. Even if you decide to use cvmfs_server instead of the wrapper, it is best to specify the repository in the command as this server could potentially host several repositories
  • Remember to log out

Deployment of LbEnv

Install new versions

LbEnv is the set of LHCb environment scripts and tools used for development. It will be released to CVMFS by a CRON running on cvmfs-lhcb.cern.ch, but in the meantime some tools have been prepared for this task (namely update_lbenv.sh).

First check in DRYRUN mode whether the tool is going to update correctly:

CVMFS-Shared ~ > DRYRUN=1 update_lbenv.sh
2019-03-07T16:39:30+0100 Will install:
lbenv-kit.stable.x86_64-centos7.347-4a87636109.tar.bz2
lbenv-kit.stable.x86_64-slc6.347-4a87636109.tar.bz2
lbenv-kit.unstable.x86_64-centos7.347-68420333c0.tar.bz2
lbenv-kit.unstable.x86_64-slc6.347-68420333c0.tar.bz2
2019-03-07T16:39:30+0100 Unpacking lbenv-kit.stable.x86_64-centos7.347-4a87636109.tar.bz2 ...
2019-03-07T16:39:30+0100 DRYRUN mode. Would add link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/testing/x86_64-centos7 -> ../347/stable/x86_64-centos7
2019-03-07T16:39:30+0100 Unpacking lbenv-kit.stable.x86_64-slc6.347-4a87636109.tar.bz2 ...
2019-03-07T16:39:30+0100 DRYRUN mode. Would add link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/testing/x86_64-slc6 -> ../347/stable/x86_64-slc6
2019-03-07T16:39:30+0100 Unpacking lbenv-kit.unstable.x86_64-centos7.347-68420333c0.tar.bz2 ...
2019-03-07T16:39:30+0100 DRYRUN mode. Would add link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/unstable/x86_64-centos7 -> ../347/unstable/x86_64-centos7
2019-03-07T16:39:30+0100 Unpacking lbenv-kit.unstable.x86_64-slc6.347-68420333c0.tar.bz2 ...
2019-03-07T16:39:30+0100 DRYRUN mode. Would add link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/unstable/x86_64-slc6 -> ../347/unstable/x86_64-slc6

If this seems satisfactory:

CVMFS-Shared ~ > update_lbenv.sh
Starting TRANSACTION
Created TRANSACTION
2019-03-07T16:39:37+0100 Will install:
lbenv-kit.stable.x86_64-centos7.347-4a87636109.tar.bz2
lbenv-kit.stable.x86_64-slc6.347-4a87636109.tar.bz2
lbenv-kit.unstable.x86_64-centos7.347-68420333c0.tar.bz2
lbenv-kit.unstable.x86_64-slc6.347-68420333c0.tar.bz2
2019-03-07T16:39:37+0100 Unpacking lbenv-kit.stable.x86_64-centos7.347-4a87636109.tar.bz2 ...
2019-03-07T16:39:37+0100 Downloading https://lhcb-rpm.web.cern.ch/lhcb-rpm/lbenv-kits/lhcb.cern.ch/lbenv-kit.stable.x86_64-centos7.347-4a87636109.tar.bz2
tar: Removing leading `/' from member names
2019-03-07T16:39:43+0100 Adding link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/testing/x86_64-centos7 -> ../347/stable/x86_64-centos7
`/cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/testing/x86_64-centos7' -> `../347/stable/x86_64-centos7'
2019-03-07T16:39:43+0100 Unpacking lbenv-kit.stable.x86_64-slc6.347-4a87636109.tar.bz2 ...
2019-03-07T16:39:43+0100 Downloading https://lhcb-rpm.web.cern.ch/lhcb-rpm/lbenv-kits/lhcb.cern.ch/lbenv-kit.stable.x86_64-slc6.347-4a87636109.tar.bz2
tar: Removing leading `/' from member names
2019-03-07T16:39:47+0100 Adding link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/testing/x86_64-slc6 -> ../347/stable/x86_64-slc6
`/cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/testing/x86_64-slc6' -> `../347/stable/x86_64-slc6'
2019-03-07T16:39:47+0100 Unpacking lbenv-kit.unstable.x86_64-centos7.347-68420333c0.tar.bz2 ...
2019-03-07T16:39:47+0100 Downloading https://lhcb-rpm.web.cern.ch/lhcb-rpm/lbenv-kits/lhcb.cern.ch/lbenv-kit.unstable.x86_64-centos7.347-68420333c0.tar.bz2
tar: Removing leading `/' from member names
2019-03-07T16:39:51+0100 Adding link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/unstable/x86_64-centos7 -> ../347/unstable/x86_64-centos7
`/cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/unstable/x86_64-centos7' -> `../347/unstable/x86_64-centos7'
2019-03-07T16:39:51+0100 Unpacking lbenv-kit.unstable.x86_64-slc6.347-68420333c0.tar.bz2 ...
2019-03-07T16:39:51+0100 Downloading https://lhcb-rpm.web.cern.ch/lhcb-rpm/lbenv-kits/lhcb.cern.ch/lbenv-kit.unstable.x86_64-slc6.347-68420333c0.tar.bz2
tar: Removing leading `/' from member names
2019-03-07T16:39:57+0100 Adding link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/unstable/x86_64-slc6 -> ../347/unstable/x86_64-slc6
`/cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/unstable/x86_64-slc6' -> `../347/unstable/x86_64-slc6'
`.installed_kits.txt.2019-03-07T16:39:37+0100' -> `/cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/.installed_kits.txt.2019-03-07T16:39:37+0100'
removed `.installed_kits.txt.2019-03-07T16:39:37+0100'
`/cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/.installed_kits.txt' -> `.installed_kits.txt.2019-03-07T16:39:37+0100'
Starting PUBLISH =====================================
gio  7 mar 2019, 16.39.57, CET
Using auto tag 'generic-2019-03-07T15:39:57Z'
Processing changes...
...........

Update stable version

When a new testing version of LbEnv is ready for production, the release shifter is asked to update the prod version, in which case he/she has to run from cvmfs-lhcb.cern.ch:

update_lbenv_stable.sh
which starts a transaction update the prod version and publish the transaction if successful.

WARNING: in case of failure the transaction is kept open for the shifter to fix the problem or abort it.

 

Special Instructions and troubleshooting

Installation on AFS

Line: 887 to 493
 ./onlinelbpkr install OnlineFarmMeta
Deleted:
<
<

Releases of the nightly builds on /cvmfs/lhcbdev.cern.ch

The releases should be completely automated, but here are the details needed to pcheck the system, perform manual installations/add slots etc...

To connect:

ssh cvmfs-lhcbdev.cern.ch 
sudo -i -u cvlhcbdev

Then you can start a transaction and copy files to /cvmfs/lhcbdev.cern.ch

Preparing the RPM and releasing a new version of CMake

LHCb uses the Linux_x86_64 binaries from Kitware:

https://cmake.org/download/

The tools to repqckage the RPM can be found in:

https://gitlab.cern.ch/lhcb-core/rpm-recipes

In the cmake folder.

The RPM prepared just needs to be copied to the yum repository and the metadata rebuilt.

  -- MarcoClemencic - 30 Jun 2014

Revision 1982019-08-30 - JoelClosier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Added:
>
>
This page is moving to here
 

Scope

Line: 912 to 913
  The RPM prepared just needs to be copied to the yum repository and the metadata rebuilt.
Deleted:
<
<

Installing new generators

The simulation team tests generators on /cvmfs/lhcbdev.cern.ch. Once validated they need to be installed on the production cvmfs.

The list will be expressed as a YAML file such as:

packages:   
- name: 'lhapdf'     
  LCG: [ '88']     
  versions: [ '6.1.6.cxxstd' ]
- name: 'pythia8'
  LCG: [ '88' ]
  versions: [ '230' ]   
- name: 'rivet'
  LCG: [ '88' ]
  versions: [ '2.5.2' ]   
- name: 'thepeg'
  LCG: [ '88' ]
  versions: [ '2.0.3' ]   

To perform the installation, it is first necessary to derive the list of RPM packages to install. This can be done in the following manner:

virtualenv yamlToPackages
. ./yamlToPackages/bin/activate
pip install --extra-index-url https://lbmultipython03.cern.ch/simple/ --trusted-host lbmultipython03.cern.ch lbinstall lbdevmanager

lbGetPackagesFromYAML <YAML FILE>

This will print out a list of packages. It is then necessary to login to cvmfs-lhcb and install them using the cvmfslbinstall command, e.g.

cvmfs_transaction
cvmfslbinstall install <packages>
cvmfs_publish

CVMFS Appendix

IT CVMFS Information

All information from IT Department can be found at https://twiki.cern.ch/twiki/bin/view/CvmFS/Installers

LHCB Software on cvmfs-lhcb

Scripts

All updates to software deployed on cvmfs should be done under account "cvlhcb". "${HOME}/bin" contains the necessary scripts to add/remove software:

  • cvmfs_transaction: To start a transaction to deploy software on the server

  • install_software.sh: To call install_project to add new projects/versions

  • remove_software.sh: To Remove software versions

  • cvmfs_publish: Synchronize the CVMFS file system

Some other scripts are also available (same name as above but prefixed with std_ perform the same action without logging)

The output of those command is automatically logged to "${HOME}/logs/install.log

Sometimes a package is not known to the install_software.sh script. In that case

<!-- SyntaxHighlightingPlugin -->
cvmfslbinstall install DBASE_HAT_PKG_vXrY
<!-- end SyntaxHighlightingPlugin -->
may help.

CRON

The local CRON is running on this host to automatically update SQLDDDB every hour. The logs for this cron is in: ${HOME}/logs/update_sqldddb.log

A backup of the CRONTAB is available in ${HOME}/conf.crontab.backup

  -- MarcoClemencic - 30 Jun 2014

Revision 1972019-04-17 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 32 to 32
 
  • register the project/version to the software database
    lb-sdb-import <Project> <version>
    
Added:
>
>
* Check that the import into the software configuration was correct by running:
 lb-sdb-query listReleaseStacks
This should show the projects to release and the platforms for which they should be released. You can select which platforms to release for by using lb-sdb-addplatform --release (they should of course be a subset of the platforms offered by the projects this one depends on).
  Within 10-15 min the build should start to appear in the nightlies page https://lhcb-nightlies.cern.ch/release. Note that, for a dependency tree of projects, lb-sdb-import can be called just once for the top level project as it will pull in all the dependencies (unless you need to provide the --sourceurl option, in which case each project has to be added individually, in the dependency order).
  • if there is a problem in the build, it can be re-started via the dedicated button after you must log in (the build will not restart by itself after a retag)

Revision 1962019-04-16 - MarcoCattaneo

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 7 to 7
 

Scope

The following documentation applies to the release of LHCb Physics projects as well as data packages.

Changed:
<
<
Ganga is installed on a separate CVMFS volume, and there are specific instructions for LHCbDIrac which can be found here
>
>
Ganga is installed on a separate CVMFS volume, and there are specific instructions for LHCbDIrac which can be found here
  Warning, important These instructions do not apply to LbScripts. Use LbScriptsRelease instead.

Revision 1952019-04-16 - FedericoStagni

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 7 to 7
 

Scope

The following documentation applies to the release of LHCb Physics projects as well as data packages.

Changed:
<
<
Ganga is installed on a separate CVMFS volume, and there are specific instructions for DIRAC and LHCbDIrac.
>
>
Ganga is installed on a separate CVMFS volume, and there are specific instructions for LHCbDIrac which can be found here
  Warning, important These instructions do not apply to LbScripts. Use LbScriptsRelease instead.
Line: 179 to 179
 
    • Note:
      1. in case of problems or missing dependencies you can alter the options to lbinstall by setting the OPTS environment variable e.g
         OPTS=--no-strict install_software.sh <Project> <version>
Deleted:
<
<
      1. This dependency problem occurs systematically during LHCbDirac deployments. This is a known issue that we are not planning to fix as LHCbDirac will soon be deployed differently.
 
      1. If the binary RPMs do not get deployed (packages with x86_64[...]), check wether the SoftwareConfigurationDB was updated correctly as per the following instructions. The DB is the source of information for the list of platforms to deploy.
Line: 205 to 204
 ls -ltr /cvmfs/lhcb.cern.ch/lib/lhcb//_/InstallArea/
Changed:
<
<
  • Prepare the web pages links (NOT for DIRAC and LHCbDirac)
>
>
  • Prepare the web pages links
 /eos/project/l/lhcbwebsites/www/projects/scripts/addrel.py

Revision 1942019-03-29 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 163 to 163
 
  • sudo into the all powerful account
    sudo -i -u cvlhcb
    
Deleted:
<
<
  • Start logging of the current shell
    logging_shell
    
 
  • Start a transaction on the server to be able to write to the disk
    cvmfs_transaction
    

Revision 1932019-03-28 - JoelClosier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 48 to 48
 

Special information

Starting from 13 October 2016, you should set your environment to DEV as : ". /cvmfs/lhcb.cern.ch/lib/LbLoginDev.sh"
Added:
>
>
*Starting from 28 March 2019, you don't need to set your environment to DEV.
 

Prerequisites for new release shifters

Line: 79 to 81
 
    • connect to lxplus.cern.ch
      ssh lxplus7.cern.ch
      bash
Deleted:
<
<
. /cvmfs/lhcb.cern.ch/lib/LbLoginDev.sh
 kinit ${USER}@CERN.CH
Line: 250 to 251
 
    • connect to lxplus.cern.ch
      ssh lxplus7.cern.ch
      bash
Deleted:
<
<
. /cvmfs/lhcb.cern.ch/lib/LbLoginDev.sh
 kinit ${USER}@CERN.CH

Revision 1922019-03-28 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 162 to 162
 
  • sudo into the all powerful account
    sudo -i -u cvlhcb
    
Added:
>
>
  • Start logging of the current shell
    logging_shell
    
 
  • Start a transaction on the server to be able to write to the disk
    cvmfs_transaction
    

Revision 1912019-03-07 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 327 to 327
  First check in DRYRUN mode whether the tool is going to update correctly:
Changed:
<
<
$ DRYRUN=1 update_lbenv.sh 2018-12-18T16:39:35+0100 Will install: lbenv-kit.dev.x86_64-centos7.283-ca05b181d0.tar.bz2 lbenv-kit.dev.x86_64-slc6.283-ca05b181d0.tar.bz2 lbenv-kit.prod.x86_64-centos7.283-3b0a5c49cf.tar.bz2 lbenv-kit.prod.x86_64-slc6.283-3b0a5c49cf.tar.bz2 2018-12-18T16:39:35+0100 Unpacking lbenv-kit.dev.x86_64-centos7.283-ca05b181d0.tar.bz2 ... 2018-12-18T16:39:35+0100 DRYRUN mode. Would add link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/dev/x86_64-centos7 -> ../283/dev/x86_64-centos7 2018-12-18T16:39:35+0100 Unpacking lbenv-kit.dev.x86_64-slc6.283-ca05b181d0.tar.bz2 ... 2018-12-18T16:39:35+0100 DRYRUN mode. Would add link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/dev/x86_64-slc6 -> ../283/dev/x86_64-slc6 2018-12-18T16:39:35+0100 Unpacking lbenv-kit.prod.x86_64-centos7.283-3b0a5c49cf.tar.bz2 ... 2018-12-18T16:39:35+0100 DRYRUN mode. Would add link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/pre-prod/x86_64-centos7 -> ../283/prod/x86_64-centos7 2018-12-18T16:39:35+0100 Unpacking lbenv-kit.prod.x86_64-slc6.283-3b0a5c49cf.tar.bz2 ... 2018-12-18T16:39:35+0100 DRYRUN mode. Would add link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/pre-prod/x86_64-slc6 -> ../283/prod/x86_64-slc6
>
>
CVMFS-Shared ~ > DRYRUN=1 update_lbenv.sh 2019-03-07T16:39:30+0100 Will install: lbenv-kit.stable.x86_64-centos7.347-4a87636109.tar.bz2 lbenv-kit.stable.x86_64-slc6.347-4a87636109.tar.bz2 lbenv-kit.unstable.x86_64-centos7.347-68420333c0.tar.bz2 lbenv-kit.unstable.x86_64-slc6.347-68420333c0.tar.bz2 2019-03-07T16:39:30+0100 Unpacking lbenv-kit.stable.x86_64-centos7.347-4a87636109.tar.bz2 ... 2019-03-07T16:39:30+0100 DRYRUN mode. Would add link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/testing/x86_64-centos7 -> ../347/stable/x86_64-centos7 2019-03-07T16:39:30+0100 Unpacking lbenv-kit.stable.x86_64-slc6.347-4a87636109.tar.bz2 ... 2019-03-07T16:39:30+0100 DRYRUN mode. Would add link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/testing/x86_64-slc6 -> ../347/stable/x86_64-slc6 2019-03-07T16:39:30+0100 Unpacking lbenv-kit.unstable.x86_64-centos7.347-68420333c0.tar.bz2 ... 2019-03-07T16:39:30+0100 DRYRUN mode. Would add link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/unstable/x86_64-centos7 -> ../347/unstable/x86_64-centos7 2019-03-07T16:39:30+0100 Unpacking lbenv-kit.unstable.x86_64-slc6.347-68420333c0.tar.bz2 ... 2019-03-07T16:39:30+0100 DRYRUN mode. Would add link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/unstable/x86_64-slc6 -> ../347/unstable/x86_64-slc6
 

If this seems satisfactory:

Changed:
<
<
> update_lbenv.sh
>
>
CVMFS-Shared ~ > update_lbenv.sh
 Starting TRANSACTION Created TRANSACTION
Changed:
<
<
2018-12-18T16:49:16+0100 Will install: lbenv-kit.dev.x86_64-centos7.283-ca05b181d0.tar.bz2 lbenv-kit.dev.x86_64-slc6.283-ca05b181d0.tar.bz2 lbenv-kit.prod.x86_64-centos7.283-3b0a5c49cf.tar.bz2 lbenv-kit.prod.x86_64-slc6.283-3b0a5c49cf.tar.bz2 2018-12-18T16:49:16+0100 Unpacking lbenv-kit.dev.x86_64-centos7.283-ca05b181d0.tar.bz2 ... 2018-12-18T16:49:16+0100 Downloading https://lhcb-rpm.web.cern.ch/lhcb-rpm/lbenv-kits/lhcb.cern.ch/lbenv-kit.dev.x86_64-centos7.283-ca05b181d0.tar.bz2
>
>
2019-03-07T16:39:37+0100 Will install: lbenv-kit.stable.x86_64-centos7.347-4a87636109.tar.bz2 lbenv-kit.stable.x86_64-slc6.347-4a87636109.tar.bz2 lbenv-kit.unstable.x86_64-centos7.347-68420333c0.tar.bz2 lbenv-kit.unstable.x86_64-slc6.347-68420333c0.tar.bz2 2019-03-07T16:39:37+0100 Unpacking lbenv-kit.stable.x86_64-centos7.347-4a87636109.tar.bz2 ... 2019-03-07T16:39:37+0100 Downloading https://lhcb-rpm.web.cern.ch/lhcb-rpm/lbenv-kits/lhcb.cern.ch/lbenv-kit.stable.x86_64-centos7.347-4a87636109.tar.bz2
 tar: Removing leading `/' from member names
Changed:
<
<
2018-12-18T16:49:23+0100 Adding link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/dev/x86_64-centos7 -> ../283/dev/x86_64-centos7 `/cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/dev/x86_64-centos7' -> `../283/dev/x86_64-centos7' 2018-12-18T16:49:23+0100 Unpacking lbenv-kit.dev.x86_64-slc6.283-ca05b181d0.tar.bz2 ... 2018-12-18T16:49:23+0100 Downloading https://lhcb-rpm.web.cern.ch/lhcb-rpm/lbenv-kits/lhcb.cern.ch/lbenv-kit.dev.x86_64-slc6.283-ca05b181d0.tar.bz2
>
>
2019-03-07T16:39:43+0100 Adding link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/testing/x86_64-centos7 -> ../347/stable/x86_64-centos7 `/cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/testing/x86_64-centos7' -> `../347/stable/x86_64-centos7' 2019-03-07T16:39:43+0100 Unpacking lbenv-kit.stable.x86_64-slc6.347-4a87636109.tar.bz2 ... 2019-03-07T16:39:43+0100 Downloading https://lhcb-rpm.web.cern.ch/lhcb-rpm/lbenv-kits/lhcb.cern.ch/lbenv-kit.stable.x86_64-slc6.347-4a87636109.tar.bz2
 tar: Removing leading `/' from member names
Changed:
<
<
2018-12-18T16:49:27+0100 Adding link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/dev/x86_64-slc6 -> ../283/dev/x86_64-slc6 `/cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/dev/x86_64-slc6' -> `../283/dev/x86_64-slc6' 2018-12-18T16:49:27+0100 Unpacking lbenv-kit.prod.x86_64-centos7.283-3b0a5c49cf.tar.bz2 ... 2018-12-18T16:49:27+0100 Downloading https://lhcb-rpm.web.cern.ch/lhcb-rpm/lbenv-kits/lhcb.cern.ch/lbenv-kit.prod.x86_64-centos7.283-3b0a5c49cf.tar.bz2
>
>
2019-03-07T16:39:47+0100 Adding link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/testing/x86_64-slc6 -> ../347/stable/x86_64-slc6 `/cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/testing/x86_64-slc6' -> `../347/stable/x86_64-slc6' 2019-03-07T16:39:47+0100 Unpacking lbenv-kit.unstable.x86_64-centos7.347-68420333c0.tar.bz2 ... 2019-03-07T16:39:47+0100 Downloading https://lhcb-rpm.web.cern.ch/lhcb-rpm/lbenv-kits/lhcb.cern.ch/lbenv-kit.unstable.x86_64-centos7.347-68420333c0.tar.bz2
 tar: Removing leading `/' from member names
Changed:
<
<
2018-12-18T16:49:31+0100 Adding link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/pre-prod/x86_64-centos7 -> ../283/prod/x86_64-centos7 `/cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/pre-prod/x86_64-centos7' -> `../283/prod/x86_64-centos7' 2018-12-18T16:49:31+0100 Unpacking lbenv-kit.prod.x86_64-slc6.283-3b0a5c49cf.tar.bz2 ... 2018-12-18T16:49:31+0100 Downloading https://lhcb-rpm.web.cern.ch/lhcb-rpm/lbenv-kits/lhcb.cern.ch/lbenv-kit.prod.x86_64-slc6.283-3b0a5c49cf.tar.bz2
>
>
2019-03-07T16:39:51+0100 Adding link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/unstable/x86_64-centos7 -> ../347/unstable/x86_64-centos7 `/cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/unstable/x86_64-centos7' -> `../347/unstable/x86_64-centos7' 2019-03-07T16:39:51+0100 Unpacking lbenv-kit.unstable.x86_64-slc6.347-68420333c0.tar.bz2 ... 2019-03-07T16:39:51+0100 Downloading https://lhcb-rpm.web.cern.ch/lhcb-rpm/lbenv-kits/lhcb.cern.ch/lbenv-kit.unstable.x86_64-slc6.347-68420333c0.tar.bz2
 tar: Removing leading `/' from member names
Changed:
<
<
2018-12-18T16:49:36+0100 Adding link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/pre-prod/x86_64-slc6 -> ../283/prod/x86_64-slc6 `/cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/pre-prod/x86_64-slc6' -> `../283/prod/x86_64-slc6' `.installed_kits.txt.2018-12-18T16:49:15+0100' -> `/cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/.installed_kits.txt.2018-12-18T16:49:15+0100' removed `.installed_kits.txt.2018-12-18T16:49:15+0100' `/cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/.installed_kits.txt' -> `.installed_kits.txt.2018-12-18T16:49:15+0100'
>
>
2019-03-07T16:39:57+0100 Adding link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/unstable/x86_64-slc6 -> ../347/unstable/x86_64-slc6 `/cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/unstable/x86_64-slc6' -> `../347/unstable/x86_64-slc6' `.installed_kits.txt.2019-03-07T16:39:37+0100' -> `/cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/.installed_kits.txt.2019-03-07T16:39:37+0100' removed `.installed_kits.txt.2019-03-07T16:39:37+0100' `/cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/.installed_kits.txt' -> `.installed_kits.txt.2019-03-07T16:39:37+0100'
 Starting PUBLISH =================================
Changed:
<
<
Die Dez 18 16:49:36 CET 2018 Using auto tag 'generic-2018-12-18T15:49:37Z'
>
>
gio 7 mar 2019, 16.39.57, CET Using auto tag 'generic-2019-03-07T15:39:57Z'
 Processing changes...
Changed:
<
<
...
>
>
...........
 

Update stable version

Revision 1902019-02-28 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 385 to 385
 
Changed:
<
<

Update prod version

>
>

Update stable version

 
Changed:
<
<
When a new pre-prod version of LbEnv is ready for production, the release shifter is asked to update the prod version, in which case he/she has to run from cvmfs-lhcb.cern.ch:
>
>
When a new testing version of LbEnv is ready for production, the release shifter is asked to update the prod version, in which case he/she has to run from cvmfs-lhcb.cern.ch:
 
Changed:
<
<
update_lbenv_prod.sh
>
>
update_lbenv_stable.sh
  which starts a transaction update the prod version and publish the transaction if successful.

Revision 1892019-02-21 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 132 to 132
 You can ignore the warnings such as: WARNING: Error encountered while acquiring CERN SSO cookie for Ariadne. Attempting to connect with no SSO.
Changed:
<
<
You can now check whether the platforms for your project were updated correctly:
>
>
  • Check whether the platforms for all projects in the stack being deployed were updated correctly, so for each project do:
 lb-sdb-query listPlatforms
Added:
>
>
which should show the complete list of platforms for which your project was released. If it's not the case add the missing ones with
lb-sdb-add-platforms <Project> <version> <platform1> [<platform2>...]
  • Check that the projects are correctly flagged as released: call
    lb-sdb-query listReleases
    
    and check that they are not in the list, otherwise call
    lb-sdb-release -r <Project> <version>
    
    to manually flag them as released.
 
Deleted:
<
<
Should show the complete list of platforms for which your project was released.
 

Software Installation (CVMFS)

Line: 196 to 205
 

  • Prepare the web pages links (NOT for DIRAC and LHCbDirac)
Changed:
<
<
$LHCBDOC/scripts/addrel.py
>
>
/eos/project/l/lhcbwebsites/www/projects/scripts/addrel.py
 

Deployment of Data Packages

Revision 1882019-02-21 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 311 to 311
 

Deployment of LbEnv

Added:
>
>

Install new versions

 LbEnv is the set of LHCb environment scripts and tools used for development. It will be released to CVMFS by a CRON running on cvmfs-lhcb.cern.ch, but in the meantime some tools have been prepared for this task (namely update_lbenv.sh).
Line: 374 to 376
 
Added:
>
>

Update prod version

When a new pre-prod version of LbEnv is ready for production, the release shifter is asked to update the prod version, in which case he/she has to run from cvmfs-lhcb.cern.ch:

update_lbenv_prod.sh
which starts a transaction update the prod version and publish the transaction if successful.

WARNING: in case of failure the transaction is kept open for the shifter to fix the problem or abort it.

 

Special Instructions and troubleshooting

Installation on AFS

Revision 1872019-01-14 - StefanGabrielChitic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 87 to 87
 build_id=1234
      • Try to copy the RPMs to the repository
Changed:
<
<
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018 /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2019 /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
 
      • If everything is fine, do the real copy and update the repository info
Changed:
<
<
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018 --copy /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2019 --copy /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
 
    • Check it's all there
Changed:
<
<
ls -ltr /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018/ | tail -10
>
>
ls -ltr /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2019/ | tail -10
 In case of problems see here

BE CAREFUL: Since 20141010, the release procedure automatically bumps up the release number for packages for which there is already a version in the RPM repo.

Line: 102 to 102
  N.B. If you want to release all rpms EXCEPT some matching a specific string (e.g. gcc62) you can use:
Changed:
<
<
lb-release-rpm -c --rpm-regex '^(?!.*(gcc62)).*' --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018 /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
>
>
lb-release-rpm -c --rpm-regex '^(?!.*(gcc62)).*' --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2019 /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
 
Line: 246 to 246
 build_id=1234
      • Try to copy the RPMs to the repository
Changed:
<
<
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018 /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2019 /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
 
        • For old stacks only released as tar files:
          lb-release-oldtar /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/olddist
          
      • If everything is fine, do the real copy and update the repository info
Changed:
<
<
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018 --copy /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2019 --copy /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
 
        • For old stacks only released as tar files:
          lb-release-oldtar --copy /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/olddist
          
    • Check it's all there
Changed:
<
<
ls -ltr /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018/ | tail -10
>
>
ls -ltr /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2019/ | tail -10
 In case of problems see here

BE CAREFUL: Since 20141010, the release procedure automatically bumps up the release number for packages for which there is already a version in the RPM repo.

Line: 267 to 267
  N.B. If you want to release all rpms EXCEPT some matching a specific string (e.g. gcc62) you can use:
Changed:
<
<
lb-release-rpm -c --rpm-regex '^(?!.*(gcc62)).*' --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018 /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
>
>
lb-release-rpm -c --rpm-regex '^(?!.*(gcc62)).*' --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2019 /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
 

Software Installation (CVMFS)

Line: 854 to 854
 

Now copying it to the RPM repository:

Changed:
<
<
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018 /tmp/rpmbuild/RPMS/noarch/ lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018 --copy /tmp/rpmbuild/RPMS/noarch/
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2019 /tmp/rpmbuild/RPMS/noarch/ lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2019 --copy /tmp/rpmbuild/RPMS/noarch/
 

Now install on plus:

Revision 1862018-12-19 - MichalKreps

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 372 to 372
 Processing changes... ...
Changed:
<
<
>
>
 

Special Instructions and troubleshooting

Revision 1852018-12-18 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 309 to 309
 
      1. Even if you decide to use cvmfs_server instead of the wrapper, it is best to specify the repository in the command as this server could potentially host several repositories
  • Remember to log out
Added:
>
>

Deployment of LbEnv

LbEnv is the set of LHCb environment scripts and tools used for development. It will be released to CVMFS by a CRON running on cvmfs-lhcb.cern.ch, but in the meantime some tools have been prepared for this task (namely update_lbenv.sh).

First check in DRYRUN mode whether the tool is going to update correctly:

 $ DRYRUN=1 update_lbenv.sh
2018-12-18T16:39:35+0100 Will install:
lbenv-kit.dev.x86_64-centos7.283-ca05b181d0.tar.bz2
lbenv-kit.dev.x86_64-slc6.283-ca05b181d0.tar.bz2
lbenv-kit.prod.x86_64-centos7.283-3b0a5c49cf.tar.bz2
lbenv-kit.prod.x86_64-slc6.283-3b0a5c49cf.tar.bz2
2018-12-18T16:39:35+0100 Unpacking lbenv-kit.dev.x86_64-centos7.283-ca05b181d0.tar.bz2 ...
2018-12-18T16:39:35+0100 DRYRUN mode. Would add link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/dev/x86_64-centos7 -> ../283/dev/x86_64-centos7
2018-12-18T16:39:35+0100 Unpacking lbenv-kit.dev.x86_64-slc6.283-ca05b181d0.tar.bz2 ...
2018-12-18T16:39:35+0100 DRYRUN mode. Would add link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/dev/x86_64-slc6 -> ../283/dev/x86_64-slc6
2018-12-18T16:39:35+0100 Unpacking lbenv-kit.prod.x86_64-centos7.283-3b0a5c49cf.tar.bz2 ...
2018-12-18T16:39:35+0100 DRYRUN mode. Would add link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/pre-prod/x86_64-centos7 -> ../283/prod/x86_64-centos7
2018-12-18T16:39:35+0100 Unpacking lbenv-kit.prod.x86_64-slc6.283-3b0a5c49cf.tar.bz2 ...
2018-12-18T16:39:35+0100 DRYRUN mode. Would add link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/pre-prod/x86_64-slc6 -> ../283/prod/x86_64-slc6

If this seems satisfactory:

> update_lbenv.sh
Starting TRANSACTION
Created TRANSACTION
2018-12-18T16:49:16+0100 Will install:
lbenv-kit.dev.x86_64-centos7.283-ca05b181d0.tar.bz2
lbenv-kit.dev.x86_64-slc6.283-ca05b181d0.tar.bz2
lbenv-kit.prod.x86_64-centos7.283-3b0a5c49cf.tar.bz2
lbenv-kit.prod.x86_64-slc6.283-3b0a5c49cf.tar.bz2
2018-12-18T16:49:16+0100 Unpacking lbenv-kit.dev.x86_64-centos7.283-ca05b181d0.tar.bz2 ...
2018-12-18T16:49:16+0100 Downloading https://lhcb-rpm.web.cern.ch/lhcb-rpm/lbenv-kits/lhcb.cern.ch/lbenv-kit.dev.x86_64-centos7.283-ca05b181d0.tar.bz2
tar: Removing leading `/' from member names
2018-12-18T16:49:23+0100 Adding link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/dev/x86_64-centos7 -> ../283/dev/x86_64-centos7
`/cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/dev/x86_64-centos7' -> `../283/dev/x86_64-centos7'
2018-12-18T16:49:23+0100 Unpacking lbenv-kit.dev.x86_64-slc6.283-ca05b181d0.tar.bz2 ...
2018-12-18T16:49:23+0100 Downloading https://lhcb-rpm.web.cern.ch/lhcb-rpm/lbenv-kits/lhcb.cern.ch/lbenv-kit.dev.x86_64-slc6.283-ca05b181d0.tar.bz2
tar: Removing leading `/' from member names
2018-12-18T16:49:27+0100 Adding link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/dev/x86_64-slc6 -> ../283/dev/x86_64-slc6
`/cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/dev/x86_64-slc6' -> `../283/dev/x86_64-slc6'
2018-12-18T16:49:27+0100 Unpacking lbenv-kit.prod.x86_64-centos7.283-3b0a5c49cf.tar.bz2 ...
2018-12-18T16:49:27+0100 Downloading https://lhcb-rpm.web.cern.ch/lhcb-rpm/lbenv-kits/lhcb.cern.ch/lbenv-kit.prod.x86_64-centos7.283-3b0a5c49cf.tar.bz2
tar: Removing leading `/' from member names
2018-12-18T16:49:31+0100 Adding link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/pre-prod/x86_64-centos7 -> ../283/prod/x86_64-centos7
`/cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/pre-prod/x86_64-centos7' -> `../283/prod/x86_64-centos7'
2018-12-18T16:49:31+0100 Unpacking lbenv-kit.prod.x86_64-slc6.283-3b0a5c49cf.tar.bz2 ...
2018-12-18T16:49:31+0100 Downloading https://lhcb-rpm.web.cern.ch/lhcb-rpm/lbenv-kits/lhcb.cern.ch/lbenv-kit.prod.x86_64-slc6.283-3b0a5c49cf.tar.bz2
tar: Removing leading `/' from member names
2018-12-18T16:49:36+0100 Adding link /cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/pre-prod/x86_64-slc6 -> ../283/prod/x86_64-slc6
`/cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/pre-prod/x86_64-slc6' -> `../283/prod/x86_64-slc6'
`.installed_kits.txt.2018-12-18T16:49:15+0100' -> `/cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/.installed_kits.txt.2018-12-18T16:49:15+0100'
removed `.installed_kits.txt.2018-12-18T16:49:15+0100'
`/cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/.installed_kits.txt' -> `.installed_kits.txt.2018-12-18T16:49:15+0100'
Starting PUBLISH =====================================
Die Dez 18 16:49:36 CET 2018
Using auto tag 'generic-2018-12-18T15:49:37Z'
Processing changes...
...

<verbatim></verbatim>
<nop>
 

Special Instructions and troubleshooting

Revision 1842018-10-22 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 77 to 77
 
  • If there are problems, ask the project manager to fix them, and restart the build
  • Update the RPMs repository (Warning, important to be done from any host)
    • connect to lxplus.cern.ch
Changed:
<
<
ssh lxplus.cern.ch
>
>
ssh lxplus7.cern.ch
 bash . /cvmfs/lhcb.cern.ch/lib/LbLoginDev.sh kinit ${USER}@CERN.CH
Line: 236 to 236
 
  • If there are problems, ask the project manager to fix them, and restart the build
  • Update the RPMs repository (Warning, important *to be done from any host *)
    • connect to lxplus.cern.ch
Changed:
<
<
ssh lxplus.cern.ch
>
>
ssh lxplus7.cern.ch
 bash . /cvmfs/lhcb.cern.ch/lib/LbLoginDev.sh kinit ${USER}@CERN.CH

Revision 1832018-10-22 - JoelClosier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 75 to 75
 Actions:
Changed:
<
<
  • Update the RPMs repository (Warning, important to be done from the host lhcb-archive.cern.ch)
    • connect to lhcb-archive.cern.ch
      ssh lhcb-archive.cern.ch
>
>
  • Update the RPMs repository (Warning, important to be done from any host)
    • connect to lxplus.cern.ch
      ssh lxplus.cern.ch
 bash . /cvmfs/lhcb.cern.ch/lib/LbLoginDev.sh kinit ${USER}@CERN.CH
Line: 234 to 234
 Actions:
  • If you didn't trigger the build, check that everything is OK (see trigger data package build section)
  • If there are problems, ask the project manager to fix them, and restart the build
Changed:
<
<
  • Update the RPMs repository (Warning, important to be done from the host lhcb-archive.cern.ch)
    • connect to lhcb-archive.cern.ch
      ssh lhcb-archive.cern.ch
>
>
  • Update the RPMs repository (Warning, important *to be done from any host *)
    • connect to lxplus.cern.ch
      ssh lxplus.cern.ch
 bash . /cvmfs/lhcb.cern.ch/lib/LbLoginDev.sh kinit ${USER}@CERN.CH
Line: 448 to 448
 
      • build log: summaries.<CMTCONFIG>/<Project>/build_log.html
      • test reports: summaries.<CMTCONFIG>/<Project>/html
Changed:
<
<
  • Publish the RPMs (from lhcb-archive.cern.ch)
>
>
  • Publish the RPMs (from lxplus.cern.ch)
 cp -v -n artifacts/MYPROJECT*.rpm $LHCBTAR/rpm/lhcb
Changed:
<
<
then, from lhcb-archive.cern.ch
>
>
then, from lxplus.cern.ch
 createrepo --workers=20 --update $LHCBTAR/rpm/lhcb
Changed:
<
<
  • If lhcb-archive.cern.ch works, continue with the procedure to install to AFS, otherwise you should install on AFS from the generated tar files, then continue with the normal procedure
>
>
  • If lxplus.cern.ch works, continue with the procedure to install to AFS, otherwise you should install on AFS from the generated tar files, then continue with the normal procedure
 

If lb-release-rpm prints messages like "error: not an rpm package"

Revision 1822018-10-11 - MarcoCattaneo

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 323 to 323
 

Removing Software (archival)

Projects

Changed:
<
<
  • Archive the project documentation (step to be defined, for the time being try python $LHCBDOC/scripts/gc_archive.py <project> <version>, noting that <project> must be in all lowercase)
>
>
  • Archive the project documentation (step to be defined, for the time being try python $LHCBDOC/scripts/gc_archive.py <project> <version>, noting that <project> must be in all lowercase). This must be done before removing the software!
 
  • On cvmfs-lhcb, start a transaction in the usual way, then list the RPMs you want to remove using the cvmfslbinstall command:

Revision 1812018-10-10 - MarcoCattaneo

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 323 to 323
 

Removing Software (archival)

Projects

Changed:
<
<
  • Archive the project documentation (step to be defined, ignore for the moment)
>
>
  • Archive the project documentation (step to be defined, for the time being try python $LHCBDOC/scripts/gc_archive.py <project> <version>, noting that <project> must be in all lowercase)
 
  • On cvmfs-lhcb, start a transaction in the usual way, then list the RPMs you want to remove using the cvmfslbinstall command:

Revision 1802018-08-03 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 323 to 323
 

Removing Software (archival)

Projects

Changed:
<
<
  • On lxplus or one of the lxbuild machines (AFS and CVMFS access, as well as the afs_admin command are needed)
    • Set up the AFS environment with the command
      afsLbLoginDev
    • Run the command:
      archive_project <Project> <version>
      This removes the software from the AFS release area and from the old Grid distribution area, and archives the tar files to castor. You need a grid proxy, since the first thing the script does is to contact the book-keeping to check that there isn't an active production step using the software to be archived. You have to hit several times to confirm that you want to remove the tar balls (don't say yes if there was some problem copying to castor)
>
>
  • Archive the project documentation (step to be defined, ignore for the moment)
 
  • On cvmfs-lhcb, start a transaction in the usual way, then list the RPMs you want to remove using the cvmfslbinstall command:
Line: 346 to 342
 URANIA_v6r2p1_x86_64_slc6_gcc62_opt 1.0.0 1 local
Changed:
<
<
Then you can remove the packages using cvmfslbinstall remove
>
>
Then you can remove the packages using cvmfslbinstall remove, passing it the list of files to be removed (it will order them correctly and check dependencies). Please note that lbinstall does NOT remove directories if they contain files that do not belong to the RPM packages (e.g. generated files). Therefore it is necessary to check the project top directory if still existing and check what is left and remove manually if necessary.
  N.B. after removing the packages, the files created by hand that do not belong to any package are NOT removed. You may need to check the directory for those.
Added:
>
>
N.B. Be careful about the packages returned, cvmfslbinstall list URANIA_v6r2 returns the packages for v6r2 and v6r2p1...
  Remember to run cvmfs_publish at the end, as usual, to close the transaction.

Data Packages

Deleted:
<
<
  • unlock the package directory
    unlockAFSDir -R /afs/cern.ch/lhcb/software/releases/DBASE/<Hat>/<Package>/<version>
  • remove the package
    /afs/cern.ch/lhcb/software/lbinstall/lbinstall remove <package_name>
 
  • got to the CVMFS publishing machine (cvmfs-lhcb) and uninstall
    cvmfs_transaction
Changed:
<
<
remove_software.sh
>
>
cvmfslbinstall remove ___
 cvmfs_publish

Revision 1792018-07-30 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Revision 1782018-06-06 - ChristopherRJones

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 82 to 82
 kinit ${USER}@CERN.CH
Changed:
<
<
      • try to copy the RPMs to the repository
>
>
      • Define the build ID to extract RPMS from
 build_id=1234
Added:
>
>
      • Try to copy the RPMs to the repository
 lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018 /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
Changed:
<
<
      • if everything is fine, do the real copy and update the repository info
>
>
      • If everything is fine, do the real copy and update the repository info
 lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018 --copy /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
    • Check it's all there
Line: 239 to 241
 kinit ${USER}@CERN.CH
Changed:
<
<
      • try to copy the RPMs to the repository
>
>
      • Define the build ID to extract RPMS from
 build_id=1234
Added:
>
>
      • Try to copy the RPMs to the repository
 lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018 /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
        • For old stacks only released as tar files:
          lb-release-oldtar /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/olddist
          
Changed:
<
<
      • if everything is fine, do the real copy and update the repository info
>
>
      • If everything is fine, do the real copy and update the repository info
 lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018 --copy /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
        • For old stacks only released as tar files:

Revision 1772018-06-05 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 117 to 117
 lb-sdb-import
Added:
>
>
 
  • Update the software database.
    First try in dry-run mode:
    lb-deployment-updatesdb /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id
    
    If more platforms/projects are being updated than necessary you can limit using the options of the lb-deployment-updatesdb command.If all ok, just run:
Line: 128 to 129
 You can ignore the warnings such as: WARNING: Error encountered while acquiring CERN SSO cookie for Ariadne. Attempting to connect with no SSO.
Added:
>
>
You can now check whether the platforms for your project were updated correctly:
lb-sdb-query listPlatforms <Project> <version>

Should show the complete list of platforms for which your project was released.

 

Software Installation (CVMFS)

Line: 156 to 163
 
  • Install the project (all registered platforms)
    install_software.sh <Project> <version>
    
Changed:
<
<
    • in case of problems or missing dependencies you can alter the options to lbinstall by setting the OPTS environment variable e.g.
      OPTS=--no-strict install_software.sh <Project> <version>
      
>
>
    • Note:
      1. in case of problems or missing dependencies you can alter the options to lbinstall by setting the OPTS environment variable e.g
         OPTS=--no-strict install_software.sh <Project> <version>
      2. This dependency problem occurs systematically during LHCbDirac deployments. This is a known issue that we are not planning to fix as LHCbDirac will soon be deployed differently.
      3. If the binary RPMs do not get deployed (packages with x86_64[...]), check wether the SoftwareConfigurationDB was updated correctly as per the following instructions. The DB is the source of information for the list of platforms to deploy.
 
  • Check that it all went fine with
    ls -ltr /cvmfs/lhcb.cern.ch/lib/lhcb/<PROJECT>/<PROJECT>_<version>/InstallArea/
    

Revision 1762018-06-01 - MarcoCattaneo

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 475 to 475
 /afs/cern.ch/lhcb/software/lbinstall/lbinstall list ${MyProject^^}_${MyVersion}
    • actually remove them
Changed:
<
<
/afs/cern.ch/lhcb/software/lbinstall/lbinstal remove $(/afs/cern.ch/lhcb/software/rpmrel/afslbpkr list ${MyProject^^}_${MyVersion})
>
>
/afs/cern.ch/lhcb/software/lbinstall/lbinstall remove $(/afs/cern.ch/lhcb/software/rpmrel/afslbpkr list ${MyProject^^}_${MyVersion})
 
  • remove the AFS volume in the release area
    afs_admin delete /afs/cern.ch/lhcb/software/releases/${MyProject^^}/${MyProject^^}_${MyVersion}

Revision 1752018-05-31 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 18 to 18
 
  1. Release manager tags a project (or several) and triggers the build by importing the projects to the software configuration database
  2. Jenkins builds the project(s) and prepares the RPM packages
Changed:
<
<
  1. Release manager checks the outcome of the build and either restarts the build or requests a release in JIRA
>
>
  1. Release manager checks the outcome of the build and either restarts the build or requests a release in JIRA
 
  1. Release shifter copies the RPM packages to the official LHCb package repository (YUM format). At this point, anybody can perform local installations using lbinstall
  2. Release shifter marks the RPM as "released" in the software configuration database, and specifies which platforms have effectively been released. This is necessary for the build system to know what is released and isn't when building stacks, but also for the automated release procedures on CVMFS to know which platforms should be installed
  3. Release shifter performs the installation on CVMFS and updates the documentation
Line: 35 to 35
  Within 10-15 min the build should start to appear in the nightlies page https://lhcb-nightlies.cern.ch/release. Note that, for a dependency tree of projects, lb-sdb-import can be called just once for the top level project as it will pull in all the dependencies (unless you need to provide the --sourceurl option, in which case each project has to be added individually, in the dependency order).
  • if there is a problem in the build, it can be re-started via the dedicated button after you must log in (the build will not restart by itself after a retag)
Changed:
<
<
  • once satisfied by the build, take note of the build id (you can use the direct link icon) and make the request via LHCb Deployment Request form
>
>
  • once satisfied by the build, take note of the build id (you can use the direct link icon) and make the request via LHCb Deployment Request form
 

For Data Packages

Changed:
<
<
>
>
 

For Release Shifters - Deployment Prerequisites

Line: 52 to 52
 

Prerequisites for new release shifters

If you are a new release shifter, please make sure you have the prerequisites that are needed at some point of the release procedure.
Changed:
<
<
>
>
 
  • You must be added to the lhcb-cvmfs-librarians E-group (for access to the IT managed CVMFS installation machine and for communication with other release shifters)
  • You must have a valid grid certificate (For release of DecFiles package)
  • You need to be registered as a software deployment shifter in the shifts database
Line: 358 to 358
 Then increase quota with
<!-- SyntaxHighlightingPlugin -->
afs_admin sq /afs/cern.ch/lhcb/software/releases/<PROJECT>/<PROJECT>_<version>/*
<!-- end SyntaxHighlightingPlugin -->
Changed:
<
<
In rpms... on 2/12/2016 Ben did (see JIRA):
>
>
In rpms... on 2/12/2016 Ben did (see JIRA):
 %SYNTAX{ syntax="sh"}% for r in $LHCBTAR/rpm/lhcb/__*rpm; do for p in `rpm -qp --requires $r | grep LCG`;

Revision 1742018-04-16 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 303 to 303
 

Deployment from Old Tarfiles

Very old versions of software projects cannot be deployed with RPMs, in which case it's possible to refer to the instructions at ProjectReleaseOld.
Added:
>
>
 

Removing Software (archival)

Projects

  • On lxplus or one of the lxbuild machines (AFS and CVMFS access, as well as the afs_admin command are needed)

Revision 1732018-04-16 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 311 to 311
  This removes the software from the AFS release area and from the old Grid distribution area, and archives the tar files to castor. You need a grid proxy, since the first thing the script does is to contact the book-keeping to check that there isn't an active production step using the software to be archived. You have to hit several times to confirm that you want to remove the tar balls (don't say yes if there was some problem copying to castor)
Changed:
<
<
  • On cvmfs-lhcb, start a transaction in the usual way, then run the command:
    remove_software.sh <Project> <version>
    . Unfortunately this is currently (28th Nov 2017) broken, see https://its.cern.ch/jira/browse/LBCORE-1488.
>
>
  • On cvmfs-lhcb, start a transaction in the usual way, then list the RPMs you want to remove using the cvmfslbinstall command:

cvmfslbinstall list URANIA_v6r2p1
URANIA_v6r2p1                            1.0.0   1    local
URANIA_v6r2p1_index                      1.0.0   1    local
URANIA_v6r2p1_x86_64_centos7_gcc62_dbg   1.0.0   1    local
URANIA_v6r2p1_x86_64_centos7_gcc62_do0   1.0.0   1    local
URANIA_v6r2p1_x86_64_centos7_gcc62_opt   1.0.0   1    local
URANIA_v6r2p1_x86_64_slc6_gcc49_dbg      1.0.0   1    local
URANIA_v6r2p1_x86_64_slc6_gcc49_do0      1.0.0   1    local
URANIA_v6r2p1_x86_64_slc6_gcc49_opt      1.0.0   1    local
URANIA_v6r2p1_x86_64_slc6_gcc62_dbg      1.0.0   1    local
URANIA_v6r2p1_x86_64_slc6_gcc62_do0      1.0.0   1    local
URANIA_v6r2p1_x86_64_slc6_gcc62_opt      1.0.0   1    local

Then you can remove the packages using cvmfslbinstall remove

N.B. after removing the packages, the files created by hand that do not belong to any package are NOT removed. You may need to check the directory for those.

  Remember to run cvmfs_publish at the end, as usual, to close the transaction.

Revision 1722018-02-13 - MichalKreps

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 292 to 292
 
      1. Even if you decide to use cvmfs_server instead of the wrapper, it is best to specify the repository in the command as this server could potentially host several repositories
  • Remember to log out
Deleted:
<
<

Special Case: Gen/DecFiles

From lxplus, once the changes on CVMFS have been propagated (check with ls /cvmfs/lhcb.cern.ch/lib/lhcb/DBASE/Gen/DecFiles), you have to update the bookkeeping.

  • Get a valid Grid proxy
    lhcb-proxy-init
    
  • Update the bookkeeping
    cd /cvmfs/lhcb.cern.ch/lib/lhcb/DBASE/Gen/DecFiles/<version>
    lb-run -c best LHCbDirac dirac-bookkeeping-eventtype-mgt-insert doc/table_event.sql
    lb-run -c best LHCbDirac dirac-bookkeeping-eventtype-mgt-update doc/table_event.sql
    lb-run -c best LHCbDirac dirac-bookkeeping-eventtype-mgt-update doc/table_obsolete.sql
    

The dirac-bookkeeping-eventtype-mgt-insert command will produce many errors like

{'EVTTYPEID': '12165431', 'DESCRIPTION': 'Bu_D02pi+pi-,K+pi-pi0=PHSP,DecProdCut', 'PRIMARY': '[B+ -> pi+ pi+ pi- (D0bar -> K+ pi- pi0)]cc'} : Excution failed.: ( ORA-00001: unique constraint (LHCB_DIRACBOOKKEEPING.SYS_C00302542) violated
ORA-06512: at "LHCB_DIRACBOOKKEEPING.BOOKKEEPINGORACLEDB", line 1131
ORA-06512: at line 1
 ) 
This is normal as most event types already exist in the Oracle table. Things are OK if the last line is like
The following event types are inserted: ['42102201', '42102200', '42163203', '42102221', '42102220', '15104451', '42100210', '42100211', '42100230', '42100231', '42142201', '42142200', '42102210', '42102211', '42162203', '42162202', '15574100', '42100221', '42100220', '12163436', '12163437', '42102601', '42102600', '12163236', '42163202', '12163235', '42112200', '42112201'] 
 

Special Instructions and troubleshooting

Revision 1712018-01-22 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 42 to 42
 
Changed:
<
<

For Release Shifters - Release of the build

>
>

For Release Shifters - Deployment Prerequisites

 See the LHCb shift database to find who is this week's shifter.
Line: 50 to 50
 Starting from 13 October 2016, you should set your environment to DEV as : ". /cvmfs/lhcb.cern.ch/lib/LbLoginDev.sh"

Prerequisites for new release shifters

Changed:
<
<
If you are a new release shifter, please make sure you have the prerequisites that are needed at some point of the release procedure.
<!--/twistyPlugin twikiMakeVisibleInline-->
>
>
If you are a new release shifter, please make sure you have the prerequisites that are needed at some point of the release procedure.
 
  • You should be added as a "developer" in the LHCb Deployment Jira project (https://sft.its.cern.ch/jira/browse/LHCBDEP/), to receive and track release requests
  • You must be added to the lhcb-cvmfs-librarians E-group (for access to the IT managed CVMFS installation machine and for communication with other release shifters)
Line: 60 to 60
 
  • Check that the CMTPATH environment variable is not defined in the shell which you are using

As shifter, you can self-assign shifts through the shift database shift database. Usually this is done by block-booking weeks for individual shifters.

Deleted:
<
<
<!--/twistyPlugin-->
 
Changed:
<
<

Release of Projects

>
>

Deployment of Projects

 

Update of the RPM repository

Line: 176 to 176
 Note: this is not about Doxygen documentation (automatically generated, with some delay, after the installation)

Actions:

Added:
>
>
  • Wait for the project to be visible in CVMFS:
    ls -ltr /cvmfs/lhcb.cern.ch/lib/lhcb/<PROJECT>/<PROJECT>_<version>/InstallArea/
    
 
  • Prepare the web pages links (NOT for DIRAC and LHCbDirac)
    $LHCBDOC/scripts/addrel.py <Project> <version>
    
Changed:
<
<

Release of Data Packages

>
>

Deployment of Data Packages

  Warning, important WARNING The procedure for the release of data packages via RPMs is still under heavy development and the automation is not yet in place.

Revision 1702018-01-22 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 7 to 7
 

Scope

The following documentation applies to the release of LHCb Physics projects as well as data packages.

Changed:
<
<
Ganga is installed on a separate CVMFS volume, and there are specific instructions for DIRAC and LHCbDIrac.
>
>
Ganga is installed on a separate CVMFS volume, and there are specific instructions for DIRAC and LHCbDIrac.
 
Changed:
<
<
Warning, important These instructions do not apply to LbScripts. Use LbScriptsRelease instead.
>
>
Warning, important These instructions do not apply to LbScripts. Use LbScriptsRelease instead.
 

Overview of the procedure

Changed:
<
<
The software is build by the Jenkins continuous integration engine.
>
>
The software is built by the Jenkins continuous integration engine.
 The configuration of the projects, their dependencies and which stacks should be built is defined in the software configuration database, with the typical procedure including the following steps:
Changed:
<
<
  1. Release manager tags a project (or several) and triggers the build by imported the projects to the software configuration database.
>
>
  1. Release manager tags a project (or several) and triggers the build by importing the projects to the software configuration database
 
  1. Jenkins builds the project(s) and prepares the RPM packages
  2. Release manager checks the outcome of the build and either restarts the build or requests a release in JIRA
  3. Release shifter copies the RPM packages to the official LHCb package repository (YUM format). At this point, anybody can perform local installations using lbinstall
Changed:
<
<
  1. Release shifter marks the RPM as "released" in the software configuration database, and specifies which platforms have effectively been released. This is necessary for the build system to know what is released and isn't when building stacks, but also for the automated release procedures on AFS and CVMFS to know which platforms should be installed.
  2. Release shifter performs the installation on AFS and updates the documentation (still AFS dependent).
  3. Release shifter performs the installation on CVMFS
>
>
  1. Release shifter marks the RPM as "released" in the software configuration database, and specifies which platforms have effectively been released. This is necessary for the build system to know what is released and isn't when building stacks, but also for the automated release procedures on CVMFS to know which platforms should be installed
  2. Release shifter performs the installation on CVMFS and updates the documentation
 
Changed:
<
<
N.B. Installations on CVMFS and AFS can be performed in parallel.

Detailed instructions for all the steps are detailed below.

>
>
Instructions for all the steps are detailed below.
 

For Release Managers - Preparation of the build

For Projects

  • prepare the project as usual (release notes, tags, etc.), following the instructions at PrepareProjectReleaseGit
  • register the project/version to the software database
Changed:
<
<
lb-sdb-import MyProject vXrY
>
>
lb-sdb-import
 
Deleted:
<
<
(If the tag is in git while the project is still by default maintained in SVN you need to tell lb-sdb-import to look in git, e.g.: lb-sdb-import  --sourceuri="gitlab-cern:LHCb/Panoramix#v23r2" Panoramix v23r2)
  Within 10-15 min the build should start to appear in the nightlies page https://lhcb-nightlies.cern.ch/release. Note that, for a dependency tree of projects, lb-sdb-import can be called just once for the top level project as it will pull in all the dependencies (unless you need to provide the --sourceurl option, in which case each project has to be added individually, in the dependency order).
Changed:
<
<
  • if there is a problem in the build, it can be re-started via the dedicated button (it will not restart by itself after a retag)
>
>
  • if there is a problem in the build, it can be re-started via the dedicated button after you must log in (the build will not restart by itself after a retag)
 
  • once satisfied by the build, take note of the build id (you can use the direct link icon) and make the request via LHCb Deployment Request form

For Data Packages

Changed:
<
<
>
>
 

For Release Shifters - Release of the build

Line: 51 to 47
 

Special information

Changed:
<
<
Starting from 13 October 2016, you should set your environment to DEV as : ". /afs/cern.ch/lhcb/software/releases/LBSCRIPTS/dev/InstallArea/scripts/LbLogin.sh"
>
>
Starting from 13 October 2016, you should set your environment to DEV as : ". /cvmfs/lhcb.cern.ch/lib/LbLoginDev.sh"
 

Prerequisites for new release shifters

If you are a new release shifter, please make sure you have the prerequisites that are needed at some point of the release procedure.
<!--/twistyPlugin twikiMakeVisibleInline-->

Deleted:
<
<
  • You must be member of the z5:librarians AFS group in order to have the correct access to the AFS areas where the software is installed.
  • You must be member or the _lhcb_ AFS group to have to have the AFS administrator rights that allow you to create AFS volumes for LHCb and to install the rpms in the LHCb rpm repository
  • You must have a valid grid certificate (For release of DecFiles package)
 
  • You must be added to the lhcb-cvmfs-librarians E-group (for access to the IT managed CVMFS installation machine and for communication with other release shifters)
Changed:
<
<
  • Check that the CMTPATH environment variable is not defined in the shell which you are using.
>
>
  • You must have a valid grid certificate (For release of DecFiles package)
 
  • You need to be registered as a software deployment shifter in the shifts database
  • You need to be registered to use Jenkins, to be able to trigger release builds from the web interface (for data packages)
Added:
>
>
  • Check that the CMTPATH environment variable is not defined in the shell which you are using
 
Changed:
<
<
As the shifter you can self-assign shifts through the shift database shift database. Usually this is done by block-booking weeks for individual shifters.
>
>
As shifter, you can self-assign shifts through the shift database shift database. Usually this is done by block-booking weeks for individual shifters.
 
<!--/twistyPlugin-->
Changed:
<
<

Using the checklist tool

The script lb-deployment-checklist simplifies the release shifter task by checking what needs to be done and either prints the next command to run or points to this page.

So connect to lxplus (or equivalent SLC6 machine) and run

lb-deployment-checklist MyProject vXrY
and follow the instructions.

The latest version of lb-deployment-checklist may cache some data to speed up the execution. If you think it's showing wrong messages, try to remove the directory ~/.cache/lb-deployment/.

Release of projects

>
>

Release of Projects

 

Update of the RPM repository

The goal of this step is to make sure that:

Changed:
<
<
  1. The RPMs have been copied to the YUM repository (curently in EOS)
>
>
  1. The RPMs have been copied to the YUM repository (currently in EOS)
 
  1. The YUM metadata has been updated (the lb-release-rpm internally uses the "createrepo" yum command).
Changed:
<
<

Warning, important This step has to be performed from the host lhcb-archive.cern.ch

  • go there
>
>
Actions:
  • Check on the release build summaries page if the build is OK: https://lhcb-nightlies.cern.ch/release
  • If there are problems, ask the project manager to fix them, and restart the build
  • Update the RPMs repository (Warning, important to be done from the host lhcb-archive.cern.ch)
    • connect to lhcb-archive.cern.ch
 ssh lhcb-archive.cern.ch bash
Changed:
<
<
. /afs/cern.ch/lhcb/software/releases/LBSCRIPTS/dev/InstallArea/scripts/LbLogin.sh kinit Your_Username@CERNNOSPAMPLEASE.CH
>
>
. /cvmfs/lhcb.cern.ch/lib/LbLoginDev.sh kinit ${USER}@CERN.CH
 
Deleted:
<
<
 
  • publish the produced RPMs (all of them, unless explicitly requested otherwise, see partial install section)
    • try to copy the RPMs to the repository
      build_id=1234
Line: 106 to 86
 build_id=1234 lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018 /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
Deleted:
<
<
      • For old stacks only released as tar files:
        lb-release-oldtar /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/olddist
               
 
    • if everything is fine, do the real copy and update the repository info
      lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018 --copy /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
      
Deleted:
<
<
      • For old stacks only released as tar files:
        lb-release-oldtar --copy /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/olddist
        
 
  • Check it's all there
    ls -ltr /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018/ | tail -10
    
    In case of problems see here
Line: 138 to 110
 
  1. The project is referenced properly in the Software configuration DB
  2. The list of platforms released for the project have been updated
Changed:
<
<
>
>
Actions:
 
  • Verify that the software configuration DB has been updated:
Changed:
<
<
MyProject= MyVersion=vXrY echo ${MyProject^^} ${MyVersion} lb-sdb-query d $MyProject $MyVersion
>
>
lb-sdb-query d
 This should show projects in the slot or already released. If some are missing re-run:
Changed:
<
<
lb-sdb-import $MyProject $MyVersion
>
>
lb-sdb-import
 
Changed:
<
<
  • update the software database.
    First try in dry-run mode:
>
>
  • Update the software database.
    First try in dry-run mode:
 lb-deployment-updatesdb /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id If more platforms/projects are being updated than necessary you can limit using the options of the lb-deployment-updatesdb command.If all ok, just run:
lb-deployment-updatesdb --update /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id
Line: 157 to 126
 export SDBURL=http://localhost:12345/db/data/ lb-deployment-updatesdb [...] You can ignore the warnings such as:
Changed:
<
<
WARNING: Error encountered while acquiring CERN SSO cookie for Ariadne. Attempting to connect with no SSO..
>
>
WARNING: Error encountered while acquiring CERN SSO cookie for Ariadne. Attempting to connect with no SSO.
 
Deleted:
<
<
  • From here on you will be installing the software on AFS and on CVMFS. The two installations can be done in parallel*
  • For Dirac and LHCbDirac you can ONLY do CVMFS installation
 
Added:
>
>

Software Installation (CVMFS)

 
Changed:
<
<

AFS Software Installation

Warning, important This step has to be performed from the host lhcb-archive.cern.ch

Install on AFS

  • First, set some environment variables that define the project name and version to be installed. Used in the following commands (so you can simply cut and paste them directly). Works for bash shells, so recommend switching to this if it isn't your default
    MyProject=<ProjectName>
    MyVersion=vXrY
    
    To check the following commands will work for you, now run
    echo ${MyProject^^} ${MyVersion}
    
    which should return the project name, in upper case.
  • prepare the AFS volume
    lb-project-manage-volume -c $MyProject $MyVersion
    
  • install the new project
    lb-deployment-afs-install $MyProject $MyVersion
    
    In case of problems or missing dependencies you can alter the options to lbinstall by setting the OPTS environment variable e.g.:
    OPTS=--no-strict lb-deployment-afs-install $MyProject $MyVersion
>
>
Actions:
  • Connect to "cvlhcb" account on host "cvmfs-lhcb"
    ssh yourusername@cvmfs-lhcb
 
Changed:
<
<
  • Check it's all there
    ls -l /afs/cern.ch/lhcb/software/releases/${MyProject^^}/${MyProject^^}_${MyVersion}/
    ls -l /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018/ | grep ${MyProject^^} | grep ${MyVersion}
    
    In case of problems see here

All cases:

  • release and lock the AFS volume
    lb-project-manage-volume -r $MyProject $MyVersion
    lb-project-manage-volume -l $MyProject $MyVersion
>
>
  • Check that nobody else is connected to install software
    who -q ; pgrep -fl -u cvlhcb
 
Changed:
<
<
  • prepare the web pages links (NOT for DIRAC and LHCbDirac)
    $LHCBDOC/scripts/addrel.py $MyProject $MyVersion
>
>
if someone else is connected and if there are processes running as cvlhcb, perhaps try sending them an email to see what they are doing before proceeding
  • sudo into the all powerful account
    sudo -i -u cvlhcb
 
Added:
>
>
  • Start a transaction on the server to be able to write to the disk
    cvmfs_transaction
    
    • if the command returns that a transaction was already created, somebody else is installing at the same time. Check with the who command before going any further.
    • Note:
      1. cvmfs_transaction is a wrapper around the CVMFS command that creates the transaction on the repository lhcb.cern.ch, and logs the command and result. It is best to use it instead of using the direct command cvmfs_server transaction lhcb.cern.ch
      2. even if you decide to use cvmfs_server instead of the wrapper, it is best to specify the repository in the command as this server could potentially host several repositories in the future.
 
Added:
>
>
  • Install the project (all registered platforms)
    install_software.sh <Project> <version>
    
    • in case of problems or missing dependencies you can alter the options to lbinstall by setting the OPTS environment variable e.g.
      OPTS=--no-strict install_software.sh <Project> <version>
      
  • Check that it all went fine with
    ls -ltr /cvmfs/lhcb.cern.ch/lib/lhcb/<PROJECT>/<PROJECT>_<version>/InstallArea/
    
    the platforms you have installed should all be there.
  • Update the CVMFS catalogs
    cvmfs_publish
    
    • Note:
      1. cvmfs_publish is a wrapper around the CVMFS command that publishes the transaction on the repository lhcb.cern.ch, and logs the command and result. It is best to use it instead of using the direct command cvmfs_server publish lhcb.cern.ch
      2. Even if you decide to use cvmfs_server instead of the wrapper, it is best to specify the repository in the command as this server could potentially host several repositories
  • Remember to log out
 
Changed:
<
<
  • FOR OLD STACKS NOT RELEASED VIA RPM: For Gauss, it is necessary to create the Genser tarball
    mkMCGentarFromRPM ${MyProject} ${MyVersion} ${CMTCONFIG}
    
    but make sure that ${CMTCONFIG} is a platform valid for Gauss. Else use another one.

CVMFS Software Installation

>
>

Update Software Project Web Page

Note: this is not about Doxygen documentation (automatically generated, with some delay, after the installation)
 
Changed:
<
<
  • install the software on CVMFS if not already done in parallel. See here for instructions
>
>
Actions:
  • Prepare the web pages links (NOT for DIRAC and LHCbDirac)
    $LHCBDOC/scripts/addrel.py <Project> <version>
    
 
Changed:
<
<

Release of Data packages

>
>

Release of Data Packages

 Warning, important WARNING The procedure for the release of data packages via RPMs is still under heavy development and the automation is not yet in place.
Deleted:
<
<

Specifics

 Data packages are different from standard projects:
  • They do not need to be compiled for several platforms (make is run nonetheless to perform basic actions)
  • They are part of the DBASE or PARAM projects
Line: 220 to 192
  The release steps are therefore different from normal projects.
Changed:
<
<

Procedure

Trigger by hand the lhcb-release Jenkins job (if needed)

>
>

Trigger by hand the lhcb-release Jenkins job (if needed)

 
Line: 231 to 202
 
Changed:
<
<
    • click on the small Jenkins icon in the "Project" column of the build summary look for the package build log (after the line "...:INFO... building Hat/Package")
>
>
    • click on the project name (DBASE or PARAM) in the build summary, and look for the package build log (after the line "building data packages in...")

Update of the RPM repository

The goal of this step is to make sure that:

  1. The RPMs have been copied to the YUM repository (currently in EOS)
  2. The YUM metadata has been updated (the lb-release-rpm internally uses the "createrepo" yum command).
 
Changed:
<
<

publish the RPMs

  • If the build was fine, publish the RPMs (from lhcb-archive.cern.ch)
    • Since default SW is now CVMFS, switch to AFS
>
>
Actions:
  • If you didn't trigger the build, check that everything is OK (see trigger data package build section)
  • If there are problems, ask the project manager to fix them, and restart the build
  • Update the RPMs repository (Warning, important to be done from the host lhcb-archive.cern.ch)
    • connect to lhcb-archive.cern.ch
 ssh lhcb-archive.cern.ch bash
Changed:
<
<
/afs/cern.ch/lhcb/software/releases/LBSCRIPTS/dev/InstallArea/scripts/LbLogin.sh kinit Your_Username@CERNNOSPAMPLEASE.CH
>
>
. /cvmfs/lhcb.cern.ch/lib/LbLoginDev.sh kinit ${USER}@CERN.CH
 
Added:
>
>
 
    • try to copy the RPMs to the repository
      build_id=1234
Changed:
<
<
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018 /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018 /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
 
Changed:
<
<
    • if everything is fine, do the real copy and update the repository info
      lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018 \
           --copy /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
>
>
        • For old stacks only released as tar files:
          lb-release-oldtar /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/olddist
 
Changed:
<
<
    • check it's all there
      ls -ltr /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018/ | tail -10

Install the new RPM on AFS

(must be done from lhcb-archive)

/afs/cern.ch/lhcb/software/lbinstall/afslbinstall install DBASE_Hat_Package_version
>
>
      • if everything is fine, do the real copy and update the repository info
        lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018 --copy /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
 
Changed:
<
<
<!--/twistyPlugin twikiMakeVisibleInline-->
Gen/DecFiles usually come in pairs (two versions for one JIRA task): one for dev and one for legacy Gauss versions. Michal will give instructions in this case (always follow his instructions, which take precedence over what is written below).
/afs/cern.ch/lhcb/software/lbinstall/afslbinstall install DBASE_Gen_DecFiles
will install the latest version. To get the legacy version do
/afs/cern.ch/lhcb/software/lbinstall/afslbinstall install DBASE_Gen_DecFiles_vXXrYY
>
>
        • For old stacks only released as tar files:
          lb-release-oldtar --copy /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/olddist
 
Changed:
<
<
where XX, YY are given in the JIRA task.
>
>
    • Check it's all there
      ls -ltr /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018/ | tail -10
      
      In case of problems see here
 
Changed:
<
<
Then one needs an extra step to update the bookkeeping. It requires to have a valid Grid certificate
  1. Get your Grid proxy:
              lb-run LHCbDirac lhcb-proxy-init
         
  2. Update the bookkeeping:
             cd /afs/cern.ch/lhcb/software/releases/DBASE/Gen/DecFiles/<version>/cmt  
             lb-run LHCbDirac dirac-bookkeeping-eventtype-mgt-insert ../doc/table_event.sql
             lb-run LHCbDirac dirac-bookkeeping-eventtype-mgt-update ../doc/table_event.sql
             lb-run LHCbDirac dirac-bookkeeping-eventtype-mgt-update ../doc/table_obsolete.sql
         
>
>
BE CAREFUL: Since 20141010, the release procedure automatically bumps up the release number for packages for which there is already a version in the RPM repo. If your slot contains more projects that the ones requested for release, you need to ONLY copy the needed ones to the RPM repo. In that case the --rpm-regex option of lb-release-rpm should be used.
 
Changed:
<
<
The dirac-bookkeeping-eventtype-mgt-insert command will produce many errors like
{'EVTTYPEID': '12165431', 'DESCRIPTION': 'Bu_D02pi+pi-,K+pi-pi0=PHSP,DecProdCut', 'PRIMARY': '[B+ -> pi+ pi+ pi- (D0bar -> K+ pi- pi0)]cc'} : Excution failed.: ( ORA-00001: unique constraint (LHCB_DIRACBOOKKEEPING.SYS_C00302542) violated
ORA-06512: at "LHCB_DIRACBOOKKEEPING.BOOKKEEPINGORACLEDB", line 1131
ORA-06512: at line 1
 ) 
This is normal as most event types already exist in the Oracle table. Things are OK if the last line is like
The following event types are inserted: ['42102201', '42102200', '42163203', '42102221', '42102220', '15104451', '42100210', '42100211', '42100230', '42100231', '42142201', '42142200', '42102210', '42102211', '42162203', '42162202', '15574100', '42100221', '42100220', '12163436', '12163437', '42102601', '42102600', '12163236', '42163202', '12163235', '42112200', '42112201'] 
>
>
N.B. If you want to release all rpms EXCEPT some matching a specific string (e.g. gcc62) you can use:
lb-release-rpm -c --rpm-regex '^(?!.*(gcc62)).*'  --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018 /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
 
Deleted:
<
<
<!--/twistyPlugin-->

Note that in the commands below, the name <Package> does not contain the hat (i.e. <Package>=SQLDDDB and not Det/SQLDDDB)

  • Check all is fine
              ls -l /afs/cern.ch/lhcb/software/releases/DBASE/<Hat>/<Package>/<version>
  • Lock the AFS directory. After the build of the package release, the directory holding it can be locked. This will remove the write permissions to the directory and this can be achieved with:
              lockAFSDir -R /afs/cern.ch/lhcb/software/releases/DBASE/<Hat>/<Package>/<version>
  • Next? If this is not a special case, the next thing to do is probably to deploy on CVMFS, instructions here.

Install the new RPM on CVMFS

Deployment on CVMFS of a data package is similar to the installation of a project;

 
Added:
>
>

Software Installation (CVMFS)

 
Deleted:
<
<

CVMFS Software Installation

Connection to "cvlhcb" account on host "cvmfs-lhcb"

All installation of software is done on host "cvmfs-lhcb", under the account called "cvlhcb".

 
Changed:
<
<
So first connect to "cvmfs-lhcb":
>
>
Actions:
  • Connect to "cvlhcb" account on host "cvmfs-lhcb"
 ssh yourusername@cvmfs-lhcb
Changed:
<
<
Then check that nobody else is connected to install software:
$ who -q ; pgrep -fl -u cvlhcb
>
>
  • Check that nobody else is connected to install software
    who -q ; pgrep -fl -u cvlhcb
 
Changed:
<
<
If someone else is connected and if there are processes running as cvlhcb, perhaps try sending them an email to see what they are doing before proceeding. If nobody else is connected, sudo into the all powerful account...
$ sudo -i -u cvlhcb
>
>
if someone else is connected and if there are processes running as cvlhcb, perhaps try sending them an email to see what they are doing before proceeding
  • sudo into the all powerful account
    sudo -i -u cvlhcb
 
Changed:
<
<
You need to start a transaction on the server to be able to write to the disk.
$ cvmfs_transaction
>
>
  • Start a transaction on the server to be able to write to the disk
    cvmfs_transaction
 
Changed:
<
<
If the command returns that a transaction was already created, somebody else is installing at the same time.
>
>
    • if the command returns that a transaction was already created, somebody else is installing at the same time.
 Check with the who command before going any further.
Changed:
<
<
N.B.1 cvmfs_transaction is a wrapper around the CVMFS command that creates the transaction on the repository lhcb.cern.ch, and logs the command and result.
>
>
    • Note:
      1. cvmfs_transaction is a wrapper around the CVMFS command that creates the transaction on the repository lhcb.cern.ch, and logs the command and result.
 It is best to use it instead of using the direct command cvmfs_server transaction lhcb.cern.ch
Added:
>
>
      1. even if you decide to use cvmfs_server instead of the wrapper, it is best to specify the repository in the command as this server could potentially host several repositories in the future.
 
Changed:
<
<
N.B.2 Even if you decide to use cvmfs_server instead of the wrapper, it is best to specify the repository in the command as this server could potentially host several repositories in the future.

Install using standard script

Data packages and Projects are installed on CVMFS roughly the same way (except for the substitution <Project> ==> <Package> for Data packages).

For stacks >= LHCb v35r4 (with Gaudi v23r6), the debug version will be installed on CVMFS. The software should be installed the scripts from the ${HOME}/bin directory.

At this stage you may need to determine which stack a software release is based on. In this case in a different window on any lxplus machine run:

<!-- SyntaxHighlightingPlugin -->
lb-sdb-query --readonly d <Project> <version> 
<!-- end SyntaxHighlightingPlugin -->

Install a project from recent stacks, for the default platforms

If the stack is the most recent, and only default platforms are required

<!-- SyntaxHighlightingPlugin -->
install_software.sh <Project> <version> 
<!-- end SyntaxHighlightingPlugin -->

In case of problems or missing dependencies you can alter the options to lbinstall by setting the OPTS environment variable e.g.:

<!-- SyntaxHighlightingPlugin -->
OPTS=--no-strict install_software.sh $MyProject $MyVersion
<!-- end SyntaxHighlightingPlugin -->

Install a Data package

You need to know which project the data package is part of (DBASE or PARAM) and the optional subdirectory (called "hat") in which it was copied (e.g. WG). Then you can run:

<!-- SyntaxHighlightingPlugin -->
cvmfslbinstall install PROJECT_Hat_Package_version
<!-- end SyntaxHighlightingPlugin -->

e.g.

<!-- SyntaxHighlightingPlugin -->
cvmfslbinstall install DBASE_WG_CharmConfig_v3r51
<!-- end SyntaxHighlightingPlugin -->

or

<!-- SyntaxHighlightingPlugin -->
cvmfslbinstall install DBASE_PRConfig_v1r19
<!-- end SyntaxHighlightingPlugin -->

Check and Update

Check: Sometimes what seems to be a warning is actually a fatal error. You can then check it all went fine with

<!-- SyntaxHighlightingPlugin -->
ls -ltr /cvmfs/lhcb.cern.ch/lib/lhcb/<PROJECT>/<PROJECT>_<version>/InstallArea/
<!-- end SyntaxHighlightingPlugin -->
The platforms you have installed should be there, the default platforms are currently only slc6-gcc48, opt and dbg. (the directory to check is different for a Data package, but should be easy to guess.)

Update: Once you have done all the installs, you have to update the global catalog (needs to be done just once at the end, to publish everything that you have installed)

<!-- SyntaxHighlightingPlugin -->
cvmfs_publish
<!-- end SyntaxHighlightingPlugin -->

N.B.1 cvmfs_publish is a wrapper around the CVMFS command that publishes the transaction on the repository lhcb.cern.ch, and logs the command and result.

>
>
  • Install the data package
    cvmfslbinstall install <PROJECT>_<Hat>_<Package>_<version>
    
  • Check that it all went fine with
    ls -ltr /cvmfs/lhcb.cern.ch/lib/lhcb/<PROJECT>/<Hat>/<Package>/<version>/
    
  • Update the CVMFS catalogs
    cvmfs_publish
    
    • Note:
      1. cvmfs_publish is a wrapper around the CVMFS command that publishes the transaction on the repository lhcb.cern.ch, and logs the command and result.
 It is best to use it instead of using the direct command cvmfs_server publish lhcb.cern.ch
Added:
>
>
      1. Even if you decide to use cvmfs_server instead of the wrapper, it is best to specify the repository in the command as this server could potentially host several repositories
  • Remember to log out
 
Changed:
<
<
N.B.2 Even if you decide to use cvmfs_server instead of the wrapper, it is best to specify the repository in the command as this server could potentially host several repositories

Exit: When done, do not forget to exit the sudo session so that others can proceed with other installations if necessary

Note: the various install_software and cvmfs_publish commands can be chained by separating them with ;

CVMFS Appendix

IT CVMFS Information

>
>

Special Case: Gen/DecFiles

From lxplus, once the changes on CVMFS have been propagated (check with ls /cvmfs/lhcb.cern.ch/lib/lhcb/DBASE/Gen/DecFiles), you have to update the bookkeeping.
 
Changed:
<
<
All information from IT Department can be found at https://twiki.cern.ch/twiki/bin/view/CvmFS/Installers

LHCB Software on cvmfs-lhcb

Scripts

All updates to software deployed on cvmfs should be done under account "cvlhcb". "${HOME}/bin" contains the necessary scripts to add/remove software:

  • cvmfs_transaction: To start a transaction to deploy software on the server

  • install_software.sh: To call install_project to add new projects/versions

  • remove_software.sh: To Remove software versions

  • cvmfs_publish: Synchronize the CVMFS file system

Some other scripts are also available (same name as above but prefixed with std_ perform the same action without logging)

The output of those command is automatically logged to "${HOME}/logs/install.log

>
>
  • Get a valid Grid proxy
    lhcb-proxy-init
    
  • Update the bookkeeping
    cd /cvmfs/lhcb.cern.ch/lib/lhcb/DBASE/Gen/DecFiles/<version>
    lb-run -c best LHCbDirac dirac-bookkeeping-eventtype-mgt-insert doc/table_event.sql
    lb-run -c best LHCbDirac dirac-bookkeeping-eventtype-mgt-update doc/table_event.sql
    lb-run -c best LHCbDirac dirac-bookkeeping-eventtype-mgt-update doc/table_obsolete.sql
    
 
Changed:
<
<
Sometimes a package is not known to the install_software.sh script. In that case
<!-- SyntaxHighlightingPlugin -->
cvmfslbinstall install DBASE_HAT_PKG_vXrY
<!-- end SyntaxHighlightingPlugin -->
may help.
>
>
The dirac-bookkeeping-eventtype-mgt-insert command will produce many errors like
{'EVTTYPEID': '12165431', 'DESCRIPTION': 'Bu_D02pi+pi-,K+pi-pi0=PHSP,DecProdCut', 'PRIMARY': '[B+ -> pi+ pi+ pi- (D0bar -> K+ pi- pi0)]cc'} : Excution failed.: ( ORA-00001: unique constraint (LHCB_DIRACBOOKKEEPING.SYS_C00302542) violated
ORA-06512: at "LHCB_DIRACBOOKKEEPING.BOOKKEEPINGORACLEDB", line 1131
ORA-06512: at line 1
 ) 
This is normal as most event types already exist in the Oracle table. Things are OK if the last line is like
The following event types are inserted: ['42102201', '42102200', '42163203', '42102221', '42102220', '15104451', '42100210', '42100211', '42100230', '42100231', '42142201', '42142200', '42102210', '42102211', '42162203', '42162202', '15574100', '42100221', '42100220', '12163436', '12163437', '42102601', '42102600', '12163236', '42163202', '12163235', '42112200', '42112201'] 
 
Added:
>
>

Special Instructions and troubleshooting

 
Changed:
<
<

CRON

>
>

Installation on AFS

Since 2018-01-22, installation on AFS is not needed, but the old instructions are still available at ProjectReleaseOld.
 
Changed:
<
<
The local CRON is running on this host to automatically update SQLDDDB every hour. The logs for this cron is in: ${HOME}/logs/update_sqldddb.log
>
>
Note that only librarians will be able to change the content of AFS, so contact <lhcb-software-librarians@cern.ch>.
 
Changed:
<
<
A backup of the CRONTAB is available in ${HOME}/conf.crontab.backup
>
>

Deployment from Old Tarfiles

Very old versions of software projects cannot be deployed with RPMs, in which case it's possible to refer to the instructions at ProjectReleaseOld.
 
Deleted:
<
<

Special Instructions and troubleshooting

 

Removing Software (archival)

Projects

  • On lxplus or one of the lxbuild machines (AFS and CVMFS access, as well as the afs_admin command are needed)
Line: 810 to 704
 

Releasing a new version of LCG Grid

Changed:
<
<
After LHCbDirac v8r0 (before then mkLCGCMTTar was necessary)
>
>
After LHCbDirac v8r0 (before then mkLCGCMTTar was necessary)
 mkLCGCMTtarFromRPM LHCbDirac vXrY $CMTCONFIG --nonatives
Line: 953 to 847
 cvmfs_publish
Added:
>
>

CVMFS Appendix

IT CVMFS Information

All information from IT Department can be found at https://twiki.cern.ch/twiki/bin/view/CvmFS/Installers

LHCB Software on cvmfs-lhcb

Scripts

All updates to software deployed on cvmfs should be done under account "cvlhcb". "${HOME}/bin" contains the necessary scripts to add/remove software:

  • cvmfs_transaction: To start a transaction to deploy software on the server

  • install_software.sh: To call install_project to add new projects/versions

  • remove_software.sh: To Remove software versions

  • cvmfs_publish: Synchronize the CVMFS file system

Some other scripts are also available (same name as above but prefixed with std_ perform the same action without logging)

The output of those command is automatically logged to "${HOME}/logs/install.log

Sometimes a package is not known to the install_software.sh script. In that case

<!-- SyntaxHighlightingPlugin -->
cvmfslbinstall install DBASE_HAT_PKG_vXrY
<!-- end SyntaxHighlightingPlugin -->
may help.

CRON

The local CRON is running on this host to automatically update SQLDDDB every hour. The logs for this cron is in: ${HOME}/logs/update_sqldddb.log

A backup of the CRONTAB is available in ${HOME}/conf.crontab.backup

 

-- MarcoClemencic - 30 Jun 2014

Line: 956 to 886
 

-- MarcoClemencic - 30 Jun 2014

Added:
>
>
 -- MarcoCattaneo - 3 Jun 2017

Added:
>
>
-- MarcoClemencic - 2018-01-22
 
META FILEATTACHMENT attachment="rebuildRPMdb.py.txt" attr="" comment="Script to list commands to recreate RPM DB" date="1461683382" name="rebuildRPMdb.py.txt" path="rebuildRPMdb.py.txt" size="4322" user="bcouturi" version="1"
META TOPICMOVED by="cattanem" date="1412348243" from="LHCb.ProjectReleaseNew" to="LHCb.ProjectRelease"

Revision 1692018-01-12 - PatrickSKoppenburg

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 243 to 243
 
    • try to copy the RPMs to the repository
      build_id=1234
Changed:
<
<
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018 /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018 /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
 
    • if everything is fine, do the real copy and update the repository info
Changed:
<
<
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018 --copy /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018 --copy /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
 
    • check it's all there
      ls -ltr /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018/ | tail -10

Revision 1682018-01-10 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 104 to 104
 
  • publish the produced RPMs (all of them, unless explicitly requested otherwise, see partial install section)
    • try to copy the RPMs to the repository
      build_id=1234
Changed:
<
<
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018 /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
 
      • For old stacks only released as tar files:
        lb-release-oldtar /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/olddist
               
    • if everything is fine, do the real copy and update the repository info
Changed:
<
<
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 --copy /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018 --copy /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
 
      • For old stacks only released as tar files:
        lb-release-oldtar --copy /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/olddist
        
  • Check it's all there
Changed:
<
<
ls -ltr /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017/ | tail -10
>
>
ls -ltr /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018/ | tail -10
 In case of problems see here

BE CAREFUL: Since 20141010, the release procedure automatically bumps up the release number for packages for which there is already a version in the RPM repo.

Line: 126 to 126
  N.B. If you want to release all rpms EXCEPT some matching a specific string (e.g. gcc62) you can use:
Changed:
<
<
lb-release-rpm -c --rpm-regex '^(?!.*(gcc62)).*' --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
>
>
lb-release-rpm -c --rpm-regex '^(?!.*(gcc62)).*' --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018 /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
 
Line: 186 to 186
 
  • Check it's all there
    ls -l /afs/cern.ch/lhcb/software/releases/${MyProject^^}/${MyProject^^}_${MyVersion}/
Changed:
<
<
ls -l /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017/ | grep ${MyProject^^} | grep ${MyVersion}
>
>
ls -l /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018/ | grep ${MyProject^^} | grep ${MyVersion}
 In case of problems see here

All cases:

Line: 243 to 243
 
    • try to copy the RPMs to the repository
      build_id=1234
Changed:
<
<
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018 /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
 
    • if everything is fine, do the real copy and update the repository info
Changed:
<
<
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 --copy /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018 --copy /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
 
Changed:
<
<
    • check it's all there
      ls -ltr /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017/ | tail -10
>
>
    • check it's all there
      ls -ltr /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018/ | tail -10
 

Install the new RPM on AFS

Line: 876 to 876
 

Now copying it to the RPM repository:

Changed:
<
<
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 /tmp/rpmbuild/RPMS/noarch/ lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 --copy /tmp/rpmbuild/RPMS/noarch/
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018 /tmp/rpmbuild/RPMS/noarch/ lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2018 --copy /tmp/rpmbuild/RPMS/noarch/
 

Now install on plus:

Revision 1672017-12-11 - JoelClosier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 160 to 160
 WARNING: Error encountered while acquiring CERN SSO cookie for Ariadne. Attempting to connect with no SSO..

  • From here on you will be installing the software on AFS and on CVMFS. The two installations can be done in parallel*
Added:
>
>
  • For Dirac and LHCbDirac you can ONLY do CVMFS installation
 

AFS Software Installation

Revision 1662017-11-28 - MarcoCattaneo

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 289 to 289
 Note that in the commands below, the name <Package> does not contain the hat (i.e. <Package>=SQLDDDB and not Det/SQLDDDB)

  • Check all is fine
Changed:
<
<
ls -l /afs/cern.ch/lhcb/software/releases//DBASE///
>
>
ls -l /afs/cern.ch/lhcb/software/releases/DBASE///
 
  • Lock the AFS directory. After the build of the package release, the directory holding it can be locked. This will remove the write permissions to the directory and this can be achieved with:
Changed:
<
<
lockAFSDir -R /afs/cern.ch/lhcb/software/releases//DBASE///
>
>
lockAFSDir -R /afs/cern.ch/lhcb/software/releases/DBASE///
 
  • Next? If this is not a special case, the next thing to do is probably to deploy on CVMFS, instructions here.

Install the new RPM on CVMFS

Line: 432 to 432
  This removes the software from the AFS release area and from the old Grid distribution area, and archives the tar files to castor. You need a grid proxy, since the first thing the script does is to contact the book-keeping to check that there isn't an active production step using the software to be archived. You have to hit several times to confirm that you want to remove the tar balls (don't say yes if there was some problem copying to castor)
Changed:
<
<
  • On cvmfs-lhcb, start a transaction in the usual way, then run the command:
    remove_software.sh <Project> <version>
>
>
  • On cvmfs-lhcb, start a transaction in the usual way, then run the command:
    remove_software.sh <Project> <version>
    . Unfortunately this is currently (28th Nov 2017) broken, see https://its.cern.ch/jira/browse/LBCORE-1488.
  Remember to run cvmfs_publish at the end, as usual, to close the transaction.

Revision 1652017-11-15 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 910 to 910
  The RPM prepared just needs to be copied to the yum repository and the metadata rebuilt.
Added:
>
>

Installing new generators

The simulation team tests generators on /cvmfs/lhcbdev.cern.ch. Once validated they need to be installed on the production cvmfs.

The list will be expressed as a YAML file such as:

packages:   
- name: 'lhapdf'     
  LCG: [ '88']     
  versions: [ '6.1.6.cxxstd' ]
- name: 'pythia8'
  LCG: [ '88' ]
  versions: [ '230' ]   
- name: 'rivet'
  LCG: [ '88' ]
  versions: [ '2.5.2' ]   
- name: 'thepeg'
  LCG: [ '88' ]
  versions: [ '2.0.3' ]   

To perform the installation, it is first necessary to derive the list of RPM packages to install. This can be done in the following manner:

virtualenv yamlToPackages
. ./yamlToPackages/bin/activate
pip install --extra-index-url https://lbmultipython03.cern.ch/simple/ --trusted-host lbmultipython03.cern.ch lbinstall lbdevmanager

lbGetPackagesFromYAML <YAML FILE>

This will print out a list of packages. It is then necessary to login to cvmfs-lhcb and install them using the cvmfslbinstall command, e.g.

cvmfs_transaction
cvmfslbinstall install <packages>
cvmfs_publish
  -- MarcoClemencic - 30 Jun 2014 -- MarcoCattaneo - 3 Jun 2017

Revision 1642017-11-13 - JoelClosier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 104 to 104
 
  • publish the produced RPMs (all of them, unless explicitly requested otherwise, see partial install section)
    • try to copy the RPMs to the repository
      build_id=1234
Changed:
<
<
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 /data/archive/artifacts/release/lhcb-release/$build_id/rpms
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
 
      • For old stacks only released as tar files:

Changed:
<
<
lb-release-oldtar /data/archive/artifacts/release/lhcb-release/$build_id/olddist
>
>
lb-release-oldtar /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/olddist
 
    • if everything is fine, do the real copy and update the repository info
Changed:
<
<
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 --copy /data/archive/artifacts/release/lhcb-release/$build_id/rpms
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 --copy /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
 
      • For old stacks only released as tar files:
Changed:
<
<
lb-release-oldtar --copy /data/archive/artifacts/release/lhcb-release/$build_id/olddist
>
>
lb-release-oldtar --copy /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/olddist
 
  • Check it's all there
    ls -ltr /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017/ | tail -10
Line: 126 to 126
  N.B. If you want to release all rpms EXCEPT some matching a specific string (e.g. gcc62) you can use:
Changed:
<
<
lb-release-rpm -c --rpm-regex '^(?!.*(gcc62)).*' --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 /data/archive/artifacts/release/lhcb-release/$build_id/rpms
>
>
lb-release-rpm -c --rpm-regex '^(?!.*(gcc62)).*' --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
 
Line: 149 to 149
 

  • update the software database.
    First try in dry-run mode:
Changed:
<
<
lb-deployment-updatesdb /data/archive/artifacts/release/lhcb-release/$build_id
>
>
lb-deployment-updatesdb /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id
 If more platforms/projects are being updated than necessary you can limit using the options of the lb-deployment-updatesdb command.If all ok, just run:
Changed:
<
<
lb-deployment-updatesdb --update /data/archive/artifacts/release/lhcb-release/$build_id
>
>
lb-deployment-updatesdb --update /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id
 In case of SSO, or certificate problems:
ssh -fN  -L12345:localhost:7474 lbariadne
export SDBURL=http://localhost:12345/db/data/
Line: 242 to 242
 
    • try to copy the RPMs to the repository
      build_id=1234
Changed:
<
<
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 /data/archive/artifacts/release/lhcb-release/$build_id/rpms
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
 
    • if everything is fine, do the real copy and update the repository info
Changed:
<
<
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 --copy /data/archive/artifacts/release/lhcb-release/$build_id/rpms
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 --copy /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/$build_id/rpms
 
    • check it's all there
      ls -ltr /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017/ | tail -10
Line: 548 to 548
 This could happen during the copy, in which case it is enough to remove the corrupted files from $LHCBTAR/rpm/lhcb and call again lb-release-rpm.

If it still does not work, check that there are no permanent problems in $LHCBTAR/rpm/lhcb (e.q. quota) and that the files are copied correctly with something like (bash)

Changed:
<
<
for f in /data/artifacts/release/lhcb-release/${build_id}/*.rpm ; do
>
>
for f in /eos/project/l/lhcbwebsites/www/lhcb-nightlies-artifacts/release/lhcb-release/${build_id}/*.rpm ; do
  cmp $f $LHCBTAR/rpm/lhcb/$(basename $f) done

Revision 1632017-10-26 - MarcoCattaneo

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 98 to 98
 ssh lhcb-archive.cern.ch bash . /afs/cern.ch/lhcb/software/releases/LBSCRIPTS/dev/InstallArea/scripts/LbLogin.sh
Added:
>
>
kinit Your_Username@CERNNOSPAMPLEASE.CH
 
Deleted:
<
<
Warning, important kinit Your_Username
 
  • publish the produced RPMs (all of them, unless explicitly requested otherwise, see partial install section)
    • try to copy the RPMs to the repository

Revision 1622017-10-23 - MarcoCattaneo

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 242 to 242
 
    • try to copy the RPMs to the repository
      build_id=1234
Changed:
<
<
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 /data/archive/artifacts/release/lhcb-release/$build_id
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 /data/archive/artifacts/release/lhcb-release/$build_id/rpms
 
    • if everything is fine, do the real copy and update the repository info
Changed:
<
<
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 --copy /data/archive/artifacts/release/lhcb-release/$build_id
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 --copy /data/archive/artifacts/release/lhcb-release/$build_id/rpms
 
    • check it's all there
      ls -ltr /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017/ | tail -10
Line: 356 to 356
 You need to know which project the data package is part of (DBASE or PARAM) and the optional subdirectory (called "hat") in which it was copied (e.g. WG). Then you can run: %SYNTAX{ syntax="sh"}%
Changed:
<
<
cvmfslbinstall PROJECT_Hat_Package_version%ENDSYNTAX%
>
>
cvmfslbinstall install PROJECT_Hat_Package_version%ENDSYNTAX%
  e.g. %SYNTAX{ syntax="sh"}%
Changed:
<
<
cvmfslbinstall DBASE_WG_CharmConfig_v3r51%ENDSYNTAX%
>
>
cvmfslbinstall install DBASE_WG_CharmConfig_v3r51%ENDSYNTAX%
  or

%SYNTAX{ syntax="sh"}%

Changed:
<
<
cvmfslbinstall DBASE_PRConfig_v1r19%ENDSYNTAX%
>
>
cvmfslbinstall install DBASE_PRConfig_v1r19%ENDSYNTAX%
 

Revision 1612017-10-13 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 126 to 126
  N.B. If you want to release all rpms EXCEPT some matching a specific string (e.g. gcc62) you can use:
Changed:
<
<
lb-release-rpm -c --rpm-regex '^(?!.*(gcc62)).*' --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 /data/archive/artifacts/release/lhcb-release/$build_id
>
>
lb-release-rpm -c --rpm-regex '^(?!.*(gcc62)).*' --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 /data/archive/artifacts/release/lhcb-release/$build_id/rpms
 
Line: 296 to 296
 

Install the new RPM on CVMFS

Changed:
<
<
Deployment on CVMFS of a data package is the same as for a project, but replace <Project> ==> <Package>
>
>
Deployment on CVMFS of a data package is similar to the installation of a project;
 

CVMFS Software Installation

Line: 339 to 339
 At this stage you may need to determine which stack a software release is based on. In this case in a different window on any lxplus machine run:
<!-- SyntaxHighlightingPlugin -->
lb-sdb-query --readonly d <Project> <version> 
<!-- end SyntaxHighlightingPlugin -->
Changed:
<
<

Install a project or Data package from recent stacks, for the default platforms

>
>

Install a project from recent stacks, for the default platforms

  If the stack is the most recent, and only default platforms are required
Line: 351 to 351
 OPTS=--no-strict install_software.sh $MyProject $MyVersion %ENDSYNTAX%
Added:
>
>

Install a Data package

You need to know which project the data package is part of (DBASE or PARAM) and the optional subdirectory (called "hat") in which it was copied (e.g. WG). Then you can run:

<!-- SyntaxHighlightingPlugin -->
cvmfslbinstall PROJECT_Hat_Package_version
<!-- end SyntaxHighlightingPlugin -->

e.g.

<!-- SyntaxHighlightingPlugin -->
cvmfslbinstall DBASE_WG_CharmConfig_v3r51
<!-- end SyntaxHighlightingPlugin -->

or

<!-- SyntaxHighlightingPlugin -->
cvmfslbinstall DBASE_PRConfig_v1r19
<!-- end SyntaxHighlightingPlugin -->

 

Check and Update

Check: Sometimes what seems to be a warning is actually a fatal error. You can then check it all went fine with

<!-- SyntaxHighlightingPlugin -->
ls -ltr /cvmfs/lhcb.cern.ch/lib/lhcb/<PROJECT>/<PROJECT>_<version>/InstallArea/
<!-- end SyntaxHighlightingPlugin -->

Revision 1602017-10-12 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 104 to 104
 
  • publish the produced RPMs (all of them, unless explicitly requested otherwise, see partial install section)
    • try to copy the RPMs to the repository
      build_id=1234
Changed:
<
<
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 /data/archive/artifacts/release/lhcb-release/$build_id
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 /data/archive/artifacts/release/lhcb-release/$build_id/rpms
 
      • For old stacks only released as tar files:
        lb-release-oldtar /data/archive/artifacts/release/lhcb-release/$build_id/olddist
               
    • if everything is fine, do the real copy and update the repository info
Changed:
<
<
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 --copy /data/archive/artifacts/release/lhcb-release/$build_id
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 --copy /data/archive/artifacts/release/lhcb-release/$build_id/rpms
 
      • For old stacks only released as tar files:
        lb-release-oldtar --copy /data/archive/artifacts/release/lhcb-release/$build_id/olddist

Revision 1592017-10-03 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 124 to 124
 If your slot contains more projects that the ones requested for release, you need to ONLY copy the needed ones to the RPM repo. In that case the --rpm-regex option of lb-release-rpm should be used.
Added:
>
>
N.B. If you want to release all rpms EXCEPT some matching a specific string (e.g. gcc62) you can use:
lb-release-rpm -c --rpm-regex '^(?!.*(gcc62)).*'  --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 /data/archive/artifacts/release/lhcb-release/$build_id
 

Update of the Software configuration Database

Revision 1582017-09-29 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 713 to 713
 The metadata files are kept and versioned in https://gitlab.cern.ch/lhcb-core/rpm-recipes (LHCBEXTERNALS sub directory).
Changed:
<
<
Follow the instructions in the README.md to get the list of rpms to install.
>
>
Follow the instructions in the README.md to get the list of rpms to install, and to install them on CVMFS and AFS.

Then update the software configuration DB, e.g.

lb-sdb-addplatform LCG 91 x86_64-slc6-gcc62-opt
lb-sdb-addplatform LCG 91 x86_64-slc6-gcc62-dbg
lb-sdb-addplatform LCG 91 x86_64-centos7-gcc62-dbg
lb-sdb-addplatform LCG 91 x86_64-centos7-gcc62-opt
lb-sdb-addplatform LCG 91 x86_64-centos7-gcc7-opt
lb-sdb-addplatform LCG 91 x86_64-centos7-gcc7-dbg
 

Updating the Software Configuration DB for a new LCG or Gaudi

Revision 1572017-09-29 - PatrickSKoppenburg

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 292 to 292
  Deployment on CVMFS of a data package is the same as for a project, but replace <Project> ==> <Package>
Deleted:
<
<

Special Instructions and troubleshooting

 

CVMFS Software Installation

Line: 400 to 397
  A backup of the CRONTAB is available in ${HOME}/conf.crontab.backup
Added:
>
>

Special Instructions and troubleshooting

 

Removing Software (archival)

Projects

  • On lxplus or one of the lxbuild machines (AFS and CVMFS access, as well as the afs_admin command are needed)

Revision 1562017-09-26 - PatrickSKoppenburg

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 228 to 228
 

publish the RPMs

  • If the build was fine, publish the RPMs (from lhcb-archive.cern.ch)
Changed:
<
<
    • Since default SW is now CVMFS, switch to AFS
%SYNTAX{ syntax="sh"}%
>
>
    • Since default SW is now CVMFS, switch to AFS
 ssh lhcb-archive.cern.ch bash /afs/cern.ch/lhcb/software/releases/LBSCRIPTS/dev/InstallArea/scripts/LbLogin.sh kinit Your_Username@CERNNOSPAMPLEASE.CH
Changed:
<
<
%ENDSYNTAX%
>
>
 
    • try to copy the RPMs to the repository
      build_id=1234
      lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017  /data/archive/artifacts/release/lhcb-release/$build_id
Line: 283 to 282
  Note that in the commands below, the name <Package> does not contain the hat (i.e. <Package>=SQLDDDB and not Det/SQLDDDB)
Changed:
<
<
  • Lock the AFS directory
After the build of the package release, the directory holding it can be locked. This will remove the write permissions to the directory and this can be achieved with:
>
>
  • Check all is fine
              ls -l /afs/cern.ch/lhcb/software/releases//DBASE/<Hat>/<Package>/<version>
  • Lock the AFS directory. After the build of the package release, the directory holding it can be locked. This will remove the write permissions to the directory and this can be achieved with:
  lockAFSDir -R /afs/cern.ch/lhcb/software/releases//DBASE///
Changed:
<
<
  • Next?
If this is not a special case, the next thing to do is probably to deploy on CVMFS, instructions here.
>
>
  • Next? If this is not a special case, the next thing to do is probably to deploy on CVMFS, instructions here.
 

Install the new RPM on CVMFS

Revision 1552017-09-26 - PatrickSKoppenburg

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Software Release Guide

Line: 136 to 136
 
  • Verify that the software configuration DB has been updated:
    MyProject=<ProjectName>
    MyVersion=vXrY
Added:
>
>
echo ${MyProject^^} ${MyVersion}
 lb-sdb-query d $MyProject $MyVersion This should show projects in the slot or already released. If some are missing re-run:
lb-sdb-import $MyProject $MyVersion

Revision 1542017-08-29 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Changed:
<
<

Project Release Guide

>
>

Software Release Guide

 
Changed:
<
<

Introduction

>
>

Scope

The following documentation applies to the release of LHCb Physics projects as well as data packages. Ganga is installed on a separate CVMFS volume, and there are specific instructions for DIRAC and LHCbDIrac.

  Warning, important These instructions do not apply to LbScripts. Use LbScriptsRelease instead.
Changed:
<
<

For Release Managers

>
>

Overview of the procedure

The software is build by the Jenkins continuous integration engine. The configuration of the projects, their dependencies and which stacks should be built is defined in the software configuration database, with the typical procedure including the following steps:

  1. Release manager tags a project (or several) and triggers the build by imported the projects to the software configuration database.
  2. Jenkins builds the project(s) and prepares the RPM packages
  3. Release manager checks the outcome of the build and either restarts the build or requests a release in JIRA
  4. Release shifter copies the RPM packages to the official LHCb package repository (YUM format). At this point, anybody can perform local installations using lbinstall
  5. Release shifter marks the RPM as "released" in the software configuration database, and specifies which platforms have effectively been released. This is necessary for the build system to know what is released and isn't when building stacks, but also for the automated release procedures on AFS and CVMFS to know which platforms should be installed.
  6. Release shifter performs the installation on AFS and updates the documentation (still AFS dependent).
  7. Release shifter performs the installation on CVMFS

N.B. Installations on CVMFS and AFS can be performed in parallel.

Detailed instructions for all the steps are detailed below.

For Release Managers - Preparation of the build

 
Changed:
<
<
For Projects:
>
>

For Projects

 
  • prepare the project as usual (release notes, tags, etc.), following the instructions at PrepareProjectReleaseGit
  • register the project/version to the software database
    lb-sdb-import MyProject vXrY
Line: 22 to 42
 
  • once satisfied by the build, take note of the build id (you can use the direct link icon) and make the request via LHCb Deployment Request form
Changed:
<
<
For Data Packages:
>
>

For Data Packages

 
Changed:
<
<

For Release Shifters

>
>

For Release Shifters - Release of the build

 See the LHCb shift database to find who is this week's shifter.
Added:
>
>
 

Special information

Starting from 13 October 2016, you should set your environment to DEV as : ". /afs/cern.ch/lhcb/software/releases/LBSCRIPTS/dev/InstallArea/scripts/LbLogin.sh"
Line: 47 to 68
 As the shifter you can self-assign shifts through the shift database shift database. Usually this is done by block-booking weeks for individual shifters. </>
<!--/twistyPlugin-->
Changed:
<
<
<!--

Standard procedure

>
>

Using the checklist tool

  The script lb-deployment-checklist simplifies the release shifter task by checking what needs to be done and either prints the next command to run or points to this page.
Line: 58 to 78
 and follow the instructions.

The latest version of lb-deployment-checklist may cache some data to speed up the execution. If you think it's showing wrong messages, try to remove the directory ~/.cache/lb-deployment/.

Deleted:
<
<
-->
 

Release of projects

Added:
>
>

Update of the RPM repository

The goal of this step is to make sure that:

  1. The RPMs have been copied to the YUM repository (curently in EOS)
  2. The YUM metadata has been updated (the lb-release-rpm internally uses the "createrepo" yum command).
 
Deleted:
<
<
  Warning, important This step has to be performed from the host lhcb-archive.cern.ch
Line: 74 to 101
  Warning, important kinit Your_Username
Deleted:
<
<
  • Verify that the software configuration DB has been updated:
    MyProject=<ProjectName>
    MyVersion=vXrY
    lb-sdb-query d $MyProject $MyVersion
    
    This should show projects in the slot or already released. If some are missing re-run:
    lb-sdb-import $MyProject $MyVersion
    
 
  • publish the produced RPMs (all of them, unless explicitly requested otherwise, see partial install section)
    • try to copy the RPMs to the repository
      build_id=1234
Line: 105 to 124
 If your slot contains more projects that the ones requested for release, you need to ONLY copy the needed ones to the RPM repo. In that case the --rpm-regex option of lb-release-rpm should be used.
Added:
>
>

Update of the Software configuration Database

 
Added:
>
>
The goal of this step is to make sure that:
  1. The project is referenced properly in the Software configuration DB
  2. The list of platforms released for the project have been updated

  • Verify that the software configuration DB has been updated:
    MyProject=<ProjectName>
    MyVersion=vXrY
    lb-sdb-query d $MyProject $MyVersion
    
    This should show projects in the slot or already released. If some are missing re-run:
    lb-sdb-import $MyProject $MyVersion
    
 
  • update the software database.
    First try in dry-run mode:
    lb-deployment-updatesdb /data/archive/artifacts/release/lhcb-release/$build_id
    
    If more platforms/projects are being updated than necessary you can limit using the options of the lb-deployment-updatesdb command.If all ok, just run:
Line: 119 to 154
 
  • From here on you will be installing the software on AFS and on CVMFS. The two installations can be done in parallel*
Changed:
<
<
>
>
 

AFS Software Installation

Added:
>
>
  Warning, important This step has to be performed from the host lhcb-archive.cern.ch
Line: 159 to 195
 mkMCGentarFromRPM ${MyProject} ${MyVersion} ${CMTCONFIG} but make sure that ${CMTCONFIG} is a platform valid for Gauss. Else use another one.
Added:
>
>

CVMFS Software Installation

 
  • install the software on CVMFS if not already done in parallel. See here for instructions
Changed:
<
<

Data packages

>
>

Release of Data packages

 Warning, important WARNING The procedure for the release of data packages via RPMs is still under heavy development and the automation is not yet in place.
Changed:
<
<
  • Trigger by hand the lhcb-release Jenkins job:
>
>

Specifics

Data packages are different from standard projects:

  • They do not need to be compiled for several platforms (make is run nonetheless to perform basic actions)
  • They are part of the DBASE or PARAM projects
  • They are not registered in the software configuration Database and the builds may not always be triggered by the release manager.

The release steps are therefore different from normal projects.

Procedure

Trigger by hand the lhcb-release Jenkins job (if needed)

 
    • go to https://jenkins-lhcb-nightlies.web.cern.ch/job/nightly-builds/job/release/build
    • add the list of packages (with their versions) in the field packages_list (e.g. RawEventFormat v1r1 Gen/DecFiles vXrY), the name of the container can be specified with something like "container":"name" (e.g. PARAM:TMVAWeights v1r0). The container can be omitted for DBASE.
    • click on the Build button (leave all the others fields empty)
Line: 175 to 225
 
    • go to https://lhcb-nightlies.cern.ch/release/
    • click on the small Jenkins icon in the "Project" column of the build summary look for the package build log (after the line "...:INFO... building Hat/Package")
Added:
>
>

publish the RPMs

 
  • If the build was fine, publish the RPMs (from lhcb-archive.cern.ch)
    • Since default SW is now CVMFS, switch to AFS
%SYNTAX{ syntax="sh"}%
Line: 192 to 243
 
    • check it's all there
      ls -ltr /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017/ | tail -10
Changed:
<
<
  • Install the new RPM on AFS (must be done from lhcb-archive)
>
>

Install the new RPM on AFS

(must be done from lhcb-archive)

 /afs/cern.ch/lhcb/software/lbinstall/afslbinstall install DBASE_Hat_Package_version
Line: 235 to 288
 
  • Next?
If this is not a special case, the next thing to do is probably to deploy on CVMFS, instructions here.
Changed:
<
<
Deployment on CVMFS of a DBASE package is the same as for a project, but replace <Project> ==> <Package>
>
>

Install the new RPM on CVMFS

Deployment on CVMFS of a data package is the same as for a project, but replace <Project> ==> <Package>

 
Added:
>
>

Special Instructions and troubleshooting

 

CVMFS Software Installation

Revision 1532017-08-29 - MarcoCattaneo

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 72 to 72
 bash . /afs/cern.ch/lhcb/software/releases/LBSCRIPTS/dev/InstallArea/scripts/LbLogin.sh
Changed:
<
<
Warning, important kinit Your_Username@CERNNOSPAMPLEASE.CH
>
>
Warning, important kinit Your_Username
 
  • Verify that the software configuration DB has been updated:
    MyProject=<ProjectName>

Revision 1522017-08-04 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 660 to 660
  Follow the instructions in the README.md to get the list of rpms to install.
Added:
>
>

Updating the Software Configuration DB for a new LCG or Gaudi

First import Gaudi without asking for a release:

lb-sdb-import --norelease Gaudi v28r3

If we are dealing with a new LCG, add the corresponding platforms:

lb-sdb-addplatform LCG 89 x86_64-slc6-gcc62-opt
lb-sdb-addplatform LCG 89 x86_64-slc6-gcc62-dbg
lb-sdb-addplatform LCG 89 x86_64-centos7-gcc62-dbg
lb-sdb-addplatform LCG 89 x86_64-centos7-gcc62-opt
lb-sdb-addplatform LCG 89 x86_64-centos7-gcc7-opt
lb-sdb-addplatform LCG 89 x86_64-centos7-gcc7-dbg

If Gaudi features more platforms than LCG, then specify the list of platforms to release:

lb-sdb-addplatform --release Gaudi v28r3 x86_64-slc6-gcc62-opt
[...]

Then schedule Gaudi for release:

lb-sdb-release Gaudi v28r3

Then check if everything is ok:

$ lb-sdb-query show Gaudi v28r3
Node 382601 Properties
------------------------------
project   : GAUDI
version   : v28r3

Node 382601 relationships
------------------------------
2683451:REQUIRES(O)     -> (ID:382602, project:LCG, version:89)
2683462:REQUESTED_PLATFORM(O) -> (ID:382306, platform:x86_64-centos7-gcc62-dbg)
2683463:REQUESTED_PLATFORM(O) -> (ID:382305, platform:x86_64-centos7-gcc62-opt)
2683464:REQUESTED_PLATFORM(O) -> (ID:382607, platform:x86_64-centos7-gcc7-dbg)
2683466:REQUESTED_PLATFORM(O) -> (ID:382608, platform:x86_64-centos7-gcc7-opt)
2683468:REQUESTED_PLATFORM(O) -> (ID:382303, platform:x86_64-slc6-gcc62-dbg)
2683469:REQUESTED_PLATFORM(O) -> (ID:382302, platform:x86_64-slc6-gcc62-opt)
2683470:REQUESTED_PLATFORM(O) -> (ID:382307, platform:x86_64-centos7-gcc62-do0)
2683471:REQUESTED_PLATFORM(O) -> (ID:382609, platform:x86_64-centos7-gcc7-do0)
2683473:REQUESTED_PLATFORM(O) -> (ID:382304, platform:x86_64-slc6-gcc62-do0)
2683449:PROJECT(I)      <- (ID:122379, project:GAUDI, sourceuri:gitlab-cern:gaudi/Gaudi)
2683474:RELEASEREQ(I)   <- (ID:122366, project:NONE, type:RELEASE, version:NONE)
 

Releasing a new version of LCG Grid

After LHCbDirac v8r0 (before then mkLCGCMTTar was necessary)

Line: 751 to 804
 Then you can start a transaction and copy files to /cvmfs/lhcbdev.cern.ch
Added:
>
>

Preparing the RPM and releasing a new version of CMake

LHCb uses the Linux_x86_64 binaries from Kitware:

https://cmake.org/download/

The tools to repqckage the RPM can be found in:

https://gitlab.cern.ch/lhcb-core/rpm-recipes

In the cmake folder.

 
Added:
>
>
The RPM prepared just needs to be copied to the yum repository and the metadata rebuilt.
 

-- MarcoClemencic - 30 Jun 2014

Revision 1512017-07-28 - PatrickSKoppenburg

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 130 to 130
 MyProject= MyVersion=vXrY To check the following commands will work for you, now run
Changed:
<
<
echo ${MyProject^^}
>
>
echo ${MyProject^^} ${MyVersion}
  which should return the project name, in upper case.
  • prepare the AFS volume
    lb-project-manage-volume -c $MyProject $MyVersion

Revision 1502017-07-28 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 77 to 77
 
  • Verify that the software configuration DB has been updated:
    MyProject=<ProjectName>
    MyVersion=vXrY
Changed:
<
<
lb-sdb-query $MyProject $MyVersion
>
>
lb-sdb-query d $MyProject $MyVersion
 This should show projects in the slot or already released. If some are missing re-run:
lb-sdb-import $MyProject $MyVersion
Changed:
<
<
  • publish the produced RPMs
>
>
 
    • try to copy the RPMs to the repository
      build_id=1234
      lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 /data/archive/artifacts/release/lhcb-release/$build_id

Revision 1492017-07-27 - PatrickSKoppenburg

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 75 to 75
 Warning, important kinit Your_Username@CERNNOSPAMPLEASE.CH

  • Verify that the software configuration DB has been updated:
Added:
>
>
MyProject= MyVersion=vXrY
 lb-sdb-query $MyProject $MyVersion This should show projects in the slot or already released. If some are missing re-run:
lb-sdb-import $MyProject $MyVersion

Revision 1482017-07-27 - PatrickSKoppenburg

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 60 to 60
 
The latest version of lb-deployment-checklist may cache some data to speed up the execution. If you think it's showing wrong messages, try to remove the directory ~/.cache/lb-deployment/.
-->
Changed:
<
<

Details of the procedure

>
>

Release of projects

 
Line: 118 to 118
 
  • From here on you will be installing the software on AFS and on CVMFS. The two installations can be done in parallel*

Changed:
<
<

AFS Software Installation

>
>

AFS Software Installation

  Warning, important This step has to be performed from the host lhcb-archive.cern.ch
Line: 167 to 167
 
    • go to https://jenkins-lhcb-nightlies.web.cern.ch/job/nightly-builds/job/release/build
    • add the list of packages (with their versions) in the field packages_list (e.g. RawEventFormat v1r1 Gen/DecFiles vXrY), the name of the container can be specified with something like "container":"name" (e.g. PARAM:TMVAWeights v1r0). The container can be omitted for DBASE.
    • click on the Build button (leave all the others fields empty)
Added:
>
>
 

Revision 1472017-07-26 - PatrickSKoppenburg

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 95 to 95
 
      • For old stacks only released as tar files:
        lb-release-oldtar --copy /data/archive/artifacts/release/lhcb-release/$build_id/olddist
        
Added:
>
>
  • Check it's all there
    ls -ltr /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017/ | tail -10
    
    In case of problems see here
  BE CAREFUL: Since 20141010, the release procedure automatically bumps up the release number for packages for which there is already a version in the RPM repo. If your slot contains more projects that the ones requested for release, you need to ONLY copy the needed ones to the RPM repo.
Line: 137 to 140
 
  • Check it's all there
    ls -l /afs/cern.ch/lhcb/software/releases/${MyProject^^}/${MyProject^^}_${MyVersion}/
Deleted:
<
<
ls -ltr /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017/ | tail -10
 ls -l /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017/ | grep ${MyProject^^} | grep ${MyVersion} In case of problems see here

Revision 1462017-07-25 - PatrickSKoppenburg

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 52 to 52
  The script lb-deployment-checklist simplifies the release shifter task by checking what needs to be done and either prints the next command to run or points to this page.
Changed:
<
<
So connect to lxplus (or equivalent SLC6 machine) and run
>
>
So connect to lxplus (or equivalent SLC6 machine) and run
 lb-deployment-checklist MyProject vXrY and follow the instructions.
Line: 75 to 74
  Warning, important kinit Your_Username@CERNNOSPAMPLEASE.CH
Changed:
<
<
  • Verify that the software configuration DB has been updated:
>
>
  • Verify that the software configuration DB has been updated:
 lb-sdb-query $MyProject $MyVersion
Changed:
<
<

Should show projects in the slot or already released. If some are missing re-run:

>
>
This should show projects in the slot or already released. If some are missing re-run:
 lb-sdb-import $MyProject $MyVersion
Line: 97 to 92
 
    • if everything is fine, do the real copy and update the repository info
      lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 --copy /data/archive/artifacts/release/lhcb-release/$build_id
      
Changed:
<
<
      • For old stacks only released as tar files:

>
>
      • For old stacks only released as tar files:
 lb-release-oldtar --copy /data/archive/artifacts/release/lhcb-release/$build_id/olddist
Line: 107 to 101
 In that case the --rpm-regex option of lb-release-rpm should be used.

Changed:
<
<
  • update the software database

First try in dry-run mode:

>
>
  • update the software database.
    First try in dry-run mode:
 lb-deployment-updatesdb /data/archive/artifacts/release/lhcb-release/$build_id
Changed:
<
<
If more platforms/projects are being updated than necessary you can limit using the options of the lb-deployment-updatesdb command. If all ok, just run:
>
>
If more platforms/projects are being updated than necessary you can limit using the options of the lb-deployment-updatesdb command.If all ok, just run:
 lb-deployment-updatesdb --update /data/archive/artifacts/release/lhcb-release/$build_id
Changed:
<
<

In case of SSO, or certificate problems:

>
>
In case of SSO, or certificate problems:
 ssh -fN -L12345:localhost:7474 lbariadne export SDBURL=http://localhost:12345/db/data/ lb-deployment-updatesdb [...]
Changed:
<
<

You can ignore the warnings such as:

>
>
You can ignore the warnings such as:
 WARNING: Error encountered while acquiring CERN SSO cookie for Ariadne. Attempting to connect with no SSO..

From here on you will be installing the software on AFS and on CVMFS. The two installations can be done in parallel

Line: 135 to 119
  Warning, important This step has to be performed from the host lhcb-archive.cern.ch
Changed:
<
<
  • install on AFS
>
>
Install on AFS
 
    • First, set some environment variables that define the project name and version to be installed.
Changed:
<
<
Used in the following commands (so you can simply cut and paste them directly). Works for bash shells, so recommend switching to this if it isn't your default.
>
>
Used in the following commands (so you can simply cut and paste them directly). Works for bash shells, so recommend switching to this if it isn't your default
 MyProject= MyVersion=vXrY To check the following commands will work for you, now run
Line: 148 to 132
 
    • install the new project
      lb-deployment-afs-install $MyProject $MyVersion
Changed:
<
<

In case of problems or missing dependencies you can alter the options to lbinstall by setting the OPTS environment variable e.g.:

>
>
In case of problems or missing dependencies you can alter the options to lbinstall by setting the OPTS environment variable e.g.:
 OPTS=--no-strict lb-deployment-afs-install $MyProject $MyVersion
Changed:
<
<

Check it's all there

>
>
  • Check it's all there
 ls -l /afs/cern.ch/lhcb/software/releases/${MyProject^^}/${MyProject^^}_${MyVersion}/ ls -ltr /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017/ | tail -10 ls -l /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017/ | grep ${MyProject^^} | grep ${MyVersion}
Changed:
<
<

In case of problems see here

>
>
In case of problems see here
  All cases:
  • release and lock the AFS volume
Line: 176 to 151
 
Changed:
<
<
  • FOR OLD STACKS NOT RELEASED VIA RPM: For Gauss, it is necessary to create the Genser tarball

>
>
  • FOR OLD STACKS NOT RELEASED VIA RPM: For Gauss, it is necessary to create the Genser tarball
 mkMCGentarFromRPM ${MyProject} ${MyVersion} ${CMTCONFIG} but make sure that ${CMTCONFIG} is a platform valid for Gauss. Else use another one.
Line: 227 to 201
 where XX, YY are given in the JIRA task.

Then one needs an extra step to update the bookkeeping. It requires to have a valid Grid certificate

Changed:
<
<
  1. Get your Grid proxy:

>
>
  1. Get your Grid proxy:
  lb-run LHCbDirac lhcb-proxy-init
Changed:
<
<
  1. Update the bookkeeping:

>
>
  1. Update the bookkeeping:
  cd /afs/cern.ch/lhcb/software/releases/DBASE/Gen/DecFiles//cmt lb-run LHCbDirac dirac-bookkeeping-eventtype-mgt-insert ../doc/table_event.sql lb-run LHCbDirac dirac-bookkeeping-eventtype-mgt-update ../doc/table_event.sql lb-run LHCbDirac dirac-bookkeeping-eventtype-mgt-update ../doc/table_obsolete.sql
Changed:
<
<
The dirac-bookkeeping-eventtype-mgt-insert command will produce many errors like
>
>
The dirac-bookkeeping-eventtype-mgt-insert command will produce many errors like
 {'EVTTYPEID': '12165431', 'DESCRIPTION': 'Bu_D02pi+pi-,K+pi-pi0=PHSP,DecProdCut', 'PRIMARY': '[B+ -> pi+ pi+ pi- (D0bar -> K+ pi- pi0)]cc'} : Excution failed.: ( ORA-00001: unique constraint (LHCB_DIRACBOOKKEEPING.SYS_C00302542) violated ORA-06512: at "LHCB_DIRACBOOKKEEPING.BOOKKEEPINGORACLEDB", line 1131 ORA-06512: at line 1 )
Changed:
<
<
This is normal as most event types already exist in the Oracle table. Things are OK if the last line is like
>
>
This is normal as most event types already exist in the Oracle table. Things are OK if the last line is like
 The following event types are inserted: ['42102201', '42102200', '42163203', '42102221', '42102220', '15104451', '42100210', '42100211', '42100230', '42100231', '42142201', '42142200', '42102210', '42102211', '42162203', '42162202', '15574100', '42100221', '42100220', '12163436', '12163437', '42102601', '42102600', '12163236', '42163202', '12163235', '42112200', '42112201'] </>
<!--/twistyPlugin-->
Line: 255 to 225
 Note that in the commands below, the name <Package> does not contain the hat (i.e. <Package>=SQLDDDB and not Det/SQLDDDB)

  • Lock the AFS directory
Changed:
<
<
After the build of the package release, the directory holding it can be locked. This will remove the write permissions to the directory and this can be achieved with:

>
>
After the build of the package release, the directory holding it can be locked. This will remove the write permissions to the directory and this can be achieved with:
  lockAFSDir -R /afs/cern.ch/lhcb/software/releases//DBASE///
  • Next?
If this is not a special case, the next thing to do is probably to deploy on CVMFS, instructions here.
Line: 271 to 240
  All installation of software is done on host "cvmfs-lhcb", under the account called "cvlhcb".
Changed:
<
<
So first connect to "cvmfs-lhcb":
>
>
So first connect to "cvmfs-lhcb":
 ssh yourusername@cvmfs-lhcb
Changed:
<
<
Then check that nobody else is connected to install software:
>
>
Then check that nobody else is connected to install software:
 $ who -q ; pgrep -fl -u cvlhcb
Changed:
<
<
If someone else is connected and if there are processes running as cvlhcb, perhaps try sending them an email to see what they are doing before proceeding. If nobody else is connected, sudo into the all powerful account...
>
>
If someone else is connected and if there are processes running as cvlhcb, perhaps try sending them an email to see what they are doing before proceeding. If nobody else is connected, sudo into the all powerful account...
 $ sudo -i -u cvlhcb
Changed:
<
<
You need to start a transaction on the server to be able to write to the disk.
>
>
You need to start a transaction on the server to be able to write to the disk.
 $ cvmfs_transaction
Line: 379 to 341
 

Removing Software (archival)

Projects

  • On lxplus or one of the lxbuild machines (AFS and CVMFS access, as well as the afs_admin command are needed)
Changed:
<
<
    • Set up the AFS environment with the command
      afsLbLoginDev
    • Run the command:
      archive_project <Project> <version>
>
>
    • Set up the AFS environment with the command
      afsLbLoginDev
    • Run the command:
      archive_project <Project> <version>
  This removes the software from the AFS release area and from the old Grid distribution area, and archives the tar files to castor. You need a grid proxy, since the first thing the script does is to contact the book-keeping to check that there isn't an active production step using the software to be archived. You have to hit several times to confirm that you want to remove the tar balls (don't say yes if there was some problem copying to castor)
Changed:
<
<
  • On cvmfs-lhcb, start a transaction in the usual way, then run the command:
    remove_software.sh <Project> <version>
>
>
  • On cvmfs-lhcb, start a transaction in the usual way, then run the command:
    remove_software.sh <Project> <version>
  Remember to run cvmfs_publish at the end, as usual, to close the transaction.

Line: 502 to 461
  This could happen during the copy, in which case it is enough to remove the corrupted files from $LHCBTAR/rpm/lhcb and call again lb-release-rpm.
Changed:
<
<
If it still does not work, check that there are no permanent problems in $LHCBTAR/rpm/lhcb (e.q. quota) and that the files are copied correctly with something like (bash)
>
>
If it still does not work, check that there are no permanent problems in $LHCBTAR/rpm/lhcb (e.q. quota) and that the files are copied correctly with something like (bash)
 for f in /data/artifacts/release/lhcb-release/${build_id}/*.rpm ; do cmp $f $LHCBTAR/rpm/lhcb/$(basename $f) done
Line: 549 to 507
 Until Gaudi v23r2, the dependencies to be included in LCGCMT were included in the Gaudi requirements files. The procedure to build the tarball was therefore the following:
Changed:
<
<
For Gaudi, one more setup needs to be done: the generation of the tarball of its dependencies. This has to be done for each and every optimized >binary< (x86-64-slc6-gcc46-opt, x86-64-slc5-gcc46-opt, x86-64-slc5-gcc43-opt, i686-slc5-gcc43-opt)

>
>
For Gaudi, one more setup needs to be done: the generation of the tarball of its dependencies. This has to be done for each and every optimized >binary< (x86-64-slc6-gcc46-opt, x86-64-slc5-gcc46-opt, x86-64-slc5-gcc43-opt, i686-slc5-gcc43-opt)
  cd $Gaudi_release_area mkLCGCMTtar -n GAUDI_ -b This script produces a log file in the local directory from where it has been run. Check it to see if everything was OK.
Line: 566 to 523
  LCGCMT 63a -> LHCbExternals v63r1 etc...
Changed:
<
<
Release LHCbExternals to AFS:

>
>
Release LHCbExternals to AFS:
  cd /afs/cern.ch/lhcb/software/releases/ mkproject -p LHCbExternals -v vXrY -a ngc mkproject -p LHCbExternals -v vXrY -a K
Line: 576 to 532
  Now prepare the tar ball of LCGCMT, which has to be done differently depending on the version of LCGCMT:
Changed:
<
<
* Up to LCGCMT 66:

>
>
* Up to LCGCMT 66:
  cd /afs/cern.ch/lhcb/software/releases/ mkLCGCMTtar -n LHCBEXTERNALS_ -b

This script produces a log file in the local directory from where it has been run. Check it to see if everything was OK.

Changed:
<
<
* As from LCGCMT 68:

>
>
* As from LCGCMT 68:
  mkLCGCMTtarFromRPM LHCbExternals v69r0p1 $CMTCONFIG
Line: 593 to 547
  We do not extract dependencies any more, the LHCBEXTERNALS project is abandoned.
Changed:
<
<
To Generate the install project tar files LCGCMT for x86_64-slc6-gcc49-opt, just run (LbScript v8r7 at least is needed)
>
>
To Generate the install project tar files LCGCMT for x86_64-slc6-gcc49-opt, just run (LbScript v8r7 at least is needed)
 makeTarFromRPM LCGCMT v86r0 x86_64-slc6-gcc49-opt ./LHCBEXTERNALS_86.json
Changed:
<
<
Where LHCBEXTERNALS_86.json is a file of the form:
>
>
Where LHCBEXTERNALS_86.json is a file of the form:
 { "heptools": { "version": 86,
Line: 652 to 604
  The metadata is contained in files such as:
Changed:
<
<
LHCBEXTERNALS_88.json:
>
>
LHCBEXTERNALS_88.json:
 { "heptools": { "version": 88,
Line: 706 to 657
 

Releasing a new version of LCG Grid

Changed:
<
<
After LHCbDirac v8r0 (before then mkLCGCMTTar was necessary)
>
>
After LHCbDirac v8r0 (before then mkLCGCMTTar was necessary)
 mkLCGCMTtarFromRPM LHCbDirac vXrY $CMTCONFIG --nonatives

If some RPMs are missing from the middleware, it is possible to create them and to add them to the LHCb externals repository in $LHCBTAR/rpm/lcg

Changed:
<
<
The code necessary is in the LCGRPM repo. It can be used to create the spec.
>
>
The code necessary is in the LCGRPM repo. It can be used to create the spec.
 git clone https://gitlab.cern.ch/lhcb-core/LCGRPM.git cd LCGRPM/package ./createExternalRPMSpec.py -s Grid -o ext.spec voms 2.0.12-3 x86_64-slc6-gcc48-opt rpmbuild -bb ext.spec
Changed:
<
<
After that step, the file should be copied to the RPM repo and the RPM repo should be reindexed:
>
>
After that step, the file should be copied to the RPM repo and the RPM repo should be reindexed:
 cp /tmp/rpmbuild/RPMS/noarch/voms_2.0.12_3_x86_64_slc6_gcc48_opt-1.0.0-1.noarch.rpm $LHCBTAR/rpm/lcg createrepo --workers=20 --update $LHCBTAR/rpm/lcg
Changed:
<
<
N.B. /tmp/rpmbuild is a default of createExternalRPMSpec, and the -b option can be used to put the files to another directory e.g.
>
>
N.B. /tmp/rpmbuild is a default of createExternalRPMSpec, and the -b option can be used to put the files to another directory e.g.
 ./createExternalRPMSpec.py -b /tmp/lben/toto -s Grid -o ext.spec voms 2.0.12-3 x86_64-slc6-gcc48-opt && rpmbuild -bb ext.spec
Line: 737 to 683
 

Preparing the meta RPM with the software to be installed locally on the Online farm

Changed:
<
<
First get a local version of LbNightlyTools:
>
>
First get a local version of LbNightlyTools:
 mkdir -p /build/$USER cd /build/$USER git clone http://git.cern.ch/pub/LbNightlyTools
Line: 749 to 693
  Now prepare the Meta RPM spec. There are two ways:
Changed:
<
<
  • By specifying explicitly the packages to be included:
>
>
  • By specifying explicitly the packages to be included:
 lbn-generate-metaspec -o ofm.spec OnlineFarmMeta 1.1.1 MOOREONLINE_v23r6_x86_64_slc6_gcc48_opt [...] (You need to pass all necessary applications RPMson the command lines, dependencies are dealt with automatically)
Changed:
<
<
And build it:
>
>
And build it:
 rpmbuild -bb ofm.spec

  • Using the ONLINE tag in the software configuration DB:
Changed:
<
<
First add the project/version/combination to the soft configuration DB and check that the update worked:
>
>
First add the project/version/combination to the soft configuration DB and check that the update worked:
 lb-sdb-tag -p x86_64-slc6-gcc48-opt MooreOnline v23r7p15 ONLINE lb-sdb-query listTag ONLINE
Changed:
<
<
Removal can be done with
>
>
Removal can be done with
 lb-sdb-tag -r -p x86_64-slc6-gcc48-opt MooreOnline v23r7pq5 ONLINE lb-sdb-query listTag ONLINE
Changed:
<
<
Then produce the RPM :
>
>
Then produce the RPM :
 . lb-sdb-env.sh lbn-generate-metaspec -o ofm.spec OnlineFarmMeta 1.1.1 -t ONLINE (You need to pass all necessary applications RPMson the command lines, dependencies are dealt with automatically)
Changed:
<
<
Build the RPM:
>
>
Build the RPM:
 rpmbuild -bb ofm.spec
Changed:
<
<
Now copying it to the RPM repository:
>
>
Now copying it to the RPM repository:
 lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 /tmp/rpmbuild/RPMS/noarch/ lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 --copy /tmp/rpmbuild/RPMS/noarch/
Changed:
<
<
Now install on plus:
>
>
Now install on plus:
 cd /group/hltsw ./onlinelbpkr install OnlineFarmMeta
Line: 802 to 738
  The releases should be completely automated, but here are the details needed to pcheck the system, perform manual installations/add slots etc...
Changed:
<
<
To connect:
>
>
To connect:
 ssh cvmfs-lhcbdev.cern.ch sudo -i -u cvlhcbdev
Changed:
<
<
Then you can start a transaction and copy files to /cvmfs/lhcbdev.cern.ch
>
>
Then you can start a transaction and copy files to /cvmfs/lhcbdev.cern.ch
 

Revision 1452017-07-24 - PatrickSKoppenburg

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 199 to 199
 
  • If the build was fine, publish the RPMs (from lhcb-archive.cern.ch)
    • Since default SW is now CVMFS, switch to AFS
%SYNTAX{ syntax="sh"}%
Added:
>
>
ssh lhcb-archive.cern.ch
 bash /afs/cern.ch/lhcb/software/releases/LBSCRIPTS/dev/InstallArea/scripts/LbLogin.sh kinit Your_Username@CERNNOSPAMPLEASE.CH

Revision 1442017-07-14 - StefanGabrielChitic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 88 to 88
 
  • publish the produced RPMs
    • try to copy the RPMs to the repository
      build_id=1234
Changed:
<
<
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb /data/archive/artifacts/release/lhcb-release/$build_id
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 /data/archive/artifacts/release/lhcb-release/$build_id
 
      • For old stacks only released as tar files:
        lb-release-oldtar /data/archive/artifacts/release/lhcb-release/$build_id/olddist
               
    • if everything is fine, do the real copy and update the repository info
Changed:
<
<
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb --copy /data/archive/artifacts/release/lhcb-release/$build_id
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 --copy /data/archive/artifacts/release/lhcb-release/$build_id
 
      • For old stacks only released as tar files:

Line: 160 to 160
 Check it's all there
ls -l /afs/cern.ch/lhcb/software/releases/${MyProject^^}/${MyProject^^}_${MyVersion}/
Changed:
<
<
ls -ltr /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb/ | tail -10 ls -l /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb/ | grep ${MyProject^^} | grep ${MyVersion}
>
>
ls -ltr /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017/ | tail -10 ls -l /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017/ | grep ${MyProject^^} | grep ${MyVersion}
 

In case of problems see here

Line: 205 to 205
 %ENDSYNTAX%
    • try to copy the RPMs to the repository
      build_id=1234
Changed:
<
<
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb /data/archive/artifacts/release/lhcb-release/$build_id
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 /data/archive/artifacts/release/lhcb-release/$build_id
 
    • if everything is fine, do the real copy and update the repository info
Changed:
<
<
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb --copy /data/archive/artifacts/release/lhcb-release/$build_id
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 --copy /data/archive/artifacts/release/lhcb-release/$build_id
 
Changed:
<
<
    • check it's all there
      ls -ltr /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb/ | tail -10
>
>
    • check it's all there
      ls -ltr /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017/ | tail -10
 
  • Install the new RPM on AFS (must be done from lhcb-archive)
    /afs/cern.ch/lhcb/software/lbinstall/afslbinstall install DBASE_Hat_Package_version
Line: 787 to 787
  Now copying it to the RPM repository:
Changed:
<
<
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb /tmp/rpmbuild/RPMS/noarch/ lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb --copy /tmp/rpmbuild/RPMS/noarch/
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 /tmp/rpmbuild/RPMS/noarch/ lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb2017 --copy /tmp/rpmbuild/RPMS/noarch/
 

Now install on plus:

Revision 1432017-07-12 - MarcoCattaneo

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 185 to 185
 
  • install the software on CVMFS if not already done in parallel. See here for instructions

Data packages

Changed:
<
<
Warning, important WARNING The procedure for the release of data packages via RPMs is still under heavy development and the automation is not yet in place. Moreover, the new procedure cover only the checkout and build steps, while special cases, tarballs, locking and cvmfs have do be done using the standard instructions at ProjectReleaseOld.
>
>
Warning, important WARNING The procedure for the release of data packages via RPMs is still under heavy development and the automation is not yet in place.
 
Line: 198 to 198
 
  • If the build was fine, publish the RPMs (from lhcb-archive.cern.ch)
    • Since default SW is now CVMFS, switch to AFS
Changed:
<
<
>
>
%SYNTAX{ syntax="sh"}%
 bash
Changed:
<
<
. /afs/cern.ch/lhcb/software/releases/LBSCRIPTS/dev/InstallArea/scripts/LbLogin.sh Warning, important kinit Your_Username@CERNNOSPAMPLEASE.CH

>
>
/afs/cern.ch/lhcb/software/releases/LBSCRIPTS/dev/InstallArea/scripts/LbLogin.sh kinit Your_Username@CERNNOSPAMPLEASE.CH %ENDSYNTAX%
 
    • try to copy the RPMs to the repository
      build_id=1234
      lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb  /data/archive/artifacts/release/lhcb-release/$build_id

Revision 1422017-07-12 - MarcoCattaneo

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 6 to 6
 

Introduction

Deleted:
<
<
Warning, important These instructions are a snapshot of the current state of the tools. The procedure will be simplified in the coming weeks.
 Warning, important These instructions do not apply to LbScripts. Use LbScriptsRelease instead.
Deleted:
<
<
Some of the old documentation at ProjectReleaseOld is still valid, in particular about troubleshooting. If you find something useful there copied it over here.
 

For Release Managers

For Projects:

Line: 31 to 27
 

For Release Shifters

Added:
>
>
See the LHCb shift database to find who is this week's shifter.
 

Special information

Starting from 13 October 2016, you should set your environment to DEV as : ". /afs/cern.ch/lhcb/software/releases/LBSCRIPTS/dev/InstallArea/scripts/LbLogin.sh"
Added:
>
>

Prerequisites for new release shifters

If you are a new release shifter, please make sure you have the prerequisites that are needed at some point of the release procedure.
<!--/twistyPlugin twikiMakeVisibleInline-->

  • You should be added as a "developer" in the LHCb Deployment Jira project (https://sft.its.cern.ch/jira/browse/LHCBDEP/), to receive and track release requests
  • You must be member of the z5:librarians AFS group in order to have the correct access to the AFS areas where the software is installed.
  • You must be member or the _lhcb_ AFS group to have to have the AFS administrator rights that allow you to create AFS volumes for LHCb and to install the rpms in the LHCb rpm repository
  • You must have a valid grid certificate (For release of DecFiles package)
  • You must be added to the lhcb-cvmfs-librarians E-group (for access to the IT managed CVMFS installation machine and for communication with other release shifters)
  • Check that the CMTPATH environment variable is not defined in the shell which you are using.
  • You need to be registered as a software deployment shifter in the shifts database
  • You need to be registered to use Jenkins, to be able to trigger release builds from the web interface (for data packages)

As the shifter you can self-assign shifts through the shift database shift database. Usually this is done by block-booking weeks for individual shifters.

<!--/twistyPlugin-->
 
<!--

Standard procedure

Revision 1412017-06-30 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 63 to 63
  Warning, important kinit Your_Username@CERNNOSPAMPLEASE.CH
Added:
>
>
  • Verify that the software configuration DB has been updated:
lb-sdb-query $MyProject $MyVersion

Should show projects in the slot or already released. If some are missing re-run:

lb-sdb-import $MyProject $MyVersion
 
  • publish the produced RPMs
    • try to copy the RPMs to the repository
Line: 99 to 108
  In case of SSO, or certificate problems:

Changed:
<
<
ssh lbariadne export SDBURL=http://localhost:7474/db/data/
>
>
ssh -fN -L12345:localhost:7474 lbariadne export SDBURL=http://localhost:12345/db/data/
 lb-deployment-updatesdb [...]
Line: 129 to 138
 lb-deployment-afs-install $MyProject $MyVersion
Added:
>
>
In case of problems or missing dependencies you can alter the options to lbinstall by setting the OPTS environment variable e.g.:
OPTS=--no-strict lb-deployment-afs-install $MyProject $MyVersion
 Check it's all there
ls -l /afs/cern.ch/lhcb/software/releases/${MyProject^^}/${MyProject^^}_${MyVersion}/
Line: 143 to 159
 lb-project-manage-volume -r $MyProject $MyVersion lb-project-manage-volume -l $MyProject $MyVersion
Changed:
<
<
  • prepare the web pages links
>
>
  • prepare the web pages links (NOT for DIRAC and LHCbDirac)
 $LHCBDOC/scripts/addrel.py $MyProject $MyVersion
Line: 291 to 307
 
<!-- SyntaxHighlightingPlugin -->
install_software.sh <Project> <version> 
<!-- end SyntaxHighlightingPlugin -->
Added:
>
>
In case of problems or missing dependencies you can alter the options to lbinstall by setting the OPTS environment variable e.g.:
<!-- SyntaxHighlightingPlugin -->
OPTS=--no-strict install_software.sh $MyProject $MyVersion
<!-- end SyntaxHighlightingPlugin -->
 

Check and Update

Revision 1402017-06-28 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 429 to 429
 

If the nightly builds do not work

The nightly build system is nothing more than wrappers around simple tools that can be called by any user, so it is possible to prepare the RPMs for the release by hand.
Changed:
<
<
  • Go to an SLC6 machine with AFS (e.g. lxplus or a build machine), to a temporary working directory (e.g. /build/$USER)
>
>
  • Go to an Centos7 machine with AFS (e.g. lxplus or a build machine), to a temporary working directory (e.g. /build/$USER)
 
  • Prepare the configuration file to drive the build
Changed:
<
<
lbn-gen-release-config --cmake MyProject vXrY > lhcb-release.json
>
>
lbn-gen-release-config MyProject vXrY > lhcb-release.json
 or
Changed:
<
<
lbn-gen-release-config --cmt --pack 'Hat/Package vXrY > lhcb-release.json
>
>
lbn-gen-release-config --cmt --pack 'Hat/Package vXrY' > lhcb-release.json
  for data packages

  • Checkout the sources and prepare the shared RPM

Revision 1392017-06-22 - PatrickSKoppenburg

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 333 to 333
  The output of those command is automatically logged to "${HOME}/logs/install.log
Added:
>
>
Sometimes a package is not known to the install_software.sh script. In that case
<!-- SyntaxHighlightingPlugin -->
cvmfslbinstall install DBASE_HAT_PKG_vXrY
<!-- end SyntaxHighlightingPlugin -->
may help.
 

CRON

Revision 1382017-06-20 - PatrickSKoppenburg

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 183 to 183
 
    • if everything is fine, do the real copy and update the repository info
      lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb --copy /data/archive/artifacts/release/lhcb-release/$build_id
      
Changed:
<
<
    • check it's all there
      ls -ltr $LHCBTAR/rpm/lhcb/ | tail -10
>
>
    • check it's all there
      ls -ltr /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb/ | tail -10
 
  • Install the new RPM on AFS (must be done from lhcb-archive)
    /afs/cern.ch/lhcb/software/lbinstall/afslbinstall install DBASE_Hat_Package_version

Revision 1372017-06-20 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 132 to 132
 Check it's all there
ls -l /afs/cern.ch/lhcb/software/releases/${MyProject^^}/${MyProject^^}_${MyVersion}/
Changed:
<
<
ls -ltr $LHCBTAR/rpm/lhcb/ | tail -10 ls -l $LHCBTAR/rpm/lhcb/ | grep ${MyProject^^} | grep ${MyVersion}
>
>
ls -ltr /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb/ | tail -10 ls -l /eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb/ | grep ${MyProject^^} | grep ${MyVersion}
 

In case of problems see here

Revision 1362017-06-12 - JoelClosier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 61 to 61
 bash . /afs/cern.ch/lhcb/software/releases/LBSCRIPTS/dev/InstallArea/scripts/LbLogin.sh
Added:
>
>
Warning, important kinit Your_Username@CERNNOSPAMPLEASE.CH
 
  • publish the produced RPMs
    • try to copy the RPMs to the repository
Line: 171 to 173
 
bash
. /afs/cern.ch/lhcb/software/releases/LBSCRIPTS/dev/InstallArea/scripts/LbLogin.sh
Added:
>
>
Warning, important kinit Your_Username@CERNNOSPAMPLEASE.CH
 
    • try to copy the RPMs to the repository
      build_id=1234

Revision 1352017-06-09 - StefanGabrielChitic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 65 to 65
 
  • publish the produced RPMs
    • try to copy the RPMs to the repository
      build_id=1234
Changed:
<
<
lb-release-rpm /data/archive/artifacts/release/lhcb-release/$build_id
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb /data/archive/artifacts/release/lhcb-release/$build_id
 
      • For old stacks only released as tar files:
        lb-release-oldtar /data/archive/artifacts/release/lhcb-release/$build_id/olddist
               
    • if everything is fine, do the real copy and update the repository info
Changed:
<
<
lb-release-rpm --copy /data/archive/artifacts/release/lhcb-release/$build_id
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb --copy /data/archive/artifacts/release/lhcb-release/$build_id
 
      • For old stacks only released as tar files:

Line: 174 to 174
 
    • try to copy the RPMs to the repository
      build_id=1234
Changed:
<
<
lb-release-rpm /data/archive/artifacts/release/lhcb-release/$build_id
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb /data/archive/artifacts/release/lhcb-release/$build_id
 
    • if everything is fine, do the real copy and update the repository info
Changed:
<
<
lb-release-rpm --copy /data/archive/artifacts/release/lhcb-release/$build_id
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb --copy /data/archive/artifacts/release/lhcb-release/$build_id
 
    • check it's all there
      ls -ltr $LHCBTAR/rpm/lhcb/ | tail -10
Line: 749 to 749
  Now copying it to the RPM repository:
Changed:
<
<
lb-release-rpm /tmp/rpmbuild/RPMS/noarch/ lb-release-rpm --copy /tmp/rpmbuild/RPMS/noarch/
>
>
lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb /tmp/rpmbuild/RPMS/noarch/ lb-release-rpm --rpm-dir=/eos/project/l/lhcbwebsites/www/lhcb-rpm/lhcb --copy /tmp/rpmbuild/RPMS/noarch/
 

Now install on plus:

Revision 1342017-06-03 - MarcoCattaneo

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 134 to 134
 ls -l $LHCBTAR/rpm/lhcb/ | grep ${MyProject^^} | grep ${MyVersion}
Changed:
<
<
In case of problems with dependencies see here
>
>
In case of problems see here
  All cases:
  • release and lock the AFS volume
Line: 382 to 382
  done %ENDSYNTAX%
Changed:
<
<

In case of problems with missing external dependencies

>
>

In case of problems with incomplete installs

 
Changed:
<
<
Sometimes, for certain platforms, the external dependencies may be missing. Depending on the reason for the missing dependencies, you may want to ignore the missing dependencies and install anyway, or just install those platforms for which the dependencies are OK.
>
>
Sometimes the automatic installation may fail, for example because not all platforms were correctly built in the release build, or because some of the needed external dependencies are missing. Depending on the reason for the incomplete install, you may want to ignore the missing dependencies, and install anyway, or just install those platforms for which the dependencies are OK.
 
  • First find the available packages:
Changed:
<
<
/afs/cern.ch/lhcb/software/lbinstall/afslbinstall  query "${MyProject^^}_${MyVersion}"
>
>
<!-- SyntaxHighlightingPlugin -->
/afs/cern.ch/lhcb/software/lbinstall/afslbinstall  query "${MyProject^^}_${MyVersion}"
<!-- end SyntaxHighlightingPlugin -->
 
Changed:
<
<
  • If required, force installation of RPMs with missing dependencies:
/afs/cern.ch/lhcb/software/lbinstall/afslbinstall install --no-strict <package_name>
>
>
  • There are two useful options of afslbinstall to help with partial deployment of the packages returned by the above query:
<!-- SyntaxHighlightingPlugin -->
/afs/cern.ch/lhcb/software/lbinstall/afslbinstall install --nodeps <package_name>
<!-- end SyntaxHighlightingPlugin -->
installs just the package but none of its dependencies. Be careful with this because you have to deploy by hand, one package at a time, the whole dependency stack for the deployment to be useful
 
Changed:
<
<
  • Then install manually all the other RPMs with:
/afs/cern.ch/lhcb/software/lbinstall/afslbinstall install <package_name>
>
>
<!-- SyntaxHighlightingPlugin -->
/afs/cern.ch/lhcb/software/lbinstall/afslbinstall install --no-strict <package_name>
<!-- end SyntaxHighlightingPlugin -->
installs the complete set of dependencies, but skipping any missing dependencies
 
Changed:
<
<
  • Hint: all the above can be condensed in a single command:
>
>
For completeness, afslbinstall called without options installs the complete set of dependencies but stops with an error if a dependency is missing. It is what is called internally by lb-deployment-afs-install
<!-- SyntaxHighlightingPlugin -->
/afs/cern.ch/lhcb/software/lbinstall/afslbinstall install <package_name>
<!-- end SyntaxHighlightingPlugin -->

  • Hint: all the above can be condensed in a single command to install all platforms ignoring missing dependencies:
%SYNTAX{ syntax="sh"}%
 for r in $(/afs/cern.ch/lhcb/software/lbinstall/afslbinstall query "${MyProject^^}_${MyVersion}"); do
Changed:
<
<
echo /afs/cern.ch/lhcb/software/lbinstall/afslbinstall install --no-strict $r
>
>
/afs/cern.ch/lhcb/software/lbinstall/afslbinstall install --no-strict $r
 done
Changed:
<
<
>
>
%ENDSYNTAX%
 
Changed:
<
<
  • Or, if you want to exclude the platform with missing dependencies:
for r in $(/afs/cern.ch/lhcb/software/lbinstall/afslbinstall query "${MyProject^^}_${MyVersion}" | grep -v ThePlatfromToExclude); do
echo /afs/cern.ch/lhcb/software/lbinstall/afslbinstall install $r
>
>
  • Or, if you want to exclude a platform (e.g. because it has missing dependencies):
%SYNTAX{ syntax="sh"}% for r in $(/afs/cern.ch/lhcb/software/lbinstall/afslbinstall query "${MyProject^^}_${MyVersion}" | grep -v ThePlatformToExclude); do /afs/cern.ch/lhcb/software/lbinstall/afslbinstall install $r
 done
Changed:
<
<
>
>
%ENDSYNTAX%
 
  • Don't forget to do the same on cvmfs, where you replace /afs/cern.ch/lhcb/software/lbinstall/afslbinstall with cvmfslbinstall
Line: 780 to 777
 

-- MarcoClemencic - 30 Jun 2014

Changed:
<
<
>
>
-- MarcoCattaneo - 3 Jun 2017
 

META FILEATTACHMENT attachment="rebuildRPMdb.py.txt" attr="" comment="Script to list commands to recreate RPM DB" date="1461683382" name="rebuildRPMdb.py.txt" path="rebuildRPMdb.py.txt" size="4322" user="bcouturi" version="1"

Revision 1332017-06-02 - MarcoCattaneo

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 134 to 134
 ls -l $LHCBTAR/rpm/lhcb/ | grep ${MyProject^^} | grep ${MyVersion}
Changed:
<
<
In case of problems, you can find out the list of packages available:

/afs/cern.ch/lhcb/software/lbinstall/afslbinstall  query "${MyProject^^}_${MyVersion}"

and install them manually for each RPM with:

/afs/cern.ch/lhcb/software/lbinstall/afslbinstall --nodeps  install <package_name>

But that disables dependency checking and at that point, you are on your own to ensure everything needed is installed.

>
>
In case of problems with dependencies see here
  All cases:
  • release and lock the AFS volume
Line: 395 to 382
  done %ENDSYNTAX%
Added:
>
>

In case of problems with missing external dependencies

Sometimes, for certain platforms, the external dependencies may be missing. Depending on the reason for the missing dependencies, you may want to ignore the missing dependencies and install anyway, or just install those platforms for which the dependencies are OK.

  • First find the available packages:
/afs/cern.ch/lhcb/software/lbinstall/afslbinstall  query "${MyProject^^}_${MyVersion}"

  • If required, force installation of RPMs with missing dependencies:
/afs/cern.ch/lhcb/software/lbinstall/afslbinstall install --no-strict <package_name>

  • Then install manually all the other RPMs with:
/afs/cern.ch/lhcb/software/lbinstall/afslbinstall install <package_name>

  • Hint: all the above can be condensed in a single command:
for r in $(/afs/cern.ch/lhcb/software/lbinstall/afslbinstall query "${MyProject^^}_${MyVersion}"); do
echo /afs/cern.ch/lhcb/software/lbinstall/afslbinstall install --no-strict $r
done

  • Or, if you want to exclude the platform with missing dependencies:
for r in $(/afs/cern.ch/lhcb/software/lbinstall/afslbinstall query "${MyProject^^}_${MyVersion}" | grep -v ThePlatfromToExclude); do
echo /afs/cern.ch/lhcb/software/lbinstall/afslbinstall install $r
done

  • Don't forget to do the same on cvmfs, where you replace /afs/cern.ch/lhcb/software/lbinstall/afslbinstall with cvmfslbinstall
 

Troubleshooting (For Librarians)

Revision 1322017-06-02 - MarcoCattaneo

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 105 to 105
 You can ignore the warnings such as: WARNING: Error encountered while acquiring CERN SSO cookie for Ariadne. Attempting to connect with no SSO..
Added:
>
>
From here on you will be installing the software on AFS and on CVMFS. The two installations can be done in parallel
 
Added:
>
>

AFS Software Installation

  Warning, important This step has to be performed from the host lhcb-archive.cern.ch
Line: 163 to 165
 but make sure that ${CMTCONFIG} is a platform valid for Gauss. Else use another one.
Changed:
<
<
  • install the software on CVMFS from the tarballs as usual. See below for instructions
>
>
  • install the software on CVMFS if not already done in parallel. See here for instructions
 

Data packages

Warning, important WARNING The procedure for the release of data packages via RPMs is still under heavy development and the automation is not yet in place. Moreover, the new procedure cover only the checkout and build steps, while special cases, tarballs, locking and cvmfs have do be done using the standard instructions at ProjectReleaseOld.
Line: 243 to 245
 Deployment on CVMFS of a DBASE package is the same as for a project, but replace <Project> ==> <Package>
Changed:
<
<
>
>
 

CVMFS Software Installation

Connection to "cvlhcb" account on host "cvmfs-lhcb"

Revision 1312017-05-31 - MarcoCattaneo

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 350 to 350
 

Removing Software (archival)

Projects

Changed:
<
<
  • On lxplus or one of the lxbuild machines (AFS and CVMFS access, as well as the afs_admin command are needed) run the command:
>
>
  • On lxplus or one of the lxbuild machines (AFS and CVMFS access, as well as the afs_admin command are needed)
    • Set up the AFS environment with the command
      afsLbLoginDev
    • Run the command:
 
archive_project <Project> <version>
This removes the software from the AFS release area and from the old Grid distribution area, and archives the tar files to castor. You need a grid proxy, since the first thing the script does is to contact the book-keeping to check that there isn't an active production step using the software to be archived. You have to hit several times to confirm that you want to remove the tar balls (don't say yes if there was some problem copying to castor)

Revision 1302017-05-23 - JoelClosier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 406 to 406
 
  • Checkout the sources and prepare the shared RPM
    lbn-checkout --verbose lhcb-release.json
Changed:
<
<
lbn-rpm --shared --verbose lhcb-release.json
>
>
lbn-rpm --shared --builddir tmp/checkout --verbose lhcb-release.json
 

  • For each required platform build, test and prepare the RPM (not needed for data packages)

Revision 1292017-05-23 - PatrickSKoppenburg

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 212 to 212
 
  1. Update the bookkeeping:

Changed:
<
<
cd $LHCBRELEASES/DBASE/Gen/DecFiles//cmt
>
>
cd /afs/cern.ch/lhcb/software/releases/DBASE/Gen/DecFiles//cmt
  lb-run LHCbDirac dirac-bookkeeping-eventtype-mgt-insert ../doc/table_event.sql lb-run LHCbDirac dirac-bookkeeping-eventtype-mgt-update ../doc/table_event.sql lb-run LHCbDirac dirac-bookkeeping-eventtype-mgt-update ../doc/table_obsolete.sql
Line: 236 to 236
 
  • Lock the AFS directory
After the build of the package release, the directory holding it can be locked. This will remove the write permissions to the directory and this can be achieved with:

Changed:
<
<
lockAFSDir -R $LHCBRELEASES/DBASE///
>
>
lockAFSDir -R /afs/cern.ch/lhcb/software/releases//DBASE///
 
  • Next?
If this is not a special case, the next thing to do is probably to deploy on CVMFS, instructions here.
Line: 362 to 362
 

Data Packages

Changed:
<
<
  • unlock the package directory
    unlockAFSDir -R $LHCBRELEASES/DBASE/<Hat>/<Package>/<version>
>
>
  • unlock the package directory
    unlockAFSDir -R /afs/cern.ch/lhcb/software/releases/DBASE/<Hat>/<Package>/<version>
 
  • remove the package
    /afs/cern.ch/lhcb/software/lbinstall/lbinstall remove <package_name>
  • got to the CVMFS publishing machine (cvmfs-lhcb) and uninstall
    cvmfs_transaction
    remove_software.sh <Package> <version>
Line: 377 to 377
 
<!-- SyntaxHighlightingPlugin -->
afs_admin sq $LHCBTAR/<PROJECT> <quota>
<!-- end SyntaxHighlightingPlugin -->

In a project:

Changed:
<
<
<!-- SyntaxHighlightingPlugin -->
fs lq $LHCBRELAESES/<PROJECT>/*
<!-- end SyntaxHighlightingPlugin -->
>
>
<!-- SyntaxHighlightingPlugin -->
fs lq /afs/cern.ch/lhcb/software/releases/<PROJECT>/*
<!-- end SyntaxHighlightingPlugin -->
 Then increase quota with
Changed:
<
<
<!-- SyntaxHighlightingPlugin -->
afs_admin sq $LHCBRELAESES/<PROJECT>/<PROJECT>_<version>/*
<!-- end SyntaxHighlightingPlugin -->
>
>
<!-- SyntaxHighlightingPlugin -->
afs_admin sq /afs/cern.ch/lhcb/software/releases/<PROJECT>/<PROJECT>_<version>/*
<!-- end SyntaxHighlightingPlugin -->
  In rpms... on 2/12/2016 Ben did (see JIRA): %SYNTAX{ syntax="sh"}%
Line: 467 to 467
 /afs/cern.ch/lhcb/software/lbinstall/lbinstal remove $(/afs/cern.ch/lhcb/software/rpmrel/afslbpkr list ${MyProject^^}_${MyVersion})
  • remove the AFS volume in the release area
Changed:
<
<
afs_admin delete $LHCBRELEASES/${MyProject^^}/${MyProject^^}_${MyVersion}
>
>
afs_admin delete /afs/cern.ch/lhcb/software/releases/${MyProject^^}/${MyProject^^}_${MyVersion}
 
  • remove the RPMs from the repository
    rm -i $LHCBTAR/rpm/lhcb/${MyProject^^}_${MyVersion}*.rpm
Line: 502 to 502
  Release LHCbExternals to AFS:

Changed:
<
<
cd $LHCBRELEASE
>
>
cd /afs/cern.ch/lhcb/software/releases/
  mkproject -p LHCbExternals -v vXrY -a ngc mkproject -p LHCbExternals -v vXrY -a K
Line: 512 to 512
  * Up to LCGCMT 66:

Changed:
<
<
cd $LHCBRELEASE
>
>
cd /afs/cern.ch/lhcb/software/releases/
  mkLCGCMTtar -n LHCBEXTERNALS_ -b

This script produces a log file in the local directory from where it has been run. Check it to see if everything was OK.

Revision 1282017-05-22 - PatrickSKoppenburg

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 238 to 238
 
          lockAFSDir -R $LHCBRELEASES/DBASE/<Hat>/<Package>/<version>
  • Next?
Changed:
<
<
If this is not a special case, the next thing to do is probably to deploy on CVMFS, instructions here. Or return to the summary section summary.
>
>
If this is not a special case, the next thing to do is probably to deploy on CVMFS, instructions here.
  Deployment on CVMFS of a DBASE package is the same as for a project, but replace <Project> ==> <Package>

Revision 1272017-05-05 - MichalKreps

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 178 to 178
 
    • click on the small Jenkins icon in the "Project" column of the build summary look for the package build log (after the line "...:INFO... building Hat/Package")

  • If the build was fine, publish the RPMs (from lhcb-archive.cern.ch)
Added:
>
>
    • Since default SW is now CVMFS, switch to AFS
bash
. /afs/cern.ch/lhcb/software/releases/LBSCRIPTS/dev/InstallArea/scripts/LbLogin.sh
 
    • try to copy the RPMs to the repository
      build_id=1234
      lb-release-rpm /data/archive/artifacts/release/lhcb-release/$build_id

Revision 1262017-05-05 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 147 to 147
  But that disables dependency checking and at that point, you are on your own to ensure everything needed is installed.
Deleted:
<
<
<!--/twistyPlugin twikiMakeVisibleInline-->
Gaudi requires its own documentation to be built and installed on AFS before locking the release directory.
  • go to a temporary directory
    cd $TMPDIR
    
  • install a local copy of the sources of Gaudi (using LbNightlyTools)
    lbn-install --project Gaudi --platform src --flavour release lhcb-release $build_id
    
  • build the project and the Doxygen documentation (requires Doxygen 1.8.7)
    cd $TMPDIR/lhcb-release/$build_id/GAUDI/GAUDI_$MyVersion
    make && make doc
    
  • install on AFS the generated documentation
    mkdir -p $GAUDISOFT/GAUDI/GAUDI_$MyVersion/InstallArea/doc
    rsync -a $TMPDIR/lhcb-release/$build_id/GAUDI/GAUDI_$MyVersion/build.$CMTCONFIG/doxygen/ $GAUDISOFT/GAUDI/GAUDI_$MyVersion/InstallArea/doc/
    
  • clean up your temporary directory
    rm -rf $TMPDIR/lhcb-release
    
<!--/twistyPlugin-->
 All cases:
  • release and lock the AFS volume
    lb-project-manage-volume -r $MyProject $MyVersion

Revision 1252017-04-27 - PatrickSKoppenburg

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 127 to 127
  Check it's all there
Changed:
<
<
ls -ltr /afs/cern.ch/lhcb/software/releases/${MyProject^^}/${MyProject^^}_${MyVersion}/
>
>
ls -l /afs/cern.ch/lhcb/software/releases/${MyProject^^}/${MyProject^^}_${MyVersion}/
 ls -ltr $LHCBTAR/rpm/lhcb/ | tail -10
Changed:
<
<
ls -ltr $LHCBTAR/rpm/lhcb/ | grep ${MyProject^^} | grep ${MyVersion}
>
>
ls -l $LHCBTAR/rpm/lhcb/ | grep ${MyProject^^} | grep ${MyVersion}
 

In case of problems, you can find out the list of packages available:

Revision 1242017-04-06 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 539 to 539
  mkLCGCMTtarFromRPM LHCbExternals v69r0p1 $CMTCONFIG
Changed:
<
<
* As from LCG 86
>
>

As from LCG 86

  We do not extract dependencies any more, the LHCBEXTERNALS project is abandoned.
Line: 595 to 596
 
Added:
>
>

As from LCG 87

We now just need to pre-install the build dependencies so that the nightly builds can continue working. The RPM needed at install time are pulled in as needed.

The metadata is contained in files such as:

LHCBEXTERNALS_88.json:

{
    "heptools": {
    "version": 88,
    "packages":[
        "AIDA",
        "Boost",
        "eigen",
   "CLHEP",
   "COOL",
   "CORAL",
   "CppUnit",
   "Frontier_Client",
   "GSL",
   "HepMC",
   "HepPDT",
   "Python",
   "QMtest",
   "Qt",
   "RELAX",
   "ROOT",
   "XercesC",
   "fastjet",
   "fftw",
   "graphviz",
   "libunwind",
   "neurobayes_expert",
   "oracle",
   "pyanalysis",
   "pygraphics",
   "pytools",
   "sqlite",
   "tcmalloc",
   "vdt",
   "xqilla",
   "xrootd",
   "tbb",
   "rangev3",
   "cppgsl",
   "ipython"]

    }
}

The metadata files are kept and versioned in https://gitlab.cern.ch/lhcb-core/rpm-recipes (LHCBEXTERNALS sub directory).

Follow the instructions in the README.md to get the list of rpms to install.

 

Releasing a new version of LCG Grid

After LHCbDirac v8r0 (before then mkLCGCMTTar was necessary)

Revision 1232017-03-23 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 83 to 83
 If your slot contains more projects that the ones requested for release, you need to ONLY copy the needed ones to the RPM repo. In that case the --rpm-regex option of lb-release-rpm should be used.
Added:
>
>
  • update the software database

First try in dry-run mode:

lb-deployment-updatesdb /data/archive/artifacts/release/lhcb-release/$build_id
If more platforms/projects are being updated than necessary you can limit using the options of the lb-deployment-updatesdb command. If all ok, just run:
lb-deployment-updatesdb --update /data/archive/artifacts/release/lhcb-release/$build_id

In case of SSO, or certificate problems:

ssh lbariadne
export SDBURL=http://localhost:7474/db/data/
lb-deployment-updatesdb [...]

You can ignore the warnings such as: WARNING: Error encountered while acquiring CERN SSO cookie for Ariadne. Attempting to connect with no SSO..

 

Warning, important This step has to be performed from the host lhcb-archive.cern.ch

Line: 160 to 183
 
mkMCGentarFromRPM ${MyProject} ${MyVersion} ${CMTCONFIG}
but make sure that ${CMTCONFIG} is a platform valid for Gauss. Else use another one.
Deleted:
<
<
  • update the software database
    lb-deployment-updatesdb $MyProject $MyVersion
    
    (you may get weird error messages, in that case just try again).

In case of SSO, or certificate problems:

ssh lbariadne
export SDBURL=http://localhost:7474/db/data/
lb-deployment-updatesdb $MyProject $MyVersion

You can ignore the warnings such as: WARNING: Error encountered while acquiring CERN SSO cookie for Ariadne. Attempting to connect with no SSO..

 

  • install the software on CVMFS from the tarballs as usual. See below for instructions

Revision 1222017-03-15 - JoelClosier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 598 to 598
 The code necessary is in the LCGRPM repo. It can be used to create the spec.
Changed:
<
<
git clone https://:@git.cern.ch/kerberos/LCGRPM
>
>
git clone https://gitlab.cern.ch/lhcb-core/LCGRPM.git
 cd LCGRPM/package ./createExternalRPMSpec.py -s Grid -o ext.spec voms 2.0.12-3 x86_64-slc6-gcc48-opt rpmbuild -bb ext.spec

Revision 1212017-03-07 - MarcoCattaneo

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 104 to 104
  Check it's all there
Deleted:
<
<
ls -ltr /afs/cern.ch/lhcb/distribution//${MyProject^^}
 ls -ltr /afs/cern.ch/lhcb/software/releases/${MyProject^^}/${MyProject^^}_${MyVersion}/ ls -ltr $LHCBTAR/rpm/lhcb/ | tail -10 ls -ltr $LHCBTAR/rpm/lhcb/ | grep ${MyProject^^} | grep ${MyVersion}

Revision 1202017-02-22 - MichalKreps

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 211 to 211
 Gen/DecFiles usually come in pairs (two versions for one JIRA task): one for dev and one for legacy Gauss versions. Michal will give instructions in this case (always follow his instructions, which take precedence over what is written below).
/afs/cern.ch/lhcb/software/lbinstall/afslbinstall install DBASE_Gen_DecFiles
will install the latest version. To get the legacy version do
Changed:
<
<
/afs/cern.ch/lhcb/software/rpmrel/afslbpkr rpm -- -ivh --oldpackage /afs/cern.ch/lhcb/distribution/rpm/lhcb/DBASE_Gen_DecFiles-XX.YY.Z-1.noarch.rpm
>
>
/afs/cern.ch/lhcb/software/lbinstall/afslbinstall install DBASE_Gen_DecFiles_vXXrYY
 
Changed:
<
<
where XX, YY, Z are given in the JIRA task.
>
>
where XX, YY are given in the JIRA task.
  Then one needs an extra step to update the bookkeeping. It requires to have a valid Grid certificate
  1. Get your Grid proxy:

Revision 1192017-02-16 - ChristopherRJones

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 102 to 102
 lb-deployment-afs-install $MyProject $MyVersion
Changed:
<
<
Check it's all there
ls -ltr /afs/cern.ch/lhcb/distribution//${MyProject^^}
>
>
Check it's all there
ls -ltr /afs/cern.ch/lhcb/distribution//${MyProject^^}
 ls -ltr /afs/cern.ch/lhcb/software/releases/${MyProject^^}/${MyProject^^}_${MyVersion}/
Changed:
<
<
ls -ltr $LHCBTAR/rpm/lhcb/ | tail -10
>
>
ls -ltr $LHCBTAR/rpm/lhcb/ | tail -10 ls -ltr $LHCBTAR/rpm/lhcb/ | grep ${MyProject^^} | grep ${MyVersion}
  In case of problems, you can find out the list of packages available:

Revision 1182017-01-27 - PatrickSKoppenburg

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 72 to 72
 lb-release-oldtar /data/archive/artifacts/release/lhcb-release/$build_id/olddist
    • if everything is fine, do the real copy and update the repository info
Deleted:
<
<
build_id=1234
 lb-release-rpm --copy /data/archive/artifacts/release/lhcb-release/$build_id
    • For old stacks only released as tar files:
Line: 195 to 194
 lb-release-rpm /data/archive/artifacts/release/lhcb-release/$build_id
    • if everything is fine, do the real copy and update the repository info
Deleted:
<
<
build_id=1234
 lb-release-rpm --copy /data/archive/artifacts/release/lhcb-release/$build_id
Changed:
<
<
    • check it's all there
      ls -ltr /afs/cern.ch/lhcb/distribution/DBASE/
      ls -ltr $LHCBTAR/rpm/lhcb/ | tail -10
>
>
    • check it's all there
      ls -ltr $LHCBTAR/rpm/lhcb/ | tail -10
 
  • Install the new RPM on AFS (must be done from lhcb-archive)
    /afs/cern.ch/lhcb/software/lbinstall/afslbinstall install DBASE_Hat_Package_version

Revision 1172017-01-25 - PatrickSKoppenburg

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 209 to 209
 
<!--/twistyPlugin twikiMakeVisibleInline-->
Gen/DecFiles usually come in pairs (two versions for one JIRA task): one for dev and one for legacy Gauss versions. Michal will give instructions in this case (always follow his instructions, which take precedence over what is written below).
/afs/cern.ch/lhcb/software/lbinstall/afslbinstall install DBASE_Gen_DecFiles
Changed:
<
<
will install the latest version. To get the legacy version do
>
>
will install the latest version. To get the legacy version do
 /afs/cern.ch/lhcb/software/rpmrel/afslbpkr rpm -- -ivh --oldpackage /afs/cern.ch/lhcb/distribution/rpm/lhcb/DBASE_Gen_DecFiles-XX.YY.Z-1.noarch.rpm where XX, YY, Z are given in the JIRA task.

Revision 1162017-01-25 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 193 to 193
 
    • try to copy the RPMs to the repository
      build_id=1234
      lb-release-rpm /data/archive/artifacts/release/lhcb-release/$build_id
Deleted:
<
<
lb-release-oldtar /data/archive/artifacts/release/lhcb-release/$build_id/olddist
 
    • if everything is fine, do the real copy and update the repository info
      build_id=1234
      lb-release-rpm --copy /data/archive/artifacts/release/lhcb-release/$build_id
Deleted:
<
<
lb-release-oldtar --copy /data/archive/artifacts/release/lhcb-release/$build_id/olddist
 
    • check it's all there
      ls -ltr /afs/cern.ch/lhcb/distribution/DBASE/
      ls -ltr $LHCBTAR/rpm/lhcb/ | tail -10

  • Install the new RPM on AFS (must be done from lhcb-archive)
    /afs/cern.ch/lhcb/software/lbinstall/afslbinstall install DBASE_Hat_Package_version
Changed:
<
<
If this is not possible because of a lock file, try figuring out why there is a lock (someone else releasing?) and then
rm /afs/cern.ch/lhcb/software/rpmrel/var/lib/rpm/.dbenv.lock
>
>
 
<!--/twistyPlugin twikiMakeVisibleInline-->
Line: 310 to 308
 install_software.sh %ENDSYNTAX%
Deleted:
<
<

Older stacks, with no debug option

For stacks < LHCb v35r4 (before Gaudi v23r6), no debug platforms were put on cvmfs. This instruction is now a little obsolete:
<!-- SyntaxHighlightingPlugin -->
install_software.sh <Project> <version> --without-dbg 
<!-- end SyntaxHighlightingPlugin -->

Not default platforms, some older stack

For older stacks, or for installing non-default platforms, you should install them one at-a-time, with the following command:

<!-- SyntaxHighlightingPlugin -->
install_software_direct.sh --binary=<cmtconfig> <Project> <version> 
<!-- end SyntaxHighlightingPlugin -->
 

Check and Update

Check: Sometimes what seems to be a warning is actually a fatal error. You can then check it all went fine with

<!-- SyntaxHighlightingPlugin -->
ls -ltr /cvmfs/lhcb.cern.ch/lib/lhcb/<PROJECT>/<PROJECT>_<version>/InstallArea/
<!-- end SyntaxHighlightingPlugin -->

Revision 1152017-01-25 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 302 to 302
 At this stage you may need to determine which stack a software release is based on. In this case in a different window on any lxplus machine run:
<!-- SyntaxHighlightingPlugin -->
lb-sdb-query --readonly d <Project> <version> 
<!-- end SyntaxHighlightingPlugin -->
Changed:
<
<

Install a project from recent stacks, for the default platforms

>
>

Install a project or Data package from recent stacks, for the default platforms

  If the stack is the most recent, and only default platforms are required
Line: 310 to 310
 install_software.sh %ENDSYNTAX%
Deleted:
<
<

Install a Data Package

In that case, the simplest way is to install directly the RPM package, e.g. for PRConfig v1r23 from DBASE (n.b the version format):

<!-- SyntaxHighlightingPlugin -->
install_software_direct.sh DBASE_PRConfig 1.23.0 
<!-- end SyntaxHighlightingPlugin -->
 

Older stacks, with no debug option

For stacks < LHCb v35r4 (before Gaudi v23r6), no debug platforms were put on cvmfs. This instruction is now a little obsolete:
<!-- SyntaxHighlightingPlugin -->
install_software.sh <Project> <version> --without-dbg 
<!-- end SyntaxHighlightingPlugin -->

Revision 1142017-01-24 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 204 to 204
 ls -ltr $LHCBTAR/rpm/lhcb/ | tail -10

  • Install the new RPM on AFS (must be done from lhcb-archive)
Changed:
<
<
/afs/cern.ch/lhcb/software/lbinstall/afslbinstall install DBASE_Hat_Package
>
>
/afs/cern.ch/lhcb/software/lbinstall/afslbinstall install DBASE_Hat_Package_version
 If this is not possible because of a lock file, try figuring out why there is a lock (someone else releasing?) and then
rm /afs/cern.ch/lhcb/software/rpmrel/var/lib/rpm/.dbenv.lock

Revision 1132017-01-24 - PatrickSKoppenburg

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 56 to 56
  Warning, important This step has to be performed from the host lhcb-archive.cern.ch
Added:
>
>
  • go there
    ssh lhcb-archive.cern.ch
    bash
    . /afs/cern.ch/lhcb/software/releases/LBSCRIPTS/dev/InstallArea/scripts/LbLogin.sh
    
 
  • publish the produced RPMs
    • try to copy the RPMs to the repository
      build_id=1234
      lb-release-rpm /data/archive/artifacts/release/lhcb-release/$build_id
      
Changed:
<
<
For old stacks only released as tar files:
>
>
    • For old stacks only released as tar files:
 
lb-release-oldtar /data/archive/artifacts/release/lhcb-release/$build_id/olddist
       
Line: 71 to 75
 build_id=1234 lb-release-rpm --copy /data/archive/artifacts/release/lhcb-release/$build_id
Changed:
<
<
For old stacks only released as tar files:
>
>
    • For old stacks only released as tar files:
 
lb-release-oldtar --copy /data/archive/artifacts/release/lhcb-release/$build_id/olddist
Line: 99 to 103
 lb-deployment-afs-install $MyProject $MyVersion
Changed:
<
<
Check it's all there
 ls -ltr /afs/cern.ch/lhcb/distribution//${MyProject^^}
>
>
Check it's all there
ls -ltr /afs/cern.ch/lhcb/distribution//${MyProject^^}
ls -ltr /afs/cern.ch/lhcb/software/releases/${MyProject^^}/${MyProject^^}_${MyVersion}/
ls -ltr $LHCBTAR/rpm/lhcb/ | tail -10
  In case of problems, you can find out the list of packages available:
Line: 194 to 200
 lb-release-rpm --copy /data/archive/artifacts/release/lhcb-release/$build_id lb-release-oldtar --copy /data/archive/artifacts/release/lhcb-release/$build_id/olddist
Changed:
<
<
    • check it's all there
       ls -ltr /afs/cern.ch/lhcb/distribution/DBASE/
>
>
    • check it's all there
      ls -ltr /afs/cern.ch/lhcb/distribution/DBASE/
      ls -ltr $LHCBTAR/rpm/lhcb/ | tail -10
 
  • Install the new RPM on AFS (must be done from lhcb-archive)
    /afs/cern.ch/lhcb/software/lbinstall/afslbinstall install DBASE_Hat_Package

Revision 1122017-01-13 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 61 to 61
 build_id=1234 lb-release-rpm /data/archive/artifacts/release/lhcb-release/$build_id
Added:
>
>
For old stacks only released as tar files:
lb-release-oldtar /data/archive/artifacts/release/lhcb-release/$build_id/olddist
       
 
    • if everything is fine, do the real copy and update the repository info
      build_id=1234
      lb-release-rpm --copy /data/archive/artifacts/release/lhcb-release/$build_id
      
Added:
>
>
For old stacks only released as tar files:
lb-release-oldtar --copy /data/archive/artifacts/release/lhcb-release/$build_id/olddist
  BE CAREFUL: Since 20141010, the release procedure automatically bumps up the release number for packages for which there is already a version in the RPM repo. If your slot contains more projects that the ones requested for release, you need to ONLY copy the needed ones to the RPM repo.

Revision 1112017-01-13 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 60 to 60
 
    • try to copy the RPMs to the repository
      build_id=1234
      lb-release-rpm /data/archive/artifacts/release/lhcb-release/$build_id
Deleted:
<
<
lb-release-oldtar /data/archive/artifacts/release/lhcb-release/$build_id/olddist
 
    • if everything is fine, do the real copy and update the repository info
      build_id=1234
      lb-release-rpm --copy /data/archive/artifacts/release/lhcb-release/$build_id
Deleted:
<
<
lb-release-oldtar --copy /data/archive/artifacts/release/lhcb-release/$build_id/olddist
 

BE CAREFUL: Since 20141010, the release procedure automatically bumps up the release number for packages for which there is already a version in the RPM repo.

Revision 1102017-01-12 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 140 to 140
 
Changed:
<
<
  • For Gauss, it is necessary to create the Genser tarball
>
>
  • FOR OLD STACKS NOT RELEASED VIA RPM: For Gauss, it is necessary to create the Genser tarball
 
mkMCGentarFromRPM ${MyProject} ${MyVersion} ${CMTCONFIG}
but make sure that ${CMTCONFIG} is a platform valid for Gauss. Else use another one.

Revision 1092017-01-09 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 285 to 285
 For stacks >= LHCb v35r4 (with Gaudi v23r6), the debug version will be installed on CVMFS. The software should be installed the scripts from the ${HOME}/bin directory.

At this stage you may need to determine which stack a software release is based on. In this case in a different window on any lxplus machine run:%SYNTAX{ syntax="sh"}%

Changed:
<
<
svnProjectDeps -P %ENDSYNTAX%
>
>
lb-sdb-query --readonly d %ENDSYNTAX%
 
Changed:
<
<

Most recent stack, and only default platforms (or Data package)

>
>

Install a project from recent stacks, for the default platforms

  If the stack is the most recent, and only default platforms are required

<!-- SyntaxHighlightingPlugin -->
install_software.sh <Project> <version> 
<!-- end SyntaxHighlightingPlugin -->
Added:
>
>

Install a Data Package

In that case, the simplest way is to install directly the RPM package, e.g. for PRConfig v1r23 from DBASE (n.b the version format):

<!-- SyntaxHighlightingPlugin -->
install_software_direct.sh DBASE_PRConfig 1.23.0 
<!-- end SyntaxHighlightingPlugin -->
 

Older stacks, with no debug option

For stacks < LHCb v35r4 (before Gaudi v23r6), no debug platforms were put on cvmfs. This instruction is now a little obsolete:
<!-- SyntaxHighlightingPlugin -->
install_software.sh <Project> <version> --without-dbg 
<!-- end SyntaxHighlightingPlugin -->

Revision 1082016-12-21 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 35 to 35
 

Special information

Starting from 13 October 2016, you should set your environment to DEV as : ". /afs/cern.ch/lhcb/software/releases/LBSCRIPTS/dev/InstallArea/scripts/LbLogin.sh"
Added:
>
>
<!--
 

Standard procedure

Deleted:
<
<
 The script lb-deployment-checklist simplifies the release shifter task by checking what needs to be done and either prints the next command to run or points to this page.

So connect to lxplus (or equivalent SLC6 machine) and run

Line: 47 to 47
 and follow the instructions.

The latest version of lb-deployment-checklist may cache some data to speed up the execution. If you think it's showing wrong messages, try to remove the directory ~/.cache/lb-deployment/.

Changed:
<
<
>
>
-->
 

Details of the procedure

Line: 88 to 88
 lb-project-manage-volume -c $MyProject $MyVersion
    • install the new project
Changed:
<
<
lb-deployment-afs-install $MyProject $MyVersion x86_64-slc6-gcc4{8,9}-{opt,dbg} x86_64-slc6-gcc49-do0 x86_64-centos7-gcc49-{opt,dbg} of if gcc49 / centos7 is not available:
lb-deployment-afs-install $MyProject $MyVersion x86_64-slc6-gcc48-{opt,dbg}
>
>
lb-deployment-afs-install $MyProject $MyVersion
 

Check it's all there

 ls -ltr /afs/cern.ch/lhcb/distribution//${MyProject^^}

Revision 1072016-12-16 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 103 to 103
  and install them manually for each RPM with:
Changed:
<
<
/afs/cern.ch/lhcb/software/lbinstall/afslbinstall install
>
>
/afs/cern.ch/lhcb/software/lbinstall/afslbinstall --nodeps install
 
Line: 152 to 152
  (you may get weird error messages, in that case just try again).
Added:
>
>
In case of SSO, or certificate problems:
ssh lbariadne
export SDBURL=http://localhost:7474/db/data/
lb-deployment-updatesdb $MyProject $MyVersion

You can ignore the warnings such as: WARNING: Error encountered while acquiring CERN SSO cookie for Ariadne. Attempting to connect with no SSO..

 
  • install the software on CVMFS from the tarballs as usual. See below for instructions

Data packages

Revision 1062016-12-02 - PatrickSKoppenburg

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 93 to 93
 lb-deployment-afs-install $MyProject $MyVersion x86_64-slc6-gcc48-{opt,dbg}
Added:
>
>
Check it's all there
 ls -ltr /afs/cern.ch/lhcb/distribution//${MyProject^^}
 In case of problems, you can find out the list of packages available:
Line: 294 to 296
 

Check and Update

Changed:
<
<
Check: Sometimes what seems to be a warning is actually a fatal error. You can then check it all went fine with
<!-- SyntaxHighlightingPlugin -->
ls -l /cvmfs/lhcb.cern.ch/lib/lhcb/<PROJECT>/<PROJECT>_<version>/InstallArea/
<!-- end SyntaxHighlightingPlugin -->
>
>
Check: Sometimes what seems to be a warning is actually a fatal error. You can then check it all went fine with
<!-- SyntaxHighlightingPlugin -->
ls -ltr /cvmfs/lhcb.cern.ch/lib/lhcb/<PROJECT>/<PROJECT>_<version>/InstallArea/
<!-- end SyntaxHighlightingPlugin -->
 The platforms you have installed should be there, the default platforms are currently only slc6-gcc48, opt and dbg. (the directory to check is different for a Data package, but should be easy to guess.)

Update: Once you have done all the installs, you have to update the global catalog (needs to be done just once at the end, to publish everything that you have installed)

<!-- SyntaxHighlightingPlugin -->
cvmfs_publish
<!-- end SyntaxHighlightingPlugin -->

Revision 1052016-12-02 - PatrickSKoppenburg

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 8 to 8
  Warning, important These instructions are a snapshot of the current state of the tools. The procedure will be simplified in the coming weeks.
Changed:
<
<
Warning, important These instructions do not apply to LbScripts. Use LbScriptsRelease instead.
>
>
Warning, important These instructions do not apply to LbScripts. Use LbScriptsRelease instead.

Some of the old documentation at ProjectReleaseOld is still valid, in particular about troubleshooting. If you find something useful there copied it over here.

 

For Release Managers

Line: 87 to 89
 
    • install the new project
      lb-deployment-afs-install $MyProject $MyVersion x86_64-slc6-gcc4{8,9}-{opt,dbg} x86_64-slc6-gcc49-do0 x86_64-centos7-gcc49-{opt,dbg}
Changed:
<
<
>
>
of if gcc49 / centos7 is not available:
 lb-deployment-afs-install $MyProject $MyVersion x86_64-slc6-gcc48-{opt,dbg}
Changed:
<
<
if gcc49 / centos7 is not available.
>
>
  In case of problems, you can find out the list of packages available:
Line: 360 to 362
 cvmfs_publish
Changed:
<
<

Troubleshooting (AKA For Librarians)

>
>

Troubleshooting (For Releasers)

If there is a quota problem

When copying tars. First check what's wrong
<!-- SyntaxHighlightingPlugin -->
fs lq $LHCBTAR/*
<!-- end SyntaxHighlightingPlugin -->
Then increase quota with
<!-- SyntaxHighlightingPlugin -->
afs_admin sq $LHCBTAR/<PROJECT> <quota>
<!-- end SyntaxHighlightingPlugin -->

In a project:

<!-- SyntaxHighlightingPlugin -->
fs lq $LHCBRELAESES/<PROJECT>/*
<!-- end SyntaxHighlightingPlugin -->
Then increase quota with
<!-- SyntaxHighlightingPlugin -->
afs_admin sq $LHCBRELAESES/<PROJECT>/<PROJECT>_<version>/*
<!-- end SyntaxHighlightingPlugin -->

In rpms... on 2/12/2016 Ben did (see JIRA):

<!-- SyntaxHighlightingPlugin -->
for r in $LHCBTAR/rpm/lhcb/<PROJECT>_<version>_*rpm; do 
          for p in `rpm -qp --requires $r | grep LCG`; 
               do ./afslbinstall --disable-yum-check --just-db install  $p; 
           done; 
    done
<!-- end SyntaxHighlightingPlugin -->

Troubleshooting (For Librarians)

 

If the nightly builds do not work

The nightly build system is nothing more than wrappers around simple tools that can be called by any user, so it is possible to prepare the RPMs for the release by hand.

Revision 1042016-12-01 - PatrickSKoppenburg

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 179 to 179
 /afs/cern.ch/lhcb/software/lbinstall/afslbinstall install DBASE_Hat_Package If this is not possible because of a lock file, try figuring out why there is a lock (someone else releasing?) and then
rm /afs/cern.ch/lhcb/software/rpmrel/var/lib/rpm/.dbenv.lock
Deleted:
<
<
Post-processing
 
<!--/twistyPlugin twikiMakeVisibleInline-->
Changed:
<
<
For Gen/DecFiles one needs an extra step to update the bookkeeping. It requires to have a valid Grid certificate
>
>
Gen/DecFiles usually come in pairs (two versions for one JIRA task): one for dev and one for legacy Gauss versions. Michal will give instructions in this case (always follow his instructions, which take precedence over what is written below).
/afs/cern.ch/lhcb/software/lbinstall/afslbinstall install DBASE_Gen_DecFiles
will install the latest version. To get the legacy version do /afs/cern.ch/lhcb/software/rpmrel/afslbpkr rpm -- -ivh --oldpackage /afs/cern.ch/lhcb/distribution/rpm/lhcb/DBASE_Gen_DecFiles-XX.YY.Z-1.noarch.rpm where XX, YY, Z are given in the JIRA task.

Then one needs an extra step to update the bookkeeping. It requires to have a valid Grid certificate

 
  1. Get your Grid proxy:
              lb-run LHCbDirac lhcb-proxy-init

Revision 1032016-11-30 - PatrickSKoppenburg

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 173 to 173
 lb-release-rpm --copy /data/archive/artifacts/release/lhcb-release/$build_id lb-release-oldtar --copy /data/archive/artifacts/release/lhcb-release/$build_id/olddist
Added:
>
>
    • check it's all there
       ls -ltr /afs/cern.ch/lhcb/distribution/DBASE/
 
  • Install the new RPM on AFS (must be done from lhcb-archive)
    /afs/cern.ch/lhcb/software/lbinstall/afslbinstall install DBASE_Hat_Package

Revision 1022016-11-30 - PatrickSKoppenburg

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Revision 1012016-11-11 - PatrickSKoppenburg

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 176 to 176
 
  • Install the new RPM on AFS (must be done from lhcb-archive)
    /afs/cern.ch/lhcb/software/lbinstall/afslbinstall install DBASE_Hat_Package
Changed:
<
<
>
>
If this is not possible because of a lock file, try figuring out why there is a lock (someone else releasing?) and then
rm /afs/cern.ch/lhcb/software/rpmrel/var/lib/rpm/.dbenv.lock
  Post-processing

Revision 1002016-11-11 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 485 to 485
  mkLCGCMTtarFromRPM LHCbExternals v69r0p1 $CMTCONFIG
Changed:
<
<

Preparing the LHCb Externals RPM

>
>
* As from LCG 86
 
Changed:
<
<
When the LHCbExternals versions have been tagged, it is necessary to prepare the RPMs, using the commands provided by the LbNightlyTools repository:
>
>
We do not extract dependencies any more, the LHCBEXTERNALS project is abandoned.
 
Added:
>
>
To Generate the install project tar files LCGCMT for x86_64-slc6-gcc49-opt, just run (LbScript v8r7 at least is needed)
 
Changed:
<
<
git clone http://git.cern.ch/pub/LbNightlyTools cd LbNightlyTools . ./setup.sh lbn-generate-extspec LHCbExternals v70r0p1 $CMTCONFIG -o lcg70root6.spec rpmbuild -bb lcg70root6.spec

Check the RPM produced and copying it to the LHCb YUM repository:

cp /tmp/rpmbuild/RPMS/noarch/LHCBEXTERNALS_v70r0p1_x86_64_slc6_gcc48_opt-1.0.0-1.noarch.rpm /afs/cern.ch/lhcb/distribution/rpm/lhcb
/afs/cern.ch/lhcb/distribution/rpm/update.sh
>
>
makeTarFromRPM LCGCMT v86r0 x86_64-slc6-gcc49-opt ./LHCBEXTERNALS_86.json
 
Added:
>
>
Where LHCBEXTERNALS_86.json is a file of the form:
{
    "heptools": {
    "version": 86,
    "packages":[
        "AIDA",
        "Boost",
        "eigen",
   "CLHEP",
   "COOL",
   "CORAL",
   "CppUnit",
   "Frontier_Client",
   "GSL",
   "HepMC",
   "HepPDT",
   "Python",
   "QMtest",
   "Qt",
   "RELAX",
   "ROOT",
   "XercesC",
   "fastjet",
   "fftw",
   "graphviz",
   "libunwind",
   "neurobayes_expert",
   "oracle",
   "pyanalysis",
   "pygraphics",
   "pytools",
   "sqlite",
   "tcmalloc",
   "vdt",
   "xqilla",
   "xrootd",
   "tbb",
   "rangev3",
   "cppgsl",
   "ipython"]
 
Changed:
<
<
Then you can install them on AFS:

cd /afs/cern.ch/lhcb/software/rpmrel/
./afslbpkr rpm -- -ivh --nodeps $LHCBTAR/rpm/lhcb/LHCBEXTERNALS_v71r0p1_*.rpm
>
>
} }
 
Deleted:
<
<
See that there is a --nodeps, we don;t want to reinstall the dependencies on AFS.

BEWARE: ALWAYS RUN MKLCGCMTTAR (OR [...]FROMRPM) ON THE CORRECT PLATFORM (or the gcc version may be wrong)

 

Releasing a new version of LCG Grid

Revision 992016-11-10 - PatrickSKoppenburg

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 185 to 185
 For Gen/DecFiles one needs an extra step to update the bookkeeping. It requires to have a valid Grid certificate
  1. Get your Grid proxy:

Changed:
<
<
SetupProject LHCbDirac lhcb-proxy-init
>
>
lb-run LHCbDirac lhcb-proxy-init
 
  1. Update the bookkeeping:
             cd $LHCBRELEASES/DBASE/Gen/DecFiles/<version>/cmt  
Changed:
<
<
dirac-bookkeeping-eventtype-mgt-insert ../doc/table_event.sql dirac-bookkeeping-eventtype-mgt-update ../doc/table_event.sql dirac-bookkeeping-eventtype-mgt-update ../doc/table_obsolete.sql
>
>
lb-run LHCbDirac dirac-bookkeeping-eventtype-mgt-insert ../doc/table_event.sql lb-run LHCbDirac dirac-bookkeeping-eventtype-mgt-update ../doc/table_event.sql lb-run LHCbDirac dirac-bookkeeping-eventtype-mgt-update ../doc/table_obsolete.sql
 

The dirac-bookkeeping-eventtype-mgt-insert command will produce many errors like

Revision 982016-11-08 - PatrickSKoppenburg

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 82 to 82
  To check the following commands will work for you, now run
echo ${MyProject^^}
which should return the project name, in upper case.
Changed:
<
<
    • prepare the AFS volume
>
>
    • prepare the AFS volume
 lb-project-manage-volume -c $MyProject $MyVersion
Changed:
<
<
    • install the new project
>
>
    • install the new project
 lb-deployment-afs-install $MyProject $MyVersion x86_64-slc6-gcc4{8,9}-{opt,dbg} x86_64-slc6-gcc49-do0 x86_64-centos7-gcc49-{opt,dbg}
Changed:
<
<
>
>
 lb-deployment-afs-install $MyProject $MyVersion x86_64-slc6-gcc48-{opt,dbg} if gcc49 / centos7 is not available.
Line: 113 to 107
 But that disables dependency checking and at that point, you are on your own to ensure everything needed is installed.

Changed:
<
<
<!--/twistyPlugin twikiMakeVisibleInline-->
Gaudi requires it's own documentation to be built and installed on AFS before locking the release directory.
>
>
<!--/twistyPlugin twikiMakeVisibleInline-->
Gaudi requires its own documentation to be built and installed on AFS before locking the release directory.
 
  • go to a temporary directory
    cd $TMPDIR
    
Line: 135 to 128
 
<!--/twistyPlugin-->
Added:
>
>
All cases:
 
  • release and lock the AFS volume
    lb-project-manage-volume -r $MyProject $MyVersion
    lb-project-manage-volume -l $MyProject $MyVersion
Line: 147 to 141
 
  • For Gauss, it is necessary to create the Genser tarball
    mkMCGentarFromRPM ${MyProject} ${MyVersion} ${CMTCONFIG}
Changed:
<
<
>
>
but make sure that ${CMTCONFIG} is a platform valid for Gauss. Else use another one.
 
  • update the software database
    lb-deployment-updatesdb $MyProject $MyVersion
Line: 185 to 178
 /afs/cern.ch/lhcb/software/lbinstall/afslbinstall install DBASE_Hat_Package
Changed:
<
<
  • Post-processing
    • Special case: Gen/DecFiles
>
>
Post-processing
 
Changed:
<
<
This one needs an extra step to update the bookkeeping. It requires to have a valid Grid certificate
>
>
<!--/twistyPlugin twikiMakeVisibleInline-->
For Gen/DecFiles one needs an extra step to update the bookkeeping. It requires to have a valid Grid certificate
 
  1. Get your Grid proxy:
              SetupProject LHCbDirac
Line: 213 to 207
 
The following event types are inserted: ['42102201', '42102200', '42163203', '42102221', '42102220', '15104451', '42100210', '42100211', '42100230', '42100231', '42142201', '42142200', '42102210', '42102211', '42162203', '42162202', '15574100', '42100221', '42100220', '12163436', '12163437', '42102601', '42102600', '12163236', '42163202', '12163235', '42112200', '42112201'] 
Added:
>
>
<!--/twistyPlugin-->
  Note that in the commands below, the name <Package> does not contain the hat (i.e. <Package>=SQLDDDB and not Det/SQLDDDB)
Changed:
<
<
    • All cases: Lock the AFS directory
>
>
  • Lock the AFS directory
 After the build of the package release, the directory holding it can be locked. This will remove the write permissions to the directory and this can be achieved with:
          lockAFSDir -R $LHCBRELEASES/DBASE/<Hat>/<Package>/<version>

Revision 972016-10-28 - JoelClosier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 143 to 143
 $LHCBDOC/scripts/addrel.py $MyProject $MyVersion
Deleted:
<
<
  • create the old tarballs .Warning, important This has to be run on the same platform as the one chose in the CMTCONFIG: e.g. generate tar files for slc6 on slc6 hosts. You may need to connect to the appropriate cluster to do so (lbjenkins is running centos7 and NOT appropriate to generate slc6 tars)
    lb-deployment-oldtarballs $MyProject $MyVersion
    
 
  • For Gauss, it is necessary to create the Genser tarball

Revision 962016-10-25 - JoelClosier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 219 to 219
 

Note that in the commands below, the name <Package> does not contain the hat (i.e. <Package>=SQLDDDB and not Det/SQLDDDB)

Deleted:
<
<
    • All cases: Build the tarball
                cd $LHCBRELEASES
                mkLHCbtar -p <Package> -v <version>
         
 
    • All cases: Lock the AFS directory
After the build of the package release, the directory holding it can be locked. This will remove the write permissions to the directory and this can be achieved with:

Revision 952016-10-24 - JoelClosier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 177 to 177
 
    • try to copy the RPMs to the repository
      build_id=1234
      lb-release-rpm /data/archive/artifacts/release/lhcb-release/$build_id
Added:
>
>
lb-release-oldtar /data/archive/artifacts/release/lhcb-release/$build_id/olddist
 
    • if everything is fine, do the real copy and update the repository info
      build_id=1234
      lb-release-rpm --copy /data/archive/artifacts/release/lhcb-release/$build_id
Added:
>
>
lb-release-oldtar --copy /data/archive/artifacts/release/lhcb-release/$build_id/olddist
 

  • Install the new RPM on AFS (must be done from lhcb-archive)

Revision 942016-10-19 - JoelClosier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 58 to 58
 
    • try to copy the RPMs to the repository
      build_id=1234
      lb-release-rpm /data/archive/artifacts/release/lhcb-release/$build_id
Added:
>
>
lb-release-oldtar /data/archive/artifacts/release/lhcb-release/$build_id/olddist
 
    • if everything is fine, do the real copy and update the repository info
      build_id=1234
      lb-release-rpm --copy /data/archive/artifacts/release/lhcb-release/$build_id
Added:
>
>
lb-release-oldtar --copy /data/archive/artifacts/release/lhcb-release/$build_id/olddist
 

BE CAREFUL: Since 20141010, the release procedure automatically bumps up the release number for packages for which there is already a version in the RPM repo.

Revision 932016-10-19 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 33 to 33
 

Special information

Starting from 13 October 2016, you should set your environment to DEV as : ". /afs/cern.ch/lhcb/software/releases/LBSCRIPTS/dev/InstallArea/scripts/LbLogin.sh"
Deleted:
<
<
Warning, important afslbpkr problems (2016/08/12)

We currently have AFS performance problems, with catastrophic consequences on the the releases with RPM (as it does many stats on the installation file system).

This makes the installation procedure very slow, we there found a way to extract the files directly bypassing rpm itself with the downside that:

  • The post install scripts are not invoked
  • the RPM DB is not updated
This procedure can be used in case of emergency but its use should be noted in the ticket (and contact ben.couturier@cernNOSPAMPLEASE.ch to clean up the DB). To force the installation of the RPMs:
/afs/cern.ch/lhcb/software/rpmrel/rpmextractor.py -v ${LHCBTAR}/rpm/lhcb/${MyProject^^}_${MyVersion}[-_]*
 

Standard procedure

Revision 922016-10-17 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 196 to 196
 

  • Install the new RPM on AFS (must be done from lhcb-archive)
Changed:
<
<
/afs/cern.ch/lhcb/software/lbinstall/lbinstall install DBASE_Hat_Package
>
>
/afs/cern.ch/lhcb/software/lbinstall/afslbinstall install DBASE_Hat_Package
 

  • Post-processing

Revision 912016-10-13 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 177 to 177
 Warning, important WARNING The procedure for the release of data packages via RPMs is still under heavy development and the automation is not yet in place. Moreover, the new procedure cover only the checkout and build steps, while special cases, tarballs, locking and cvmfs have do be done using the standard instructions at ProjectReleaseOld.

  • Trigger by hand the lhcb-release Jenkins job:
Changed:
<
<
>
>
 
    • add the list of packages (with their versions) in the field packages_list (e.g. RawEventFormat v1r1 Gen/DecFiles vXrY), the name of the container can be specified with something like "container":"name" (e.g. PARAM:TMVAWeights v1r0). The container can be omitted for DBASE.
Deleted:
<
<
    • make sure you specify a valid build_id, not reusing an existing one
 
    • click on the Build button (leave all the others fields empty)

  • Once the build is completed, check the checkout log

Revision 902016-10-13 - JoelClosier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 31 to 31
 

For Release Shifters

Special information

Added:
>
>
Starting from 13 October 2016, you should set your environment to DEV as : ". /afs/cern.ch/lhcb/software/releases/LBSCRIPTS/dev/InstallArea/scripts/LbLogin.sh"
  Warning, important afslbpkr problems (2016/08/12)

Revision 892016-10-13 - JoelClosier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 199 to 199
 /afs/cern.ch/lhcb/software/lbinstall/lbinstall install DBASE_Hat_Package
Changed:
<
<
>
>
  • Post-processing
    • Special case: Gen/DecFiles

This one needs an extra step to update the bookkeeping. It requires to have a valid Grid certificate

  1. Get your Grid proxy:
              SetupProject LHCbDirac
              lhcb-proxy-init
         
  2. Update the bookkeeping:
             cd $LHCBRELEASES/DBASE/Gen/DecFiles/<version>/cmt  
             dirac-bookkeeping-eventtype-mgt-insert ../doc/table_event.sql
             dirac-bookkeeping-eventtype-mgt-update ../doc/table_event.sql
             dirac-bookkeeping-eventtype-mgt-update ../doc/table_obsolete.sql
         

The dirac-bookkeeping-eventtype-mgt-insert command will produce many errors like

{'EVTTYPEID': '12165431', 'DESCRIPTION': 'Bu_D02pi+pi-,K+pi-pi0=PHSP,DecProdCut', 'PRIMARY': '[B+ -> pi+ pi+ pi- (D0bar -> K+ pi- pi0)]cc'} : Excution failed.: ( ORA-00001: unique constraint (LHCB_DIRACBOOKKEEPING.SYS_C00302542) violated
ORA-06512: at "LHCB_DIRACBOOKKEEPING.BOOKKEEPINGORACLEDB", line 1131
ORA-06512: at line 1
 ) 
This is normal as most event types already exist in the Oracle table. Things are OK if the last line is like
The following event types are inserted: ['42102201', '42102200', '42163203', '42102221', '42102220', '15104451', '42100210', '42100211', '42100230', '42100231', '42142201', '42142200', '42102210', '42102211', '42162203', '42162202', '15574100', '42100221', '42100220', '12163436', '12163437', '42102601', '42102600', '12163236', '42163202', '12163235', '42112200', '42112201'] 

Note that in the commands below, the name <Package> does not contain the hat (i.e. <Package>=SQLDDDB and not Det/SQLDDDB)

    • All cases: Build the tarball
                cd $LHCBRELEASES
                mkLHCbtar -p <Package> -v <version>
         
    • All cases: Lock the AFS directory
After the build of the package release, the directory holding it can be locked. This will remove the write permissions to the directory and this can be achieved with:
          lockAFSDir -R $LHCBRELEASES/DBASE/<Hat>/<Package>/<version>
    • Next?

If this is not a special case, the next thing to do is probably to deploy on CVMFS, instructions here. Or return to the summary section summary.

Deployment on CVMFS of a DBASE package is the same as for a project, but replace <Project> ==> <Package>

 

CVMFS Software Installation

Revision 882016-10-13 - JoelClosier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 176 to 176
 Warning, important WARNING The procedure for the release of data packages via RPMs is still under heavy development and the automation is not yet in place. Moreover, the new procedure cover only the checkout and build steps, while special cases, tarballs, locking and cvmfs have do be done using the standard instructions at ProjectReleaseOld.

  • Trigger by hand the lhcb-release Jenkins job:
Changed:
<
<
>
>
 
    • add the list of packages (with their versions) in the field packages_list (e.g. RawEventFormat v1r1 Gen/DecFiles vXrY), the name of the container can be specified with something like "container":"name" (e.g. PARAM:TMVAWeights v1r0). The container can be omitted for DBASE.
    • make sure you specify a valid build_id, not reusing an existing one
    • click on the Build button (leave all the others fields empty)

Revision 872016-10-11 - JoelClosier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 114 to 114
 /afs/cern.ch/lhcb/software/lbinstall/afslbinstall query "${MyProject^^}_${MyVersion}"
Changed:
<
<
and install them manually with:
>
>
and install them manually for each RPM with:
 
/afs/cern.ch/lhcb/software/lbinstall/afslbinstall   install <package_name>

Revision 862016-10-10 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 60 to 60
 The latest version of lb-deployment-checklist may cache some data to speed up the execution. If you think it's showing wrong messages, try to remove the directory ~/.cache/lb-deployment/.
Deleted:
<
<
Warning, important afslbpkr currently has an issue and does not deal properly with the RPM lock files. If you encounter the message
found dbenv lock (/afs/cern.ch/lhcb/software/rpmrel/var/lib/rpm/.dbenv.lock), cannot continue
First check that nobody else is currently running an installation. You can remove the aforementioned file and continue with deployment if not.

 

Details of the procedure

Line: 102 to 93
  To check the following commands will work for you, now run
echo ${MyProject^^}
which should return the project name, in upper case.
Changed:
<
<
    • prepare the AFS volume
>
>
    • prepare the AFS volume
 lb-project-manage-volume -c $MyProject $MyVersion
Changed:
<
<
    • install the new project
>
>
    • install the new project
 lb-deployment-afs-install $MyProject $MyVersion x86_64-slc6-gcc4{8,9}-{opt,dbg} x86_64-slc6-gcc49-do0 x86_64-centos7-gcc49-{opt,dbg}
Changed:
<
<
or
>
>
 lb-deployment-afs-install $MyProject $MyVersion x86_64-slc6-gcc48-{opt,dbg} if gcc49 / centos7 is not available.
Changed:
<
<
If there are problems about "conflicts", like LHCbDirac, or with the do0 platform, call /afs/cern.ch/lhcb/software/rpmrel/afslbpkr update and try again. Until the problems with Externals installation has been solved, it is possible to run:
>
>
In case of problems, you can find out the list of packages available:
 
Changed:
<
<
/afs/cern.ch/lhcb/software/rpmrel/afslbpkr rpm -- -ivh --nodeps /afs/cern.ch/lhcb/distribution/rpm/lhcb/${MyProject^^}_${MyVersion}*
>
>
/afs/cern.ch/lhcb/software/lbinstall/afslbinstall query "${MyProject^^}_${MyVersion}"
 
Changed:
<
<
or
>
>
and install them manually with:
 
Changed:
<
<
/afs/cern.ch/lhcb/software/rpmrel/afslbpkr install --nodeps ${MyProject^^}_${MyVersion}_${config}
>
>
/afs/cern.ch/lhcb/software/lbinstall/afslbinstall install
 
Line: 198 to 196
 

  • Install the new RPM on AFS (must be done from lhcb-archive)
Changed:
<
<
/afs/cern.ch/lhcb/software/rpmrel/afslbpkr install DBASE_Hat_Package
>
>
/afs/cern.ch/lhcb/software/lbinstall/lbinstall install DBASE_Hat_Package
 

Line: 327 to 325
 
remove_software.sh <Project> <version>
Remember to run cvmfs_publish at the end, as usual, to close the transaction.

Deleted:
<
<
  • if a manual removal from the grid is necessary ( i.e. when there were problems running archive_project, which also initiates the removal from the grid) use dirac-lhcb-remove-software
           SetupProject LHCbDirac  
           lhcb-proxy-init
           dirac-lhcb-remove-software <Project> <version>
 

Data Packages

  • unlock the package directory
    unlockAFSDir -R $LHCBRELEASES/DBASE/<Hat>/<Package>/<version>
Changed:
<
<
  • get the name of the rpm to remove with
    /afs/cern.ch/lhcb/software/rpmrel/afslbpkr rpm -- -qf $LHCBRELEASES/DBASE/<Hat>/<Package>/<version>
  • remove the rpm
    /afs/cern.ch/lhcb/software/rpmrel/afslbpkr rpm -- -ev <rpm_name>
>
>
  • remove the package
    /afs/cern.ch/lhcb/software/lbinstall/lbinstall remove <package_name>
 
  • got to the CVMFS publishing machine (cvmfs-lhcb) and uninstall
    cvmfs_transaction
    remove_software.sh <Package> <version>
    cvmfs_publish
Line: 384 to 376
 
  • If lhcb-archive.cern.ch works, continue with the procedure to install to AFS, otherwise you should install on AFS from the generated tar files, then continue with the normal procedure
Deleted:
<
<

If the RPMs of the base projects do not exist

If we are building a project/version for which the rest of the underlying chain was not installed through RPMs, we can still build and install the project via the RPMs.

The procedure is the same as in the regular case, except that instead of using "lbpkr install" we need to call:

/afs/cern.ch/lhcb/software/rpmrel/afslbpkr rpm -- -ivh --nodeps $LHCBTAR/rpm/lhcb/MYPROJECT_vXrY*.rpm

If lbpkr prints messages like "Fatal error, run database recovery"

It happened once that the RPM database got corrupted.

It was fixed following the instructions at http://www.rpm.org/wiki/Docs/RpmRecovery where we replaced /var/lib with /afs/cern.ch/lhcb/software/rpmrel/var/lib and the command rpm with /afs/cern.ch/lhcb/software/rpmrel/afslbpkr rpm --

If this procedure fails, then ask the admin for a recovery of the DB from the backups, or reconstruction from the list of packages. It is possible to rebuild the RPM DB from the AFS installation area by using the script:

* rebuildRPMdb.py.txt: Script to list commands to recreate RPM DB

 

If lb-release-rpm prints messages like "error: not an rpm package"

It might mean that the RPM file is probably corrupted.
Line: 433 to 405
 
  • remove web page linksTODO
  • uninstall the RPMs
    • check that you know what you want to remove
Changed:
<
<
/afs/cern.ch/lhcb/software/rpmrel/afslbpkr installed ${MyProject^^}_${MyVersion}
>
>
/afs/cern.ch/lhcb/software/lbinstall/lbinstall list ${MyProject^^}_${MyVersion}
 
    • actually remove them
Changed:
<
<
/afs/cern.ch/lhcb/software/rpmrel/afslbpkr rm $(/afs/cern.ch/lhcb/software/rpmrel/afslbpkr installed ${MyProject^^}_${MyVersion})
>
>
/afs/cern.ch/lhcb/software/lbinstall/lbinstal remove $(/afs/cern.ch/lhcb/software/rpmrel/afslbpkr list ${MyProject^^}_${MyVersion})
 
  • remove the AFS volume in the release area
    afs_admin delete $LHCBRELEASES/${MyProject^^}/${MyProject^^}_${MyVersion}

Revision 852016-10-06 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 178 to 178
 Warning, important WARNING The procedure for the release of data packages via RPMs is still under heavy development and the automation is not yet in place. Moreover, the new procedure cover only the checkout and build steps, while special cases, tarballs, locking and cvmfs have do be done using the standard instructions at ProjectReleaseOld.

  • Trigger by hand the lhcb-release Jenkins job:
Changed:
<
<
>
>
 
    • add the list of packages (with their versions) in the field packages_list (e.g. RawEventFormat v1r1 Gen/DecFiles vXrY), the name of the container can be specified with something like "container":"name" (e.g. PARAM:TMVAWeights v1r0). The container can be omitted for DBASE.
Added:
>
>
    • make sure you specify a valid build_id, not reusing an existing one
 
    • click on the Build button (leave all the others fields empty)

  • Once the build is completed, check the checkout log
Changed:
<
<
>
>
    • go to https://lhcb-nightlies.cern.ch/release/
    • click on the small Jenkins icon in the "Project" column of the build summary look for the package build log (after the line "...:INFO... building Hat/Package")
 
  • If the build was fine, publish the RPMs (from lhcb-archive.cern.ch)
    • try to copy the RPMs to the repository

Revision 842016-09-07 - MarcoCattaneo

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 17 to 17
 
  • register the project/version to the software database
    lb-sdb-import MyProject vXrY
    
Changed:
<
<
(If the tag is in git while the project is still by default maintained in SVN you need to tell lb-sdb-import to look in git, e.g.: lb-sdb-import  --sourceuri="gitlab-cern:LHCb-SVN-mirrors/Panoramix#v23r2" Panoramix v23r2)
>
>
(If the tag is in git while the project is still by default maintained in SVN you need to tell lb-sdb-import to look in git, e.g.: lb-sdb-import  --sourceuri="gitlab-cern:LHCb/Panoramix#v23r2" Panoramix v23r2)
  Within 10-15 min the build should start to appear in the nightlies page https://lhcb-nightlies.cern.ch/release. Note that, for a dependency tree of projects, lb-sdb-import can be called just once for the top level project as it will pull in all the dependencies (unless you need to provide the --sourceurl option, in which case each project has to be added individually, in the dependency order).
  • if there is a problem in the build, it can be re-started via the dedicated button (it will not restart by itself after a retag)

Revision 832016-08-12 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 29 to 29
 

For Release Shifters

Added:
>
>

Special information

Warning, important afslbpkr problems (2016/08/12)

We currently have AFS performance problems, with catastrophic consequences on the the releases with RPM (as it does many stats on the installation file system).

This makes the installation procedure very slow, we there found a way to extract the files directly bypassing rpm itself with the downside that:

  • The post install scripts are not invoked
  • the RPM DB is not updated
This procedure can be used in case of emergency but its use should be noted in the ticket (and contact ben.couturier@cernNOSPAMPLEASE.ch to clean up the DB). To force the installation of the RPMs:
/afs/cern.ch/lhcb/software/rpmrel/rpmextractor.py -v ${LHCBTAR}/rpm/lhcb/${MyProject^^}_${MyVersion}[-_]*

Standard procedure

 The script lb-deployment-checklist simplifies the release shifter task by checking what needs to be done and either prints the next command to run or points to this page.

So connect to lxplus (or equivalent SLC6 machine) and run

Line: 47 to 67
 First check that nobody else is currently running an installation. You can remove the aforementioned file and continue with deployment if not.
Changed:
<
<
Warning, important afslbpkr problems. We currently have AFS performance problems, with catastrophic consequences on the the releases with RPM (as it does many stats on the installation file system). This makes the installation procedure very slow, we there found a way to extract the files directly bypassing rpm itself with the downside that: * The post install scripts are not invoked * the RPM DB is not updated This procedure can be used in case of emergency but its use should be noted in the ticket (and contact ben.couturier@cernNOSPAMPLEASE.ch to clean up the DB). To force the installation of the RPMs:
/afs/cern.ch/lhcb/software/rpmrel/rpmextractor.py -v ${LHCBTAR}/rpm/lhcb/${MyProject^^}_${MyVersion}[-_]*
>
>
 

Details of the procedure

Revision 822016-08-11 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 55 to 55
 This procedure can be used in case of emergency but its use should be noted in the ticket (and contact ben.couturier@cernNOSPAMPLEASE.ch to clean up the DB). To force the installation of the RPMs:
Changed:
<
<
/afs/cern.ch/lhcb/software/rpmrel/rpmextractor.py -v ${LHCBTAR}/rpm/lhcb/${MyProject}^^_${Myversion}[-_]*
>
>
/afs/cern.ch/lhcb/software/rpmrel/rpmextractor.py -v ${LHCBTAR}/rpm/lhcb/${MyProject^^}_${MyVersion}[-_]*
 

Details of the procedure

Revision 812016-08-09 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 46 to 46
  First check that nobody else is currently running an installation. You can remove the aforementioned file and continue with deployment if not.
Added:
>
>
Warning, important afslbpkr problems. We currently have AFS performance problems, with catastrophic consequences on the the releases with RPM (as it does many stats on the installation file system). This makes the installation procedure very slow, we there found a way to extract the files directly bypassing rpm itself with the downside that: * The post install scripts are not invoked * the RPM DB is not updated This procedure can be used in case of emergency but its use should be noted in the ticket (and contact ben.couturier@cernNOSPAMPLEASE.ch to clean up the DB). To force the installation of the RPMs:
/afs/cern.ch/lhcb/software/rpmrel/rpmextractor.py -v ${LHCBTAR}/rpm/lhcb/${MyProject}^^_${Myversion}[-_]*
 

Details of the procedure

Revision 802016-08-08 - MarcoCattaneo

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 13 to 13
 

For Release Managers

For Projects:

Changed:
<
<
  • prepare the project as usual (release notes, tags, etc.), following the instructions at PrepareProjectRelease
>
>
 
  • register the project/version to the software database
    lb-sdb-import MyProject vXrY
    
Changed:
<
<
(If the tag is in git while the project is still by default maintained in SVN you need to tell lb-sdb-import to look in git, e.g.: lb-sdb-import  --sourceuri="gitlab-cern:LHCb-SVN-mirrors/LHCb#v40r2" LHCb v40r2)
>
>
(If the tag is in git while the project is still by default maintained in SVN you need to tell lb-sdb-import to look in git, e.g.: lb-sdb-import  --sourceuri="gitlab-cern:LHCb-SVN-mirrors/Panoramix#v23r2" Panoramix v23r2)
  Within 10-15 min the build should start to appear in the nightlies page https://lhcb-nightlies.cern.ch/release. Note that, for a dependency tree of projects, lb-sdb-import can be called just once for the top level project as it will pull in all the dependencies (unless you need to provide the --sourceurl option, in which case each project has to be added individually, in the dependency order).
  • if there is a problem in the build, it can be re-started via the dedicated button (it will not restart by itself after a retag)
Line: 25 to 25
 

For Data Packages:

Changed:
<
<
>
>
 

For Release Shifters

Revision 792016-07-26 - ChristopherRJones

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 83 to 83
 lb-project-manage-volume -c $MyProject $MyVersion
    • install the new project
Changed:
<
<
lb-deployment-afs-install $MyProject $MyVersion x86_64-slc6-gcc4{8,9}-{opt,dbg} x86_64-slc6-gcc49-do0
>
>
lb-deployment-afs-install $MyProject $MyVersion x86_64-slc6-gcc4{8,9}-{opt,dbg} x86_64-slc6-gcc49-do0 x86_64-centos7-gcc49-{opt,dbg}
 or
lb-deployment-afs-install $MyProject $MyVersion x86_64-slc6-gcc48-{opt,dbg}
Changed:
<
<
if gcc49 is not available.
>
>
if gcc49 / centos7 is not available.
  If there are problems about "conflicts", like LHCbDirac, or with the do0 platform, call /afs/cern.ch/lhcb/software/rpmrel/afslbpkr update and try again. Until the problems with Externals installation has been solved, it is possible to run:

Revision 782016-07-11 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 8 to 8
  Warning, important These instructions are a snapshot of the current state of the tools. The procedure will be simplified in the coming weeks.
Changed:
<
<
Warning, important These instructions do not apply for LbScripts, Compat and Ganga. For those, use the old instructions: ProjectReleaseOld
>
>
Warning, important These instructions do not apply to LbScripts. Use LbScriptsRelease instead.
 

For Release Managers

Line: 295 to 295
 

Removing Software (archival)

Projects

Changed:
<
<
For projects refer to ProjectReleaseOld#7_Removing_software_archival
>
>
  • On lxplus or one of the lxbuild machines (AFS and CVMFS access, as well as the afs_admin command are needed) run the command:
    archive_project <Project> <version>
    This removes the software from the AFS release area and from the old Grid distribution area, and archives the tar files to castor. You need a grid proxy, since the first thing the script does is to contact the book-keeping to check that there isn't an active production step using the software to be archived.
You have to hit several times to confirm that you want to remove the tar balls (don't say yes if there was some problem copying to castor)

  • On cvmfs-lhcb, start a transaction in the usual way, then run the command:
    remove_software.sh <Project> <version>
    Remember to run cvmfs_publish at the end, as usual, to close the transaction.

  • if a manual removal from the grid is necessary ( i.e. when there were problems running archive_project, which also initiates the removal from the grid) use dirac-lhcb-remove-software
           SetupProject LHCbDirac  
           lhcb-proxy-init
           dirac-lhcb-remove-software <Project> <version>
 

Data Packages

Line: 312 to 325
 

If the nightly builds do not work

The nightly build system is nothing more than wrappers around simple tools that can be called by any user, so it is possible to prepare the RPMs for the release by hand.
Changed:
<
<
  • Go to a work directory, get the nightly build tools, and prepare the environment, e.g. on lxbuild
    mkdir -p /build/$USER
    cd /build/$USER
    git clone http://git.cern.ch/pub/LbNightlyTools
    cd LbNightlyTools
    . setup.sh
    
>
>
  • Go to an SLC6 machine with AFS (e.g. lxplus or a build machine), to a temporary working directory (e.g. /build/$USER)
 
  • Prepare the configuration file to drive the build
Changed:
<
<
lbn-gen-release-config --cmt MyProject vXrY > lhcb-release.json
>
>
lbn-gen-release-config --cmake MyProject vXrY > lhcb-release.json or
lbn-gen-release-config --cmt  --pack 'Hat/Package vXrY > lhcb-release.json
for data packages
 
  • Checkout the sources and prepare the shared RPM
    lbn-checkout --verbose lhcb-release.json
    lbn-rpm --shared --verbose lhcb-release.json
    
Changed:
<
<
  • For each required platform build, test and prepare the RPM
>
>
  • For each required platform build, test and prepare the RPM (not needed for data packages)
 export CMTCONFIG=... lbn-build --verbose --clean --with-tests -j 4 lhcb-release.json lbn-rpm --verbose lhcb-release.json
Changed:
<
<
  • Check if the builds/tests are ok
>
>
  • Check if the builds/tests are ok (not needed for data packages)
 
    • start the Python mini web server
      cd artifacts && python -m SimpleHTTPServer
      
Changed:
<
<
>
>
 
      • build log: summaries.<CMTCONFIG>/<Project>/build_log.html
      • test reports: summaries.<CMTCONFIG>/<Project>/html
Changed:
<
<
  • Publish the RPMs
>
>
  • Publish the RPMs (from lhcb-archive.cern.ch)
 cp -v -n artifacts/MYPROJECT*.rpm $LHCBTAR/rpm/lhcb
Added:
>
>
then, from lhcb-archive.cern.ch
 createrepo --workers=20 --update $LHCBTAR/rpm/lhcb
Changed:
<
<
  • Continue with the procedure to install to AFS
>
>
  • If lhcb-archive.cern.ch works, continue with the procedure to install to AFS, otherwise you should install on AFS from the generated tar files, then continue with the normal procedure
 

If the RPMs of the base projects do not exist

If we are building a project/version for which the rest of the underlying chain was not installed through RPMs, we can still build and install the project via the RPMs.

Revision 772016-07-04 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 299 to 299
 

Data Packages

Changed:
<
<
  • unlock the package directory
    lockAFSDir -R $LHCBRELEASES/DBASE/<Hat>/<Package>/<version>
>
>
  • unlock the package directory
    unlockAFSDir -R $LHCBRELEASES/DBASE/<Hat>/<Package>/<version>
 
  • get the name of the rpm to remove with
    /afs/cern.ch/lhcb/software/rpmrel/afslbpkr rpm -- -qf $LHCBRELEASES/DBASE/<Hat>/<Package>/<version>
  • remove the rpm
    /afs/cern.ch/lhcb/software/rpmrel/afslbpkr rpm -- -ev <rpm_name>
  • got to the CVMFS publishing machine (cvmfs-lhcb) and uninstall
    cvmfs_transaction

Revision 762016-06-28 - ChristopherRJones

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 40 to 40
 The latest version of lb-deployment-checklist may cache some data to speed up the execution. If you think it's showing wrong messages, try to remove the directory ~/.cache/lb-deployment/.
Changed:
<
<
Warning, important afslbpkr currently has an issue and does not deal properly with the RPM lock files. I tou encounter the message
>
>
Warning, important afslbpkr currently has an issue and does not deal properly with the RPM lock files. If you encounter the message
 
found dbenv lock (/afs/cern.ch/lhcb/software/rpmrel/var/lib/rpm/.dbenv.lock), cannot continue

Revision 752016-06-04 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 39 to 39
  The latest version of lb-deployment-checklist may cache some data to speed up the execution. If you think it's showing wrong messages, try to remove the directory ~/.cache/lb-deployment/.
Added:
>
>
Warning, important afslbpkr currently has an issue and does not deal properly with the RPM lock files. I tou encounter the message
found dbenv lock (/afs/cern.ch/lhcb/software/rpmrel/var/lib/rpm/.dbenv.lock), cannot continue
First check that nobody else is currently running an installation. You can remove the aforementioned file and continue with deployment if not.
 

Details of the procedure

Revision 742016-06-03 - MarcoCattaneo

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 16 to 16
 
  • prepare the project as usual (release notes, tags, etc.), following the instructions at PrepareProjectRelease
  • register the project/version to the software database
    lb-sdb-import MyProject vXrY
Changed:
<
<
within 10-15 min the build should start to appear in the nightlies page https://lhcb-nightlies.cern.ch/release. Note that, for a dependency tree of projects, lb-sdb-import can be called just once for the top level project as it will pull in all the dependencies.
>
>
(If the tag is in git while the project is still by default maintained in SVN you need to tell lb-sdb-import to look in git, e.g.: lb-sdb-import  --sourceuri="gitlab-cern:LHCb-SVN-mirrors/LHCb#v40r2" LHCb v40r2)

Within 10-15 min the build should start to appear in the nightlies page https://lhcb-nightlies.cern.ch/release. Note that, for a dependency tree of projects, lb-sdb-import can be called just once for the top level project as it will pull in all the dependencies (unless you need to provide the --sourceurl option, in which case each project has to be added individually, in the dependency order).

 
  • if there is a problem in the build, it can be re-started via the dedicated button (it will not restart by itself after a retag)
  • once satisfied by the build, take note of the build id (you can use the direct link icon) and make the request via LHCb Deployment Request form

Revision 732016-05-26 - ChristopherRJones

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 122 to 122
 
  • prepare the web pages links
    $LHCBDOC/scripts/addrel.py $MyProject $MyVersion
    
Added:
>
>
 
Changed:
<
<
  • create the old tarballs
Warning, important This has to be run on the same platform as the one chose in the CMTCONFIG: e.g. generate tar files for slc6 on slc6 hosts. You may need to connect to the appropriate cluster to do so (lbjenkins is running centos7 and NOT appropriate to generate slc6 tars)
>
>
  • create the old tarballs .Warning, important This has to be run on the same platform as the one chose in the CMTCONFIG: e.g. generate tar files for slc6 on slc6 hosts. You may need to connect to the appropriate cluster to do so (lbjenkins is running centos7 and NOT appropriate to generate slc6 tars)
 lb-deployment-oldtarballs $MyProject $MyVersion

Revision 722016-05-03 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 15 to 15
 For Projects:
  • prepare the project as usual (release notes, tags, etc.), following the instructions at PrepareProjectRelease
  • register the project/version to the software database
Changed:
<
<
lb-sdb-importsvn MyProject vXrY within 10-15 min the build should start to appear in the nightlies page https://lhcb-nightlies.cern.ch/release. Note that, for a dependency tree of projects, lb-sdb-importsvn can be called just once for the top level project as it will pull in all the dependencies.
>
>
lb-sdb-import MyProject vXrY within 10-15 min the build should start to appear in the nightlies page https://lhcb-nightlies.cern.ch/release. Note that, for a dependency tree of projects, lb-sdb-import can be called just once for the top level project as it will pull in all the dependencies.
 
  • if there is a problem in the build, it can be re-started via the dedicated button (it will not restart by itself after a retag)
  • once satisfied by the build, take note of the build id (you can use the direct link icon) and make the request via LHCb Deployment Request form

Revision 712016-04-26 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 354 to 354
  It was fixed following the instructions at http://www.rpm.org/wiki/Docs/RpmRecovery where we replaced /var/lib with /afs/cern.ch/lhcb/software/rpmrel/var/lib and the command rpm with /afs/cern.ch/lhcb/software/rpmrel/afslbpkr rpm --
Added:
>
>
If this procedure fails, then ask the admin for a recovery of the DB from the backups, or reconstruction from the list of packages. It is possible to rebuild the RPM DB from the AFS installation area by using the script:

* rebuildRPMdb.py.txt: Script to list commands to recreate RPM DB

 

If lb-release-rpm prints messages like "error: not an rpm package"

It might mean that the RPM file is probably corrupted.
Line: 591 to 598
  -- MarcoClemencic - 30 Jun 2014
Added:
>
>

META FILEATTACHMENT attachment="rebuildRPMdb.py.txt" attr="" comment="Script to list commands to recreate RPM DB" date="1461683382" name="rebuildRPMdb.py.txt" path="rebuildRPMdb.py.txt" size="4322" user="bcouturi" version="1"
 
META TOPICMOVED by="cattanem" date="1412348243" from="LHCb.ProjectReleaseNew" to="LHCb.ProjectRelease"

Revision 702016-03-15 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 99 to 99
 
  • go to a temporary directory
    cd $TMPDIR
    
Changed:
<
<
  • install a local copy of the build of Gaudi (using LbNightlyTools)
    lbn-install --project Gaudi --platform $CMTCONFIG --flavour release lhcb-release $build_id
>
>
  • install a local copy of the sources of Gaudi (using LbNightlyTools)
    lbn-install --project Gaudi --platform src --flavour release lhcb-release $build_id
 
Changed:
<
<
  • build the Doxygen documentation (requires Doxygen 1.8.7)
>
>
  • build the project and the Doxygen documentation (requires Doxygen 1.8.7)
 cd $TMPDIR/lhcb-release/$build_id/GAUDI/GAUDI_$MyVersion
Changed:
<
<
export USE_CMAKE=1 make doc
>
>
make && make doc
 
  • install on AFS the generated documentation
    mkdir -p $GAUDISOFT/GAUDI/GAUDI_$MyVersion/InstallArea/doc

Revision 692016-02-29 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 78 to 78
 lb-deployment-afs-install $MyProject $MyVersion x86_64-slc6-gcc48-{opt,dbg} if gcc49 is not available.
Changed:
<
<
If there are problems about "conflicts", like LHCbDirac, call /afs/cern.ch/lhcb/software/rpmrel/afslbpkr update and try again.
>
>
If there are problems about "conflicts", like LHCbDirac, or with the do0 platform, call /afs/cern.ch/lhcb/software/rpmrel/afslbpkr update and try again.
 Until the problems with Externals installation has been solved, it is possible to run:
/afs/cern.ch/lhcb/software/rpmrel/afslbpkr rpm -- -ivh --nodeps /afs/cern.ch/lhcb/distribution/rpm/lhcb/${MyProject^^}_${MyVersion}*
Added:
>
>
or
/afs/cern.ch/lhcb/software/rpmrel/afslbpkr install --nodeps ${MyProject^^}_${MyVersion}_${config}
 But that disables dependency checking and at that point, you are on your own to ensure everything needed is installed.

Revision 682016-02-24 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 117 to 117
 $LHCBDOC/scripts/addrel.py $MyProject $MyVersion
Changed:
<
<
  • create the old tarballs
>
>
  • create the old tarballs
Warning, important This has to be run on the same platform as the one chose in the CMTCONFIG: e.g. generate tar files for slc6 on slc6 hosts. You may need to connect to the appropriate cluster to do so (lbjenkins is running centos7 and NOT appropriate to generate slc6 tars)
 lb-deployment-oldtarballs $MyProject $MyVersion

Revision 672016-02-22 - JoelClosier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 73 to 73
 lb-project-manage-volume -c $MyProject $MyVersion
    • install the new project
Changed:
<
<
lb-deployment-afs-install $MyProject $MyVersion x86_64-slc6-gcc4{8,9}-{opt,dbg} x86_64-slc6-gcc48-do0
>
>
lb-deployment-afs-install $MyProject $MyVersion x86_64-slc6-gcc4{8,9}-{opt,dbg} x86_64-slc6-gcc49-do0
 or
lb-deployment-afs-install $MyProject $MyVersion x86_64-slc6-gcc48-{opt,dbg}
if gcc49 is not available.

Revision 662016-02-11 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 157 to 157
 lb-release-rpm --copy /data/archive/artifacts/release/lhcb-release/$build_id
Changed:
<
<
  • Install the new RPM on AFS (N.B. must be done from and SLC6 machine - i.e. not lhcb-archive which runs CentOS7)
>
>
  • Install the new RPM on AFS (must be done from lhcb-archive)
 /afs/cern.ch/lhcb/software/rpmrel/afslbpkr install DBASE_Hat_Package

Revision 652016-02-09 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 40 to 40
 
Changed:
<
<
  • publish the produced RPMs (from lhcb-archive.cern.ch)
>
>
Warning, important This step has to be performed from the host lhcb-archive.cern.ch

  • publish the produced RPMs
 
    • try to copy the RPMs to the repository
      build_id=1234
      lb-release-rpm /data/archive/artifacts/release/lhcb-release/$build_id
Line: 55 to 58
 In that case the --rpm-regex option of lb-release-rpm should be used.

Changed:
<
<
  • install on AFS (N.B. must be done from and SLC6 machine - i.e. not lhcb-archive which runs CentOS7)
>
>
Warning, important This step has to be performed from the host lhcb-archive.cern.ch

  • install on AFS
 
    • First, set some environment variables that define the project name and version to be installed. Used in the following commands (so you can simply cut and paste them directly). Works for bash shells, so recommend switching to this if it isn't your default.
      MyProject=<ProjectName>

Revision 642016-01-25 - MarcoCattaneo

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 55 to 55
 In that case the --rpm-regex option of lb-release-rpm should be used.

Changed:
<
<
  • install on AFS
>
>
  • install on AFS (N.B. must be done from and SLC6 machine - i.e. not lhcb-archive which runs CentOS7)
 
    • First, set some environment variables that define the project name and version to be installed. Used in the following commands (so you can simply cut and paste them directly). Works for bash shells, so recommend switching to this if it isn't your default.
      MyProject=<ProjectName>
Line: 151 to 151
 lb-release-rpm --copy /data/archive/artifacts/release/lhcb-release/$build_id
Changed:
<
<
  • Install the new RPM on AFS
>
>
  • Install the new RPM on AFS (N.B. must be done from and SLC6 machine - i.e. not lhcb-archive which runs CentOS7)
 /afs/cern.ch/lhcb/software/rpmrel/afslbpkr install DBASE_Hat_Package

Revision 632015-12-04 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 270 to 270
  A backup of the CRONTAB is available in ${HOME}/conf.crontab.backup
Added:
>
>

Removing Software (archival)

Projects

For projects refer to ProjectReleaseOld#7_Removing_software_archival

Data Packages

  • unlock the package directory
    lockAFSDir -R $LHCBRELEASES/DBASE/<Hat>/<Package>/<version>
  • get the name of the rpm to remove with
    /afs/cern.ch/lhcb/software/rpmrel/afslbpkr rpm -- -qf $LHCBRELEASES/DBASE/<Hat>/<Package>/<version>
  • remove the rpm
    /afs/cern.ch/lhcb/software/rpmrel/afslbpkr rpm -- -ev <rpm_name>
  • got to the CVMFS publishing machine (cvmfs-lhcb) and uninstall
    cvmfs_transaction
    remove_software.sh <Package> <version>
    cvmfs_publish
    
 

Troubleshooting (AKA For Librarians)

Revision 622015-11-26 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 28 to 28
 

For Release Shifters

The script lb-deployment-checklist simplifies the release shifter task by checking what needs to be done and either prints the next command to run or points to this page.
Changed:
<
<
So connect to lhcb-archive.cern.ch (or lxplus) and run
>
>
So connect to lxplus (or equivalent SLC6 machine) and run
 
lb-deployment-checklist MyProject vXrY
and follow the instructions.
Deleted:
<
<
Warning, important the commands involving afslbpkr must be run on lxplus. The only command that should be run on lhcb-archive.cern.ch is lb-release-rpm.
 The latest version of lb-deployment-checklist may cache some data to speed up the execution. If you think it's showing wrong messages, try to remove the directory ~/.cache/lb-deployment/.

Details of the procedure

Revision 612015-11-25 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 34 to 34
  and follow the instructions.
Added:
>
>
Warning, important the commands involving afslbpkr must be run on lxplus. The only command that should be run on lhcb-archive.cern.ch is lb-release-rpm.
 The latest version of lb-deployment-checklist may cache some data to speed up the execution. If you think it's showing wrong messages, try to remove the directory ~/.cache/lb-deployment/.

Details of the procedure

Revision 602015-11-06 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 28 to 28
 

For Release Shifters

The script lb-deployment-checklist simplifies the release shifter task by checking what needs to be done and either prints the next command to run or points to this page.
Changed:
<
<
So connect to buildlhcb.cern.ch (or lxplus) and run
>
>
So connect to lhcb-archive.cern.ch (or lxplus) and run
 
lb-deployment-checklist MyProject vXrY
Line: 40 to 40
 
Changed:
<
<
  • publish the produced RPMs (from buildlhcb.cern.ch)
>
>
  • publish the produced RPMs (from lhcb-archive.cern.ch)
 
    • try to copy the RPMs to the repository
      build_id=1234
Changed:
<
<
lb-release-rpm /data/artifacts/release/lhcb-release/$build_id
>
>
lb-release-rpm /data/archive/artifacts/release/lhcb-release/$build_id
 
    • if everything is fine, do the real copy and update the repository info
      build_id=1234
Changed:
<
<
lb-release-rpm --copy /data/artifacts/release/lhcb-release/$build_id
>
>
lb-release-rpm --copy /data/archive/artifacts/release/lhcb-release/$build_id
 

BE CAREFUL: Since 20141010, the release procedure automatically bumps up the release number for packages for which there is already a version in the RPM repo.

Line: 132 to 132
 Warning, important WARNING The procedure for the release of data packages via RPMs is still under heavy development and the automation is not yet in place. Moreover, the new procedure cover only the checkout and build steps, while special cases, tarballs, locking and cvmfs have do be done using the standard instructions at ProjectReleaseOld.

  • Trigger by hand the lhcb-release Jenkins job:
Changed:
<
<
>
>
 
    • add the list of packages (with their versions) in the field packages_list (e.g. RawEventFormat v1r1 Gen/DecFiles vXrY), the name of the container can be specified with something like "container":"name" (e.g. PARAM:TMVAWeights v1r0). The container can be omitted for DBASE.
    • click on the Build button (leave all the others fields empty)

  • Once the build is completed, check the checkout log
Changed:
<
<
>
>
 
    • click on the latest lhcb-release build in the left box (Build History)
    • click on the "Console Output" link and look for the package build log (after the line "...:INFO... building Hat/Package")
Changed:
<
<
  • If the build was fine, publish the RPMs (from buildlhcb.cern.ch)
>
>
  • If the build was fine, publish the RPMs (from lhcb-archive.cern.ch)
 
    • try to copy the RPMs to the repository
      build_id=1234
Changed:
<
<
lb-release-rpm /data/artifacts/release/lhcb-release/$build_id
>
>
lb-release-rpm /data/archive/artifacts/release/lhcb-release/$build_id
 
    • if everything is fine, do the real copy and update the repository info
      build_id=1234
Changed:
<
<
lb-release-rpm --copy /data/artifacts/release/lhcb-release/$build_id
>
>
lb-release-rpm --copy /data/archive/artifacts/release/lhcb-release/$build_id
 

  • Install the new RPM on AFS

Revision 592015-11-02 - JoelClosier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 72 to 72
 lb-deployment-afs-install $MyProject $MyVersion x86_64-slc6-gcc48-{opt,dbg} if gcc49 is not available.
Changed:
<
<
If there are problems about "conflicts", call /afs/cern.ch/lhcb/software/rpmrel/afslbpkr update and try again.
>
>
If there are problems about "conflicts", like LHCbDirac, call /afs/cern.ch/lhcb/software/rpmrel/afslbpkr update and try again.
 Until the problems with Externals installation has been solved, it is possible to run:
/afs/cern.ch/lhcb/software/rpmrel/afslbpkr rpm -- -ivh --nodeps /afs/cern.ch/lhcb/distribution/rpm/lhcb/${MyProject^^}_${MyVersion}*

Revision 582015-10-20 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 16 to 16
 
  • prepare the project as usual (release notes, tags, etc.), following the instructions at PrepareProjectRelease
  • register the project/version to the software database
    lb-sdb-importsvn MyProject vXrY
Changed:
<
<
within 10-15 min the build should start to appear in the nightlies page https://buildlhcb.cern.ch/nightlies-release/. Note that, for a dependency tree of projects, lb-sdb-importsvn can be called just once for the top level project as it will pull in all the dependencies.
>
>
within 10-15 min the build should start to appear in the nightlies page https://lhcb-nightlies.cern.ch/release. Note that, for a dependency tree of projects, lb-sdb-importsvn can be called just once for the top level project as it will pull in all the dependencies.
 
  • if there is a problem in the build, it can be re-started via the dedicated button (it will not restart by itself after a retag)
  • once satisfied by the build, take note of the build id (you can use the direct link icon) and make the request via LHCb Deployment Request form
Line: 37 to 37
 The latest version of lb-deployment-checklist may cache some data to speed up the execution. If you think it's showing wrong messages, try to remove the directory ~/.cache/lb-deployment/.

Details of the procedure

Changed:
<
<
>
>
 
  • if there are problems, ask the project manager to fix them, and restart the build
  • publish the produced RPMs (from buildlhcb.cern.ch)
Line: 339 to 339
 done
Changed:
<
<
If everything looks correct, try to rebuild the project via the rebuild button in the dashboard.
>
>
If everything looks correct, try to rebuild the project via the rebuild button in the dashboard.
 

If everything has gone so wrong that you you want to sit in a corner and cry... or restart from scratch

Revision 572015-10-12 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 87 to 87
 cd $TMPDIR
Changed:
<
<
lbn-install --project Gaudi --platform $CMTCONFIG --artifacts-root https://buildlhcb.cern.ch/artifacts/release lhcb-release $build_id
>
>
lbn-install --project Gaudi --platform $CMTCONFIG --flavour release lhcb-release $build_id
 
  • build the Doxygen documentation (requires Doxygen 1.8.7)
    cd $TMPDIR/lhcb-release/$build_id/GAUDI/GAUDI_$MyVersion

Revision 562015-08-31 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 126 to 126
  (you may get weird error messages, in that case just try again).
Changed:
<
<
  • install the software on CVMFS from the tarballs as usual. For the time being the instructions for CVMFS are available on the Old Release twiki.
>
>
  • install the software on CVMFS from the tarballs as usual. See below for instructions
 

Data packages

Warning, important WARNING The procedure for the release of data packages via RPMs is still under heavy development and the automation is not yet in place. Moreover, the new procedure cover only the checkout and build steps, while special cases, tarballs, locking and cvmfs have do be done using the standard instructions at ProjectReleaseOld.
Line: 157 to 157
 
Added:
>
>

CVMFS Software Installation

Connection to "cvlhcb" account on host "cvmfs-lhcb"

All installation of software is done on host "cvmfs-lhcb", under the account called "cvlhcb".

So first connect to "cvmfs-lhcb":

ssh yourusername@cvmfs-lhcb

Then check that nobody else is connected to install software:

$ who -q ; pgrep -fl -u cvlhcb

If someone else is connected and if there are processes running as cvlhcb, perhaps try sending them an email to see what they are doing before proceeding. If nobody else is connected, sudo into the all powerful account...

$ sudo -i -u cvlhcb

You need to start a transaction on the server to be able to write to the disk.

$ cvmfs_transaction

If the command returns that a transaction was already created, somebody else is installing at the same time. Check with the who command before going any further.

N.B.1 cvmfs_transaction is a wrapper around the CVMFS command that creates the transaction on the repository lhcb.cern.ch, and logs the command and result. It is best to use it instead of using the direct command cvmfs_server transaction lhcb.cern.ch

N.B.2 Even if you decide to use cvmfs_server instead of the wrapper, it is best to specify the repository in the command as this server could potentially host several repositories in the future.

Install using standard script

Data packages and Projects are installed on CVMFS roughly the same way (except for the substitution <Project> ==> <Package> for Data packages).

For stacks >= LHCb v35r4 (with Gaudi v23r6), the debug version will be installed on CVMFS. The software should be installed the scripts from the ${HOME}/bin directory.

At this stage you may need to determine which stack a software release is based on. In this case in a different window on any lxplus machine run:

<!-- SyntaxHighlightingPlugin -->
svnProjectDeps -P <Project> <version> 
<!-- end SyntaxHighlightingPlugin -->

Most recent stack, and only default platforms (or Data package)

If the stack is the most recent, and only default platforms are required

<!-- SyntaxHighlightingPlugin -->
install_software.sh <Project> <version> 
<!-- end SyntaxHighlightingPlugin -->

Older stacks, with no debug option

For stacks < LHCb v35r4 (before Gaudi v23r6), no debug platforms were put on cvmfs. This instruction is now a little obsolete:
<!-- SyntaxHighlightingPlugin -->
install_software.sh <Project> <version> --without-dbg 
<!-- end SyntaxHighlightingPlugin -->

Not default platforms, some older stack

For older stacks, or for installing non-default platforms, you should install them one at-a-time, with the following command:

<!-- SyntaxHighlightingPlugin -->
install_software_direct.sh --binary=<cmtconfig> <Project> <version> 
<!-- end SyntaxHighlightingPlugin -->

Check and Update

Check: Sometimes what seems to be a warning is actually a fatal error. You can then check it all went fine with

<!-- SyntaxHighlightingPlugin -->
ls -l /cvmfs/lhcb.cern.ch/lib/lhcb/<PROJECT>/<PROJECT>_<version>/InstallArea/
<!-- end SyntaxHighlightingPlugin -->
The platforms you have installed should be there, the default platforms are currently only slc6-gcc48, opt and dbg. (the directory to check is different for a Data package, but should be easy to guess.)

Update: Once you have done all the installs, you have to update the global catalog (needs to be done just once at the end, to publish everything that you have installed)

<!-- SyntaxHighlightingPlugin -->
cvmfs_publish
<!-- end SyntaxHighlightingPlugin -->

N.B.1 cvmfs_publish is a wrapper around the CVMFS command that publishes the transaction on the repository lhcb.cern.ch, and logs the command and result. It is best to use it instead of using the direct command cvmfs_server publish lhcb.cern.ch

N.B.2 Even if you decide to use cvmfs_server instead of the wrapper, it is best to specify the repository in the command as this server could potentially host several repositories

Exit: When done, do not forget to exit the sudo session so that others can proceed with other installations if necessary

Note: the various install_software and cvmfs_publish commands can be chained by separating them with ;

CVMFS Appendix

IT CVMFS Information

All information from IT Department can be found at https://twiki.cern.ch/twiki/bin/view/CvmFS/Installers

LHCB Software on cvmfs-lhcb

Scripts

All updates to software deployed on cvmfs should be done under account "cvlhcb". "${HOME}/bin" contains the necessary scripts to add/remove software:

  • cvmfs_transaction: To start a transaction to deploy software on the server

  • install_software.sh: To call install_project to add new projects/versions

  • remove_software.sh: To Remove software versions

  • cvmfs_publish: Synchronize the CVMFS file system

Some other scripts are also available (same name as above but prefixed with std_ perform the same action without logging)

The output of those command is automatically logged to "${HOME}/logs/install.log

CRON

The local CRON is running on this host to automatically update SQLDDDB every hour. The logs for this cron is in: ${HOME}/logs/update_sqldddb.log

A backup of the CRONTAB is available in ${HOME}/conf.crontab.backup

 

Troubleshooting (AKA For Librarians)

If the nightly builds do not work

Revision 552015-08-26 - JoelClosier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 347 to 347
 The code necessary is in the LCGRPM repo. It can be used to create the spec.
Changed:
<
<
git checkout https://:@git.cern.ch/kerberos/LCGRPM
>
>
git clone https://:@git.cern.ch/kerberos/LCGRPM
 cd LCGRPM/package ./createExternalRPMSpec.py -s Grid -o ext.spec voms 2.0.12-3 x86_64-slc6-gcc48-opt rpmbuild -bb ext.spec

Revision 542015-08-25 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 338 to 338
 

Releasing a new version of LCG Grid

Changed:
<
<
Before LHCbDIrac v8:
>
>
After LHCbDirac v8r0 (before then mkLCGCMTTar was necessary)
 
Changed:
<
<
mkLCGCMTtarFromRPM LHCbDirac vXrY $CMTCONFIG
>
>
mkLCGCMTtarFromRPM LHCbDirac vXrY $CMTCONFIG --nonatives
 
Changed:
<
<
After LHCbDirac v8r0:
>
>
If some RPMs are missing from the middleware, it is possible to create them and to add them to the LHCb externals repository in $LHCBTAR/rpm/lcg The code necessary is in the LCGRPM repo. It can be used to create the spec.
 
Changed:
<
<
mkLCGCMTtarFromRPM LHCbDirac vXrY $CMTCONFIG --nonatives
>
>
git checkout https://:@git.cern.ch/kerberos/LCGRPM cd LCGRPM/package ./createExternalRPMSpec.py -s Grid -o ext.spec voms 2.0.12-3 x86_64-slc6-gcc48-opt rpmbuild -bb ext.spec

After that step, the file should be copied to the RPM repo and the RPM repo should be reindexed:

cp /tmp/rpmbuild/RPMS/noarch/voms_2.0.12_3_x86_64_slc6_gcc48_opt-1.0.0-1.noarch.rpm $LHCBTAR/rpm/lcg
 createrepo --workers=20 --update $LHCBTAR/rpm/lcg
 
Added:
>
>
N.B. /tmp/rpmbuild is a default of createExternalRPMSpec, and the -b option can be used to put the files to another directory e.g.
./createExternalRPMSpec.py -b /tmp/lben/toto -s Grid -o ext.spec voms 2.0.12-3 x86_64-slc6-gcc48-opt && rpmbuild -bb ext.spec

Creates the RPM: /tmp/lben/toto/rpmbuild/RPMS/noarch/voms_2.0.12_3_x86_64_slc6_gcc48_opt-1.0.0-1.noarch.rpm

 

Preparing the meta RPM with the software to be installed locally on the Online farm

First get a local version of LbNightlyTools:

Revision 532015-08-25 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 8 to 8
  Warning, important These instructions are a snapshot of the current state of the tools. The procedure will be simplified in the coming weeks.
Changed:
<
<
Warning, important These instructions do not apply for LbScripts, Compat, *Dirac, Ganga and data packages. For those, use the old instructions: ProjectReleaseOld
>
>
Warning, important These instructions do not apply for LbScripts, Compat and Ganga. For those, use the old instructions: ProjectReleaseOld
 

For Release Managers

Revision 522015-07-23 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 227 to 227
  If everything looks correct, try to rebuild the project via the rebuild button in the dashboard.
Added:
>
>

If everything has gone so wrong that you you want to sit in a corner and cry... or restart from scratch

THIS SHOULD NOT BE DONE, but I document it in any case.

To undo a release, so that it can be replayed from the beginning you should (not that it is a somehow reverse order to what done in the normal procedure):

  • uninstall from CVMFS... you shouldn't have gone that far... TODO
  • revert the software database (should not be needed) TODO
  • remove the old-style tarballs and html files
    rm -i $LHCBTAR/html/${MyProject^^}_${MyProject^^}_${MyVersion}*
    rm -i $LHCBTAR/${MyProject^^}/${MyProject^^}_${MyProject^^}_${MyVersion}*
    
  • remove web page linksTODO
  • uninstall the RPMs
    • check that you know what you want to remove
      /afs/cern.ch/lhcb/software/rpmrel/afslbpkr installed ${MyProject^^}_${MyVersion}
      
    • actually remove them
      /afs/cern.ch/lhcb/software/rpmrel/afslbpkr rm $(/afs/cern.ch/lhcb/software/rpmrel/afslbpkr installed ${MyProject^^}_${MyVersion})
      
  • remove the AFS volume in the release area
    afs_admin delete $LHCBRELEASES/${MyProject^^}/${MyProject^^}_${MyVersion}
    
  • remove the RPMs from the repository
    rm -i $LHCBTAR/rpm/lhcb/${MyProject^^}_${MyVersion}*.rpm
    createrepo --update --workers=20 $LHCBTAR/rpm/lhcb
    
 

For the Librarian

Releasing a new version of LCG externals

Revision 512015-07-23 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 351 to 351
 
lb-sdb-tag -p x86_64-slc6-gcc48-opt MooreOnline v23r7p15 ONLINE
lb-sdb-query listTag ONLINE
Changed:
<
<
>
>
  Removal can be done with
lb-sdb-tag -r -p x86_64-slc6-gcc48-opt MooreOnline v23r7pq5 ONLINE
lb-sdb-query listTag ONLINE
Changed:
<
<
>
>
  Then produce the RPM :

Revision 502015-07-22 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 332 to 332
 . setup.sh
Changed:
<
<
Now prepare the Meta RPM spec:
>
>
Now prepare the Meta RPM spec. There are two ways:

  • By specifying explicitly the packages to be included:
 
lbn-generate-metaspec -o ofm.spec OnlineFarmMeta 1.1.1 MOOREONLINE_v23r6_x86_64_slc6_gcc48_opt [...]
Line: 342 to 344
 rpmbuild -bb ofm.spec
Added:
>
>
  • Using the ONLINE tag in the software configuration DB:

First add the project/version/combination to the soft configuration DB and check that the update worked:

lb-sdb-tag -p x86_64-slc6-gcc48-opt MooreOnline v23r7p15 ONLINE
lb-sdb-query listTag ONLINE
<verbatim>

Removal can be done with 
<verbatim>
lb-sdb-tag -r -p x86_64-slc6-gcc48-opt MooreOnline v23r7pq5 ONLINE
lb-sdb-query listTag ONLINE
<verbatim>

 Then produce the RPM : 
<verbatim>
. lb-sdb-env.sh
lbn-generate-metaspec -o ofm.spec OnlineFarmMeta 1.1.1 -t ONLINE
</verbatim>
(You need to pass all necessary applications RPMson the command lines, dependencies are dealt with automatically)
Build the RPM:
<verbatim>
rpmbuild -bb ofm.spec
</verbatim>

</verbatim>
<nop>
 Now copying it to the RPM repository:
lb-release-rpm /tmp/rpmbuild/RPMS/noarch/
lb-release-rpm --copy /tmp/rpmbuild/RPMS/noarch/
Added:
>
>
Now install on plus:
cd /group/hltsw
./onlinelbpkr install OnlineFarmMeta
 

Releases of the nightly builds on /cvmfs/lhcbdev.cern.ch

The releases should be completely automated, but here are the details needed to pcheck the system, perform manual installations/add slots etc...

Revision 492015-07-22 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 34 to 34
  and follow the instructions.
Added:
>
>
The latest version of lb-deployment-checklist may cache some data to speed up the execution. If you think it's showing wrong messages, try to remove the directory ~/.cache/lb-deployment/.
 

Details of the procedure

Line: 211 to 213
 It was fixed following the instructions at http://www.rpm.org/wiki/Docs/RpmRecovery where we replaced /var/lib with /afs/cern.ch/lhcb/software/rpmrel/var/lib and the command rpm with /afs/cern.ch/lhcb/software/rpmrel/afslbpkr rpm --
Added:
>
>

If lb-release-rpm prints messages like "error: not an rpm package"

It might mean that the RPM file is probably corrupted.

This could happen during the copy, in which case it is enough to remove the corrupted files from $LHCBTAR/rpm/lhcb and call again lb-release-rpm.

If it still does not work, check that there are no permanent problems in $LHCBTAR/rpm/lhcb (e.q. quota) and that the files are copied correctly with something like (bash)

for f in /data/artifacts/release/lhcb-release/${build_id}/*.rpm ; do
  cmp $f $LHCBTAR/rpm/lhcb/$(basename $f)
done

If everything looks correct, try to rebuild the project via the rebuild button in the dashboard.

 

For the Librarian

Revision 482015-07-21 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 65 to 65
 lb-project-manage-volume -c $MyProject $MyVersion
    • install the new project
Changed:
<
<
/afs/cern.ch/lhcb/software/rpmrel/afslbpkr install ${MyProject^^}_${MyVersion}_{x86_64_slc6_gcc4{8,9}_{opt,dbg},index} (note the '_' instead of '-'), or
/afs/cern.ch/lhcb/software/rpmrel/afslbpkr install ${MyProject^^}_${MyVersion}_{x86_64_slc6_gcc48_{opt,dbg},index}
>
>
lb-deployment-afs-install $MyProject $MyVersion x86_64-slc6-gcc4{8,9}-{opt,dbg} x86_64-slc6-gcc48-do0 or
lb-deployment-afs-install $MyProject $MyVersion x86_64-slc6-gcc48-{opt,dbg}
  if gcc49 is not available.

If there are problems about "conflicts", call /afs/cern.ch/lhcb/software/rpmrel/afslbpkr update and try again.

Revision 472015-07-16 - MarcoCattaneo

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 153 to 153
 /afs/cern.ch/lhcb/software/rpmrel/afslbpkr install DBASE_Hat_Package
Changed:
<
<
>
>
 

Troubleshooting (AKA For Librarians)

Revision 462015-06-25 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 25 to 25
 
Deleted:
<
<
If the nightly build system is too busy, it is possible to speed up the build by (re)starting the Jenkins job lhcb-release specifying slc6-urgent in the field os_label.
 

For Release Shifters

The script lb-deployment-checklist simplifies the release shifter task by checking what needs to be done and either prints the next command to run or points to this page.

Revision 452015-05-28 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 335 to 335
 lb-release-rpm --copy /tmp/rpmbuild/RPMS/noarch/
Added:
>
>

Releases of the nightly builds on /cvmfs/lhcbdev.cern.ch

The releases should be completely automated, but here are the details needed to pcheck the system, perform manual installations/add slots etc...

To connect:

ssh cvmfs-lhcbdev.cern.ch 
sudo -i -u cvlhcbdev

Then you can start a transaction and copy files to /cvmfs/lhcbdev.cern.ch

 

-- MarcoClemencic - 30 Jun 2014

Revision 442015-05-21 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 87 to 87
 cd $TMPDIR
Changed:
<
<
lbn-install --project Gaudi --paltform $CMTCONFIG --artifacts-root https://buildlhcb.cern.ch/artifacts/release lhcb-release $build_id cd lhcb-release/$build_id/GAUDI/GAUDI_$MyVersion export USE_CMAKE=1 make doc
>
>
lbn-install --project Gaudi --platform $CMTCONFIG --artifacts-root https://buildlhcb.cern.ch/artifacts/release lhcb-release $build_id
 
  • build the Doxygen documentation (requires Doxygen 1.8.7)
    cd $TMPDIR/lhcb-release/$build_id/GAUDI/GAUDI_$MyVersion

Revision 432015-05-21 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 303 to 303
 Before LHCbDIrac v8:
mkLCGCMTtarFromRPM  LHCbDirac vXrY $CMTCONFIG
Changed:
<
<
>
>
  After LHCbDirac v8r0:
mkLCGCMTtarFromRPM  LHCbDirac vXrY $CMTCONFIG --nonatives
Added:
>
>

Preparing the meta RPM with the software to be installed locally on the Online farm

First get a local version of LbNightlyTools:

mkdir -p /build/$USER
cd /build/$USER
git clone http://git.cern.ch/pub/LbNightlyTools
cd LbNightlyTools
. setup.sh

Now prepare the Meta RPM spec:

lbn-generate-metaspec -o ofm.spec OnlineFarmMeta 1.1.1 MOOREONLINE_v23r6_x86_64_slc6_gcc48_opt [...]
(You need to pass all necessary applications RPMson the command lines, dependencies are dealt with automatically) And build it:
 
Added:
>
>
rpmbuild -bb ofm.spec

Now copying it to the RPM repository:

lb-release-rpm /tmp/rpmbuild/RPMS/noarch/
lb-release-rpm --copy /tmp/rpmbuild/RPMS/noarch/
 

Revision 422015-05-08 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 24 to 24
 For Data Packages:
Added:
>
>
If the nightly build system is too busy, it is possible to speed up the build by (re)starting the Jenkins job lhcb-release specifying slc6-urgent in the field os_label.
 

For Release Shifters

The script lb-deployment-checklist simplifies the release shifter task by checking what needs to be done and either prints the next command to run or points to this page.

Revision 412015-04-08 - ChristopherRJones

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 126 to 126
  (you may get weird error messages, in that case just try again).
Changed:
<
<
  • install the software on CVMFS from the tarballs as usual. For the time being the instructions for CVMFS are available on the Old Release twiki.
>
>
  • install the software on CVMFS from the tarballs as usual. For the time being the instructions for CVMFS are available on the Old Release twiki.
 

Data packages

Warning, important WARNING The procedure for the release of data packages via RPMs is still under heavy development and the automation is not yet in place. Moreover, the new procedure cover only the checkout and build steps, while special cases, tarballs, locking and cvmfs have do be done using the standard instructions at ProjectReleaseOld.

Revision 402015-04-07 - ChristopherRJones

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 67 to 67
 /afs/cern.ch/lhcb/software/rpmrel/afslbpkr install ${MyProject^^}_${MyVersion}_{x86_64_slc6_gcc4{8,9}_{opt,dbg},index} (note the '_' instead of '-'), or
/afs/cern.ch/lhcb/software/rpmrel/afslbpkr install ${MyProject^^}_${MyVersion}_{x86_64_slc6_gcc48_{opt,dbg},index}
Changed:
<
<
if gcc49 is not available. If there are problems about "conflicts", call /afs/cern.ch/lhcb/software/rpmrel/afslbpkr update and try again.
>
>
if gcc49 is not available.
 
Added:
>
>
If there are problems about "conflicts", call /afs/cern.ch/lhcb/software/rpmrel/afslbpkr update and try again.
 Until the problems with Externals installation has been solved, it is possible to run:
/afs/cern.ch/lhcb/software/rpmrel/afslbpkr rpm -- -ivh --nodeps /afs/cern.ch/lhcb/distribution/rpm/lhcb/${MyProject^^}_${MyVersion}*
Changed:
<
<
But that disables dependency checking and at that point, you are on your own to ensure everything needed ios installed.
>
>
But that disables dependency checking and at that point, you are on your own to ensure everything needed is installed.
 
<!--/twistyPlugin twikiMakeVisibleInline-->
Line: 126 to 126
  (you may get weird error messages, in that case just try again).
Changed:
<
<
  • install the software on CVMFS from the tarballs as usual.
>
>
  • install the software on CVMFS from the tarballs as usual. For the time being the instructions for CVMFS are available on the Old Release twiki.
 

Data packages

Warning, important WARNING The procedure for the release of data packages via RPMs is still under heavy development and the automation is not yet in place. Moreover, the new procedure cover only the checkout and build steps, while special cases, tarballs, locking and cvmfs have do be done using the standard instructions at ProjectReleaseOld.

Revision 392015-02-17 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 96 to 96
 
  • install on AFS the generated documentation
    mkdir -p $GAUDISOFT/GAUDI/GAUDI_$MyVersion/InstallArea/doc
Changed:
<
<
rsync -avP $TMPDIR/lhcb-release/$build_id/GAUDI/GAUDI_$MyVersion/build.$CMTCONFIG/doxygen/ $GAUDISOFT/GAUDI/GAUDI_$MyVersion/InstallArea/doc/
>
>
rsync -a $TMPDIR/lhcb-release/$build_id/GAUDI/GAUDI_$MyVersion/build.$CMTCONFIG/doxygen/ $GAUDISOFT/GAUDI/GAUDI_$MyVersion/InstallArea/doc/
 
  • clean up your temporary directory
    rm -rf $TMPDIR/lhcb-release

Revision 382015-02-17 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 76 to 76
  But that disables dependency checking and at that point, you are on your own to ensure everything needed ios installed.
Added:
>
>
 
<!--/twistyPlugin twikiMakeVisibleInline-->
Gaudi requires it's own documentation to be built and installed on AFS before locking the release directory.

Revision 372015-02-16 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 296 to 296
 

Releasing a new version of LCG Grid

Added:
>
>
Before LHCbDIrac v8:
mkLCGCMTtarFromRPM  LHCbDirac vXrY $CMTCONFIG
<verbatim>

After LHCbDirac v8r0:
<verbatim>
mkLCGCMTtarFromRPM  LHCbDirac vXrY $CMTCONFIG --nonatives
<verbatim>

</verbatim>
<nop>
  -- MarcoClemencic - 30 Jun 2014

Revision 362015-02-16 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 76 to 76
  But that disables dependency checking and at that point, you are on your own to ensure everything needed ios installed.
Added:
>
>
<!--/twistyPlugin twikiMakeVisibleInline-->
Gaudi requires it's own documentation to be built and installed on AFS before locking the release directory.

  • go to a temporary directory
    cd $TMPDIR
    
  • install a local copy of the build of Gaudi (using LbNightlyTools)
    lbn-install --project Gaudi --paltform $CMTCONFIG --artifacts-root https://buildlhcb.cern.ch/artifacts/release lhcb-release $build_id
    cd lhcb-release/$build_id/GAUDI/GAUDI_$MyVersion
    export USE_CMAKE=1
    make doc
    
  • build the Doxygen documentation (requires Doxygen 1.8.7)
    cd $TMPDIR/lhcb-release/$build_id/GAUDI/GAUDI_$MyVersion
    export USE_CMAKE=1
    make doc
    
  • install on AFS the generated documentation
    mkdir -p $GAUDISOFT/GAUDI/GAUDI_$MyVersion/InstallArea/doc
    rsync -avP $TMPDIR/lhcb-release/$build_id/GAUDI/GAUDI_$MyVersion/build.$CMTCONFIG/doxygen/ $GAUDISOFT/GAUDI/GAUDI_$MyVersion/InstallArea/doc/
    
  • clean up your temporary directory
    rm -rf $TMPDIR/lhcb-release
    
<!--/twistyPlugin-->
 
    • release and lock the AFS volume
      lb-project-manage-volume -r $MyProject $MyVersion

Revision 352015-02-13 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 205 to 205
 

As from Gaudi v23r3...

Changed:
<
<
The generation of the tarball of its dependencies. This has to be done for each and every optimized >binary< (x86-64-slc6-gcc46-opt, x86-64-slc5-gcc46-opt, x86-64-slc5-gcc43-opt, i686-slc5-gcc43-opt)
>
>
The generation of the tarball of its dependencies. This has to be done for each and every optimized and debug >binary< (x86-64-slc6-gcc48-opt, x86-64-slc6-gcc48-dbg, x86-64-slc6-gcc49-opt, x86-64-slc6-gcc49-dbg).
 The project LHCbExternals contains the list of dependencies to be included in LCGCMT. It is therefore necessary to:
  1. Make sure that the dependency list is up to date
  2. Tag a new version of LHCbExternals, with a version matching LCGCMT e.g.
Line: 213 to 214
  LCGCMT 63a -> LHCbExternals v63r1 etc...
Changed:
<
<
Release LHCbEXternals to AFS:
>
>
Release LHCbExternals to AFS:
 
          cd $LHCBRELEASE
          mkproject -p LHCbExternals -v vXrY -a ngc 
Line: 221 to 222
 
Changed:
<
<
Now prepare the tar ball of LCGCMT, which has to be done dofferently depending on the version of LCGCMT:
>
>
Now prepare the tar ball of LCGCMT, which has to be done differently depending on the version of LCGCMT:
  * Up to LCGCMT 66:

Revision 342015-01-26 - ChristopherRJones

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 33 to 33
  and follow the instructions.
Changed:
<
<

Details of the procedre

>
>

Details of the procedure

 
Line: 48 to 48
 

BE CAREFUL: Since 20141010, the release procedure automatically bumps up the release number for packages for which there is already a version in the RPM repo.

Changed:
<
<
If your slot contains more projects that the ones requested for release, you need to ONLY copy the needed ones to teh RPM repo.
>
>
If your slot contains more projects that the ones requested for release, you need to ONLY copy the needed ones to the RPM repo.
 In that case the --rpm-regex option of lb-release-rpm should be used.

Line: 103 to 103
 
  • install the software on CVMFS from the tarballs as usual.

Data packages

Changed:
<
<
Warning, important WARNING The procedure for the release of data packages via RPMs is still under heavy development and the automation is not yet in place. Moreover, the new procedure cover only the checkout and build steps, while special cases, tarballs, locking and cvmfs have do be done using the standard instructions at ProjectRelease
>
>
Warning, important WARNING The procedure for the release of data packages via RPMs is still under heavy development and the automation is not yet in place. Moreover, the new procedure cover only the checkout and build steps, while special cases, tarballs, locking and cvmfs have do be done using the standard instructions at ProjectReleaseOld.
 

Revision 332015-01-16 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 25 to 25
 

For Release Shifters

Added:
>
>
The script lb-deployment-checklist simplifies the release shifter task by checking what needs to be done and either prints the next command to run or points to this page.
 
Added:
>
>
So connect to buildlhcb.cern.ch (or lxplus) and run
lb-deployment-checklist MyProject vXrY
and follow the instructions.

Details of the procedre

 
Line: 78 to 86
 
  • create the old tarballs
Changed:
<
<
cd $LHCBRELEASES mkLHCbtar -p $MyProject -v $MyVersion for platform in `ls $LHCBRELEASES/${MyProject^^}/${MyProject^^}_${MyVersion}/InstallArea` ; do mkLHCbtar -p $MyProject -v $MyVersion -b $platform ; done
>
>
lb-deployment-oldtarballs $MyProject $MyVersion
 

  • For Gauss, it is necessary to create the Genser tarball
Line: 90 to 96
 
  • update the software database
Changed:
<
<
for platform in `ls $LHCBRELEASES/${MyProject^^}/${MyProject^^}_${MyVersion}/InstallArea` ; do lb-sdb-addplatform $MyProject $MyVersion $platform ; done lb-sdb-release -r $MyProject $MyVersion
>
>
lb-deployment-updatesdb $MyProject $MyVersion
 
Added:
>
>
(you may get weird error messages, in that case just try again).
 
  • install the software on CVMFS from the tarballs as usual.

Revision 322015-01-15 - MarcoCattaneo

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 16 to 16
 
  • prepare the project as usual (release notes, tags, etc.), following the instructions at PrepareProjectRelease
  • register the project/version to the software database
    lb-sdb-importsvn MyProject vXrY
Changed:
<
<
within 10-15 min the build should start to appear in the nightlies page https://buildlhcb.cern.ch/nightlies-release/. Note that, for a dependency tree of projects, lb-sdb-importsvn can be called just once for the top level project as it will pull in all the dependencies, whereas lb-sdb-release must be called once for each project.
>
>
within 10-15 min the build should start to appear in the nightlies page https://buildlhcb.cern.ch/nightlies-release/. Note that, for a dependency tree of projects, lb-sdb-importsvn can be called just once for the top level project as it will pull in all the dependencies.
 
  • if there is a problem in the build, it can be re-started via the dedicated button (it will not restart by itself after a retag)
  • once satisfied by the build, take note of the build id (you can use the direct link icon) and make the request via LHCb Deployment Request form

Revision 312015-01-13 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 16 to 16
 
  • prepare the project as usual (release notes, tags, etc.), following the instructions at PrepareProjectRelease
  • register the project/version to the software database
    lb-sdb-importsvn MyProject vXrY
Deleted:
<
<
lb-sdb-release MyProject vXrY
  within 10-15 min the build should start to appear in the nightlies page https://buildlhcb.cern.ch/nightlies-release/. Note that, for a dependency tree of projects, lb-sdb-importsvn can be called just once for the top level project as it will pull in all the dependencies, whereas lb-sdb-release must be called once for each project.
  • if there is a problem in the build, it can be re-started via the dedicated button (it will not restart by itself after a retag)
  • once satisfied by the build, take note of the build id (you can use the direct link icon) and make the request via LHCb Deployment Request form

Revision 302015-01-09 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 57 to 57
 lb-project-manage-volume -c $MyProject $MyVersion
    • install the new project
Changed:
<
<
/afs/cern.ch/lhcb/software/rpmrel/lbpkr install ${MyProject^^}_${MyVersion}_{x86_64_slc6_gcc4{8,9}_{opt,dbg},index}
>
>
/afs/cern.ch/lhcb/software/rpmrel/afslbpkr install ${MyProject^^}_${MyVersion}_{x86_64_slc6_gcc4{8,9}_{opt,dbg},index}
  (note the '_' instead of '-'), or
Changed:
<
<
/afs/cern.ch/lhcb/software/rpmrel/lbpkr install ${MyProject^^}_${MyVersion}_{x86_64_slc6_gcc48_{opt,dbg},index} if gcc49 is not available. If there are problems about "conflicts", call /afs/cern.ch/lhcb/software/rpmrel/lbpkr update and try again.
>
>
/afs/cern.ch/lhcb/software/rpmrel/afslbpkr install ${MyProject^^}_${MyVersion}_{x86_64_slc6_gcc48_{opt,dbg},index} if gcc49 is not available. If there are problems about "conflicts", call /afs/cern.ch/lhcb/software/rpmrel/afslbpkr update and try again.
  Until the problems with Externals installation has been solved, it is possible to run:
Changed:
<
<
/afs/cern.ch/lhcb/software/rpmrel/lbpkr rpm -- -ivh --nodeps /afs/cern.ch/lhcb/distribution/rpm/lhcb/${MyProject^^}_${MyVersion}*
>
>
/afs/cern.ch/lhcb/software/rpmrel/afslbpkr rpm -- -ivh --nodeps /afs/cern.ch/lhcb/distribution/rpm/lhcb/${MyProject^^}_${MyVersion}*
 

But that disables dependency checking and at that point, you are on your own to ensure everything needed ios installed.

Line: 121 to 121
 

  • Install the new RPM on AFS
Changed:
<
<
/afs/cern.ch/lhcb/software/rpmrel/lbpkr install DBASE_Hat_Package
>
>
/afs/cern.ch/lhcb/software/rpmrel/afslbpkr install DBASE_Hat_Package
 

Line: 173 to 173
 If we are building a project/version for which the rest of the underlying chain was not installed through RPMs, we can still build and install the project via the RPMs.

The procedure is the same as in the regular case, except that instead of using "lbpkr install" we need to call:

Changed:
<
<
/afs/cern.ch/lhcb/software/rpmrel/lbpkr rpm -- -ivh --nodeps $LHCBTAR/rpm/lhcb/MYPROJECT_vXrY*.rpm
>
>
/afs/cern.ch/lhcb/software/rpmrel/afslbpkr rpm -- -ivh --nodeps $LHCBTAR/rpm/lhcb/MYPROJECT_vXrY*.rpm
 

If lbpkr prints messages like "Fatal error, run database recovery"

It happened once that the RPM database got corrupted.
Changed:
<
<
It was fixed following the instructions at http://www.rpm.org/wiki/Docs/RpmRecovery where we replaced /var/lib with /afs/cern.ch/lhcb/software/rpmrel/var/lib and the command rpm with /afs/cern.ch/lhcb/software/rpmrel/lbpkr rpm --
>
>
It was fixed following the instructions at http://www.rpm.org/wiki/Docs/RpmRecovery where we replaced /var/lib with /afs/cern.ch/lhcb/software/rpmrel/var/lib and the command rpm with /afs/cern.ch/lhcb/software/rpmrel/afslbpkr rpm --
 
Line: 253 to 253
 
cd /afs/cern.ch/lhcb/software/rpmrel/
Changed:
<
<
./lbpkr rpm -- -ivh --nodeps $LHCBTAR/rpm/lhcb/LHCBEXTERNALS_v71r0p1_*.rpm
>
>
./afslbpkr rpm -- -ivh --nodeps $LHCBTAR/rpm/lhcb/LHCBEXTERNALS_v71r0p1_*.rpm
 

See that there is a --nodeps, we don;t want to reinstall the dependencies on AFS.

Revision 292014-12-18 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 29 to 29
 
Changed:
<
<
>
>
 
  • publish the produced RPMs (from buildlhcb.cern.ch)
Deleted:
<
<
 
    • try to copy the RPMs to the repository
      build_id=1234
      lb-release-rpm /data/artifacts/release/lhcb-release/$build_id
Line: 45 to 44
 If your slot contains more projects that the ones requested for release, you need to ONLY copy the needed ones to teh RPM repo. In that case the --rpm-regex option of lb-release-rpm should be used.
Added:
>
>
 
  • install on AFS
Deleted:
<
<
 
    • First, set some environment variables that define the project name and version to be installed. Used in the following commands (so you can simply cut and paste them directly). Works for bash shells, so recommend switching to this if it isn't your default.
      MyProject=<ProjectName>
Line: 78 to 75
 lb-project-manage-volume -l $MyProject $MyVersion
    • prepare the web pages links
Changed:
<
<
$LHCBRELEASES/DOC/scripts/addrel.py $MyProject $MyVersion
>
>
$LHCBDOC/scripts/addrel.py $MyProject $MyVersion
 
Changed:
<
<
>
>
 
  • create the old tarballs
    cd $LHCBRELEASES
    mkLHCbtar -p $MyProject -v $MyVersion
Line: 92 to 89
 mkMCGentarFromRPM ${MyProject} ${MyVersion} ${CMTCONFIG}
Added:
>
>
 
  • update the software database
    for platform in `ls $LHCBRELEASES/${MyProject^^}/${MyProject^^}_${MyVersion}/InstallArea` ; do lb-sdb-addplatform $MyProject $MyVersion $platform ; done
    lb-sdb-release -r $MyProject $MyVersion

Revision 282014-12-16 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 60 to 60
 

    • install the new project
Changed:
<
<
/afs/cern.ch/lhcb/software/rpmrel/lbpkr install ${MyProject^^}_${MyVersion}_x86_64_slc6_gcc48_opt /afs/cern.ch/lhcb/software/rpmrel/lbpkr install ${MyProject^^}_${MyVersion}_x86_64_slc6_gcc48_dbg /afs/cern.ch/lhcb/software/rpmrel/lbpkr install ${MyProject^^}_${MyVersion}_index (note the '_' instead of '-'). If there are problems about "conflicts", call /afs/cern.ch/lhcb/software/rpmrel/lbpkr update and try again.
>
>
/afs/cern.ch/lhcb/software/rpmrel/lbpkr install ${MyProject^^}_${MyVersion}_{x86_64_slc6_gcc4{8,9}_{opt,dbg},index} (note the '_' instead of '-'), or
/afs/cern.ch/lhcb/software/rpmrel/lbpkr install ${MyProject^^}_${MyVersion}_{x86_64_slc6_gcc48_{opt,dbg},index}
if gcc49 is not available. If there are problems about "conflicts", call /afs/cern.ch/lhcb/software/rpmrel/lbpkr update and try again.
  Until the problems with Externals installation has been solved, it is possible to run:

Revision 272014-12-15 - JoelClosier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 93 to 93
 

  • update the software database
Changed:
<
<
for platform in `ls $LHCBRELEASES/${MyProject^^}/${MyProject^^}_${MyVersion}/InstallArea` ; do lb-sdb-addplatform MyProject $MyVersion $platform ; done
>
>
for platform in `ls $LHCBRELEASES/${MyProject^^}/${MyProject^^}_${MyVersion}/InstallArea` ; do lb-sdb-addplatform $MyProject $MyVersion $platform ; done
 lb-sdb-release -r $MyProject $MyVersion

Revision 262014-12-09 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 250 to 250
 /afs/cern.ch/lhcb/distribution/rpm/update.sh
Added:
>
>
Then you can install them on AFS:

cd /afs/cern.ch/lhcb/software/rpmrel/
./lbpkr rpm -- -ivh --nodeps $LHCBTAR/rpm/lhcb/LHCBEXTERNALS_v71r0p1_*.rpm

See that there is a --nodeps, we don;t want to reinstall the dependencies on AFS.

 BEWARE: ALWAYS RUN MKLCGCMTTAR (OR [...]FROMRPM) ON THE CORRECT PLATFORM (or the gcc version may be wrong)

Revision 252014-11-26 - MarcoCattaneo

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"

Project Release Guide

Line: 104 to 104
 
Changed:
<
<
    • add the list of packages (with their versions) in the field packages_list (e.g. RawEventFormat v1r1 Gen/DecFiles vXrY), the name of the container can be specified with something like "container":"name" (e.g. PARAM:TMVAWeights v1r0)
>
>
    • add the list of packages (with their versions) in the field packages_list (e.g. RawEventFormat v1r1 Gen/DecFiles vXrY), the name of the container can be specified with something like "container":"name" (e.g. PARAM:TMVAWeights v1r0). The container can be omitted for DBASE.
 
    • click on the Build button (leave all the others fields empty)

  • Once the build is completed, check the checkout log

Revision 242014-11-26 - ChristopherRJones

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Added:
>
>

Project Release Guide

Introduction

 Warning, important These instructions are a snapshot of the current state of the tools. The procedure will be simplified in the coming weeks.

Warning, important These instructions do not apply for LbScripts, Compat, *Dirac, Ganga and data packages. For those, use the old instructions: ProjectReleaseOld

Changed:
<
<

For Release Managers

>
>

For Release Managers

  For Projects:
  • prepare the project as usual (release notes, tags, etc.), following the instructions at PrepareProjectRelease
Line: 21 to 25
 For Data Packages:
Added:
>
>

For Release Shifters

 
Deleted:
<
<

For Release Shifters

 
Line: 124 to 128
 
Changed:
<
<

Troubleshooting (AKA For Librarians)

If the nightly builds do not work

>
>

Troubleshooting (AKA For Librarians)

If the nightly builds do not work

 The nightly build system is nothing more than wrappers around simple tools that can be called by any user, so it is possible to prepare the RPMs for the release by hand.

  • Go to a work directory, get the nightly build tools, and prepare the environment, e.g. on lxbuild
Line: 166 to 171
 
  • Continue with the procedure to install to AFS
Changed:
<
<

If the RPMs of the base projects do not exist

>
>

If the RPMs of the base projects do not exist

 If we are building a project/version for which the rest of the underlying chain was not installed through RPMs, we can still build and install the project via the RPMs.

The procedure is the same as in the regular case, except that instead of using "lbpkr install" we need to call:

/afs/cern.ch/lhcb/software/rpmrel/lbpkr rpm -- -ivh --nodeps $LHCBTAR/rpm/lhcb/MYPROJECT_vXrY*.rpm
Changed:
<
<

If lbpkr prints messages like "Fatal error, run database recovery"

>
>

If lbpkr prints messages like "Fatal error, run database recovery"

 It happened once that the RPM database got corrupted.

It was fixed following the instructions at http://www.rpm.org/wiki/Docs/RpmRecovery where we replaced /var/lib with /afs/cern.ch/lhcb/software/rpmrel/var/lib and the command rpm with /afs/cern.ch/lhcb/software/rpmrel/lbpkr rpm --

Changed:
<
<

For the Librarian

>
>

For the Librarian

 
Changed:
<
<

Releasing a new version of LCG externals

>
>

Releasing a new version of LCG externals

 
Changed:
<
<

Until Gaudi v23r2

>
>

Until Gaudi v23r2

  Until Gaudi v23r2, the dependencies to be included in LCGCMT were included in the Gaudi requirements files. The procedure to build the tarball was therefore the following:
Line: 195 to 200
  mkLCGCMTtar -n GAUDI_ -b This script produces a log file in the local directory from where it has been run. Check it to see if everything was OK.
Changed:
<
<

As from Gaudi v23r3...

>
>

As from Gaudi v23r3...

  The generation of the tarball of its dependencies. This has to be done for each and every optimized >binary< (x86-64-slc6-gcc46-opt, x86-64-slc5-gcc46-opt, x86-64-slc5-gcc43-opt, i686-slc5-gcc43-opt) The project LHCbExternals contains the list of dependencies to be included in LCGCMT. It is therefore necessary to:
Line: 227 to 232
  mkLCGCMTtarFromRPM LHCbExternals v69r0p1 $CMTCONFIG
Changed:
<
<

Preparing the LHCb Externals RPM

>
>

Preparing the LHCb Externals RPM

  When the LHCbExternals versions have been tagged, it is necessary to prepare the RPMs, using the commands provided by the LbNightlyTools repository:
Line: 248 to 253
 BEWARE: ALWAYS RUN MKLCGCMTTAR (OR [...]FROMRPM) ON THE CORRECT PLATFORM (or the gcc version may be wrong)
Changed:
<
<

Releasing a new version of LCG Grid

>
>

Releasing a new version of LCG Grid

 

-- MarcoClemencic - 30 Jun 2014

Revision 232014-11-21 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Warning, important These instructions are a snapshot of the current state of the tools. The procedure will be simplified in the coming weeks.
Line: 7 to 7
 

For Release Managers

Added:
>
>
For Projects:
 
  • prepare the project as usual (release notes, tags, etc.), following the instructions at PrepareProjectRelease
  • register the project/version to the software database
    lb-sdb-importsvn MyProject vXrY
Line: 15 to 17
 
  • if there is a problem in the build, it can be re-started via the dedicated button (it will not restart by itself after a retag)
  • once satisfied by the build, take note of the build id (you can use the direct link icon) and make the request via LHCb Deployment Request form
Added:
>
>
For Data Packages:
 

For Release Shifters

Revision 222014-11-13 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Warning, important These instructions are a snapshot of the current state of the tools. The procedure will be simplified in the coming weeks.
Line: 241 to 241
 BEWARE: ALWAYS RUN MKLCGCMTTAR (OR [...]FROMRPM) ON THE CORRECT PLATFORM (or the gcc version may be wrong)
Added:
>
>

Releasing a new version of LCG Grid

 -- MarcoClemencic - 30 Jun 2014

META TOPICMOVED by="cattanem" date="1412348243" from="LHCb.ProjectReleaseNew" to="LHCb.ProjectRelease"

Revision 212014-11-10 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Warning, important These instructions are a snapshot of the current state of the tools. The procedure will be simplified in the coming weeks.
Line: 30 to 30
 lb-release-rpm --copy /data/artifacts/release/lhcb-release/$build_id
Added:
>
>
BE CAREFUL: Since 20141010, the release procedure automatically bumps up the release number for packages for which there is already a version in the RPM repo. If your slot contains more projects that the ones requested for release, you need to ONLY copy the needed ones to teh RPM repo. In that case the --rpm-regex option of lb-release-rpm should be used.
 
  • install on AFS

    • First, set some environment variables that define the project name and version to be installed.

Revision 202014-11-07 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Warning, important These instructions are a snapshot of the current state of the tools. The procedure will be simplified in the coming weeks.
Line: 89 to 89
 
Changed:
<
<
    • add the list of packages (with their versions) in the field packages_list (e.g. RawEventFormat v1r1 Gen/DecFiles vXrY)
>
>
    • add the list of packages (with their versions) in the field packages_list (e.g. RawEventFormat v1r1 Gen/DecFiles vXrY), the name of the container can be specified with something like "container":"name" (e.g. PARAM:TMVAWeights v1r0)
 
    • click on the Build button (leave all the others fields empty)

  • Once the build is completed, check the checkout log

Revision 192014-10-30 - MarcoCattaneo

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Warning, important These instructions are a snapshot of the current state of the tools. The procedure will be simplified in the coming weeks.
Line: 11 to 11
 
  • register the project/version to the software database
    lb-sdb-importsvn MyProject vXrY
    lb-sdb-release MyProject vXrY
Changed:
<
<
within 10-15 min the build should start to appear in the nightlies page https://buildlhcb.cern.ch/nightlies-release/
>
>
within 10-15 min the build should start to appear in the nightlies page https://buildlhcb.cern.ch/nightlies-release/. Note that, for a dependency tree of projects, lb-sdb-importsvn can be called just once for the top level project as it will pull in all the dependencies, whereas lb-sdb-release must be called once for each project.
 
  • if there is a problem in the build, it can be re-started via the dedicated button (it will not restart by itself after a retag)
  • once satisfied by the build, take note of the build id (you can use the direct link icon) and make the request via LHCb Deployment Request form

Revision 182014-10-21 - MarcoClemencic

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Warning, important These instructions are a snapshot of the current state of the tools. The procedure will be simplified in the coming weeks.
Line: 231 to 231
 Check the RPM produced and copying it to the LHCb YUM repository:
cp /tmp/rpmbuild/RPMS/noarch/LHCBEXTERNALS_v70r0p1_x86_64_slc6_gcc48_opt-1.0.0-1.noarch.rpm /afs/cern.ch/lhcb/distribution/rpm/lhcb
Changed:
<
<
/afs/cern.ch/lhcb/distribution/rpm/lhcb/update.sh
>
>
/afs/cern.ch/lhcb/distribution/rpm/update.sh
 

BEWARE: ALWAYS RUN MKLCGCMTTAR (OR [...]FROMRPM) ON THE CORRECT PLATFORM (or the gcc version may be wrong)

Revision 172014-10-17 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Warning, important These instructions are a snapshot of the current state of the tools. The procedure will be simplified in the coming weeks.
Line: 168 to 168
 It was fixed following the instructions at http://www.rpm.org/wiki/Docs/RpmRecovery where we replaced /var/lib with /afs/cern.ch/lhcb/software/rpmrel/var/lib and the command rpm with /afs/cern.ch/lhcb/software/rpmrel/lbpkr rpm --
Added:
>
>

For the Librarian

Releasing a new version of LCG externals

Until Gaudi v23r2

Until Gaudi v23r2, the dependencies to be included in LCGCMT were included in the Gaudi requirements files. The procedure to build the tarball was therefore the following:

For Gaudi, one more setup needs to be done: the generation of the tarball of its dependencies. This has to be done for each and every optimized >binary< (x86-64-slc6-gcc46-opt, x86-64-slc5-gcc46-opt, x86-64-slc5-gcc43-opt, i686-slc5-gcc43-opt)

          cd $Gaudi_release_area
          mkLCGCMTtar -n GAUDI_<version> -b <binary>
This script produces a log file in the local directory from where it has been run. Check it to see if everything was OK.

As from Gaudi v23r3...

The generation of the tarball of its dependencies. This has to be done for each and every optimized >binary< (x86-64-slc6-gcc46-opt, x86-64-slc5-gcc46-opt, x86-64-slc5-gcc43-opt, i686-slc5-gcc43-opt) The project LHCbExternals contains the list of dependencies to be included in LCGCMT. It is therefore necessary to:

  1. Make sure that the dependency list is up to date
  2. Tag a new version of LHCbExternals, with a version matching LCGCMT e.g. LCGCMT 63 -> LHCbExternals v63r0 LCGCMT 63a -> LHCbExternals v63r1 etc...

Release LHCbEXternals to AFS:

          cd $LHCBRELEASE
          mkproject -p LHCbExternals -v vXrY -a ngc 
          mkproject -p LHCbExternals -v vXrY -a K

Now prepare the tar ball of LCGCMT, which has to be done dofferently depending on the version of LCGCMT:

* Up to LCGCMT 66:

          cd $LHCBRELEASE
          mkLCGCMTtar -n LHCBEXTERNALS_<version> -b <binary>

This script produces a log file in the local directory from where it has been run. Check it to see if everything was OK.

* As from LCGCMT 68:

          mkLCGCMTtarFromRPM LHCbExternals v69r0p1 $CMTCONFIG

Preparing the LHCb Externals RPM

When the LHCbExternals versions have been tagged, it is necessary to prepare the RPMs, using the commands provided by the LbNightlyTools repository:

git clone http://git.cern.ch/pub/LbNightlyTools 
cd LbNightlyTools
. ./setup.sh
lbn-generate-extspec LHCbExternals v70r0p1 $CMTCONFIG -o lcg70root6.spec
rpmbuild -bb lcg70root6.spec

Check the RPM produced and copying it to the LHCb YUM repository:

cp /tmp/rpmbuild/RPMS/noarch/LHCBEXTERNALS_v70r0p1_x86_64_slc6_gcc48_opt-1.0.0-1.noarch.rpm /afs/cern.ch/lhcb/distribution/rpm/lhcb
/afs/cern.ch/lhcb/distribution/rpm/lhcb/update.sh

BEWARE: ALWAYS RUN MKLCGCMTTAR (OR [...]FROMRPM) ON THE CORRECT PLATFORM (or the gcc version may be wrong)

 -- MarcoClemencic - 30 Jun 2014

META TOPICMOVED by="cattanem" date="1412348243" from="LHCb.ProjectReleaseNew" to="LHCb.ProjectRelease"

Revision 152014-10-13 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Warning, important These instructions are a snapshot of the current state of the tools. The procedure will be simplified in the coming weeks.
Line: 72 to 72
 for platform in `ls $LHCBRELEASES/${MyProject^^}/${MyProject^^}_${MyVersion}/InstallArea` ; do mkLHCbtar -p $MyProject -v $MyVersion -b $platform ; done
Added:
>
>
  • For Gauss, it is necessary to create the Genser tarball
    mkMCGentarFromRPM ${MyProject} ${MyVersion} ${CMTCONFIG}
    
 
  • update the software database
    for platform in `ls $LHCBRELEASES/${MyProject^^}/${MyProject^^}_${MyVersion}/InstallArea` ; do lb-sdb-addplatform MyProject $MyVersion $platform ; done
    lb-sdb-release -r $MyProject $MyVersion

Revision 142014-10-08 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Warning, important These instructions are a snapshot of the current state of the tools. The procedure will be simplified in the coming weeks.
Line: 13 to 13
 lb-sdb-release MyProject vXrY within 10-15 min the build should start to appear in the nightlies page https://buildlhcb.cern.ch/nightlies-release/
  • if there is a problem in the build, it can be re-started via the dedicated button (it will not restart by itself after a retag)
Changed:
<
<
  • once satisfied by the build, take note of the build id (you can use the direct link icon) and make the request via LHCb Deployment Request form
>
>
  • once satisfied by the build, take note of the build id (you can use the direct link icon) and make the request via LHCb Deployment Request form
 

For Release Shifters

Revision 132014-10-08 - JoelClosier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Warning, important These instructions are a snapshot of the current state of the tools. The procedure will be simplified in the coming weeks.
Line: 106 to 106
 /afs/cern.ch/lhcb/software/rpmrel/lbpkr install DBASE_Hat_Package
Changed:
<
<
>
>
 

Troubleshooting (AKA For Librarians)

If the nightly builds do not work

Revision 122014-10-03 - MarcoCattaneo

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Warning, important These instructions are a snapshot of the current state of the tools. The procedure will be simplified in the coming weeks.
Changed:
<
<
Warning, important These instructions do not apply for LbScripts, Compat, *Dirac, and data packages. For those, use the old instructions: ProjectRelease
>
>
Warning, important These instructions do not apply for LbScripts, Compat, *Dirac, and data packages. For those, use the old instructions: ProjectReleaseOld
 
Line: 13 to 13
 lb-sdb-release MyProject vXrY within 10-15 min the build should start to appear in the nightlies page https://buildlhcb.cern.ch/nightlies-release/
  • if there is a problem in the build, it can be re-started via the dedicated button (it will not restart by itself after a retag)
Changed:
<
<
  • once satisfied by the build, take note of the build id (you can use the direct link icon) and make the request via LHCb Deployment Request form
>
>
  • once satisfied by the build, take note of the build id (you can use the direct link icon) and make the request via LHCb Deployment Request form
 

For Release Shifters

Line: 164 to 164
 

-- MarcoClemencic - 30 Jun 2014 \ No newline at end of file

Added:
>
>
META TOPICMOVED by="cattanem" date="1412348243" from="LHCb.ProjectReleaseNew" to="LHCb.ProjectRelease"

Revision 112014-08-25 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Warning, important These instructions are a snapshot of the current state of the tools. The procedure will be simplified in the coming weeks.
Line: 50 to 50
 /afs/cern.ch/lhcb/software/rpmrel/lbpkr install ${MyProject^^}_${MyVersion}_index (note the '_' instead of '-'). If there are problems about "conflicts", call /afs/cern.ch/lhcb/software/rpmrel/lbpkr update and try again.
Added:
>
>
Until the problems with Externals installation has been solved, it is possible to run:
/afs/cern.ch/lhcb/software/rpmrel/lbpkr rpm -- -ivh --nodeps /afs/cern.ch/lhcb/distribution/rpm/lhcb/${MyProject^^}_${MyVersion}*

But that disables dependency checking and at that point, you are on your own to ensure everything needed ios installed.

 
    • release and lock the AFS volume
      lb-project-manage-volume -r $MyProject $MyVersion
      lb-project-manage-volume -l $MyProject $MyVersion

Revision 102014-07-30 - ChristopherRJones

Line: 1 to 1
 
META TOPICPARENT name="LHCbComputing"
Warning, important These instructions are a snapshot of the current state of the tools. The procedure will be simplified in the coming weeks.
Line: 30 to 31
 

  • install on AFS
Added:
>
>
    • First, set some environment variables that define the project name and version to be installed. Used in the following commands (so you can simply cut and paste them directly). Works for bash shells, so recommend switching to this if it isn't your default.
      MyProject=<ProjectName>
      MyVersion=vXrY
      
      To check the following commands will work for you, now run
      echo ${MyProject^^}
      
      which should return the project name, in upper case.
 
    • prepare the AFS volume
Changed:
<
<
lb-project-manage-volume -c MyProject vXrY
>
>
lb-project-manage-volume -c $MyProject $MyVersion
 
Added:
>
>
 
    • install the new project
Changed:
<
<
/afs/cern.ch/lhcb/software/rpmrel/lbpkr install MYPROJECT_vXrY_x86_64_slc6_gcc48_opt /afs/cern.ch/lhcb/software/rpmrel/lbpkr install MYPROJECT_vXrY_x86_64_slc6_gcc48_dbg /afs/cern.ch/lhcb/software/rpmrel/lbpkr install MYPROJECT_vXrY_index (note the '_' instead of '-'). If there are problems about "conflicts", call /afs/cern.ch/lhcb/software/rpmrel/lbpkr update and try again.
>
>
/afs/cern.ch/lhcb/software/rpmrel/lbpkr install ${MyProject^^}_${MyVersion}_x86_64_slc6_gcc48_opt /afs/cern.ch/lhcb/software/rpmrel/lbpkr install ${MyProject^^}_${MyVersion}_x86_64_slc6_gcc48_dbg /afs/cern.ch/lhcb/software/rpmrel/lbpkr install ${MyProject^^}_${MyVersion}_index (note the '_' instead of '-'). If there are problems about "conflicts", call /afs/cern.ch/lhcb/software/rpmrel/lbpkr update and try again.
 
    • release and lock the AFS volume
Changed:
<
<
lb-project-manage-volume -r MyProject vXrY lb-project-manage-volume -l MyProject vXrY
>
>
lb-project-manage-volume -r $MyProject $MyVersion lb-project-manage-volume -l $MyProject $MyVersion
 
    • prepare the web pages links
Changed:
<
<
$LHCBRELEASES/DOC/scripts/addrel.py MyProject vXrY
>
>
$LHCBRELEASES/DOC/scripts/addrel.py $MyProject $MyVersion
 

  • create the old tarballs
    cd $LHCBRELEASES
Changed:
<
<
mkLHCb