CRAB Logo

Notes about CRAB3 release management

Instructions below are mostly tailored for patching a given release once master branch in GIT has progressed ahead. For the more normal case of creating a new release from current master branch, the tag can be simply created in github, see: https://help.github.com/articles/creating-releases/ and CMSCrabClient#Deployment_of_CRAB_client_on_CVM

Staying aligned with WMCore

Releasing a CRABClient release candidate / patch

1) Creating a tag in the CRABClient repository

Checkout the tag that needs patching from the master branch, for example:

git checkout 3.3.1608

Fetch the upstream repository branches / tags so that the patch commit is available to be cherry-picked.

git fetch upstream

Cherry pick the needed commit(s), e.g. :

git cherry-pick 89197bae12345e3unicorn8df916rainbow6d584

Create the new tag, for example 3.3.1608.patch1

git tag 3.3.1608.patch1

Push the new tag to the upstream repo.

git push upstream --tag

2) Creating a tag in the WMCore repository (if WMCore was modified)

Ask Alan (alan.malta@cernNOSPAMPLEASE.ch) or otherwise Eric (ewv@fnalNOSPAMPLEASE.gov) to create a new tag (e.g., 1.0.16.crab4) on top of the WMCore crab branch (e.g., 1.0.16_crab)

3) Modifying the .spec file and building a package

Quick tutorial about building RPMs with cmsdist scripts

Change package version and build/upload a package to the comp.$USER directory. If no package is needed ignore the commands about pkgtools, otherwise they must be issued in the CMSWEB build machines:

  • For the slc6 distribution use vocms022 as building machine, gcc493 as <gcc-version> and slc6_amd64_gcc493 as <architecture>
  • For the slc7 distribution use vocms055 as building machine, gcc630 as <gcc-version> and slc7_amd64_gcc630 as <architecture>

# prepare a build area
mkdir -p /build/$USER
cd /build/$USER
(git clone -b V00-30-XX https://github.com/cms-sw/pkgtools.git)
(git clone https://github.com/cms-sw/cmsdist.git && cd cmsdist && git checkout comp_<gcc-version>)
(cd ..)

# do some changes to the .spec file. For example, we change the WMCore version from 1.0.16.crab3 to 1.0.16.crab4
# it is also important to update the first line of the .spec file "### RPM cms crabclient 3.3.1608" in case a new CRABClient version is required
vim cmsdist/crabclient.spec 

Once your PR receive the "+1" comment from Jenkins you can merge it by adding a "merge" comment.

An automatic script will build the package that will be available in ~4h at the cvmfs link.

You can also build the package yourself and upload it to http://cmsrep.cern.ch/cmssw/repos/comp.$USER:

# building the new package (works only in the machine mentioned above)
pkgtools/cmsBuild -c cmsdist --repository comp -a <architecture> --builders 8 -j 5 --work-dir w build crabclient

# uploading it to comp.$USER (works only in the machine mentioned above)
pkgtools/cmsBuild -c cmsdist --repository comp -a <architecture> --builders 8 -j 5 --work-dir w upload crabclient --upload-user=$USER

Note that for slc6 it has been decided to use the comp.$USER repository for gcc493 and comp.pre.$USER for gcc481, therefore we pick comp as our repository.

In case of strange errors that can't be explained by logs, try to reset your environment by deleting the work directory (/build/$USER/w/) and refreshing your ~/.bashrc file in case you've modified it previously (a backup of a default .bashrc can be found at /etc/skel/.bashrc). Also make sure that you're checked out to the correct branch of the git repositories!

4) Installing and testing CRABClient on your VM

First, deploy the crabclient to your virtual machine:

cd /data
mkdir -p /data/srv/crabclient
cd /data/srv/crabclient

export VERSION=3.3.1608.patch1
export SCRAM_ARCH=<architecture> # see point 3
export REPO=comp.erupeika
 
mkdir -p $VERSION
wget -O $VERSION/bootstrap.sh http://cmsrep.cern.ch/cmssw/repos/bootstrap.sh
sh $VERSION/bootstrap.sh -architecture $SCRAM_ARCH -path $VERSION -repository $REPO setup
source $VERSION/$SCRAM_ARCH/external/apt/0.5.16/etc/profile.d/init.sh
$VERSION/common/cmspkg -a $SCRAM_ARCH upgrade
$VERSION/common/cmspkg -a $SCRAM_ARCH update
$VERSION/common/cmspkg -a $SCRAM_ARCH install cms+crabclient+$VERSION

Once it is installed, do the regular crabclient setup routine before use:

set up a $SCRAM_ARCH, although it will probably be the same as we used before

export SCRAM_ARCH=<architecture> 

install CMSSW

cd ~
mkdir CRAB3-crabclient-test
cd CRAB3-crabclient-test
cmsrel CMSSW_7_3_5_patch2

and set up the environment

cd ~/CRAB3-crabclient-test/CMSSW_7_3_5_patch2/src/ 
cmsenv

finally, source CRAB3 with the file init.sh which can be found in the deployed crabclient (command find . | grep crabclient | grep init.sh)

source /data/srv/crabclient/3.3.1608.patch1/slc6_amd64_gcc493/cms/crabclient/3.3.1608.patch1/etc/profile.d/init.sh

Now you should be able to use crab in your vm the same way you do in lxplus.

5) After testing the crabclient

If everything works as expected during testing,

  • make a pull request to cmsdist with the spec file that you used to build the new CRABClient;
  • add the corresponding version of the CRABClient to the cmsweb-rest-config.json file in the CRAB3ServerConfig repository.
    • Before pushing the updated cmsweb-rest-config.json file, its consistency must be verified: just run the python checkConfig.py command and make sure it does not throw any error.

Releasing a CRAB TaskWorker release candidate / patch

1) Creating a tag in the CRABServer repository Checkout the tag that needs updating, for example:
git checkout 3.3.1603.rc1

Fetch the upstream repository branches / tags so that the patch commit is available to be cherry-picked.

git fetch upstream

Cherry pick the needed commit, e.g. :

git cherry-pick 89197bae12345e3unicorn8df916rainbow6d584

Create the new tag, for example 3.3.1603.rc2

git tag 3.3.1603.rc2

Push the new tag to the upstream repo.

git push upstream --tag
2) Creating a tag in the WMCore repository (if WMCore was modified)

3) Modifying the specfile and building a package

Quick tutorial about building RPMs with cmsdist scripts

Building and uploading a package:

  • For the slc6 distribution use vocms022 as building machine, gcc493 as <gcc-version> and slc6_amd64_gcc493 as <architecture>.
  • For the slc7 distribution use vocms055 as building machine, gcc630 as <gcc-version> and slc7_amd64_gcc630 as <architecture>.

# prepare a build area
mkdir -p /build/$USER
cd /build/$USER
(git clone -b V00-30-XX https://github.com/cms-sw/pkgtools.git)
(git clone https://github.com/cms-sw/cmsdist.git && cd cmsdist && git checkout comp_<gcc-version>)
(cd ..)

# Do some changes to the specfile, for example we modify the first line "### RPM cms crabserver 3.3.1603.rc1" to have the version 3.3.1603.rc2
vim cmsdist/crabtaskworker.spec

Once your PR receive the "+1" comment from Jenkins you can merge it by adding a "merge" comment.

An automatic script will build the package that will be available in ~1h at the RPM repository listed below

You can also build the package yourself and upload it to http://cmsrep.cern.ch/cmssw/repos/comp.$USER:

# building the new package (works only in the machine mentioned above)
pkgtools/cmsBuild -c cmsdist --repository comp -a <architecture> --builders 8 -j 5 --work-dir w build crabtaskworker | tee logBuild

# uploading it to comp.$USER (works only in the machine mentioned above)
pkgtools/cmsBuild -c cmsdist --repository comp -a <architecture> --builders 8 -j 5 --work-dir w upload crabtaskworker --upload-user=$USER | tee logUpload
Note that for slc6 it has been decided to use the comp.$USER repository for gcc493 and comp.pre.$USER for gcc481, therefore we pick comp as our repository.

The user repository in comp.$USER only holds one release, latest one will override previous versions. If you want to upload more packages (e.g. crabtaskworker and crabserver) you need to indicate both in the same upload command like

pkgtools/cmsBuild -c cmsdist --repository comp -a <architecture> --builders 8 -j 5 --work-dir w upload crabserver crabtaskworker --upload-user=$USER
In case of strange errors that can't be explained by logs, try to reset your environment by deleting the work directory (/build/$USER/w/) and refreshing your ~/.bashrc file in case you've modified it previously (a backup of a default .bashrc can be found at /etc/skel/.bashrc). Also make sure that you're checked out to the correct commit/branch of the git repositories!

4) Deploying and testing the TaskWorker on your VM

Follow the regular deployment instructions on the twiki: https://twiki.cern.ch/twiki/bin/view/CMSPublic/CMSCrabTaskWorker

5) After testing the TaskWorker on your VM

If all looks good, request for the TaskWorker to be deployed on the testbed to be thoroughly tested before releasing it into production. Afterwards, make a pull request to the cmsdist repository with the new crabtaskworker.spec file.

Notes on RPM repositories

Edit | Attach | Watch | Print version | History: r33 < r32 < r31 < r30 < r29 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r33 - 2019-07-05 - LeonardoCristella
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback