Middleware repository update procedure
Introduction
We operate apt-get and yum repositories for the grid middleware and the CA rpms. The apt-get repository is used by yaim. The yum repositories exist in two forms, yum 2.2 and yum 2.4. The repositories all exist under the same AFS directory and are generated from the rpm lists in the middleware CVS. Updates are added manually.
Meta rpms
The middleware required for each node type is captured in the dependencies of a 'meta rpm', which itself contains no software. This meta rpm, for example lcg-CE.rpm, is generated from the rpm list in the CE-rpm file in CVS. There is also a CA-rpm file which holds the current list of certification authority rpms.
Required software
In order to manage the repositories, you need to have a shell on a machine which can access AFS and write to the relevant directories. Furthermore, you will need to have the appropriate utilities available for the generation of each repository; genbasedir, yum-arch and createrepo. These are available from the SL3 packages 'yum' and 'apt', and the SL4 package
createrepo
which can safely be installed on SL3. These utils are not available on lxplus! lxb1712 has all required tools.
Generating a repository from scratch
Update of rpm list for apt generation
The following rpm lists are used
- certification; lcgdeploy.cvs.cern.ch:/cvs/lcgdeploy/lcg-release/rpmlist_sl3
- release; lcgdeploy.cvs.cern.ch:/cvs/lcgdeploy/lcg2/rpmlist_sl3
- CA; lcgdeploy.cvs.cern.ch:/cvs/lcgdeploy/lcg2/rpmlist
The *-rpm files in these folders define what goes into the meta-rpms. These files will have to be updated in order to build a repository.
Generating the repositories
There is a script you can use to generate the repositories. Check out lcgdeploy.cvs.cern.ch:/cvs/lcgdeploy/lcg-scripts/yaim/server and use repgen.pl. This script checks out a particular tag from CVS, builds the metarpms from the files there, puts the rpms in the repository, puts all the middleware in the repository and then generates the apt and yum repositories in the directory.
The CA and release repository is held under
/afs/cern.ch/project/gd/www/gis/apt
The cert testbed one is under
/afs/cern.ch/project/gd/www/apt-cert
To build a repository based on the LCG-2_7_0 tag, producing meta rpms of version 2.7.0 for sl3, issue the following command
./repgen -t LCG-2_7_0 -v 2.7.0 -m mw -o sl3
This will produce the repository under /afs/cern.ch/project/gd/www/gis/apt/LCG-2_7_0/en/i386 with the files necessary for it to be accessible via both apt and yum.
Updating the main repository
This should not normally be done - use the updates folder. If this is necessary though, rebuild replacing the lcg_sl3.updates with lcg_sl3;
RELEASE=LCG-2_7_0
genbasedir --flat --bloat --bz2only --partial /afs/cern.ch/project/gd/www/gis/apt/$RELEASE/sl3/en/i386 lcg_sl3
yum-arch -l /afs/cern.ch/project/gd/www/gis/apt/$RELEASE/sl3/en/i386/RPMS.lcg_sl3
createrepo /afs/cern.ch/project/gd/www/gis/apt/$RELEASE/sl3/en/i386/RPMS.lcg_sl3
CAs
To create the CA repository based on the tag LCG_CA-0_28 and with the lcg-CA rpm at version 0.28, do
./repgen -t LCG_CA-0_28 -v 0.28 -m CA
Make sure that CA-rpm and security-rpm.h have the appropriate tag (in this case LCG_CA-0_28).
The repository will be built under /afs/cern.ch/project/gd/www/gis/apt/LCG_CA/en/i386/.
Procedure for CA update
The general procedure can be found here :
https://wiki.egi.eu/wiki/EGI_IGTF_Release_Process
A script which executes the steps below can be found here :
http://glite.cvs.cern.ch:8180/cgi-bin/glite.cgi/org.glite.integration.release/
For the integration team part these are the steps:
PPS Email Template
Dear all,
Please find a preview for the new CA’s version x.y-z available here. YUM repo file should contain:
[CA]
name=CAs
baseurl=http://linuxsoft.cern.ch/LCG-CAs/x.y-z
The release notes can be found under:
http://grid-deployment.web.cern.ch/grid-deployment/lcg2CAlist_new.html
As soon as it has been approved by PPS and SAM team the repo will be enabled under:
http://glitesoft.cern.ch/LCG-CAs/current/RPMS.production/
and the release notes under:
http://grid-deployment.web.cern.ch/grid-deployment/lcg2CAlist.html
Publishing updates
Updates should be added manually to the updates folder.
If any file is changed or added to an apt/yum repository, the repository must be rebuilt otherwise there will be an inconsistency between the indexed information and the directory contents.
If you have added a file to the updates folder, you can rebuild a modified repository like this.
RELEASE=LCG-2_7_0
genbasedir --flat --bloat --bz2only --partial /afs/cern.ch/project/gd/www/gis/apt/$RELEASE/sl3/en/i386 lcg_sl3.updates
yum-arch -l /afs/cern.ch/project/gd/www/gis/apt/$RELEASE/sl3/en/i386/RPMS.lcg_sl3.updates/
createrepo /afs/cern.ch/project/gd/www/gis/apt/$RELEASE/sl3/en/i386/RPMS.lcg_sl3.updates/
DO NOT put anything in the updates folder which break sites which upgrade automatically. Putting a fix here which is enabled by later admin intervention is fine.
All updates must be associated with an announcement to the ROLLOUT and the ROCs, and a news entry on the
CIC portal
. You can use their broadcast tool to do all this at once. Take a look at earlier news entries about updates to see the information required. Note that the CIC site is protected by certificate.
Repgen options
The full list of repgen options is as follows
-t TAG_name : the tag to check out from CVS
-v rpm_version : the version number of the rpms to be generated
-m [mwCA] : operate in middleware (mw) or CA (ca) mode
-o [rh73sl3] : operating system (mw mode only)
-r : recheck
RpmDir on AFS for new middleware
-b : build rpms only, do not create the repositories
-d repository top-level directory : create repository with this name, rather than the default which is the TAG supplied
-n node_type : generate the rpm for this node type only, not the full set. This option will not remove existing meta rpms from the repository
-f apt_folder : generate the whole thing under this directory, useful for testing
-c CVS path : path under lcgdeploy.cvs.cern.ch:/cvs/lcgdeploy from which to checkout
-i [i386|ia64] : architecture - default is i386
-s obsolete this tag in the meta-rpms (test this before using on a live repository, obsoletes can have interesting effects)