TWiki> EGEE Web>IntegrationProcedure>ProdRelCheck (revision 68)EditAttachPDF

Production Release Check List


The following list must be used in combination with the Integration Procedure twiki that describes all the technical details involved in the release process. This twiki aims to list the necessary steps to create a production release helping to avoid mistakes and decrease human errors.

Check List template

Please, print the following check list templates to go through the production release process:

Pre requirements

Access to machine lxbrf2713 where you have the necessary scripts to create repositories (

A machine to create the release notes where you should have:

yum install tidy                     # SL4/5 repository needed
cpan install HTML::TextToHTML        # Anwser "no" to the manual configuration. For all the other questions use the default answer and it should work
  • mysql-server
  • mysql client installed
  • java 1.5.0 (only SL4)
    • Check that you have the CERN-only repofile in /etc/yum.repos.d
    • yum -y install jpackage-utils
    • yum -y --enablerepo CERN-only install java-1.5.0-sun java-1.5.0-sun-devel
  • PyXML (only SL5)

Check List

1. Copy the production repo into the prepare area:

cp -pR /afs/ /afs/

2. For each patch in status Ready for Production:

  • Check in the list of associated bugs, if any, that the bug status is Fix certified or Fix not certified. Remove Invalid/Obsolete bugs. Contact certifiers if there are bugs in Ready for test since they should update this status after verifying the bug.

  • Check the release notes again and ask for an improved text to the patch submitter if the quality is poor or the explanations are not very useful.

3. Update the Release field of the patches with 3.1.0 Update *NN*.

4. In the machine where you are going to create the release notes populate the org.glite.integration.release/noarch/lists/metapackages.R3X.list file. The file should just contain a plain list of the affected metapackages in this update.

5. In lxbrf2713, run the script to create the production repository:  -r <release> -o <os> -a <arch> <patch1 [... patchN]>

6. In the release-notes machine, run the following script to create the xml file ( R3.X.0-NN.xml) that will contain the savannah information for every patch in the release:

./scripts/ -r 3.X.0 -x arch -u NN

You can check the spelling using:

aspell -c R3.1.0-04_i386.xml

It's a good idea to save the common words and put links to ~/.aspell.en.prepl and ~/.aspell.en.pws to somewhere in AFS to keep them between releases.

7. In the release-notes machine, launch the following script under org.glite.integration.release/noarch. It takes a long time to be executed and it will save you some time to run it at this point.

3.1 32bits: ./scripts/ --gliteVersion=3.1 --afsRepo=/afs/ 
                                                             --newReleaseDir=temp/newRepository_RpmList --metaPackFile=lists/metapackages.R3.1.list --os=sl4 --arch=i386 --reltype=prod

3.1 64bits: ./scripts/ --gliteVersion=3.1 --afsRepo=/afs/ 
                                                             --newReleaseDir=temp/newRepository_RpmList --metaPackFile=lists/metapackages.R3.1.list --os=sl4 --arch=x86_64 --reltype=prod

3.2 64bits: ./scripts/ --gliteVersion=3.2 --afsRepo=/afs/ 
                                                             --newReleaseDir=temp/newRepository_RpmList --metaPackFile=lists/metapackages.R3.2.list --os=sl5 --arch=x86_64 --reltype=prod

8. Copy the production repo in the prepare area to the production area:

cd /afs/
rm -Rf R3.X_old
cp -pR  /afs/ R3.X_new 

9. If the release affects the UI and/or the WN, copy the tarballs from the staged rollout area to the production area .

Copy from 

/afs/ or SL5_x86_64) 

/afs/,SL4_x86_64 or SL5_x86_64) 

/afs/,SL4_x86_64 or SL5_x86_64) 
/afs/,SL4_x86_64 or SL5_x86_64) 

10. Generate the Release Notes:

  • Format the file R3.X.0-NN.xml to improve the layout and the contents of the release notes.
  • Generate the release notes. Check with Operations whether the priority should be high in case any patch is marked with High in the priority field, otherwise it should be Normal:
  3.1 32bits : ./ --gliteRelease=3.1 --update=NN --patchFile=exportedfile 
                                                       --priority=Normal --arch=i386 --os=sl4 --reltype=prod

   3.1 64bits: ./ --gliteRelease=3.1 --update=NN --patchFile=exportedfile 
                                                       --priority=Normal --arch=x86_64 --os=sl4 --reltype=prod

   3.2 64bits: ./ --gliteRelease=3.2 --update=NN --patchFile=exportedfile 
                                                       --priority=Normal --arch=x86_64 --os=sl5 --reltype=prod

11. Upload the Release Notes in the web server by running:

  3.1 32bits :   ./scripts/ --gliteVersion=3.1 --user=CERN\\glbuild --arch=i386 --os=sl4 --reltype=prod

  3.1 64bits :   ./scripts/ --gliteVersion=3.1 --user=CERN\\glbuild --arch=x86_64 --os=sl4 --reltype=prod

  3.2 64bits :   ./scripts/ --gliteVersion=3.2 --user=CERN\\glbuild --arch=x86_64 --os=sl5 --reltype=prod

This will not overwrite the existing pages but will copy the new ones with different names (e.g. glite-BDII_new.asp)

12. Apply any modifications in R3.X.0-NN.xml file until you are satisfied with the release notes. Then regenerate the release pages with the --skipRPMgen option (since the part of the script that creates the list of released packages doesn't need to be repeated again). This is what you need to execute in each iteration until the contents of the release notes are OK:

  3.1 32bits : ./ --gliteRelease=3.1 --update=NN --patchFile=exportedfile 
                                                       --priority=Normal --arch=i386 --os=sl4 --reltype=prod --skipRPMgen 
                   ./scripts/ --gliteVersion=3.1 --user=CERN\\glbuild --arch=i386 --os=sl4 --reltype=prod

   3.1 64bits: ./ --gliteRelease=3.1 --update=NN --patchFile=exportedfile 
                                                       --priority=Normal --arch=x86_64 --os=sl4 --reltype=prod --skipRPMgen
                   ./scripts/ --gliteVersion=3.1 --user=CERN\\glbuild --arch=x86_64 --os=sl4 --reltype=prod

   3.2 64bits: ./ --gliteRelease=3.2 --update=NN --patchFile=exportedfile 
                                                       --priority=Normal --arch=x86_64 --os=sl5 --reltype=prod --skipRPMgen
                   ./scripts/ --gliteVersion=3.2 --user=CERN\\glbuild --arch=x86_64 --os=sl5 --reltype=prod

13. Add any known issues by editing manually the relevant metapackage pages. In order to do that you need to mount the gLite web page in your release-notes machine:

mount -t smbfs -o username=CERN\\glbuild // /mnt/glite_repository

3.1 32bits: vi /mnt/glite_repository/packages/R3.1/deployment/glite-metapackage/glite-metapackage-known-issues.asp

3.1 64bits: vi /mnt/glite_repository/packages/R3.1/x86_64/deployment/glite-metapackage/glite-metapackage-known-issues.asp

3.2 64bits: vi /mnt/glite_repository/packages/R3.2/sl5_x86_64/deployment/glite-metapackage/glite-metapackage-known-issues.asp

. 14. Do a final check of the contents of the release pages:

  • Main page:
    • gLite 3.1 32bits
    • gLite 3.1 64bits
    • gLite 3.2 64bits
      • Check that a summary for each patch is present.
      • Check the summary for each patch reflects the highlights of the patch and that contains a list of affected metapackages. Make sure the text is well presented and it's coherent within the context of the web page. Text in the Release Notes field of the Savannah patch is copied and pasted here and sometimes it needs some changes to make it fit within the overall documentation.
      • Check the list of patches is complete according to the original mail sent by Operations.
      • Check the priority of patches is correct. If a patch is priority High the corresponding affected services should be in red, stating High.
      • Check the new versions of the affected metapackages correspond with the output of print-release

  • Service specific update pages:
    • Check the release notes make sense for the service and that only the patches affecting the service are actually described.
    • Check the list of rpms is correct by comparing it with the rpm list in the relevant patches.
    • Check the list of bugs is correct by comparing it with the bug list in the relevant patches.
    • Check the URLs of the rpms are correct, in the update page and in the rpm list page (both HTML and txt).

  • Some known issues to be checked manually:
    • In the rpm.list.txt and html pages of the cream CE there is sometimes a typo on some rpm versions. Check there's no 1:. Ocurrences of 1: should be deleted.

15. Send a preview mail to project-eu-egee-middleware-emt and gd-release-team. In the subject write Preview 3.X Production Update XX. They'll check that everything is OK and they'll give the green light to publish the release. The mail should be something like:

Dear all,

Please find a preview of the production release notes here: and/or or

Any input/feedback provided as a reply to this mail would be valuable and is therefore welcome. 

The Integration Team

16. In case any of the patches provide a fix for a security issue identified by GSVS, contact Lind Cornwall to get the URL for the Security Advice. Modify the web pages to include this URL as soon as it's available.

17. Publish the new repository (every 30min past full hour glitesoft is synchronized):

cd /afs/
mv R3.X R3.X_old
mv R3.X_new R3.X

18. Publish the release notes by executing /mnt/glite_repository/packages/R3.X/((sl5)_x86_64)/ If there's some delay between the moment of publication and the moment of creation of the pages, please make sure the date associated with the update corresponds to the date when you are publishing it and not earlier.

19. Make sure there are no broken links (this has to be executed always in an SL5 machine):

  • Modify the corresponding noarch/scripts/check_links/R3.X_iarch.list where arch is either i386 or x86_64.
  • Execute ./ -r 3.X -a arch and check the results.
  • Fix any broken links.

20. For each release patch change the Savannah status to In production. Change the attached bugs status to Ready for Review. This can be done using the SCLI. Please, download the script in the location where you normally have org.glite.savannah.viewer/cli/:

svn co
cd trunk/

21. If there are new node types released, remember to:

  • Create the repo files under
  • Add in the Generic Install Guide any relevant information about the new node type where needed.
  • Add in the a new entry for the new node type.

22. Announce the Release using the CIC portal.

  • Go to the CIC portal and select Send a EGEE Broadcast.
  • Then select Broadcast Information from ROC section (although the broadcast form is the same for all).
  • Select News publication in all CIC portal views YES
  • Select CIC-on-Duty, OSG, ROC managers, VO managers, Production site admins.
  • Add in the CC:,,, gd-release-team.


 Release of UPDATE nn to gLite 3. x. Priority: [Normal/HIGH/**URGENT**] 


Dear members of the WLCG Grid Production Service,

UPDATE nn for gLite 3.x has been released to the WLCG Grid production service.
The priority of the updates is: [Normal/HIGH/**URGENT**]

The highlights of the update are:


All details of the update can be found in:
[COPY AND PASTE updates URL here]

Note that the UI and WN tarballs are available from the UI and WN pages.

Remember to report any issues with this set of updates using GGUS:

Best regards,
The gLite Middleware Team

23. Announce the release also in the LCG-ROLLOUT.

24. Put a symlink to empty in the beta repo for the released services.

Topic attachments
ISorted ascending Attachment History Action Size Date Who Comment
Microsoft Word filedoc Prod-check-list.doc r1 manage 72.0 K 2010-08-16 - 16:08 MariaALANDESPRADILLO Check list for Production Releases
Edit | Attach | Watch | Print version | History: r84 | r70 < r69 < r68 < r67 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r68 - 2010-08-18 - MariaALANDESPRADILLO
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    EGEE All webs login

This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright & by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Ask a support question or Send feedback