Building

Before doing any development or release, one shall be able to build the component in an environment, where it will be built in "official" builds.

For gLite R3.1 one shall use ETICS on any platforms.

On any Linux distributions, which are running a recent (>= 2.6.9) kernel one can set up both SLC4, SL5 and Debian test build environments via ChrootEnvs.

Building using ETICS

This build shall work on SLC4 and SL5 (both 32 and 64 bit) platforms with the default Workstation configuration.

    etics-workspace-setup
    etics-get-project org.glite
    etics-checkout --project-config=glite_branch_3_1_0 --ignorelocking org.glite.data
    etics-build --continueonerror org.glite.data

    etics-workspace-setup
    etics-get-project org.glite
    etics-checkout --project-config=glite_branch_3_2_0 --ignorelocking org.glite.data
    etics-build --continueonerror org.glite.data

ETICS Tips and Tricks

Please refer to ClientHowTo (and in general ETICS) on how to use the ETICS client tools.

However here are some ideas on how to make things smoother:

• separate the workspace from the client installation area that you can upgrade the client any time (you shall put the setting of ETICS_HOME and WS into your .bashrc script!):

        mkdir -p etics
        export ETICS_HOME=$PWD/etics
        mkdir -p ws
        export WS=$PWD/ws
        cd $ETICS_HOME
        wget "http://eticssoft.web.cern.ch/eticssoft/repository/etics-client-setup.py" -O etics-client-setup
        python etics-client-setup
        cd $WS

• separate the repository directory, thus it could be shared among workspaces

        mkdir -p repository
        sed -i.bak -e "s|^#repository|repository=$PWD/repository|" $HOME/.etics.conf

• avoid anonymous CVS check-outs, so you can commit in place:

        export CVSROOT=$LOGNAME'@glite.cvs.cern.ch:/cvs/glite'  # via SSH
        export CVSROOT=':gserver:glite.cvs.cern.ch:/cvs/glite'       # KerberosV

• ETICS modifications require X509 based authentication, so please register (see RegistrationHowTo) and set the appropriate environmental variables:

        export X509_USER_CERT=$HOME/.globus/usercert.pem
        export X509_USER_KEY=$HOME/.globus/userkey.pem   

• In order to satisfy external dependencies install oracle-client package from Cern repository which also will download occi compatible libstdc++ library, missing otherwise.

        yum install oracle-instantclient-basic

Development

The easiest way to do development is to do a build, then replace the checked out version of the desired component with the HEAD (or stable) branch from CVS:

ETICS

In an ETICS environment:

    cvs up -A org.glite.data.transfer-cli
    etics-build org.glite.data.transfer-cli

The default build option will try to re-build any dependencies, if they have been changed. You can switch it off by using the --nodeps option, however ETICS will still try to resolve all dependencies.

Theories

In theory there is a HEAD configuration in ETICS. However I would recommend not using it as ETICS HEAD configuration, because they are autogenerated at the creation of the component.

Please note that ETICS HEAD configuration is not the same as the CVS HEAD branch! While the source code in CVS HEAD should be usable there are no guarantees on the ETICS configuration.

Release

The steps of release are recommendations, not rules. However following them makes life easier!

There are helper scripts used, which are coming from org.glite.data/bin (HEAD). The easiest way is to add this directory of the checked out component to the PATH.

Component Release

Using Release Candidate Tag

A component (equivalent to a CVS module) is released by its developer or maintainer. The steps to be done:

1. update the version number in the project/version.properties or in the VERSION file!
   Please do not forget to reset module.age or RELEASE to '1', when any of the version numbers are changed! (See below other rules at 'Version Numbers')

For the following steps one can call the release-rc script to print out an aid or TODO list with the exact commands to be executed. (In case of any difference, the output of that target has preference over this document.)

1. Update the RELEASE-NOTES or CHANGES file.
   To help here the release-rc script will run cvs2cl to generate a draft ChangeLog file, what you can use to edit the RELEASE-NOTES.
2. Run a full build: whatever you need to see if this package actually builds.
3. Update the runtime dependencies. For ETICS this is the package.requires property in the project/e-c_*.ini file.
4. Commit all pending changes
5. Tag the baseline with a glite-data-_R_1_2_3_4 style CVS tag
6. Generate an ETICS configuration using ../org.glite.data/bin/e-gen-config

Example:

   cd org.glite.data.transfer-cli
   release-rc
   gvim -o RELEASE-NOTES ChangeLog
   cvs commit -m 'release candidate: glite-data-transfer-cli_R_3_4_0_4'
   cvs tag -c glite-data-transfer-cli_R_3_4_0_4
   cd ..
   etics-configuration add -i org.glite.data.transfer-cli/glite-data-transfer-cli_R_3_4_0_4.ini

Rationale:

In most cases the only information what changes from one release to the other from packaging point of view is the version number. Neither the name, run time or build time dependencies change very often.

So this release procedure is built on the principle that you only have to change the version number and everything else should be automatically generated from that.

You may try to use the ETICS tools to clone an existing configuration (in other words making a copy with a different name), however then you will have to edit the configuration name and version numbers about seven places consistently using the command line or the web interface.

Susbsystem Release

The subsystem release internal procedure is an augmentation of the EGEE/gLite procedure, described in the Developer's Guide.

For each official release there is a release branch of the 'org.glite' base component, it is usually called glite_branch_#_#_0, for example glite_branch_3_1_0.

For each official release there are two subsystem branches:

• glite-data_branch_#_#_0: which contains the tags for the subsytem release. It is created by branching HEAD:
  cvs rtag -b -r HEAD glite-data_branch_3_1_0 org.glite.data

The outline of the release process is that one sets the precise component versions in the glite-data_branch_#_#_0 branch, then updates the 'tree' branch with the proper component branches (more later).

1. Create a subsystem build environment (see above) using ETICS. In the ideal case you shall have the following environments ready to test the release:
   • SLC4 with ETICS
   • SLC4/x86_64 with ETICS
2. Update the org.glite.data module to use the branch:
   cvs co -r glite-data_branch_3_1_0 org.glite.data
3. Update the version of the subsystem and branch:
   cd org.glite.data
$EDITOR project/version.properties
module.version=3.1.18
   
4. Add the new release candidate tags:
   $EDITOR project/e-c_org.glite.data.ini
LCG-DM                                 = LCG-DM_R_1_6_4_3
org.glite.data.util-c                  = glite-data-util-c_R_1_6_4_3
   
5. Generate draft release notes by scanning the differences in the RELEASE-NOTES files between the new release candidate tags and the already released tags:
   ./bin/generate-release-notes
   This script will generate a RELEASE-NOTES. file, which shall be edited by hand for reformatting entries (see RELEASE-NOTES as an example for the style) and removing the '@' lines.
   gvim -o RELEASE-NOTES RELEASE-NOTES.3.1.18-1
   
6. Update metapackages and Quattor templates based on the new version numbers from the ETICS configuration:
   ./bin/ec2
7. Commit all changes before a test build:
   cvs commit -m '3.1.18-pre1'
   
8. Commit the changes and tag the subsystem:
   cvs commit -m '3.1.18-1'
cvs tag -c glite-data_R_3_1_18_1
   
9. For ETICS you also need to add (or upload) the generated ETICS configurations. The ./bin/e-gen-config . command will print out these commands for your convenience:
   cd ..
etics-configuration add -i org.glite.data/glite-data_R_3_1_18_1.ini
   =etics-commit=
   
10. Lock the configuration: etics-configuration lock --parent-config glite_branch_3_1_0 -c glite-data_R_3_1_79_1 org.glite.data
11. Submit a remote build: etics-submit build --register --platform slc4_x86_64_gcc346,slc4_ia32_gcc346 -c glite-data_R_3_1_79_1 org.glite.data
12. Announce the new release on the integration and on our internal mailing list.
13. Update the bug descriptions in Savannah: log into the EGEE Savannah page and execute the following script:
    ./bin/release-notes-to-savannah RELEASE-NOTES.1.6.2
    It will scan the file, and will open the bugs one-by-one in the browser, while displaying a text, which can be copy-n-pasted into the comment field of the bug.
    The script checks the $BROWSER environmental variable, which can be set to any web browser, if you are not using Mozilla Firefox.

Version Numbers

Interface Components

Interface component (...-interface): basically only the major, minor and release numbers are changes. Patch version stays as 0.

• Increase major version, when the interface changes in an incompatible way (field removed, method deleted or renamed). It is preferred not to do it often, i.e. when a method is renamed, the "old" one should be kept and the new one should be introduced with a new name.

• Increase minor version number, when the interface changes in a backward compatible way, i.e a new method is added or a class is extended with a new, optional field.

• Increase the release number, when there is a change in the build files or documentation. It shall not affect the API.

Autogenerated Library

Autogenerated Library component (...-api-...): version numbers shall not be modified, except the release number!

The autogenerated components inherit the version number of the corresponding interface (WSDL), thus their own version number is irrelevant and should not be set (usually 0.0.0).

However, when the building process or dependencies change it shall be reflected in the generated packages, thus the release number (module.age) has to be increased.

Normal Component

Implementation component: normally the major and minor version number is kept the same inside a release, only the patch and the release number changes in a branch.

• Increase the major version number, when there are major or incompatible changes.

• Increase the minor version number, when there are changes in the configuration, or there is a minor functionality change. It is recommended to increase the minor version number between gLite releases.

• Increase the patch version number, when a bug is fixed, or there is any change to the sources (code or documentation).

• Increase the release version number, when there is a runtime dependency update or build change. In general this number is to be used for integration related changes only.

If there is a situation, which is not covered by these cases, then please ask!

References

Official builds logs on SLC4 and SLC4/64.

Data management related bugs and patches.

Last edit: PedroAndrade on 2010-02-17 - 14:33

Maintainer: AkosFrohner