Integration Guide

Temporary Disclaimer

This page is a draft of the 'gLite Integration Guide'.

It is still under discussion and feedback is sought.

This will probably become a substantial part of the developer guide.

Introduction

The document describes how the gLite build and integration process works, with direct reference to the ETICS service used. It is supposed to be used in conjunction with the ETICS user guide and endeavours to explain to those involved in building middleware for release (developers and SA3 builders) how they should use ETICS and how gLite integration works.

gLite

What is gLite? Why do we have an integrated distribution? How is it defined? What are our reference platforms?

Build management

The project definition, glite_branch_3_1_0 , is maintained by the SA3 integration team. The glite_branch_3_1_0 is intended to be as close as possible to the latest production release. Thus, building against it is conceptually equivalent to installing an up to date build machine with production rpms and then building your source rpms there.

The SA3 integration team has responsibility for the project definition, setting defaults, maintaining the build and enforcing patch acceptance criteria.

Motivation

• Building your software in the environment where you want to install and run it is a good way to make something that works
• The voms 1.8 story - ETICS by default used to set runtime dependencies to be the same as what was actually used in the build. Because of this, we must ensure that builds are performed against what is in production or else the results will not be installable. While ETICS now offers the opportunity to override this behaviour, it will require all developers to establish the minimum required versions of their dependencies and explicitly set these as runtime deps. This process will take some time.
• Enabling the 'developer build'

Nightlies

A nightly build of glite_branch_3_1_0, as it is defined, will be performed, but this is not used to build releases nor to test new code. The purpose of defining it as a nightly build is to ensure that updates do not compromise the project definition and, in consequence, other builds. It is a consistency check.

glite_branch_3_1_0_dev is maintained to hold the latest tags of interest to developers, ie the configuration currently 'in work' and which is likely to subsequently be locked and used for a release candidate build. These ETICS configurations can use CVS branch tags for convenience, but in this case these must be substituted for standard tags before release. Any build related bugs can be considered fixed in this build. This build will not be used to create releasable software but provides an automatic build of the latest tags against new versions of dependencies, enabling anticipation of problems. Note that the tags contained in glite_branch_3_1_0_dev will be used to create releasable software, when locked and built against glite_branch_3_1_0. Developers can always build any configuration against glite_branch_3_1_0 in the usual way, and such a build could even be set up as a nightly if desired.

When a release is required, the 'in work' configuration in glite_branch_3_1_0_dev should be locked and built against glite_branch_3_1_0. The next configuration to be 'in work' should be communicated to the integration team so it can be added to glite_branch_3_1_0_dev. When the rpms are certified and released to PPS, the appropriate configuration will be transferred by the integration team to glite_branch_3_1_0.

Note - as ETICS defines runtime dependencies by default as '>=' the version of a library you build against, developers are encouraged to explicitly set the real runtime dependency in their ETICS configurations (which is normally '>=' an older library version). This will make sure that the release of the new build is not dependent upon the successful certification of the libraries concerned.

Release Workflow

This is what happens to the project definitions when components are released

1. ETICS subsystem config, S1, belonging to glite_branch_3_1_0_dev, locked
2. Build of subsystem S1 performed against glite_branch_3_1_0
3. < ETICS deployment test run >
4. Patch submitted, referencing artifacts
5. glite_branch_3_1_0_dev updated to S2 (unlocked)
6. Patch certified
7. glite_branch_3_1_0 cloned to produce a dated backup of the configuration
8. glite_branch_3_1_0 updated with S1
9. Patch released to validation service (PPS)
10. Patch released to production

"Developer Builds"

This is the mechanism by which a developer or subsystem administrator can produce releasable components. A release candidate build is perfomred in a controlled environment (ie is a remote build), its products cannot be changed and its build log is archived with the artifacts.

Motivation

This approach will

• extend the scope of testing by ensuring the same rpm is tested by developers as well as by certification
• remove the overhead of an expensive cross-activity process just to identify trivial errors
• reduce problems due to multiple/unreproducible builds
• speed up the process by eliminating communication delays
• allow build problems to be addressed directly by those most knowledgeable about the component.

Standard Use Cases

The following diagram illustrates the simple use case of a single component being affected.

It can be extended to the case of a single subsystem.

Other scenarios

< from Francesco, to be completed >

1. working on the next version of a piece of software that depends only on components already part of glite_branch_3_1_0. Talking to several people, it seems that everyone has devised a procedure, I wonder if we could come up with a common one.

1. working on the next version of a piece of software that depends on components that are not yet part of glite_branch_3_1_0. Such components could be themselves under development.

1. rebuild a particular rpm, changing one external dependency.

ETICS advice

< awaiting input from Alberto here >

how to create a suitable workspace

how to update it if there are changes in dependencies (especially in the second case)

how to do remote builds

how to work efficiently with ETICS, using update, merge and caching features

A note on build structure

Both ETICS and CVS are arranged on a subsystem basis. To work best, a subsystem should

• Group together components with synchronised evolution, where updates are frequently correlated
• Contain no other unrelated components, as this imposes an artificial connection and prevents independent evolution

Unfortunately, the subsystems as currently defined typically break both these rules, as for historical reasons they are arranged along administrative lines.

SA3 encourages developers to investigate the benefits of rearranging subsystems (the reorganisation of security can be taken as a model).

Patch acceptance

Patches will be accepted if the packages are

• Built remotely
• Built against the correct project definition
• Built with appropriate command line args (not overriding deps etc)
• Configuration locked (this is a pre-requisite for registration anyway)
• $ etics-submit build  --project org.glite ---project-config glite_branch_3_1_0 --register --platforms slc3_ia32_gcc323,slc4_ia32_gcc346,slc4_x86_64_gcc346 --continueonerror --versioneddeps -c
• Can we prove that the build did not override project level default dependencies?
• The ETICS configuration should not reference branch tags

Nota Bene...

• A rebuilt binary will result in the testing being started again from scratch, even if the source hasn't changed

Porting and Multiplatform support

We want to move certain service/platform combinations from 'porting' to 'multiplatform'. For 'multiplatform' the standard project configuration is used for the build, all necessary fixes are in the codebase, and support is at the same level as for any other part of gLite. The 'porting' status is basically where we are now for everything not SL3/SL4, where the porting coordinator maintains special patches and builds for the platform, and it is not supported.

Handling non backward compatible changes

This can be

• ABI change
• non backward compatible API change
• change in an component, possibly external, which requires simultaneous rebuilds across components (eg gSOAP)

The common point is a requirement for coordinated builds, and then simultaneous certification and release of affected components.

1. Update glite_branch_3_1_0_dev with the new library
2. Developers must get all 'in work' configurations there to build against the new library
   • This is not necessarily much work (eg an ABI change typically just needs all dependees to be rebuilt, no source changes are required)
3. glite_branch_3_1_0 updated with new library
4. All affected subsystems are locked and built against glite_branch_3_1_0
5. Everything is certified simultaneously
6. Configurations are moved to glite_branch_3_1_0 when certification is complete

Note that this process puts glite_branch_3_1_0 temporarily into a 'vulnerable' state as builds against it are no longer immediately releasable.

How to do a quick update when glite_branch_3_1_0 is temporarily 'ahead' of production

It can happen that a build against glite_branch_3_1_0 will not always produce an immediately releasable rpm, for example in the case of handling a non backward compatible change. In this case, if a quick release is required, the integration team will identify which of the cloned and dated project configurations corresponds to the one used to produce the rpms which must be updated. This can be done by looking at the build date of the rpms themselves. A build can then be arranged of the appropriate ETICS configuration (presumably cloned from the original and referencing a CVS patch branch) against the old project definition.

Patch tactics

Some remarks on patch composition and tactics

• A single patch may affect multiple node-types, this is fine - only one patch is required
• Where a change affects multiple platforms, we need one patch per platform (to allow prioritised release per platform)
• Where possible, patches should not share rpms with other patches. A current example at time of writing is the CREAM CE which originally included lcas/lcmaps and glexec rpms, both of which had updates already in the process. The dependency of CREAM on these other packages should
  1. already be encoded in the rpms in any case
  2. be recorded by putting a savannah dependency between the patches in question (with a comment indicating the sense of the dep, ie CREAM needs glexec, but glexec doesn't need CREAM). In this case the certification team will make sure the patches are installed together where appropriate.