Node Type Builds

The current state of the node type builds can be seen on the Integration Dashboard.

Motivation

During EGEE-II, service based releases coupled with a continuous release process has increased the efficiency and effectiveness of the gLite releases. Decoupling the release cycles of the different services enables them to be released when ready and not held back by unrelated issues. Updates enabled the services to be continuously improved responding to the production feedback in a timely manner. The time and effort required to provide an update is now only dependent on the complexity of that update and is independent of any other updates. While this process has proved effective, it is still not clear how the process works with multi-platform support

Being able to "build" a node type will go a long way to achieve multi-platform support. However, as the node type definitions are currently maintained out side of ETICS, it is impossible to "build" a node type for other platforms. It must be understood how to achieve node type builds, how the release process is expected to work with them, and how this can be implemented in ETICS. There is a general consensus to move towards node type builds and a number of methods to realize this in ETICS have been discussed.

Method

A project configuration is used to define a list of of components and versions of those components. For externals this is done by setting this as a property of the project configuration. For project components the sub configurations functionality must be used. A subsystem node contains a list of components that represent the node type build. The dependencies of this component should reflect the primary components that make up that node type. The node type is built using the project configuration. To how to update a component depends on the type of update. The different scenarios are described below.

Updating a component project wide across all platforms

To update a component project wide across all platforms, the project configuration should be updated. For external components this requires updating the property and for project components this requires updating the sub configurations.

Updating an external component project wide on one platforms

To update an external component project wide on one platform, the project property should be updated for that platform

Updating a component project wide on one platform

To update a component project wide on one platform, ether a new configuration needs to be created or the checkout field for that platform should be modified to use a different CVS tag.

Updating component for one node type on all platforms

To update a component for one node type on all platforms, update the property of the node type.

Updating component for one node type on one platforms

To update a component for one node type on one platforms, update the property of the node type only for that platform.

The updated process

Two main project configurations are used; prod and pps. The prod lists represent the components in production and should be version with the production update number. The PPS lists represent the components in Pre-production and should be version with the PPS update number. These lists are NOT used for the primary builds. Their main purpose to generate the list of components to be placed in the production and PPS repositories. They will also be used for porting activities where a third party may wish to build a node type for a particular platform. However, this should not be confused with multi platform support. A developer should work on their sub system against the prod project configuration. They may build and lock their components against this project configuration. For a major change a dev project configuration can be used which my involve many subsystems.

In order to introduce a change (patch), the production project configuration should be cloned, renamed to the patch number and the update applied to this configuration. If not already done so the components, via a node type build against this patch configuration can be built and locked. This approach may be necessary if the change affects multiple subsystems.

The patch configuration can be used to create a patch repository for the node types to be tested. Once certified, this patch can be merged with either th PPS or Prod project configuration. To do this the project configuration must first be cloned and renamed to included the new update number before the change is applied.

Requirements

Merging tool

A tool will be required to simplify the activity of merging the changes from the path project configuration into the prod or pps project configuration.

Patch concept in ETICS

The concept of a patch need to be introduced. The requirements for this will be come more clear after gaining experience with the node type build approach.

Removal of subsystems

In many cases, the use of subsystems is an unnecessary administration overhead. If such scenarios are identified, the components should be move out of the subsystem.

Component and configuration naming

The name of components and configurations should reflect the packages they create. The configuration name should be of the form "package-name_R_x_y_z_a".

Components that create multiple packages.

An effective way is required to manage components that create multiple packages.

Node Type Builds Scenarios

The following is a list of update scenarios and how they should be handled in ETICS using node type builds.

Update a component used on one or many node types

1. Create new configuration for the updated client
2. Clone the latest prod configuration for the subsystem and rename to include the patch number
3. Update the subsystem configuration to include the updated client
4. Clone latest project configuration and rename to include the patch number
5. Update the project configuration to include the new subsystem configuration
6. Certify the patch
7. Rename/Merge the subsystem configuration to production update number.
8. Clone the production configuration and rename to increment the update number
9. Update the production configuration to include the new subsystem configuration with that update number.

Example: Update glite-security-lcas on many node types

Update an external component used on many node types

1. Create new configuration for the external component
2. Clone the latest project configuration and rename to include the patch number
3. Update the project configuration properties to include the external component
4. Certify the patch
5. Updated the production configuration properties to include the external component

Update a component on only one node type

1. Create new configuration for the updated component
2. Clone the node type configuration and rename to the patch number
3. Add the configuration to the properties of the node type
4. Certify the patch
5. Rename/Merge the node type configuration to production update number
6. Clone the production configuration and rename to increment the update number
7. Update the production configuration to include the new subsystem configuration with that update number.

Update an external only for one platform

1. Create new configuration for the external component
2. Clone the latest project configuration and rename to include the patch number
3. Update the project configuration properties for that platform to include the updated external component
4. Certify the patch
5. Updated the production configuration properties for that platform to include the external component

Update a component only for one platform

1. Create new configuration for the updated component
2. Edit the checked command for that platform to use a difference tag.
3. Clone the latest prod configuration for the subsystem and rename to include the patch number
4. Update the subsystem configuration to include the updated client
5. Clone latest project configuration and rename to include the patch number
6. Update the project configuration to include the new subsystem configuration
7. Certify the patch
8. Rename/Merge the subsystem configuration to production update number.
9. Clone the production configuration and rename to increment the update number
10. Update the production configuration to include the new subsystem configuration with that update number.