Node Type Builds [ETICS Approach]

Requirements

1. Define a node type as a set of component configurations (to be built with their internal and external dependencies).
2. Create APT repositories out of a build outcome for deployments.
3. Define in some way a custom set of configurations from specific components (with dependencies) to be checked-out for development-testing purposes
4. Generate meta-packages automatically according to the ETICS definition already present for building
5. Multi-platform support
6. (Assumption??) Keep site administrator maintenance easy with only 1 repository for the whole gLite and not one per node type

First Possible implementation

Use the ETICS parent-child relation to define sets of component configurations (https://twiki.cern.ch/twiki/bin/view/EGEE/NodeTypeBuilds)

Second Possible implementation

Use the ETICS dependency relations to define sets of component configurations.

This implementation allows the whole project configuration to be kept together and adds some subsystems with custom meaning to achieve the requirements above. Inside each of these new subsystems there will be components that define sets of configurations with specific meanings. Doing so it is possible to generate meta-packages, to define node-types and to let the developers create their custom set of configurations and external dependencies to work within their workspaces.

Recommended operations

Before illustrating the method, there are some recommended operations to do:

• Clean the gLite project configurations (i.e. glite_branch_3_1_0), removing all subsystems/components that are not needed for the build.
• Start to version and lock the gLite project configurations every time a change is done. Further details in the Software Process section.

Steps to create sets

In order to create these custom sets of configurations, the following steps are needed:

• Create several subsystems with categories of sets. For instance 'metapackages' or 'node-types' or 'development-sets' or any name you like.
• Create components under each subsystem with the name of the particular group of configurations you need to build. This can be a node-type, a different view of the project for development, or any other set of configurations to build together. For instance 'WN' and 'UI' under 'metapackages', or 'WMS-Dev' or even 'Francesco-Dev' under 'development-sets'.

What is important is that within the same module (Project/Subsystem/Component) all the configurations are related to the same piece of software and have the same meaning. Configuration names define different versions of software within a module and not different modules. For instance, it is good to create configurations such as WN_3_1_27 and WN_3_1_29 under WN and UI_3_2_45 under UI. Instead configurations like metapackage_WN_3_1_2 and metapackage_UI_3_1_3 under the module 'metapackage' define different software within the same module and should be avoided. This heterogeneity creates problems in version comparison and when two configurations have the same versions but different meaning. What would happen in case two RPM with the same package name end up in an APT repository??

• Create any configuration you want under those just created components to define a particular version of that module.

• Select the components to group together using dynamic dependencies. These will be resolved according to the project and will follow the project evolution.

For all external dependencies that do not match the project definition (to be considered EXCEPTIONS ), the property 'module.DEFAULT' can be specified in the component itself and will overwrite the project definition only for that set. For internal dependencies that do not match the project definition (also to be considered EXCEPTIONS ), static dependencies can be set.

• Decide whether some subsystem/component configurations may be included in the project configuration or not. For instance I would include the metapackages to be built together with the project but not custom development-sets.

• Grant custom permissions to delegate meta-package definitions and/or to allow developers to create/clone their custom workspaces.

Outcome

Integrators can build node-types separately by building the specific component configuration and generate automatically APT repositories.

Developers can checkout and build their custom definitions, overwriting all (and only) the pieces they want to be different from the project configuration.

In this way, metapackages are automatically created by ETICS simply by building the component configuration or even by building the whole project configuration.

This allows permissions to be granted separately for metapackages, node-types and developer-sets, without giving full access to the project.

No duplication of data.

No refactoring needed.

The software process

1. Development phase.
2. Integration
3. Certification
4. Move Patch to PPS/Prod
5. Manage repositories

Development phase

Developers can use one or more custom project configurations to developed new components. These configurations can be built against one of the main project configurations (integration, certification, PPS, release) or cloned from any of them.

This allows developers to create their own custom workspaces with all and only the software they want in the combinations they want.

Integration

Once one or more changes in the development branches are ready, they are communicated to the integration team. The integration team combines the changes in the integration branch (according to their integration needs) and does smoke test building. If the smoke tests are passed, the integration configuration is cloned and passed to certification with a meaningful name representing content and status, for example giving it the name of a patch and a suffix '_cert'.

Certification

The configurations ready for certifications can be (re)built and the results (the metapackages) tested by the testers. This can be done either by building the whole project configuration or just the affected note-types. Once certification is done, the configuration is ready for pre-deployment (PPS).

Move Patch to PPS/Prod

The configurations coming from certifications must be integrated with the current PPS configuration to guarantee that it represents what is currently in PPS. The new cofniguration can be locked at this point or later, better at this point. It should also be given now a release number since it becomes a Release Candidate.The configurations and all attached metapackages (if affected by the original change) can be tested in PPS and finally released for production. At this point is mandatory to rename the configuration with a final release name and lock.

Manage repositories

At any stage in the process the APT repositories are automatically updated to contain the packages ready for the various stages (integration, certification, PPS, release). In addition, the ETICS volatile repositories can be used to create custom sets of RPMS. There should be default volatile repositories for the various stages plus as many custom repositories as necessary.

Example Update

Example: starting point is an open development branch:

org.glite ---> glite_branch_for_WMS_development

Once a given change has been developed and locally validated, it is sent to the integration team. The integration team takes the change and integrates it in the main integration branch

org.glite ---> glite_branch_3_1_0 (integration branch)

This is not what is in production, it is the starting point of the certification process. The required smoke integration tests are done (essentially builds and deployment tests). When ready the branch is cloned and given an appropriate name to represent its content and status.

org.glite ---> glite_branch_3_1_0 --> glite_3_1_0_patch_2345_cert

The new configuration is (re)built if necessary, metapackages produced, APT repositories generated. The configuration (which now contains the base + the patch delta) is moved through the process until released. When it is ready for release it is renamed and locked:

org.glite ---> glite_branch_3_1_0 --> glite_3_1_0_patch_2345_cert --> glite_3_1_0_28_pps --> glite_3_1_0_28

The final locked configuration is also the reference configuration for any later bug fix of components locked with it. Note that any of the project configurations in the various stages can be used to build against by developers, depending of what type of validation is required

Problems using Parent-Child relationship instead of Dependency relationship

Duplication of data

1. Duplication of External dependencies (module.DEFAULT properties) in each project configuration leads to errors and low maintainability. The external dependencies should be defined in a single point.
2. Duplication of subsystem configurations leads to errors and low maintainability. There should only be a subsystem with all the components and not N subsystem configurations, one per node-type. Components required by more than one node-type need to be under multiple subsystems and maintained at every update
3. Configurations with different purposes should not be under the same module because it is not possible to compare their versions (i.e. glite_WN and glite_UI). They should be organized in different modules, each having only configurations comparable by version (therefore with the same meaning). Not good to have different configurations with the same version.
4. Problems in locking of components that are part of multiple node-types. What project-config to use?? It is not guaranteed that they are they all the same.

Refactoring required

1. Moving external dependencies inside the gLite project is a lot of work and does not solve the problem because all project configurations will have exactly the same parent-child relation for each ex-external dependency considering most of those dependencies are used by many nodes. It will also move the maintenance from ETICS to gLite.
2. Removing subsystems will remove the hierarchical organization and the project will contain an infinite list of components without particular grouping (even worse if all the externals are mixed with gLite components)

Security

1. Using project configurations instead of component configurations limits the granting of permissions. With different subsystem such as 'metapackages' and for instance 'developer-sets' it is possible to grant permissions on single components instead of having to grant permissions on the project ( everybody at the end will be able to change anything they like ). Giving permissions per configuration has problems with cloning. Giving permissions on custom created components is the best way of permission management.
2. Removal of subsystems comports problems in permissions ( permissions must be granted to a lot of components).

Missing requirements / features

1. No meta-packages can be generated automatically together with the build process.
2. It will be not possible anymore to build the whole project and it will no longer be automatic to install multiple services in the same node (versions of internal and external dependencies may be different per node-type)
3. No builds per subsystems anymore available

Advantages using Parent-Child relationship instead of Dependency relationship

• It is possible to completely decouple node-types with their internal versions and external dependencies.

This can also be done with the other implementation by setting static dependencies or redefining .DEFAULT properties in the metapackage configuration.

It is not as dynamic as having different configurations but we consider this an advantage because these cases should be considered exceptions and solved as soon as possible by the integration team.

The complete decoupling has problems with a single APT repository setup because only the latest versions of packages will be installed.

A complete decoupling of configuration leads to a setup of multiple APT repositories, one per node-type. This highly affects Site administrators' ease of maintenance. This decoupling simply moves the problem from configuration stage to deployment stage and delays the discovery of problems.

Problems still to be solved in any case

• The ETICS data model does not allow tracking of packages for components that generate more than one package. This issue can be solved with virtual-packages having other components link each package generated by the original configuration. Virtual-packages require maintenance at every update of the original configuration.

Additional Features

• As discussed in the past, the ETICS system will include a release management tool to facilitate the implmentation of the described process. The transition of a configuration through the various stages (integration, certification, pps, release, or more traditionally, alfa, beta, RC, Release) will be made automatic by hiding the steps of cloning, renaming, locking, etc. It will also allow better tracking and the (semi)automated generation of release notes.