User Manual

Etics Service V. 2.0 (revision 2.0.1)

INFSOM-RI-026753 2007 © Members of ETICS collaboration 1 / 145 ETICS USER MANUAL

Copyright (c) Members of the ETICS Collaboration. # See http://www.eu-etics.org/etics/partners/ for details on the copyright holders. ETICS (“E-Infrastructure for Testing, Integration and Configuration of Software”) is a project funded by the European Union. For more information on the project, its partners and contributors please see http://www.eu-etics.org . You are permitted to copy and distribute verbatim copies of this document containing this copyright notice, but modifying this document is not allowed. You are permitted to copy this document in whole or in part into other documents if you attach the following reference to the copied elements: "Copyright (C) # Members of the ETICS Collaboration. http://www.eu-etics.org ". The information contained in this document represents the views of ETICS as of the date they are published. ETICS does not guarantee that any information contained herein is error-free, or up to date. ETICS MAKES NO WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, BY PUBLISHING THIS DOCUMENT.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 2 / 145 ETICS USER MANUAL

Content # INTRODUCTION .......................................................................................................................................... 6 1.1. OVERVIEW.................................................................................................................................................. 6 1.2. HOW ETICS REPRESENTS SOFTWARE INFORMATION ............................................................................... 8 1.2.1. Modules: Projects, Subsystems and Components............................................................................... 8 1.2.2. Configurations ................................................................................................................................... 10 1.2.3. Platforms............................................................................................................................................ 12 1.2.4. Commands and Properties ................................................................................................................ 13 1.2.5. Dependencies..................................................................................................................................... 15 1.2.6. Commands and Properties Search Order for Multiple Platforms ................................................... 16 1.2.7. Property Inheritance ......................................................................................................................... 16 1.2.8. Locking............................................................................................................................................... 17 1.3. AUTHENTICATION, AUTHORIZATION AND ROLES ................................................................................... 17 # ETICS PORTAL........................................................................................................................................... 20 2.1. OVERVIEW................................................................................................................................................ 20 2.2. LAYOUT.................................................................................................................................................... 20 2.3. PANEL – MYETICS.................................................................................................................................. 20 2.3.1. Welcome............................................................................................................................................. 21 2.3.2. My submissions.................................................................................................................................. 21 2.4. PANEL – BUILD AND TEST SYSTEM .......................................................................................................... 21 2.5. PANEL – REPOSITORY .............................................................................................................................. 21 2.6. PANEL – ADMINISTRATION ....................................................................................................................... 21 2.7. PANEL – EXTERNALS ................................................................................................................................ 22 2.8. PANEL – PROCESS NEW EXTERNAL COMPONENT REQUEST ...................................................................... 22 # THE ETICS BUILD AND TEST WEB APPLICATION ........................................................................ 24 3.1. OVERVIEW................................................................................................................................................ 24 3.2. LAYOUT.................................................................................................................................................... 25 3.3. ENABLING SECURITY ............................................................................................................................... 26 3.4. CERTIFICATE REGISTRATION ................................................................................................................... 27 # THE ETICS COMMAND-LINE CLIENT................................................................................................ 29 4.1. OVERVIEW................................................................................................................................................ 29 4.2. HOW TO INSTALL THE ETICS CLIENT ..................................................................................................... 29 4.3. HOW TO CONFIGURE THE ETICS CLIENT ................................................................................................ 30 4.4. WORKSPACES ........................................................................................................................................... 32 4.5. ETICS COMMANDS AND LIBRARIES ........................................................................................................ 32 4.6. ENABLING SECURITY ............................................................................................................................... 33 # BROWSING MODULES AND CONFIGURATION INFORMATION ............................................... 34 5.1. OVERVIEW................................................................................................................................................ 34 5.2. HOW TO BROWSE WITH THE WEB APPLICATION ..................................................................................... 34 5.2.1. Selecting a Project............................................................................................................................. 34 5.2.2. Browsing a Project, Subsystems and Components ........................................................................... 34 5.2.3. Searching Configurations.................................................................................................................. 36 5.2.4. Adding configurations to the Workspace .......................................................................................... 37 5.2.5. Browsing Configurations and Sub-Configurations .......................................................................... 37 5.2.6. Listing supported Platforms .............................................................................................................. 38 5.2.7. Viewing Commands, Properties and Dependencies ......................................................................... 39 5.3. HOW TO BROWSE WITH THE COMMAND-LINE CLIENT ............................................................................ 40 5.3.1. How to Get a Project......................................................................................................................... 40 5.3.2. How to Checkout and Browse Configurations.................................................................................. 40

INFSOM-RI-026753 2007 © Members of ETICS collaboration 3 / 145 ETICS USER MANUAL

5.3.3. How to Show the Structure of Modules and Configurations?.......................................................... 41 5.3.4. Modules, Platforms, Users, Roles and Other Objects ...................................................................... 42 # CHECKING OUT PROJECT, SUBSYSTEMS AND COMPONENTS................................................ 43 6.1. OVERVIEW................................................................................................................................................ 43 6.2. SOURCE CODE OR BINARY PACKAGES?................................................................................................... 43 6.3. THE ETICS-CHECKOUT COMMAND ........................................................................................................... 43 6.4. HOW TO CHECKOUT SOURCE CODE OR BINARY PACKAGES ................................................................... 45 6.4.1. Merging Configurations .................................................................................................................... 46 6.4.2. Updating Configurations................................................................................................................... 46 6.4.3. Forcing Checkout .............................................................................................................................. 47 6.4.4. Permanent and Volatile Repositories................................................................................................ 47 # EDITING PROJECTS ................................................................................................................................. 49 7.1. OVERVIEW................................................................................................................................................ 49 7.2. HOW TO EDIT WITH THE WEB APPLICATION ........................................................................................... 49 7.2.1. Editing Modules................................................................................................................................. 49 7.2.2. Editing Configurations ...................................................................................................................... 51 7.2.3. Editing Sub-Configuration Relationships ......................................................................................... 55 7.2.4. Attaching Platforms to Configurations ............................................................................................. 57 7.2.5. Editing Commands ............................................................................................................................ 58 7.2.6. Editing Properties and Environment ................................................................................................ 58 7.2.7. Editing Dependencies........................................................................................................................ 59 7.3. HOW TO EDIT WITH THE COMMAND-LINE CLIENT .................................................................................. 60 7.4. THE ETICS-MODULE COMMAND .................................................................................................... 63 7.4.1. How to prepare a module.................................................................................................................. 64 7.4.2. How to add a module......................................................................................................................... 65 7.4.3. How to modify a module.................................................................................................................... 66 7.4.4. How to rename a module................................................................................................................... 67 7.4.5. How to remove a module................................................................................................................... 67 7.5. THE ETICS-CONFIGURATION COMMAND .................................................................................................. 68 7.5.1. How to prepare a configuration........................................................................................................ 69 7.5.2. How to add a configuration............................................................................................................... 72 7.5.3. How to clone a configuration............................................................................................................ 73 7.5.4. How to modify a configuration.......................................................................................................... 73 7.5.5. How to remove a configuration......................................................................................................... 74 7.5.6. How to rename a configuration ........................................................................................................ 74 7.5.7. How to lock a configuration.............................................................................................................. 75 7.6. THE ETICS-COMMIT COMMAND..................................................................................................... 75 # TAGGING CONFIGURATIONS............................................................................................................... 76 8.1. OVERVIEW................................................................................................................................................ 76 8.2. THE ETICS-TAG COMMAND ...................................................................................................................... 76 8.2.1. How to Tag a Configuration ............................................................................................................. 77 8.2.2. How to Tag a Configuration Tree..................................................................................................... 77 # BUILDING CONFIGURATIONS.............................................................................................................. 79 9.1. OVERVIEW................................................................................................................................................ 79 9.2. THE ETICS-BUILD COMMAND ................................................................................................................... 79 9.2.1. How to Build a Configuration........................................................................................................... 80 9.2.2. Build Targets ..................................................................................................................................... 80 9.2.3. Forcing Build Execution.................................................................................................................... 81 9.3. THE ETICS PACKAGING SYSTEM ............................................................................................................ 81 9.3.1. How to Pass Installation Instructions to the Packager .................................................................... 86 9.4. THE ETICS PUBLISHER............................................................................................................................ 87

INFSOM-RI-026753 2007 © Members of ETICS collaboration 4 / 145 ETICS USER MANUAL

9.5. BUILD REPORTS ....................................................................................................................................... 89 # TESTING CONFIGURATIONS ................................................................................................................ 90 10.1. OVERVIEW .......................................................................................................................................... 90 10.2. THE ETICS-TEST COMMAND ............................................................................................................... 90 10.2.1. How to Test a Configuration ....................................................................................................... 91 10.2.2. Test Targets.................................................................................................................................. 91 10.3. TEST REPORTS .................................................................................................................................... 92 # SUBMITTING REMOTE BUILDS AND TESTS TO THE ETICS SYSTEM .................................... 93 11.1. SUBMITTING BUILDS AND TESTS USING THE WEB APPLICATION ..................................................... 93 11.2. SUBMITTING BUILDS AND TESTS USING THE COMMAND-LINE CLIENT ............................................ 93 11.2.1. How to submit a remote build job ............................................................................................... 99 11.2.2. How to submit a remote test job .................................................................................................. 99 # ACCOUNT INFORMATION ................................................................................................................... 100 # REPOSITORY ............................................................................................................................................ 102 13.1. VOLATILE STORAGE AREAS........................................................................................................... 102 13.2. REGISTERED STORAGE AREA ........................................................................................................... 103 13.3. REPOSITORY WEB APPLICATION...................................................................................................... 103 # ANALYSING BUILD AND TEST REPORTS ....................................................................................... 108 14.1. STANDARD REPORTS ........................................................................................................................ 108 14.2. THE ETICS-GET-REPORT COMMAND ................................................................................................. 113 14.3. THE ETICS-GET-RUNDIR COMMAND ................................................................................................. 114 14.4. SPECIFIC REPORTS ............................................................................................................................ 114 # USER AND MODULE ADMINISTRATION......................................................................................... 120 15.1. THE ETICS ADMINISTRATION APPLICATION .................................................................................. 120 15.1.1. Layout......................................................................................................................................... 120 15.1.2. Manage projects ........................................................................................................................ 121 15.1.3. Manage users ............................................................................................................................. 122 15.1.4. Manage user permissions .......................................................................................................... 123 15.2. ADMINISTRATION WITH THE COMMAND-LINE CLIENT.................................................................... 126 15.2.1. Manage users ............................................................................................................................. 126 15.2.2. Manage user permissions .......................................................................................................... 127 # CLIENT PLUGIN FRAMEWORK ......................................................................................................... 129 16.1. OVERVIEW ........................................................................................................................................ 129 16.2. PLUGIN SPECIFICATION .................................................................................................................... 130 A. PLUGIN SAMPLES................................................................................................................................... 132 B. PROPERTIES ............................................................................................................................................. 141

INFSOM-RI-026753 2007 © Members of ETICS collaboration 5 / 145 ETICS USER MANUAL

# INTRODUCTION This user manual described the ETICS Service v2.0.

1.1. Overview ETICS stands for eInfrastructure for Testing, Integration and Configuration of Software. ETICS is an on-line collaborative service for managing small and large software projects by managing their configuration, enforcing quality standards, building packages and testing them in environments as close as possible to real-world infrastructures. The ETICS tools can be used to run builds and tests on users’ computers, but also and especially to submit remote builds and tests on multiple platforms at the same time.

The process of building and testing software with ETICS goes through three phases:

# Configuration: this is the process of registering the software components to be built and tested with the ETICS system; creating configurations and attaching version control commands, build commands, test commands, properties, environment variables and dependencies. Functionalities required to accomplish this task are provided by the Build and Test Web Application and Command-Line Client mediating the access to the underlying Build and Test Service # Local build/test: once configuration information is registered in the system, the ETICS command-line client allows performing local builds and tests on a user machine. This step allows the user to validate the configuration information and make sure the components build and pass tests as expected and produce the expected results. In the case of building, the outcome of this procedure is a set of packages in various formats (typically tarballs and RPMS on Linux systems) and a set of log files, test and metrics reports. When performing system tests, a set of log files, test and metrics reports is also generated. # Remote build/test: once the components have been verified to build and/or test successfully locally, it is possible to submit and schedule builds and tests on a pool of worker nodes (i.e. remote machines) providing support for different platforms. This step allows running the builds and tests on a variety of platforms to validate portability of the code and generate a number of static and dynamic test reports under various conditions. The remote execution is implemented using the Metronome engine. Once on a worker node the build and test procedures is performed by the ETICS Command-Line Client, follow the exact same sequence as a local build and test procedure.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 6 / 145 ETICS USER MANUAL

Figure 1: The ETICS Architecture

The ETICS Services includes of the following main components (Figure 1):

* The ETICS Client: a set of command line tools to access the ETICS services. The tools are written in Python and are currently available on most platforms1 with Python >= 2.2 * The ETICS Web Application: a web application to manage the configuration of projects registered in the ETICS metadata store * The ETICS Web Service: the web service is the business core of the system. It handles requests from the command-line client and the web application, processes requests and forwards build/test jobs requests to the Metronome engine * The Metronome Execution Engine: the Metronome engine uses the Condor system to execute jobs on dedicated worker nodes. The worker nodes operates on different platforms to provide a multiplatform build/test environment * The ETICS Database: all configuration information is stored in the ETICS database, which allows creating reproducible builds, setting dependencies between projects and packages and creating quality controlled software packages according to stored policies. In this manual, we refer to the data stored in the database as the metadata store. * The ETICS Repository: software packages and test data generated by running build and test jobs in ETICS system can be automatically uploaded to the software repository, where they can be accessed by users or by the ETICS system itself. Remote build and test reports can also be browsed from the ETICS Repository or locally with a standard web browser.

1 At the time of releasing this document, the client doesn’t work on Windows.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 7 / 145 ETICS USER MANUAL

1.2. How ETICS Represents Software Information Software projects need to be registered in the ETICS metadata store in order to use the build and test service. Before using ETICS it is recommended to get familiar with a few basic concepts used by the system. The objects described in the following paragraphs are part of the ETICS data model.

1.2.1. Modules: Projects, Subsystems and Components Software projects are known in ETICS as Projects. A Project can be considered as a high level container for software components and it’s a good place to define properties and policies that must be applied by default to all components, as will be explained later. A project can be further split in Subsystems and Components. A subsystem represents a logical partition of the software architecture providing some subset of the project functionality. As for the project, subsystems can normally be considered containers for software components with special properties and policies that must be applied by default to all components of the subsystem. However, the subsystem can be a software component in its own right and normally can be used to produce meta-packages. Components represent even smaller unit of functionality in the project architecture. A component can belong to the project or to an individual subsystem in the project. Normally they are used to generate individual software packages or small families of packages. Figure 2 shows the relationships among project, subsystem and component objects. A subsystem must belong to one and only one project and a component must belong to one and only one project or subsystem. Project, subsystem and component objects are generically referred to as Module.

P

S

C

C Figure 2: The Project, Subsystem and Component objects Project The project parameters are: Table 1: Project parameters Parameter Description Mandatory name The name of the project set. It must be unique in the ETICS Yes metadata store displayName A friendlier name for the project set. It doesn’t need to be unique No description The project set description No homepage The home page of the project No logo The URL of the project logo in a format suitable to be used on a No

INFSOM-RI-026753 2007 © Members of ETICS collaboration 8 / 145 ETICS USER MANUAL

web page repository The default URL of the repository storing packages from this No project (can be overridden by subobjects) vcsroot The default root of the version control system storing source code No for this project (can be overridden by subobjects) vendor The project owner (organization, company, individual) No modifyDate The date of last modification No createDate The date of creation No

Subsystem The subsystem parameters are: Table 2: Subsystem parameters Parameter Description Mandatory name The name of the subsystem set. It must be unique in the ETICS Yes metadata store displayName A friendlier name for the subsystem set. It doesn’t need to be No unique description The subsystem set description No repository The default URL of the repository storing packages from this No subsystem (can be overridden by subobjects) vcsroot The default root of the version control system storing source code No for this subsystem (can be overridden by subobjects) vendor The subsystem owner (organization, company, individual) No modifyDate The date of last modification No createDate The date of creation No

INFSOM-RI-026753 2007 © Members of ETICS collaboration 9 / 145 ETICS USER MANUAL

Component The component parameters are: Table 3: Component parameters Parameter Description Mandatory name The name of the component set. It must be unique in the ETICS Yes metadata store displayName A friendlier name for the component set. It doesn’t need to be No unique description The component set description No packageName The name of the package produced by this component if different No from the component name homepage The home page of the project No download The URL of the download page for this component if taken No directly in binary format from the vendor licenceType The license to which this component is subject No repository The URL of the repository storing packages from this component No vcsroot The root of the version control system storing source code for this No component vendor The subsystem owner (organization, company, individual) No modifyDate The date of last modification No createDate The date of creation No

1.2.2. Configurations A Configuration in ETICS is the set of information representing a specific version of a module. For example, one can compare the ETICS modules to CVS modules and the configurations to CVS tags and branches. A configuration must be attached to one and only one module (project, subsystem or components), but each module can have one or many configurations, like in CVS a module always has the HEAD branch, but can have many other tags and branches. Configurations are organized in a hierarchy that mirrors that of the corresponding modules. Therefore if a project contains some subsystems and each subsystem contains some components, a project configuration can contain subsystem configurations and the subsystem configurations can each contain component configurations (Figure 3). However, noted that a configuration tree doesn’t have to contain configurations for all objects in the corresponding module tree. The module tree in a project represents the project structure, but a configuration tree is a particular version of the project and can contain only a subset of the configurations. Projects and subsystems configuration trees can contain any combination of the available configurations, provided they do not contain more than one configuration of the same module (this requirement is relaxed for external dependencies, since it is possible to have different versions of the same dependency used by different components in the same project). In the ETICS system the configurations are the objects on which the build and test operations are performed. When a project configuration is built, all subsystem configurations in its tree are automatically built unless differently specified. In the same way, when a subsystem configuration is built, all component configurations in its tree are built.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 10 / 145 ETICS USER MANUAL

P Cp

S Cs

C Cc

C CC Figure 3: The ETICS Modules and Configurations

The configuration parameters are: Table 4: Configuration parameters Parameter Description Mandatory name The name of the configuration set. It must be unique in the Yes ETICS metadata store displayName A friendlier name for the configuration set. It doesn’t need to be No unique description The configuration description No age The configuration age number No 1 majorVersion The configuration major version number Yes minorVersion The configuration minor version number No revisionVersion The configuration revision version number No path The path where the package file can be downloaded from No relative to the repository profile A comma-separated list of profiles No status The configuration status (alpha, beta, RC, etc) No tag The tag string of this configuration in the Version Control No System hosting the module modifyDate The date of last modification Auto createDate The date of creation Auto

In addition a configuration inherits the following parameters from its corresponding module (i.e. project, subsystem or component):

1 Standard version information can be represented in the form of: ..-

INFSOM-RI-026753 2007 © Members of ETICS collaboration 11 / 145 ETICS USER MANUAL

Table 5: Inherited module parameters Parameter Description moduleName The name of the parent module moduleDescription The description of the parent module licenseType The license of the parent module projectName The name of the project to which the parent module belongs

1.2.3. Platforms The information required to build or test a configuration may depend on the platform where the operation is executed. In order to allow the correct execution on multiple platforms, the ETICS metadata store defines platform objects representing particular combinations of OS types, architectures and compilers. Each platform is identified as a string of the form:

__

For example the following platforms are currently defined:

slc3_ia32_gcc323 slc4_ia32_gcc346 slc4_x86_64_gcc346 win32_vc71

The platform is automatically detected when running client commands, but it can be overridden for special reasons as described in Chapter 9, for example if your working platform doesn’t exactly match one of the predefined platforms. All commands and properties (described in the following sections) are attached to configurations via one or more platforms. Each configuration can have separate sets of commands and properties for each platform. In addition, a special default platform is defined in the system to which default sets of commands and properties can be attached to be used whenever a platform-specific value is not defined (e.g. when a specific behaviour for a given platform is not required). The platform parameters are: Table 6: Platform parameters Parameter Description Mandatory Name The name of the platform set. It must be unique in the ETICS Yes metadata store displayName A friendlier name for the platform set. It doesn’t need to be unique No description The platform set description No architecture The CPU architecture (ia32, x86_64, etc) No family The OS family (slc4, win32, etc) No vendor The OS vendor name (Red Hat, Microsoft, etc) No age The OS age No

INFSOM-RI-026753 2007 © Members of ETICS collaboration 12 / 145 ETICS USER MANUAL

majorVersion The OS major version number No minorVersion The OS minor version number No revisionVersion The OS revision version number No

1.2.4. Commands and Properties In order to build and test a configuration, the proper commands and properties must be defined. Commands and properties are attached to a configuration by platforms. There are three different sets of commands and two types of properties (i.e. ETICS properties and environment variables):

VCS Commands The VCS Commands are used to manage code stored in a Version Control System (VCS), for example CVS, Subversion or ClearCase. The commands are defined as: Table 7: VCS Command parameters Command name Description Mandatory description The command set description No tag Command to tag code No 1 commit Command to commit code No checkout Command to commit code No 2 branch Command to branch No

More information about the VCS Commands can be found in Chapter 6 (“Checking out Project, Subsystems and Components”).

Build Commands The Build Commands are used to build the code, generate documentation and packages and publish the build artefacts in a standard location and format. The commands are defined as: Table 8: Build Command targets Command name Description Mandatory description The command set description No clean Command to clean init Command to perform initialization operations that depend on the No build system structure (for example fetching files from other modules). This command cannot be used in packaging scripts (like the RPMS spec file configure Command to perform initialisation operations that do not depend No on the build system structure (e.g. running the ./configure script as part of the RPM post-install step, which the ETICS packager automatically inserts in the RPM generated spec file) checkstyle Command to perform code checking operations (coding No

1 Not supported in the current release 2 Not supported in the current release

INFSOM-RI-026753 2007 © Members of ETICS collaboration 13 / 145 ETICS USER MANUAL

conventions) Compile Command to compile code No Test Command to perform static tests like unit tests or coverage tests Doc Command to generate documentation No packaging Command to generate distribution packages No prepublish Commands to perform steps to be executed before publishing the No build artefacts (collecting additional files or applying transformations, etc) Publish Command to publish build artefacts to standard distribution No formats postpublish Commands to perform steps to be executed after publishing the No build artefacts (cleaning or publishing to custom locations) Install Command to install the package No (however, the ETICS Packager1 requires this target to be defined)

More information about the Build Commands can be found in Chapter 9 (“Building Configurations”).

Test Commands The Test Commands are used to perform dynamic (integration, system) tests on software packages and generate test report. The commands are defined as: Table 9: Test Command parameters Command name Description Mandatory description The command set description No clean Command to clean init Command to perform initialization operations before compiling No test Command to execute the tests No

More information about the Test Commands can be found in Chapter 10 (“Testing Configurations”).

Properties Properties are key/value pairs that can be attached to a configuration for a specific platform (or the default platform). The properties can then be used in any command or other properties as ${property-name}. The ETICS system defines a number of built-in properties (see Appendix B: Properties for details on the built-in properties) that are always available when executing the commands, but users can define any number of custom properties or in most cases override the values

1 For details on the ETICS Packager, see section 9.3 - The ETICS Packaging System

INFSOM-RI-026753 2007 © Members of ETICS collaboration 14 / 145 ETICS USER MANUAL

of the built-in properties. The properties are inherited and propagated following the algorithm described in section 1.2.7.

Environment variables Environment variables are key/value pairs that can be attached to a configuration for a specific platform (or the default platform). The environment variables are appended to the execution environment and are visible to the ETICS commands and any script or program invoked by the commands. Normally, environment variables are simply (re)set the specified by the value in the configuration. However, if the environment variable ends with the string ‘PATH’, then if the environment variable is already defined, the new value is pre-pended to the existing value. This allows the user to accumulate the values of special environment variables, for example PYTHONPATH or PATH.

1.2.5. Dependencies It is possible to define dependencies between configurations, not only within the same subsystem or project, but also across subsystems and projects. Also the dependencies are attached to configurations by platform. Therefore it is possible to have different dependencies on different platforms for the same configuration. There are two types of dependencies: static and dynamic.

Static dependencies Static dependencies are fixed, named dependencies between two configurations. They are specified by name and are not affected by build properties or policies.

Dynamic dependencies Dynamic dependencies are defined between a configuration and a module. The specific configuration (version) of the module to be used as dependency is dynamically defined at checkout using properties and may change if the same parent configuration is used in different configuration trees. This allows for example to set a dependency on a module, but always get whatever default configuration is defined at the project level without having to change the dependency explicitly when dependencies are upgraded. The dynamic dependency resolution is performed according to the following rules:

# By setting a property of the format:

<module-name>.DEFAULT

at any level or above the component having the dependency (for example this can be done at the project level so that the value is used by all configurations in the project). This method works for all dependencies both internal and external to the current project and it’s the normal way of setting dynamic dependencies on external components. # By using the current configuration tree. For example assume that SubsystemConfigA contains two components, ComponentConfigB and ComponentConfigC, and that ComponentConfigB depends dynamically on ComponentC. The value of the dependency will be set automatically by the system to ComponentConfigC, since it belongs to the current configuration tree. This method works only for dependencies internal to the project

INFSOM-RI-026753 2007 © Members of ETICS collaboration 15 / 145 ETICS USER MANUAL

and it’s the normal way of setting dynamic dependencies among components within the same project and configuration tree. As of ETICS v2.0, the default values for dynamic dependencies have been removed. Instead, users can either lock their configuration (see details in section XX), or specify a constraint by version. Constraints by version allow the user to define a constraint for the dynamic resolution of a dependency, in terms of version information and optionally an inequality. The system then uses the existing above mentioned dynamic dependency mechanisms to resolve the dynamic dependency, and if a constraint by version is defined, it then tests if the resolution is compliant the user defined constraint. In addition, both static and dynamic dependencies have a scope and can be build-time, run-time or both. Build-time dependencies are used during build operations and are automatically built if necessary, but are not added as dependencies for distribution packages (for example in RPMS spec files). Run-time dependencies are not used during a build, but are used to define dependencies in the distribution packages (again, for example in RPMS spec files).

1.2.6. Commands and Properties Search Order for Multiple Platforms When commands and properties are defined for different platforms, the system looks for them in a specific order, which is slightly different for commands and properties

Commands When a configuration is built or tested, the system first looks for commands sets defined for the current working platform (for example slc3_ia32_gcc323), if a command set is not defined it looks for an equivalent command set defined for the default platform, if none is defined, the command set is not executed. The command sets are taken as complete sets. For example if a build command set is defined for “slc3_ia32_gcc323”, and it defines the init target, but not the configure target, then configure is not executed even if the command set attached to the default platform defines it.

Properties When a configuration is built or tested, the system first looks for properties defined for the default platform, then for properties defined for the current working platform. The resulting properties set include both sets of properties. If a property is defined for both the current working platform and the default platform, the platform-specific one is used.

1.2.7. Property Inheritance Property processing in ETICS includes inheritance from the parent properties, as well as propagation of properties from children configurations and dependencies, providing both a top/bottom and bottom/up capabilities. Each configuration in ETICS inherits the properties of its parent (e.g. project and possibly subsystem for a component configuration), which means that any property defined above a configuration is available to the configuration, where the property definition closest to the configuration overrides properties further defines. For example, say a component configuration is set as a child configuration of a subsystem and project configurations (i.e. parent configurations). If both parent configurations define the same property, when processing the component configuration, the value will be taken from the subsystem configuration, since it is closest to the component configuration than the project configuration. This mechanism allows users to set properties at project level (e.g. debug level, default installation location) for all configurations to use, while allowing subsystems and component configurations to overwrite these default project settings.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 16 / 145 ETICS USER MANUAL

The bottom/up propagation of property works as follows. If a configuration has children configurations and dependencies, all of their properties will be available to it. To avoid name clashing (i.e. in the case that different configurations define the same properties), all properties defined for the children configuration and dependencies are namespace qualified with the module name of their configuration. For example, say a component configuration MyComponent has a dependency on another component configuration with the module name called MyDependency, and in turn this dependency defines a property called myProperty, then this property will be available to MyComponent as MyDependency.myProperty. Finally, the user can mix the two concepts, where a property can be defined at a higher level so that they are accessible only by a particular component down the hierarchy by qualifying the property name with the module name of the particular component. For example, assume that the configuration of project MyProject contains the configuration MySubsystem, which contains the configuration MyComponent. If the property myProperty is defined for the project it will be accessible by the project, the subsystem and the component. If the property MyComponent.myProperty is defined at the project level, it will only be accessible by the configuration MyComponent. However, note that MyComponent.myProperty will be available to all children in the project hierarchy.

The environment variables follow a similar algorithm, with the exception that no namespace qualification takes place. As mentioned in section 1.2.4, normally, environment variables are simply (re)set the specified by the value in the configuration. However, if the environment variable ends with the string ‘PATH’, then if the environment variable is already defined, the new value is pre-pended to the existing value. This allows the user to accumulate the values of special environment variables, for example PYTHONPATH or PATH.

1.2.8. Locking This feature introduced in ETICS v2.0 allows users to make a configuration self consistent. By self consistent we mean that a given "locked" configuration will contain all the required information (i.e. properties, environment variables, dependencies resolution) in order to be built/tested, independently of the context in which it is built or tested. This will guarantee the reproducibility of those configurations across over time. Several constraints come with locking configurations. In order to guarantee the above mentioned goal, locked configurations graphs can only contain locked configurations. In other words, in order to lock a project or a subsystem configuration with sub-configuration, or if any configuration has dependencies, the sub-configurations and dependencies must also be locked. This means that in order for a configuration graph to be locked in a single operation, the user must have appropriate privileges to lock all the configurations part of the graph (only Administrators, Release Managers and Developers can lock configurations). Failure to fulfil this requirement, the locking procedure will fail. Once a configuration has been locked, no further modifications are allowed to this configuration. This implies that the users will have to create a new copy (clone) of the locked configuration if he/she wants to make modifications. Finally, only the artefacts generated from locked configurations can be registered in the permanent storage of the repository. This means that users must lock their configurations prior to registering artefacts in the registered part of the repository. Since artefacts can only be registered once, it is important that the configurations used to build a given artefact do not change once it has been registered, this is guaranteed by the locking mechanism.

1.3. Authentication, Authorization and Roles

INFSOM-RI-026753 2007 © Members of ETICS collaboration 17 / 145 ETICS USER MANUAL

Security in ETICS is based on users and roles. Authorisation and authentication is achieved using x509 client digital certificates. By default all read-only operations like browsing projects, executing a checkout command and performing local builds is open to any user, even users not registered with the ETICS system. However, although the execution of VCS commands is open to anonymous access, the actual result of the operation depends on the VCS system security, which is independent from ETICS. For example, if anonymous access is disabled in CVS, the user will have to provide authentication, something that is outside the scope of ETICS1. Operations that require altering the information in the system or using system resources, like editing modules and configurations or submitting remote builds and tests, can only be performed by registered users with the appropriate role. Users are identified by the x509 Distinguished Name (DN) that appears in a client certificate issued by a recognised Certification Authority. ETICS recognises as valid the CAs that are part of the EUGridPMA distributions. In addition to being registered as users in the system, users must have one or more roles in order to perform operations. The following roles are defined:

1 A solution to secure accesses to VCS systems is for users to deploy private machine and attach them to an ETICS resource pool. This topic is outside the scope of this manual. For more information on this, please contact the ETICS Support team at etics-support@cernNOSPAMPLEASE.ch.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 18 / 145 ETICS USER MANUAL

Table 10: Roles Role name Allowed operations Administrator All operations on all projects. This is a role used only by the ETICS service managers Module Administrator Can edit modules and manage security Developer Can edit configurations and submit remote builds Integrator Can edit configurations, submit remote builds and register build artefacts in the repository Tester Can submit remote tests and register test artefacts in the repository Release Manager Can lock configurations, submit remote builds and register artefacts in the repository Guest Can only perform read-only operations

Roles apply to specific objects in the system and are inherited by objects in the same tree from top to bottom. For example a user can have the role of Module Administrator for a project, which gives him or her the same role on all subsystems and components in that project. Authentication and authorization can only occur when the ETICS system is accessed using the secure protocol ‘https’. When http is used, only Guest access is possible.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 19 / 145 ETICS USER MANUAL

2. Etics Portal

2.1. Overview The ETICS portal provides a web-based interface to all ETICS functionality. It is the main web entry point to the ETICS system. It offers an easy way to navigate between different system domains, offers common look and behaviour to facilitate the user experience. The portal is publicly available at this location: https://etics.cern.ch/eticsPortal. The application supports two main browsers: Firefox (version 2.x or higher) and Internet Explorer (version 6.x or higher). User authentication is bases on x.509 user certificates installed in the browser. The portal uses asynchronous client-server communication mechanism where only a part of a page is refreshed when new data arrive from the server. This improves significantly the application response time and lowers network bandwidth usage. On the other hand, due to asynchronous communication users cannot bookmark in their browser a given portal page or interaction state – after opening any bookmarked portal session, the portal will always restart from the default page. For the same reasons using the browser’s refresh button is not recommended – since it will restart the portal session.

2.2. Layout The portal has been divided into panels that correspond to different system areas – they are organised as tabs accessible at the top of the portal page. When selecting a panel, its content is maximised in the window. Any previously selected panel and any modification made to them are preserved and available when the user selects one of these panels again. Additionally, every main panel can be divided into sub-panels available via smaller tabs at the top of the selected panel – they work exactly in the same manner as the main panels. Here’s a screen shot of the portal and a brief layout explanation:

Figure 4: Portal layout The portal adapts itself to the user, where it shows only the panels and sub-panels that are appropriate for the current user according to his/her privileges. The next sub-sections describe one-by-one the different panels in the portal.

2.3. Panel – MyETICS The MyETICS panel is composed of sub-panels, and is the default panel of the portal.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 20 / 145 ETICS USER MANUAL

2.3.1. Welcome The Welcome sub-panel shows the following information: * The ETICS system, the portal, support channel, user manual. * The user – his/her identity in the system. * Available panels. This panel is available for every user.

2.3.2. My submissions The My submission panel shows job submitted by the current portal user. This panel is available for registered and active users only. Submissions are sorted by date starting from the most recent one; the list is divided into pages. After clicking on a row, the user can see the submission details, such as the checkout parameters. A filter on the top of the panel allows filtering submissions by given date range and a submission type. The button “apply filter” become enabled after the filter is modified. By default, the filter is set to show all the current user submissions from the last 7 days. Here’s a screenshot of the MyETICS -> My submissions panel:

Figure 5: MyETICS -> My submission layout

2.4. Panel – Build and Test system The Build system panel embeds the ETICS web application described below in chapter 3 (“The ETICS Build and test ”). This panel is available for every user.

2.5. Panel – Repository The Repository panel embeds the ETICS Repository Web Client described in more details in chapter 13 (“Repository”). This panel is available for every user.

2.6. Panel – administration The Administration panel includes the ETICS administration application described in more details in chapter 15.1 (“The ETICS Administration Application”). This panel is available only for system and module administrators.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 21 / 145 ETICS USER MANUAL

2.7. Panel – externals The Externals panel allows user to request the creation of a new components and/or a new configurations in the ETICS externals project (display name “All external components”). It is available for all registered and active users. This application is implemented as a wizard. Users can manoeuvre between the wizard steps using two buttons (back/next) located at the top and the bottom of the page. The next button is available only if all the data in the current wizard step is correctly filled. The request form consists of two main parts. Firstly, the user must either select from a tree (and optionally modify) an existing component, or create a new component by providing the following information: # Component name – to uniquely identify the component in the ETICS system. # Component description. # Component vendor. # Component license type. Secondly, the user must provide details about the component configuration which corresponds to a particular component version: # Configuration name – to uniquely identify the component in the ETICS system. # Description. # Version. # Platform – one of the ETICS platforms. # Binary location – where the ETICS team can found the configuration binaries. # Reason – a justification why the new component and configuration are needed. When these two steps are completed, the request is sent to the ETICS team for further processing. The user is then notified by e-mail after his/her request has been accepted or rejected by the ETICS support staff on-duty. Here is a screenshot of the Wizard to request a new external component:

Back/Next User input form. buttons Required fields are marked in red.

Figure 6: Externals request wizard layout

2.8. Panel – Process new external component request The Process a new external component request panel allows the ETICS system administrator to accept or reject a request for adding a new component in the ETICS external area (see previous chapter).

INFSOM-RI-026753 2007 © Members of ETICS collaboration 22 / 145 ETICS USER MANUAL

This panel is available only for system administrator after he/she opens a link from an e-mail that is send automatically by the system after the request is triggered by the user using the Externals request wizard. Additionally, in order to accept the request, the person accepting the request must have write access to CERN AFS repository1. This application is implemented as a wizard and is very similar to requesting a new external component (see previous section). The administrator checks the submitted information and corrects it if needed. If the administrator accepts the request, the wizard instructs him/her to execute manually a UNIX command on a machine having access to the infrastructure able to physically copy the component binaries to the ETICS repository. The operation will succeed only if the administrator has write access to the CERN AFS area where the ETICS repository is located. At the end, a notification e-mail is sent to a user that made the request. The following screenshot shows an example of the Wizard to process a new external component request:

Figure 7: Processing of externals request wizard layout This panel is available for every user.

1 This requirement access might eventually disappear if the new repository completely replaces the AFS-base repository.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 23 / 145 ETICS USER MANUAL

3. The Etics Build And Test Web Application

3.1. Overview The Build and Test Web Application (WA hereafter), together with the ETICS command-based tools, provides user access to build and test capabilities of the ETICS system. Besides being a tool for inspecting and modelling projects within the ETICS infrastructure, the WA also enables easy remote build and test submission and access to build and test reports and artefact. Access to the ETICS WA is granted to everybody without any authentication/authorization restriction. By default, and this can be modified for each ETICS service installation, anonymous read-only access is granted to all users. On the other hand, editing grants can be autonomously managed in a fine-grained fashion within any hosted project letting project managers the full control over authorization aspect. For authentication, X509 certificates are used to identify users. Several deployments of the ETICS system, including WA are currently available1. The main WA installation is reachable via the ETICS Portal at the following address, selecting the “Build system” panel: https://etics.cern.ch/eticsPortal

Figure 8 - Build and Test System Web Application

1 The main ETICS installations are located at CERN, CNAF/INFN and University of Wisconsin-Madison.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 24 / 145 ETICS USER MANUAL

3.2. Layout The current version of the Build and Test System Web Application welcomes the user with a project- selection page. It allows the user to browse, inspect and select the set of projects hosted by the ETICS infrastructure.

Figure 9 - Web Application Project Selector Once a project is selected, the user is redirected to the main working page where she/he can perform all the main actions concerning the project modelling and build submission. This main page is organised around a four-area layout: - a browse panel on the top-left side devoted to navigate through the structure of all projects hosted by the ETICS infrastructure. - a configuration list panel on the bottom-left side with quick search capability - a workspace panel in middle of the page hosting a number of items1 the user is currently working on. - a main edit panel on the top-right side of the page. This area allows either to display metadata for each and every object of the ETICS data model or to host editors for each of them - a toolbar at the top of the page. This area contains buttons for each action that can be performed on the currently-selected object (i.e. the element shown in the main panel, either in view or edit mode) The following picture highlights the above-described areas:

1 In the current release, only one item is allowed in the workspace.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 25 / 145 ETICS USER MANUAL

Figure 10 - Build and Test System Web Application Layout

3.3. Enabling Security In order to be authenticated by the ETICS Service, using the web application, you need to load a valid certificate in your browser. The ETICS production deployment accepts all certificates signed by certificate authorities recognised by the EUGridPMA1. For test purposes ETICS also has a certificate authority able to sign digital certificates. Users who don’t have access to a certification authority recognised by the EUGridPMA can contact the ETICS support team (etics-support@cernNOSPAMPLEASE.ch) to obtain a recognised certificate. To load a certificate in the Firefox browser, the certificate must be in the p12 format. The simplest way to load a certificate in Firefox is to: * On Linux: go to Edit -> Preferences -> Advanced -> Security * On Windows: go to Tools -> Options -> Advanced. Under the Encryption/Certificates section, click on ‘Manage certificates’. Under the tab ‘Your Certificates’ click Import. Select your p12 format certificate. When prompted, enter the private key password of your certificate. Once the certificate is loaded, close and restart Firefox. To verify that you’ve loaded the certificate successfully, connect to the ETICS Web Application again, for example: https://etics.cern.ch/eticsPortal (select the panel “Build system”). This time, you should see that you are authenticated with your certificate as indicated in the very top- right corner of the browser window, as shown in Figure 11 below.

1 http://www.eugridpma.org/

INFSOM-RI-026753 2007 © Members of ETICS collaboration 26 / 145 ETICS USER MANUAL

Figure 11: Loaded certificate in Firefox (i.e. “Etics User1”) A similar procedure exists for all browsers supporting digital certificate authentication1.

The command-line client also uses digital certificates for authentication. For more details on how to use digital certificates for the command-line client, refer to section 4.6 (“Enabling Security”).

3.4. Certificate Registration Once your certificate has been properly installed in your browser, it can be registered with the ETICS service. Point your browser to (see Figure 12): https://etics.cern.ch/eticsAdmin/public/registration/requestRegistration.jsp

1 Please note that at the time of writing, the Build and Test web application only supports Firefox. Support for Internet Explorer and other popular browsers will be added in a future release

INFSOM-RI-026753 2007 © Members of ETICS collaboration 27 / 145 ETICS USER MANUAL

Figure 12: Certificate registration form Fill in the form (all fields are mandatory). Please type a valid email as it will be used to send you the account activation link. Once you sent the request, you will receive a notification email. The request is then validated by the ETICS support team. When your request is accepted, you will receive an email with the activation link. Click on it to ACTIVATE your User Account.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 28 / 145 ETICS USER MANUAL

4. The Etics Command-line Client

4.1. Overview The ETICS Client is a set of command-line tools to access ETICS services. The tools are written in Python and can be installed on any platform where Python >= 2.2. The installation packages are available from the ETICS repository (see “Chapter 13: Repository” for details) in various formats (e.g., tarballs, RPMS). The currently recommended way of installing the ETICS Client on most platforms is by using the script:

http://eticssoft.web.cern.ch/eticssoft/repository/etics-client-setup.py

This script installs the latest stable version of the client and all required dependencies.

4.2. How to Install the ETICS Client As mentioned in the previous section, the recommended way of installing the ETICS Client is by using the etics-client-setup.py script. This script doesn’t require root privileges to be used and only expects Python >= 2.2 to be installed.

The following commands can be used to install the client:

On Unix/Linux systems

wget http://eticssoft.web.cern.ch/eticssoft/repository/etics-client- setup.py python etics-client-setup.py

The client will be installed in the ‘etics’ directory in the directory where the command is executed. An optional parameter --prefix can be used to install the client in a different directory.

On Windows systems1

Download the file from

http://eticssoft.web.cern.ch/eticssoft/repository/etics-client-setup.py

and save it in a folder of your choice, then execute: python etics-client-setup.py

The client will be installed in the folder where the command is executed. Also in this case the --prefix option can be used to redirect the installation to a different folder.

1 At the time of releasing this document, the client is not supporting Windows

INFSOM-RI-026753 2007 © Members of ETICS collaboration 29 / 145 ETICS USER MANUAL

The ETICS Client installation script installes the core client executables and libraries, the test manager and a default set of plugins. It will additionally check for the presence of the following dependencies and install or upgrade them if necessary (the upgrade doesn’t affect packages installed outside the ETICS client installation tree):

* 4Suite * pyXML * ZSI * pyOpenSSL * log4py

OpenSSL is also required and it must be already installed before installing the client.

Finally the installation script checks the presence of and installs if required the ETICS TestManager unit test execution engine. After installing the client it is recommended to set the following environment variables:

ETICS_HOME = PATH = $ETICS_HOME/bin:$PATH

Setting these parameters is required in order to use the same client installation with multiple workspaces. If these variables are not set, the client will only work within the directory where it has been installed (provided the etics-workspace-setup command has been run as well, see section 0 for more details on workspaces).

4.3. How to Configure the ETICS Client The ETICS Client can be configured using the following configuration files:

System configuration file: /etc/etics.conf User configuration file: /.etics.conf Workspace configuration file: /etics.conf

The files have exactly the same format and they are used in a specific order. Values in the workspace file have precedence on values in the user file that in turn have precedence on values in the system file. In addition, the workspace file only applies to the current workspace, the user file to all workspaces owned by the current user and the system file to all workspaces of all users sharing the same client installation. Some parameters can also be set using environment variables (see table below). In this case the environment variable has lower priority than the workspace file, but higher priority than the user file in order to allow setting per workspace configurations.

The configuration parameters are organized in three categories:

User parameters: parameters to configure how users interact with the ETICS client System parameters: parameters to configure how the client interacts with the ETICS services

INFSOM-RI-026753 2007 © Members of ETICS collaboration 30 / 145 ETICS USER MANUAL

Properties: parameters to modify the behaviour of some client commands or how builds and tests are executed

The ETICS client is installed with a preconfigured system configuration file with valid default values for read-only operations. Users can copy the system file and use it as template for the user and workspace files. Note that in the case of the user file, the file must be renamed with a leading “.”. The following table shows the currently supported configuration values and their usage: Table 11: Configuration file parameters Parameter Category Default value Description x509_user_cert user The x509_user_cert option is used to specify the location of the user public certificate file. This option can also be set by using the X509_USER_CERT environment variable. x509_user_key user The x509_user_key option is used to specify the location of the user private key of the certificate file. This option can also be set by using the X509_USER_KEY environment variable. vcsroot user The value of the version control system root to be used instead of the one specified as default in the ETICS component metadata. It is normally required to set this property in order to have write access to the version control system. This property can contain for example the value of the CVSROOT to be used. cvsproxy user The value of the CVS proxy host to be used to contact the CVS server. If the environment variable CVS_PROXY is set, it has higher priority than the values set in the system and user configuration files, but lower than the workspace configuration file updateNotification user True If this parameter is True or missing, a notification message is displayed when running any etics commands when a new version of the client is available. Set this to False not to display the message protocol system https The protocol used to connect to the ETICS service endpoint. Valid values are [http|https] If this option is not set, the default value is https (secure connection) server system etics.cern.ch The fully qualified hostname where the ETICS service is running port system 8443 The IP port used to connect to the ETICS service

INFSOM-RI-026753 2007 © Members of ETICS collaboration 31 / 145 ETICS USER MANUAL

If this option is not set, the default values are 8080 if the http protocol is used 8443 if the https protocol is used updateURL system http://eticssoft.web The default URL of the etics-client-setup .cern.ch/eticssoft/r script, used to check for new versions. If this epository/etics- parameter is missing the default value client-setup.py displayed on the left is used platform properties Automatically The platform name to be used for checkout, detected build and test operations. If this property is set here, this value overrides the automatic platform detection. It can be used in case the local platform is not a supported ETICS platform, but compatible with one of the supported platforms repository properties The path of the local repository cache. /repository Normally the repository cache is local to each workspace, but the location can be overridden here for example to redirect the cache to another location or to share the same cache among multiple workspaces

4.4. Workspaces A workspace is a directory in your system where all build and test operations are performed. It is possible to create many workspaces in the system and use them to build different projects or different configurations of the same project. The different workspaces do not interact with each other and removing a workspace removes any trace of what was built or tested in that workspace. When code is checked-out (prior to a build and/or test), all source code from version control systems (like CVS), as well as all source or binary package required as dependency are downloaded from the repositories into a local cache inside the workspace called repository, thus making the workspace self- contained and minimising the system requirements to perform builds and tests. In order to preserve disk space, it is possible to share the local repository cache among different workspace by using the ‘repository’ configuration parameter in the ETICS Client configuration files.

Workspaces are created simply by creating a directory of your choice and running the etics- workspace-setup command in it. The command will configure the workspace for first use.

4.5. ETICS Commands and Libraries The ETICS Client is composed of a number of commands and libraries. The commands are found in

<installation-root>/bin

On Linux systems they do not have the extension ‘py’, while they keep the extension on Windows. The libraries are found in

<installation-root>/lib/pythonX.Y/site-packages (Linux)

INFSOM-RI-026753 2007 © Members of ETICS collaboration 32 / 145 ETICS USER MANUAL

<installation-root>/lib/pythonXY (Windows)

where X and Y represent the installed version of python, for example ‘python2.2’. By convention all commands have a --help and a --version options that can be used to get more information about the command and the version of the client. The command

etics-version

can be also used to get the client version and other information (copyright, etc)

4.6. Enabling Security In order to be authenticated by the ETICS Service, using the command-line client, you need to install a valid digital certificate on your local machine. The ETICS production deployment accepts all certificates signed by certificate authorities recognised by the EUGridPMA1. For test purposes, ETICS also has a certificate authority able to sign digital certificates. Therefore, users who don’t have access to a certification authority recognised by the EUGridPMA, can contact the ETICS support team (etics- support@cernNOSPAMPLEASE.ch).

To instruct the client to use a certificate, copy your certificate in PEM format on your local machine and configure following parameters in the client config file: * x509_user_cert with the location of the public certificate file * x509_user_key with the location of the private certificate file

The first time you use an ETICS command, you will be prompted to enter the passphrase of your private key. Once this is done, your private key will be encrypted and stored in a secure way such that you do not have to re-enter your passphrase every time you use the client.

Here’s a recipe2 to convert a digital certificate from .p12 to .pem format on Linux: Execute the following command (you will be prompted for a password): > openssl pkcs12 -in cert.p12 -clcerts -out cert.pem where cert.p12 is a certificate in .p12 format (i.e. PKCS12) and cert.pem is the output certificate in .pem format.

Users can also define the following environment variables instead of the config file: X509_USER_KEY and X509_USER_CERT. Note that if both are set (environment variable and config file), the values from the environment variables will be taken.

1 http://www.eugridpma.org/ 2 Recipe based on the following document: http://www.openssl.org/docs/apps/pkcs12.html

INFSOM-RI-026753 2007 © Members of ETICS collaboration 33 / 145 ETICS USER MANUAL

5. Browsing Modules And Configuration Information

5.1. Overview This chapter describes how to retrieve and browse information about project, subsystems, component, configuration and all related objects using the ETICS Web Application and the ETICS Client. All methods described in this chapter can be used by anonymous users and do not require configuring the web browser or the client for secure access. However, if security is configured, also the read-only command will use it and will ask for the certificate passphrase if necessary.

5.2. How to Browse with the Web Application In this section, you will learn how to view and browse metadata using the web application.

5.2.1. Selecting a Project For most functionality, the usage of the Web Application (WA in this section) is scoped to a specific project. For that, the welcome page of the WA provides the user with the list of the projects currently managed by the ETICS infrastructure. Projects can be selected using the drop-down list in the top part of the project selector. After selecting one, details of the project are displayed just below. Once the desired project is found, the arrow- shaped button on the right-bottom gives you access to this project.

Figure 13: Project selection form

5.2.2. Browsing a Project, Subsystems and Components Once a project has been selected, the main page of the WA shows the project structure on the left, project details on the right and project configurations in the configuration list, whereas the workspace is initially empty.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 34 / 145 ETICS USER MANUAL

On the left-hand side of the page, a tree-like component shows the structure of the current project; expanding the root, subsystems and components are progressively shown1. An icon beside the module name2 tells what type the module is (i.e. project, subsystem or component).

Figure 14: Module tree A single click on a node in the tree triggers an update of the main panel on the right side of the page, as well as an update of the list of configurations available for the selected module. In the main panel, the details area is titled with the module displayName3 and description. Then, all module metadata is provided with appropriate formatting. In the configuration list, the available configurations for the selected module are displayed, sorted by last modification date.

1 A lightweight approach has been adopted for the tree component. After clicking a node the first time, child modules are not immediately available as a request for them is being sent to the ETICS server. 2 If fully qualified names are adopted, for clarity, just the most significant part of the name with respect to the parent name is displayed in the tree. The full name of the component is shown in the module details area. 3 If no displayName attribute is set for the module, the name attribute is used to title the area. In this case, the title is italicised

INFSOM-RI-026753 2007 © Members of ETICS collaboration 35 / 145 ETICS USER MANUAL

Figure 15: Module details It may happen that, while browsing a project, the corresponding data-model changes on the ETICS System. In this case, because of the proxying policy adopted in the WA, the information available could be out-to-date. To synchronize the tree cache with respect to the latest project data-model, perform a refresh of the node. This can be done by clicking the ‘refresh’ button on the top-right corner of the panel, as well as using a drop-down menu on the tree node.

Figure 16: Refresh a module

5.2.3. Searching Configurations Upon selection of modules in the project tree, the configuration list panel is automatically populated with all the available configurations for the selected project. The list of configurations is initially sorted according to the last-modification date. By selecting items in the configuration list, configuration’s metadata is displayed in the main panel on the right-side of the page. When the number of configurations for a module becomes large, finding a particular configuration can become difficult. To ease this job, a Search can be performed over the set of module configurations.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 36 / 145 ETICS USER MANUAL

The search action can be activated by specifying a keyword in the search field and pressing the ‘Enter’ key1. The configuration list is then updated with the result set, whose size is displayed in the status bar of the list. Search results are not persisted, i.e. after selecting a different module in the tree, the content of the list is discarded and updated with a new list.

5.2.4. Adding configurations to the Workspace Configurations can be added to the workspace by selecting them and then clicking the ‘bookmark’ button on the top-right side of the panel, or by using the contextual menu on the configuration. In the current release, the workspace can host only one configuration; thus adding a configuration results in the replacement of the current configuration (if any) in the workspace.

Figure 17: Selecting configuration

5.2.5. Browsing Configurations and Sub-Configurations As with modules, to inspect the configuration’s metadata, first select a configuration. Currently this can be done in two areas of the WA: the configuration list and the workspace. The configuration list is located in the bottom-left part of the page and lists all the configurations associated with a selected module. By clicking one of them, details of the configuration are displayed in the details area.

1 In the current release, the match algorithm simply looks for the search key within the configuration name.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 37 / 145 ETICS USER MANUAL

Figure 18: Listing configurations Similar to modules, configurations are hierarchical structures constrained by the associated project structure. This structure can be browsed by using the workspace available in the central area of the page. Initially, the workspace is empty; refer to the previous section to see how to change its content. Expanding nodes in the workspace, sub-configurations are shown as children1. Clicking on them, details are provided in the right-hand side of the page. When expanding configurations and sub-configurations, platform nodes, if defined, are displayed; details on this kind of nodes will be given in the following section. It may happen that, while browsing a configuration tree, the corresponding data-model changes on the ETICS System. In this case, because of the proxying policy adopted in the WA, the information available could be out-of sync. To synchronise the tree cache with respect to the latest project data- model, perform a refresh of the node, by clicking the ‘refresh’ button on the top-right corner of the workspace or by using the drop-down menu on the tree node.

5.2.6. Listing supported Platforms The expansion of a configuration node, besides revealing the configuration structure, often shows a number of ‘platform’ nodes; details of each of them can be displayed, again, by clicking on the node.

1 If fully qualified names are adopted, for clarity, just the most significant part of the name with respect to the parent name is displayed in the tree. The full name of the configuration is shown in the configuration details area.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 38 / 145 ETICS USER MANUAL

Figure 19: Supported Platforms Different specific platforms can be attached to a configuration, including the ‘Default Platform’ providing commands, properties, environments variables and dependencies to be used when no specific platform is defined.

5.2.7. Viewing Commands, Properties and Dependencies Expanding a platform node, platform-dependent items are displayed if defined for the corresponding platform. Namely VCS commands, Build Commands, Test Commands, Properties, Environment variables and Dependencies. As usual, details of any selected item are shown on the right side of the page. When a set of commands (i.e. Build, Test, VCS) is not defined for a configuration and a platform, the label of the node is rendered in italic style and gray colour. Commands, properties and environment variables viewers simply show attribute-value or key-value pairs. The Dependency viewer shows the list of required software for the current configuration. In addition, both the type (static and/or dynamic) and the scope (build and/or runtime) of each dependency are displayed beside each dependency entry.

Figure 20: Viewing configuration details

INFSOM-RI-026753 2007 © Members of ETICS collaboration 39 / 145 ETICS USER MANUAL

5.3. How to Browse with the Command-Line Client In this section, you will learn how to view and browse metadata using the command-line client. The following actions assume that the client has been correctly installed and configured and that the current workspace has been configured using the etics-workspace-setup command (see section 0).

5.3.1. How to Get a Project Before any action can be taken in a project, the project information (metadata) must be extracted from the ETICS metadata store into the workspace. The action of getting a project into the workspace can be compared to the action of setting the CVSROOT for CVS. It gives the context where all subsequent operations are performed. The command to get a project is:

etics-get-project <project-name>

If you are not sure of the project name or just want to know what’s available the following command can be used:

etics-list-project [options] [<project-name>]

This command by itself prints a list of the available projects. If the -d option is used, the properties of each project are also printed. An optional project name argument can be passed to get information only about a specific project.

All information retrieved by etics-get-project and other commands are cached locally in the store.xml file in at the root of the workspace. It is not recommended to manually edit this file, since the information may get out of synch with the ETICS server data-store.

5.3.2. How to Checkout and Browse Configurations Once a project has been inserted into the workspace, all information about the project structure (its subsystems and components) is available locally. However this information is not enough to perform builds or tests, since additional configuration information and the actual source code or binary packages are required. In order to get the configuration information, source code and packages downloaded and installed onto the workspace, the following command is used:

etics-checkout [options] [<module-name>]

The etics-checkout command has many options and its behaviour is described in details in the following chapter. For the moment we will look at a few simple cases. It is useful to note that the syntax of the etics-checkout command is similar to the syntax of the cvs checkout command. Used by itself, the etics-checkout command gets the HEAD configuration of the current project and all attached subsystems and components configurations and dependencies for the current working platform. More information about configurations is given in the following chapter (Chapter 6 - Checking out Project, Subsystems and Components), but in general we can assume that all modules in

INFSOM-RI-026753 2007 © Members of ETICS collaboration 40 / 145 ETICS USER MANUAL

the system have a HEAD configuration. The HEAD configuration is created, with each newly created module, with default values. The default HEAD configuration name is by convention:

module-name.HEAD

If the <module-name> argument is used, it is possible to check out the HEAD configuration of a specific module (subsystem or component) in the project tree:

etics-checkout <module-name>

In order to download a specific configuration from the system, the configuration name must be specified, just like you have to specify a tag name when checking out code from a CVS tag or branch:

etics-checkout -c <config-name> <module-name>

or

etics-checkout --config <config-name> <module-name>

If you are not sure of the configuration name of a module or just want to know what’s available the following command can be used:

etics-list-configuration [options] [<module-name>]

This command by itself prints a list of the available configurations for the current project. If the -d option is used the properties of each configuration are also printed. If the optional module name argument is passed, the command lists the configurations of that module. Additionally, the command

etics-list-property [options][<module-name>]

can be used to get a list of the properties of given module. If no module is provided as argument, the command prints the properties of the current working project.

5.3.3. How to Show the Structure of Modules and Configurations? The command:

etics-show-configuration-structure [options] [<module-name>]

can be used to get a visualise of the structure of a configuration. The command displays on the screen a list of all children and dependencies of the current configuration of the specified module in the order they would be built. As for the previous commands, the command etics-show- configuration-structure by itself prints the structure of the HEAD configuration of the

INFSOM-RI-026753 2007 © Members of ETICS collaboration 41 / 145 ETICS USER MANUAL

current project. A number of options are available to change the way the information is displayed (as a flat list instead of an indented hierarchical list) or to output an xml file instead of a text list. This command only work with local metadata therefore requires an etics-checkout to have been performed before.

etics-show-module-structure [options] [<module-name>]

accepts as argument the name of a module (could be a project, a subsystem or a component) and prints on the screen the module hierarchy. If no module is specified the command shows the structure of the current project. In order to get the structure of a component or a subsystem the corresponding option must be used (--component if it’s a component, --subsystem or -s if it’s a subsystem). The -d option is available to display more details.

5.3.4. Modules, Platforms, Users, Roles and Other Objects The ETICS command line client provides a set of tools to list and get information about several of the different objects in the ETICS metadata store. Some of these tools have been described earlier (etics-list-project, etics-list-configuration). The following additional commands are also available:

etics-list-platform [option] [<platform-name>]

can be used to get a list of the platforms supported by the ETICS system or get the details of a specific platform. The -d option can be used to get the platform properties. The --detect option displays the platform string corresponding to the local platform, which may or may not exist in the ETICS metadata store.

etics-list-user [option]

by default returns the list of users registered in the ETICS system. If -d option is used, it returns also the projects for which the users are registered and the roles they have on it. The --user (-u) <user-last-name> option can be used to get information about a specific user. With this option, the command lists the user DN (as defined in the user’s certificate) and the e- mail address. In addition, if the -d option is also given, the modules for which the user is registered and the related Roles are printed on the screen. Finally the --project (-p) <project-name> option can be used to return the list of users registered for a given project. If -d option is also used, more detailed information is printed out about the users and roles for each module within the project. The --user and --project options cannot be used together.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 42 / 145 ETICS USER MANUAL

# CHECKING OUT PROJECT, SUBSYSTEMS AND COMPONENTS This chapter describes how to checkout and download code onto the workspace.

6.1. Overview This section describes in more details how the etics-checkout command works to download configuration metadata, source code and binary packages from the ETICS metadata store, as well as source code and package repositories. All operations described in this chapter can be used by anonymous users and do not require configuring the client for secure access. However, if security is configured, the etics-checkout command prompt the user for the certificate passphrase if necessary.

6.2. Source Code or Binary Packages? Before going into the details of the etics-checkout command, it is necessary to understand what type of objects can checked-out. The ETICS system uses three types of data for software:

# Metadata: the configuration objects in the ETICS metadata store describing how a particular version of a module must be checked out, built or tested # Source code: the code stored in some VCS (Version Control System) like CVS or Subversion or stored as a tarball in the ETICS repository # Binary packages: pre-compiled tarballs stored in the ETICS repository.

For more information about the format of the ETICS repository, please refer to Chapter 13 (“Repository”). The metadata is always required to perform build and test operations. The choice of using source code from a version control system, source code from a tarball in the repository or binary tarballs from the repository depends on user requirements and on the availability of the packages. In the following sections we will explore the various options that affect what type of packages are fetched by the etics-checkout command. One more thing to take into account is that both the source and binary tarballs taken from the ETICS repository are cached locally in the ‘repository’ directory (by default there is one such directory in every workspace, but this can be changed by using the repository entry in the client configuration files). The repository directory structure is identical to the ETICS central repository structure, but contains expanded copies of the packages required by the current build/test operations. Conversely, code taken from version control systems is written locally in the workspace according to the instruction set by the ‘checkout’ target in the VCS Commands set of each configuration.

6.3. The etics-checkout Command As briefly seen in the previous chapter, the etics-checkout command is used to extract metadata and code or binary packages from various places. The command is at the base of all build and test operations, since no such operation can be performed without first having the required data locally. The syntax of the command is:

etics-checkout [option] [<module-name>]

INFSOM-RI-026753 2007 © Members of ETICS collaboration 43 / 145 ETICS USER MANUAL

The command etics-checkout by itself checks out the metadata and all associated source or binary packages for the HEAD configurations of the current project.

The command has several options that allow controlling how check out is executed. The following table shows the command options: Table 12: etics-checkout command options Option Description -h, --help Show the usage instructions -c, --config <configuration-name> Define a specific configuration to be used instead of the default <module-name>.HEAD. If the <module-name> argument is defined, the configuration applies to the module, otherwise it applies to the current project. --project <project-name> Specify a project name to be used instead of the current project (as defined by the etics-get-project command). Use this when checking out components configurations from other projects (for example from the 'externals' project). This option is often used with the --merge option. --project-config <project- Define a specific project configuration for the metadata. If this configuration-name> parameter is specified, the full project metadata is retrieved. This is required when checking out only parts of the project, but there is project-wide information that should be propagated. -p, --property <property=value> If already define, override existing properties, or define new ones. You can list the available properties and their default value using 'etics-list-property'. --platform <platform-name> Overwrite the local platform. --nocheckout Do not perform the actual VCS checkout and/or repository download. --nodeps Only checkout the currently specified module (do not checkout children and dependencies) --shallowbindeps When checking out using the --frombinary or --frombinaryonly options, dependencies of binary packages are not checked out --runtimedeps When checking out configurations, dependencies of run-time dependencies are also processed. This option is useful to make sure that the final list of packages contains everything needed to deploy the components --local Do not contact the server to download metadata. This will only work if the configuration information has already being downloaded (via this command, without this option). --volatile Use a named volatile repository to look for packages. If a package is not found in the volatile repository, it is searched for in the permanent repository, before giving up unless --volatileonly is used. This option cannot be used together with --defaultvolatile. --defaultvolatile Use the default volatile repository to look for packages. If a package is not found in the volatile repository, it is searched for

INFSOM-RI-026753 2007 © Members of ETICS collaboration 44 / 145 ETICS USER MANUAL

in the permanent repository, before giving up unless --volatileonly is used. This is equivalent to --volatile=default. This option cannot be used together with --volatile. --volatileonly Use only the specified volatile repository to look for packages. If a package is not found in the volatile repository, it is not searched for in the permanent repository --force Force the configuration checkout or download even if no changes are detected. Note: This option is equivalent to -- forcevcscheckout and --forcedownload. --forcevcscheckout Forces the checkout of the configuration from VCS, even if the corresponding module hasn't changed with respect to the workspace version. --forcedownload Forces the download of the configuration from the repository, even if the corresponding artefact hasn't changed with respect to the repository version --noask Doesn't ask user confirmation and assumes YES to all questions --fromsource When possible, check out source code instead of downloading binaries. --frombinary When possible, download binaries instead of checking out source code. This implies that packages will not be built. --fromsourceonly Check out source code only. This means that all the configurations, including dependencies have to be available in source code form or the operation will fail --frombinaryonly Checkout binaries only. This means that all the configurations, including dependencies have to be available in binary form. --continueonerror Continue when checkout and/or download errors are encountered. --merge Merge checkouts with current workspace. Without this option, the metadata stored in the workspace, from previous checkouts, is cleaned-up, while the code checked-out by the VCS commands and downloaded remains in the workspace. --verbose Print verbose messages. --version Display the client version.

6.4. How to Checkout Source Code or Binary Packages By default the etics-checkout command tries to extract from the VCS system or the repository the source code of the module specified as argument for the given configuration. If the module is a project or a subsystem, the command tries to check out the source code of all its children. If the source code is not available, the command tries to get binary packages. All dependencies different from the module or its children are taken in binary format if they exist, otherwise they are taken in source form. The described default behaviour is equivalent to say that the configurations of module specified as argument and its children (if any) are checked out as if the --fromsource option had been specified, while all dependencies are checked out as if the --frombinary option had been specified.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 45 / 145 ETICS USER MANUAL

This behaviour allows checking out and working on the source code of a module during a normal development session, without having to build all the external dependencies.

If one of the options --fromsource, --frombinary, --fromsourceonly or -- frombinaryonly is specified, it applies to all configurations.

For example:

etics-checkout -c etics_R_0_9_2 org.etics.build-system

checks out the Build and Test System subsystem of the ETICS system and all its children (e.g. client- py, packager, plugin-framework) in source format from the ETICS CVS repository and checks out the required external dependencies (e.g. ZSI, pyxml, Psyco) in binary format for the local platform.

etics-checkout --frombinaryonly -c etics_R_0_9_2 org.etics.build- system

checks out the Build and Test System subsystem of the ETICS system and all its children and dependencies in in binary format for the local platform. If one or more packages do not exist, the command fails and exits with a non-zero return code.

6.4.1. Merging Configurations When running the etics-checkout command, the local configuration store is normally reset to contain only the new configurations being checked out. However, there may be cases when one or more configurations have to be injected in an existing store, for example to try a different configuration of one component with the existing configurations of other components. The command:

etics-checkout --merge -c <configuration-name> <module-name>

perfoms such a merge operation. As a result of this operation, the new configuration (or configurations in case of subsystems) is added to the store, but it doesn’t replace existing configurations of the same module. Therefore it may be necessary to use the -c option when building to select one of the existing configurations (see section 9 - Building Configurations for more information).

6.4.2. Updating Configurations In case an existing configuration has changed in the repository or VCS system and it’s necessary to update the local store, a different command can be used. The etics-update command performs the same types of operation as etics-checkout but updates existing configurations using the same configuration names and version numbers.

etics-update <module-name>

INFSOM-RI-026753 2007 © Members of ETICS collaboration 46 / 145 ETICS USER MANUAL

updates the currently stored configuration of the module <module-name>. The etics-update command has the same or equivalent options as etics-checkout with the exception of the -c option.

6.4.3. Forcing Checkout The standard behaviour of the etics-checkout command is to execute any given action once and then cache some information on disk that prevents executing again the same action if nothing in the code or binary package has changed. This allows speeding up the operation of checking out components if up-to-date packages or code are already available in the workspace. However in certain occasions it may be necessary to perform the same checkout again, for example because something has changed in a place outside of ETICS’s control or the same action has to be repeated with different options. In this case, the build can be force using the --force option:

etics-checkout --force -c <configuration-name> <module-name>

It is also possible to force only the check out of code from a VCS repository without refreshing the packages in the package repository (--forcevcscheckout) or to force only the download of packages from the package repository without refreshing code from the VCS system (-- forcedownload)

6.4.4. Permanent and Volatile Repositories ETICS provides two types of repositories: * The permanent repository at http://eticssoft.web.cern.ch/eticssoft/repository is used to store packages that can be officially published by a project. The packages are stored in the repository when a remote build is submitted with the option:

--register

Packages in the permanent repository are never overwritten even if the package is rebuilt. * The volatile repositories are custom namespaced repositories at http://eticssoft.web.cern.ch/eticssoft/builds/ where namespace is a string supplied by the user. Volatile repositories are used to temporarily store packages during the development phase when they are not yet ready for public general availability and can be used to categorize the builds or provide teamwork support on specific tasks. Packages are store in the volatile repositories when submitting a remote build with the option:

--register-volatile

Packages in the volatile repositories are usually purged every two months (or less depending on available space) and the packages are overwritten if they are built again. All packages built by submitting a remote build without the --register-volatile option are registered in the default volatile repository at http://eticssoft.web.cern.ch/eticssoft/builds/default

INFSOM-RI-026753 2007 © Members of ETICS collaboration 47 / 145 ETICS USER MANUAL

The etics-checkout command provides the possibility of checking out code from a volatile repository only or from a combination of a volatile repository and the permanent repository. The command

etics-checkout -c <config-name> --volatile=<my-namespace> <module- name>

will try to checkout the configuration config-name of the module module-name from the volatile repository called <my-namespace>, if a package cannot be found in the volatile repository, the client looks in the permanent repository. The command:

etics-checkout -c <config-name> --volatile=<my-namespace> -- volatileonly <module-name>

will try to checkout the configuration config-name of the module module-name from the volatile repository called <my-namespace>, if a package cannot be found the client stops. The shortcut option --defaultvolatile can be used to indicate that packages have to be looked for in the default volatile repository http://eticssoft.web.cern.ch/eticssoft/builds/default

INFSOM-RI-026753 2007 © Members of ETICS collaboration 48 / 145 ETICS USER MANUAL

7. Editing Projects

7.1. Overview This chapter describes how to edit information (i.e. metadata) about project, subsystems, component, configuration and all related objects using the ETICS Web Application and the ETICS Client. All methods described in this chapter require the users to authenticate themselves using a digital certificate (see section 3.3 “Enabling Security” for details).

7.2. How to Edit with the Web Application This chapter describes how to edit project, subsystems, component, configuration and all related objects information (i.e. metadata) using the ETICS Web Application. All methods described in this chapter require the users to authenticate themselves using a digital certificate (see section 3.3 “Enabling Security” for details). In this section, you will learn how to view and browse metadata using the web application. All editing functionalities can be activated through the ‘edit’ button available in the main panel for most object types. Alternatively, editable items provide an ‘edit’ item in their contextual menu.

Figure 21 - Editing Etics Objects

7.2.1. Editing Modules Creating new Modules According to the ETICS data-model, projects can contain both subsystems and components, whereas subsystems can contain components only. A new subsystem or component can be created using the buttons in the toolbar as well as using the contextual menu on the parent module.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 49 / 145 ETICS USER MANUAL

Figure 22: Creating new modules An empty form will appear in the details area, allowing the user to provide metadata. When finished, click on the ‘save’ button to store the new module or ‘exit’ to abort the action.

Figure 23: Editing newly created module form Modifying existing Modules A module can also be modified after its creation using the ‘edit’ button or through the corresponding item in the contextual menu. A form will appear showing the current metadata of the selected module. Most entries can be modified. When finished, click ‘save’ to update the module metadata or ‘exit’ to abort the action.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 50 / 145 ETICS USER MANUAL

Removing Modules A module can be removed using the ‘delete’ button in the main panel or through the corresponding item in the contextual menu of the selected node.

Figure 24: Deleting modules Note: The removal of a module, also deletes its sub-modules (i.e. subsystems and components), all its configurations and related objects (commands, properties, dependencies, etc.). A project cannot be deleted via the web application. To delete a project, send a request to etics-support@cernNOSPAMPLEASE.ch.

7.2.2. Editing Configurations Configuration editing can be activated through the ‘edit’ button in the main panel or through the popup menu associated with each item in the workspace or in the configuration list. Creating new Configurations A new configuration for a module can be created using the ‘new configuration’ button in the toolbar or through the contextual menu on the selected module.

Figure 25: Creating configurations

INFSOM-RI-026753 2007 © Members of ETICS collaboration 51 / 145 ETICS USER MANUAL

An empty form will appear in the details area, allowing setting values for user-editable metadata. When finished, click on the ‘save’ button to store the new module or ‘exit’ to abort the action. Modifying existing Configurations A configuration can be modified using the ‘edit’ button or through the corresponding item in the contextual menu of the selected configuration. A form will appear showing the current metadata of the selected configuration. Most entries can be modified. When finished, click ‘save’ to update the configuration metadata or ‘exit’ to abort the action. Removing Configurations A configuration can be removed using the ‘delete’ button in the main panel or through the corresponding item in the contextual menu of the selected configuration.

Figure 26: Deleting configurations Note: The removal of a configuration, also deletes its related objects (e.g. commands, properties, dependencies), and its relationships with its sub-configurations. Further, sub-configurations are not removed and remain attached to the corresponding modules. Cloning Configurations The ‘Clone’ operation creates a new configuration with the very same metadata (with the only exception of the configuration id and name), including commands, properties, environment, sub- configurations relationships and dependencies relationships as the cloned configuration. It is worth mentioning that the ‘Clone’ operation is not recursive: related configurations are not recursively cloned, where only new relationships with existing ones are created. The ‘clone’ operation can be activated using the ‘clone’ button in the toolbar or through the corresponding item in the contextual menu of the selected configuration.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 52 / 145 ETICS USER MANUAL

Figure 27 - Cloning Configurations Locking Configurations The ‘Lock’ operation makes a configuration no longer modifiable. In particular, its metadata, commands, properties, environment, dependencies and sub-configurations are ‘frozen’ to their current status. For the lock to be effective, children configurations and dependencies must also be locked. For the children there are generally no authorization issues because the grant to lock is propagated to children items; for the dependencies, on the other hand, the user should have ‘locking’ privilege on the dependency configuration. If one of the related configurations cannot be locked, the whole lock operation will fail. The ‘lock’ operation can be activated by using the ‘lock’ button in the toolbar or through the corresponding item in the contextual menu of the selected configuration.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 53 / 145 ETICS USER MANUAL

Figure 28: Locking configuration Subsystem and Component configurations are generally built in the context of a Project/Subsystem configuration from which they inherit properties and from which dynamic dependencies are resolved. In order to reproduce the build in this context without locking the Project/Subsystem configuration itself, it’s possible to specify a ‘context’ configuration to use when locking. Specifying a context allows building a locked configuration alone as if it was built in the context of an ancestor configuration, thus providing all inherited properties and resolved dependencies. In the WA, the context configuration can be selected by using a drop down list.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 54 / 145 ETICS USER MANUAL

Figure 29: Selection of context configuration for locking Note that if the configuration being locked is not a sub-configuration of project or subsystem configuration, the ‘Context’ section of the above shown form is not displayed.

7.2.3. Editing Sub-Configuration Relationships The edit functionality for Sub-Configuration relationships is triggered using the ‘edit sub- configuration’ button in the toolbar or through the corresponding item of the popup-menu; this can be done either in the workspace or in the configuration list.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 55 / 145 ETICS USER MANUAL

Figure 30: Editing sub-configuration relationships The edit area is then updated with a view showing a list of module-configuration association. The information provided in the view is actually the same as the one provided by the configuration tree. Beside each module name, a drop-down list allows to select a configuration from a list. After the selection, the chosen configuration will then appear in the module-configuration association, replacing any previously existing configuration. At the same time, the configuration tree on the left is also updated.

Figure 31 - Editing sub-configuration relationships Removing Sub-Configurations Sub-configuration relationships can also be removed by using the drop-down list in the sub- configuration editor: this is done by selecting the entry labelled ‘—empty—‘. The previously- associated configuration will then be removed from configuration tree.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 56 / 145 ETICS USER MANUAL

Figure 32: Removing sub-configuration menu Note that removing a sub-configuration is different from deleting it, since in the case of delete the configuration is itself deleted, instead of simply removing the sub-configuration relationship.

7.2.4. Attaching Platforms to Configurations With the only exception of Sub-Configurations, most configuration-related entities (e.g. commands, properties, dependencies, etc.) are platform-dependent. Thus, before creating any of such entities, a platform has to be selected. Adding the support for a specific platform, among those available in the ETICS system, is done selecting a configuration in the workspace and then clicking the ‘Attach Platform’ button in the toolbar. The required platform can then be selected from the list displayed1:

1 Some platform entries might be greyed-out; this means that the support for the platform is already present for the selected configuration. This can be verified by expanding the configuration node in the tree.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 57 / 145 ETICS USER MANUAL

Figure 33: Attaching platforms menu After the selection, a new platform node is created under the configuration node. This node is the starting point for creating further platform-dependent entities.

7.2.5. Editing Commands Commands (i.e. VCS Commands, Build Commands and Test Commands) can be modified using the ‘edit’ button in the main panel or through corresponding item in the contextual menu of command node. If a command is not defined for a platform, the corresponding node is rendered in italic style and gray colour. In this case it’s sufficient to click on them to start editing.

Figure 34: Editing Build Command form When finished, click ‘save’ to store the command metadata or ‘exit’ to abort the action. Commands can be removed clicking the ‘remove’ button in toolbar or by using the ‘delete’ entry from the context-menu associated to the command node.

Figure 35: Deleting Build Commands

7.2.6. Editing Properties and Environment

INFSOM-RI-026753 2007 © Members of ETICS collaboration 58 / 145 ETICS USER MANUAL

For each configuration and platform, a set of properties and environment variables can be specified. When a new platform is attached to a configuration those sets are initially empty. They can be created selecting the properties or environment nodes, and then clicking the ‘edit’ button.

Figure 36: Editing environment variables menu The details area on the right will then display a list of key-value pair. New properties/environment can be created clicking on the ‘new property/environment’ button in the vertical toolbar. A form asking for property/environment name and value will appear in the editor.

Figure 37: Editing property form Existing properties/environment can also be modified by simply updating the value of fields. A newly created entry as well as a modified one will display a little orange bar on their left side. That means that there are unsaved changes for the entry. To persist the changes click the ‘save’ button associated to the entry; alternatively, the ‘cancel’ button will undo either the modification.

7.2.7. Editing Dependencies For each configuration and platform, a set of dependencies can be specified. When a new platform is attached to a configuration the dependency set is initially empty. New dependencies can be specified clicking the ‘edit’ button in the toolbar or selecting the ‘edit’ item from the context-menu associated to the dependencies node. The details area on the right will then display a list of dependencies (if any). When in edit mode, new configurations can be added as dependencies, by first selected from either the configuration list, the workspace1 or the project tree. Details of the selected configuration will be shown in a floating box. To set this configuration as dependency, click the ‘add as dependency’ button in the vertical toolbar of the floating box. As a consequence the new configuration will appear in the dependency list.

1 The content of the configuration tree and configuration list can be updated as usual

INFSOM-RI-026753 2007 © Members of ETICS collaboration 59 / 145 ETICS USER MANUAL

Figure 38: Add dependency floating box Selecting a module from the project tree will result in adding a dynamic dependency. At this point, the dynamic dependency can be defined in terms of version, with or without an inequality, as shown in the figure below.

Figure 39: Dependency definition by version Dependencies selected from the configuration list and the workspace are automatically added as static dependencies. Existing entries can be modified; in this case, an orange bar is put on the left of the dependency telling that there are unmodified changes. To persist changes click the ‘save’ button. Individual dependencies can also be removed by using the ‘remove’ button on the right side of each entry.

7.3. How to Edit with the Command-Line Client This section describes how the edit commands work. These commands are applied to all the objects part of the ETICS data model described in Section 1.2 (i.e., modules, configurations, resources, platforms, users, and role). They support the following operations: add that registers a new object; clone creates a new copy of an existing configuration (only applicable for the configurations); remove that deletes an existing object; modify that updates object information and finally prepare that serialise a module of configuration in an ini formatted file.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 60 / 145 ETICS USER MANUAL

The parameters that characterize an object in the operations add and modify, are provided in three different modes: # interactively, by requesting the user to provide the needed information; # by using a ini file; # by giving information in the command line.

Interactively One of the modes to add and modify the objects of ETICS is to provide interactively the information to the system. The interactive mode is used by default when neither the ini file nor the --param option is given. The user is then requested to provide at the command-line values for the attributes of the object that have to be added/modified. Each attribute is shown in alphabetical order following this syntax:

<name-of-the-attribute>[<default-value>]:

The name-of-the-attribute is the name used by ETICS to describe a feature of the object (see the tables in Section 2 in order to have the list of the attributes for each type of object). The default-value is the value that the system will give to the attribute if the user doesn’t change it. It could be the current value of the attribute if the operation is modify or a default value inherited by its parents if the operation is add. If the default-value is None, it means that no value is specified for that attribute.

After having printed a line like the previous one, the system waits for an input by the user. The possibilities of input by the user are: − to press Enter, in this case the user doesn’t want to change the value of the attributed and the one between square brackets is used − to write the new value and then press Enter, in this case the new value is used.

If a name is followed by ‘***’, it means that a value is required and that the user must provide a value. Some attributes cannot be changed because they are related to other objects - e.g. the attribute moduleName for a configuration. In this case ETICS shows the name of the attribute followed by its value but it doesn’t wait for an input and continues the execution to the next attribute.

Once all the attributes are processed, the system saves the new values. If the execution is stopped before the last attribute, all the changes are discarded. This schema is used for all type of objects and also for the commands.

Via Ini File Another way to provide parameters in the ETICS infrastructure is by using the ini mode during the add and modify operations. In order to use this mode the operation prepare, only applicable to module, configuration, resource and platform objects, has to be used to create an ini file in the user workspace. If the related object already exists in the ETICS data model, the file created will contain the current information; otherwise, the file created will contain default values. The default name of the ini file has the form <object-type>-<object-name>.ini, for example,

INFSOM-RI-026753 2007 © Members of ETICS collaboration 61 / 145 ETICS USER MANUAL

Component-org.glite.wms.common.ini Configuration-org.glite.wms.common.HEAD.ini Resource-joda.cnaf.infn.it Subsystem-org.glite.wms.ini

Below you will find a partial schema of the ini file. As you can see, there are several sections: the first one is common in all the object ini files; the second section is only present in the module ini file; all the other sections are presented in the configuration ini file, and the last section is not included in the component configuration ini file. The dynamicDependency entry supports two syntaxes: the former specifies just the dependency type by the scope, the latter adds a constraint

[<Object-type>-<configuration-name>] … [Parent] <parent-type>=<parent-name> [Platform-<platform-name>:BuildCommand] … [Platform-<platform-name>:TestCommand] … [Platform-:VcsCommand] … [Platform-:Property] ;var1 = None [Platform-:Environment] ;var1 = None [Platform-:StaticDependency] ;<project-name>|<module-name> = <conf-name>,<dependency-type> [Platform-:DynamicDependency] ;<project-name>|<module-name> = ;<project-name>|<module-name> = <module-name>, [Hierarchy] ;<children-module-name> = <children-conf-name>

The operation add performs the insertion of new objects in the ETICS data model and it is applicable to module, configuration, platform and resource objects. For a module object, the operation automatically creates a module configuration called <module-name>.HEAD. The operation clone performs a copy of an existing configuration. It is only applicable to configuration objects. The operation modify updates an existing object and it is applicable to module, configuration, platform, and resource objects. The operation rename changes the name of an existing module or configuration. It is only applicable to module and configuration objects. The operation remove performs the deletion of an existing object and all its children. It can fail if the object itself (or its children) is used by other objects (e.g. the configuration is a dependency of another configuration or the configuration is specified as a child of the main configuration). Currently, these operations first commit the changes remotely and then locally. This behaviour is being changed in a future release,

INFSOM-RI-026753 2007 © Members of ETICS collaboration 62 / 145 ETICS USER MANUAL

where changes will only take place locally, and an explicit command will be required to commit the changes on the server.

7.4. THE ETICS-MODULE COMMAND The ETICS command to edit a module is called etics-module. The syntax of the command is

etics-module [options] [<module-name>]

The following table shows the options: Table 13: etics-module command options Option Description -h, --help Show the usage instructions -I, --input Specify the filename from which to get module parameters (ini file) -o, --output Specify the filename where to copy module parameters (ini file) -f, --force If a destination file already exists, overwrite it without confirmation --param <param-value> Allow user to specify parameters of the object module --parent <parent-name> Specify the parentName of the specified module. It is mandatory during the operations add and remove of a component that belongs to a subsystem --component Specify that the moduleName is a component. It is mandatory during the operation add and remove -s, --subsystem Specify that the moduleName is a subsystem. It is mandatory during the operation add and remove --new-module --noask Do not ask the user to confirm the deletion. It is valid only for the remove operation --autocommit Commit module information first in the DB and then in the user work area. Its default value is True. --noautocommit Commit module information in the user work area. Its default value is False. To commit module information in the DB, the command etics-commit has to be used. --new Create a ini file for a new module. --verbose Print verbose messages.

During the operation add and modify, the module-name is mandatory in the following two cases: # the option -i, --input is not used; # the option -i, --input is used and the name of the module has to be changed.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 63 / 145 ETICS USER MANUAL

7.4.1. How to prepare a module A module template file in the ini format is created using different syntaxes of the command etics- module according to the type of module in the ETICS metadata store. This operation can be applied to module which type is a Project, or a Component or a Subsystem.

In order to create a component ini file which parent is the project, the ETICS command to be run is:

etics-module prepare --component <module-name>

This command serialises a module into a file called Component-<module-name>.ini which body contains either default values if the module doesn’t exist, or the current values of the module if it already exists in the ETICS metadata store. For instance, if a user prepares an ini file for a new component called voms.api, which parent is the project called OMII-Europe, the file Component- voms.api.ini will be created, and its body will be:

[Component-voms.api] licenseType=None displayName=Voms.api description= repository=http://eticssoft.web.cern.ch/eticssoft/repository/ download=None vendor=The OMII-Europe Consortium packageName=None homepage=None vcsroot=None

[Parent] Project=OMII-Europe

In order to create a component ini file which parent is a subsystem, the ETICS command to run is:

etics-module prepare --parent <subsystem-name> --component <module- name>

This command serialises a module into a file called Component-<module-name>.ini which body contains either default values if the module doesn’t exist, or the current values of the module if it already exists in the ETICS metadata store. For instance, if a user prepares a ini file for an existing component called voms.api-cpp, which parent is the subsystem called voms, the file Component- voms.api-cpp.ini will be created, and its body will be:

[Component-voms.api-cpp] licenseType=EGEE license displayName=voms.api-cpp description=CPP API for VOMS repository=:pserver:nonymous@glite.cvs.cern.ch:/cvs/glite

INFSOM-RI-026753 2007 © Members of ETICS collaboration 64 / 145 ETICS USER MANUAL

download=None vendor=The OMII-Europe Consortium packageName=voms.api-cpp homepage=http://voms.forge.cnaf.infn.it vcsroot=:pserver:nonymous@glite.cvs.cern.ch:/cvs/glite

[Parent] Subsystem=voms

In order to create a subsystem ini file which parent is the project, the ETICS command to be run is:

etics-module prepare -s <module-name>

This command serialises a module into a file called Subsystem-<module-name>.ini which body contains either default values if the module doesn’t exist, or the current values of the module if it already exists in the ETICS metadata store. For instance, if a user prepares a ini file for an existing subsystem called voms, which parent is the project called OMII-Europe, the file Subsystem-voms.ini will be created, and its body will be:

[Subsystem-voms] vendor=The OMII-Europe Consortium displayName=voms description=voms repository=:pserver:nonymous@glite.cvs.cern.ch:/cvs/glite vcsroot=:pserver:nonymous@glite.cvs.cern.ch:/cvs/glite

[Parent] Project=OMII-Europe

7.4.2. How to add a module The command to add a module in the ETICS infrastructure has different syntaxes according to the way the parameters are provided. In addition this operation automatically creates a configuration, called <module-name>.HEAD for the new module. This operation can be applied to a module which type is a Component or a Subsystem. Interactively The command to add a module providing its parameters interactively is one of the following:

* etics-module add --component <module-name> in order to add the component <module-name> to the project;

* etics-module add --subsystem <module-name> in order to add the subsystem <module-name> to the project;

INFSOM-RI-026753 2007 © Members of ETICS collaboration 65 / 145 ETICS USER MANUAL

* etics-module add --parent <parent-name> --component <module- name> in order to add the component <module-name> to the parent <parent-name>. Via ini file The command to add a module providing its parameters via ini file is one of the following:

etics-module add -i Component-<module-name>.ini --component <module- name> or etics-module add -i Component-<module-name>.ini in order to add the component <module-name> to the project;

etics-module add -i Subsystem-<module-name>.ini -s <module-name> or etics-module add -i Subsystem-<module-name>.ini in order to add the subsystem <module-name> to the project;

-etics-module add -i Component-<module-name>.ini --parent <parent-name> --component <module-name> or etics-module add -i Component-<module-name>.ini in order to add the component <module-name> to the parent <parent-name>. Via command line The command to add a module providing its parameters via command line is one of the following:

etics-module add --param description=”new module” --component <module-name> in order to add the component <module-name> to the project;

etics-module add --param description=”new module” --subsystem <module-name> in order to add the subsystem <module-name> to the project;

etics-module add --param description=”new module” --parent <parent- name> --component <module-name> in order to add the component <module-name> to the parent <parent-name>.

7.4.3. How to modify a module The command to modify a module in ETICS has different syntaxes according to the way the parameters are provided. This operation can be applied to all module types – i.e. project, subsystem or component. Interactively

INFSOM-RI-026753 2007 © Members of ETICS collaboration 66 / 145 ETICS USER MANUAL

The command to modify a module providing its parameters interactively is one of the following:

etics-module modify <module-name> etics-module modify --subsystem <module-name> etics-module modify --component <module-name> Via ini file The command to modify a module providing its parameters via ini file is one of the following:

etics-module modify -i Component-<module-name>.ini etics-module modify -i Subsystem-<module-name>.ini etics-module modify -i Project-<module-name>.ini Via command line The command to modify a module providing its parameters via command line is one of the following:

etics- module modify --param description=”new module” <module-name> etics-module modify --param description=”new module” --subsystem <module-name> etics-module modify --param description=”new module” --component <module-name>

7.4.4. How to rename a module The command to rename a module in ETICS has different syntaxes according to the way the parameters are provided. This operation can be applied to all module types – i.e. project, subsystem or component. Via command line The command to modify a module providing its parameters via command line is one of the following:

etics- module rename --new-module <module-name> etics-module rename --new-module --subsystem <module-name> etics-module rename --new-module --component <module-name>

7.4.5. How to remove a module The command to remove a component module is: Via command line etics-module remove --component <module-name>

The command to remove a subsystem module is: Via command line etics-module remove -s <module-name>

INFSOM-RI-026753 2007 © Members of ETICS collaboration 67 / 145 ETICS USER MANUAL

This operation can be applied to module which type is a Component or a Subsystem.

7.5. The etics-configuration Command The ETICS command to edit a module is called etics-configuration. The syntax of the command is

etics-configuration [options] [-c configuration-name] [configuration-name-cloned] <module-name>

The following table shows the options: Table 14: etics-configuration command options Option Description -h, --help Show the usage instructions -i, --input Specify the filename from which to get configuration parameters -o, --output Specify the filename where to copy configuration parameters -f, --force If a destination file already exists, overwrite it without confirmation --param <param-value> Override existing or define new properties --new-config name of either a cloned configuration or a renamed configuration. --new-input Specify the filename from which to get new configuration information. --forceremoval Force the removal when the configuration is linked by other configuration. Other configuration could be destroyed by using this option. It is valid only for the remove operation. --noask Do not ask the user to confirm the deletion. It is valid only for the remove operation -p, --project <project-name> Specify the project name to which the configuration belongs. --parent-config configuration to be locked belongs to. --recursive Force the recursion for clone operation if the configuration belongs to a project or a subsystem. Its default value is False. --recursive-suffix <suffix-value> Specify a suffix to append during a clone operation. Its default value is ‘cloned’. --autocommit Commit configuration information first in the DB and then in the user work area. Its default value is True. --noautocommit Commit configuration information in the user work area. Its default value is False. To commit configuration information in the DB, the command etics-commit has to be used. --new Create a ini file for a new configuration. --verbose Print verbose messages.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 68 / 145 ETICS USER MANUAL

During the operation add and modify, the <module-name> and the option -c <configuration-name> are mandatory in the following two cases: # the option -i, --input is not used; # the option -i, --input is used and the name of the configuration has to be changed. The option -p, --project can be used during the operations prepare, add and modify, if the configuration belongs to a module of a different project respect to what get by using the command etics-get-report. In this case, the changes done by using the operations add and modify will not be saved in the database. Therefore it is suggested to use it together with the option -- noautocommit in order to avoid users to obtain an error message from the command etics- configuration.

7.5.1. How to prepare a configuration The command to serialise a configuration into an ini formatted file is:

etics-configuration prepare -c <configuration-name> <module-name>

This command serialises a configuration into a file called Configuration-<configuration-name>.ini which body contains either default values if the configuration doesn’t exist, or the current values of the configuration if it already exists in the ETICS metadata store

For instance, if a user prepares an ini file for a new configuration called module.TEST, the file Configuration-module.TEST.ini will be created, and its body will start with:

[Configuration-module.TEST]

For instance, if a user prepares an ini file for an existing configuration such as voms.HEAD, associated with the subsystem voms, the file Configuration-voms.HEAD.ini will be created, and its body will be:

[Configuration-voms.HEAD] moduleName = voms projectName = org.glite displayName = voms.HEAD description = None age = 0 tag = HEAD path=${projectName}/${moduleName}/${version}/${platformName}/${packa geName}-${version}-${age}.tar.gz version = 1.0.0

[Platform-None:BuildCommand] ;init = None ;install = None ;checkstyle = None ;clean = None ;displayName = None

INFSOM-RI-026753 2007 © Members of ETICS collaboration 69 / 145 ETICS USER MANUAL

;compile = None ;configure = None ;prepublish = None ;packaging = None ;test = None ;doc = None ;description = None ;publish = None ;postpublish = None

[Platform-None:TestCommand] ;clean = None ;init = None ;displayName = None ;description = None ;test = None

[Platform-None:VcsCommand] ;branch = None ;tag = None ;displayName = None ;description = None ;checkout = None ;commit = None

[Platform-None:Property] ;var1 = None

[Platform-None:Environment] ;var1 = None

[Platform-None:StaticDependency] ;<project-name>|<module-name> = <conf-name>,

[Platform-None:DynamicDependency] ;<project-name>|<module-name> = ;<project-name>|<module-name> = <module- name>,

[Hierarchy] voms.all = voms.all.HEAD voms.api-cpp = voms.api-cpp.HEAD

INFSOM-RI-026753 2007 © Members of ETICS collaboration 70 / 145 ETICS USER MANUAL

Reading this file, we observe that the configuration voms.HEAD related to the subsystem voms contains two sub-configurations, one of the component voms.all and the other one of the component voms.api-cpp. We also observe that this configuration does not contain any platform.

For instance, if a user prepares an ini file for existing configuration such as voms.api-cpp.HEAD, associated with the component voms.api-cpp, the file Configuration-voms.api-cpp.HEAD.ini will be created, and its body will be:

[Configuration-voms.api-cpp.HEAD] version = 1.0.0 moduleName = voms.api-cpp projectName = org.glite displayName = voms.api-cpp.HEAD description = None age = 1 tag = Unused path = ${projectName}/${moduleName}/${version}/${platformName}/${packageNam e}-${version}-${age}.tar.gz

[Platform-default:BuildCommand] postpublish = None packaging = None displayName = None description = None doc = None prepublish = None publish = None compile = mkdir -p build; cp -R stage/lib build init = mkdir -p stage/lib; cp ${stageDir}/lib/libvomsapi* stage/lib install = cp -R build/lib ${prefix} clean = None test = None checkstyle = None configure = None

[Platform-default:VcsCommand] tag = None displayName = None description = None branch = None commit = None checkout = echo "VCS for voms.api-cpp"; mkdir -p voms.api-cpp

INFSOM-RI-026753 2007 © Members of ETICS collaboration 71 / 145 ETICS USER MANUAL

[Platform-default:TestCommand] ;clean = None ;init = None ;displayName = None ;description = None ;test = None

[Platform-default:Property] ;var1 = None

[Platform-default:Environment] ;var1 = None

[Platform-default:StaticDependency] OMII-Europe|voms.all = voms.all.HEAD,BR

[Platform-default:DynamicDependency] ;<project-name>|<module-name> =

Reading this file, we observe that the configuration voms.api-cpp.HEAD related to the component voms.api-cpp is associated to the platform default. For that Platform, this configuration has one BuildCommand, one VcsCommand, and one StaticDependency with the configuration of the component voms.all.

It is possible to have more than one platform associated to the same configuration. In this case the ini file will contain all information presented in the ETICS server data-store.

7.5.2. How to add a configuration The command to add a configuration in the ETICS infrastructure has different syntaxes according to the way the parameters are provided. Interactively The command to add a configuration providing its parameters interactively is:

etics-configuration add -c <configuration-name> <module-name>

After having added the configuration in the standard interactive mode, the system shows a menu with different options in order to allow the user to associate platforms to the new configuration. This menu will be explained in the section 6.5.4. Via ini file The command to add a configuration providing its parameters via ini file is:

etics-configuration add -i Configuration-<configuration-name>.ini

INFSOM-RI-026753 2007 © Members of ETICS collaboration 72 / 145 ETICS USER MANUAL

Via command line The command to add a configuration providing its parameters via command line is:

etics-configuration add --param description=”new configuration”-c <configuration-name> <module-name>

7.5.3. How to clone a configuration The command to clone a configuration is: Via command line etics-configuration clone -c <configuration-current-name> <configuration-new-name> <module-name>

But it is also possible to do as follows when the configuration belongs to a subsystem or a project:

etics-configuration clone --recursive -c <configuration-current- name> <configuration-new-name> <module-name>

The configuration itself with its children will be cloned and their new name will contain the suffix ‘cloned’.

etics-configuration clone --recursive --recursive-suffix <suffix- value> –new-config -c <configuration- current-name> <configuration-new-name> <module-name>

The configuration itself with its children will be cloned and their new name will contain the suffix <suffix-value>.

7.5.4. How to modify a configuration The command to modify a configuration in the ETICS infrastructure has different syntaxes according to the way the parameters are provided. Interactively The command to modify a configuration providing its parameters interactively is:

etics-configuration modify -c <configuration-name> <module-name>

The operation modify of a configuration also allows the user to edit the associated platforms, the commands, dependencies, environments and properties linked to it and to set the hierarchy of the configuration (i.e. specify children configurations). This command, when used interactively, shows a menu with different options: # Modify the general configuration parameters: this option is used for changing the attributes of the configuration. # Support a new platform for this configuration: this option is used for supporting a new platform. The system displays the platforms currently supported for the configuration and asks

INFSOM-RI-026753 2007 © Members of ETICS collaboration 73 / 145 ETICS USER MANUAL

for the name of the new one. If an already supported platform or a non existing one is given, an error is raised and it goes back to the main menu. If the name of the platform is correct, the user is requested to insert the value for the types of objects to add to the platform (i.e. commands, dependencies, properties and environment variables). None of those are mandatory but at least one of them must be added. The build, vcs and test commands are unique for each configuration and platform and therefore only one object per type can be inserted. For the other objects the system accepts multiple insertions; it move to the following type if a blank or inconsistent value is given. # Modify configuration options (build/vcs/test commands, properties, environments, dependencies): this option is used for modifying the values of the objects linked to a platform already associated to the configuration. It lists the associated platforms and the user must choose one of them. Then it shows the names of the objects defined for that platforms and allows the user to add/modify/remove them.

# Modify the configuration’s children in the configuration’s hierarchy: this option is used for defining the hierarchy of the current configuration. The children, if any, are shown and the user can delete them. It is also possible to add new children choosing them among the configurations of the modules that are children of the module of the configuration. For example, if the user is modifying a configuration of a subsystem that has two components as children, this option allows to define a child configuration for each of the components. # Exit Via ini file The command to modify a configuration providing its parameters via ini file is:

etics-configuration modify -i Configuration-<configuration-name>.ini

By default the information contained in the ini file is considered to be complete, this means that if there is more information in the server-side metadata store than in the ini file, once this command is executed only the information contained in the ini file will remain in the metadata store.

Via command line The command to modify a configuration providing its parameters via command line is:

etics-configuration modify --param description=”new configuration” - c <configuration-name> <module-name>

7.5.5. How to remove a configuration The command to remove a configuration is:

etics-configuration remove -c <configuration-name> <module-name>

7.5.6. How to rename a configuration The command to rename a configuration is:

Via command line

INFSOM-RI-026753 2007 © Members of ETICS collaboration 74 / 145 ETICS USER MANUAL

etics-configuration rename --new-config -c <configuration-name> <module-name>

But it is also possible to do as follows when the configuration belongs to a subsystem or a project:

etics-configuration rename --new-input Configuration-.ini -i Configuration- <configuration-name>.ini

The configuration itself with its children will be renamed according what it is specified in the ini file Configuration-.ini.

7.5.7. How to lock a configuration The command to lock a configuration is:

Via command line

etics-configuration lock --parent-config -c <configuration-name> <module-name>

7.6. THE ETICS-COMMIT COMMAND The ETICS command to commit changes performed by using the commands etics-module and etics-configuration is called etics-commit. The syntax of the command is

etics-commit

INFSOM-RI-026753 2007 © Members of ETICS collaboration 75 / 145 ETICS USER MANUAL

8. Tagging Configurations

8.1. Overview Previous chapters described how to create configurations using the standard command-line client. During development and maintenance users may need a quick and simple way to add new configurations based on existing ones as a consequence of fixing code bugs or making minor modifications that do not require big changes in the configuration commands or properties. In this case the etics-tag command can be used.

8.2. The etics-tag Command The etics-tag command creates a configuration from an existing one and optionally tags the code in the underlying Version Control System. This command is logically equivalent to the CVS tag command where this latter creats a new tag on an existing branch. The syntax of the command is:

etics-tag [options] [<module-name>]

The command has several options that allow controlling how the tagging is executed: Table 15: etics-tag command options Option Description -h, --help Show the usage instructions -c, --config <configuration-name> The configuration name to be created from the current one. Required to create new configurations. If this option is used, the operation only applies to the specified module (implies -- nodeps). If the <module-name> argument is provided, the configuration applies to the module, otherwise it applies to the current project. -t, --tag The VCS tag string to be used for tagging. If this option is not specified the configuration name is used as tag string --config-version The version of the configuration to be created/updated in the form x.y.z-r. -i, --inputfile An input file in the ETICS INI format to be used. If this file is provided, the --tag and --config-version options are ignored (the values are taken from the file). The operation is performed on the current configuration of the specified module if the configuration name in the INI file matches it or a new configuration is created if the name doesn't match --prepare-childlist Using this option it is possible to create a template childlist file containing an entry for each component in the subsystem (and the subsystem itself) with the current configuration name, version and tag. This file can be edited and passed to the – childlist option to tag new versions of a subsystem --childlist A file containing a list of children to be tagged with the parent. Each line in the file has the format: module-name config-name x.y.z-age

INFSOM-RI-026753 2007 © Members of ETICS collaboration 76 / 145 ETICS USER MANUAL

-u, --update If the configuration already exists, update it with the new values. If this option is not specified creating an existing configuration produces an error --notag Do not perform the actual VCS tag and configuration creation/update. --novcstag Do not perform the actual VCS tag, but perfom the configuration creation/update. --nodeps Only tag the currently specified module (do not tag children) --continueonerror Continue when tag errors are encountered. --verbose Print verbose messages --version Display the current client version number.

8.2.1. How to Tag a Configuration The standard command to tag a configuration is:

etics-tag -c <configuration-name> -t --config-version <x.y.z.- r> <module-name>

This command creates a new configuration of the specified module by cloning the current configuration in the local workspace (or HEAD if no configuration is present) using the new name, VCS tag and version. If the configuration is stored in a Version Control System and a tag target is defined in the VCS Commands, the code is also tagged in the VCS. If the module name is not specified, the command applies to the project. If a more complex configuration has to be created as result of the tag operation, an INI file in the ETICS format can be supplied with the -i option. In this case the --tag and --config- version options are ignored if specified.

8.2.2. How to Tag a Configuration Tree The standard command to tag a configuration tree is:

etics-tag -c <configuration-name> -t --config-version <x.y.z.- r> --childlist <module-name>

where <module-name> must be a subsystem name. This command creates a new configuration of the specified subsystem by cloning the current configuration in the local workspace (or HEAD if no configuration is present) using the new name, VCS tag and version. If the configuration is stored in a Version Control System and a tag target is defined in the VCS Commands, the code is also tagged in the VCS. Additionally, the command parses the supplied child list file, tags if necessary each module listed in the file and attaches the configuration to the parent subsystem configuration. Note that the child configurations listed in the child list file don’t have to be all new configurations to be tagged. If a configuration already exist, it is skipped or updated depending on the presence of the --update option. However, all listed

INFSOM-RI-026753 2007 © Members of ETICS collaboration 77 / 145 ETICS USER MANUAL

configurations are attached to the parent subsystem configuration no matter if they have been just tagged or not. If the module name is not specified, the command applies to the project. A helper option called --prepare-childlist can be used to create a template childlist file that can then be edited with new values for one or more entries. The command to generate this file is:

etics-tag --prepare-childlist -c <configuration-name> <module-name>

INFSOM-RI-026753 2007 © Members of ETICS collaboration 78 / 145 ETICS USER MANUAL

9. Building Configurations

9.1. Overview Previous chapters described how to get a project into a workspace and how to browse its structure and configurations and edit them by creating, modifying or deleting modules, configurations and related commands, properties and dependencies. Once the information is registered in the ETICS system, we can use the ETICS client to execute builds and generate software packages and reports. This chapter will describe the commands used to perform these operations and how to control how builds are performed. In the following section, it is assumed that a project has been inserted in the workspace and that a suitable configuration has been checked out.

9.2. The etics-build Command The ETICS command to perform builds is called etics-build. The syntax of the command is:

etics-build [options] [<module-name>]

The command has several options that allow controlling how builds are executed. The following table shows the most common options, while more advanced options will be described in the following sections: Table 16: etics-build command options Option Description -h, --help Show the usage instructions -c, --config <configuration-name> Define a specific configuration to be used instead of the default one, where default is the first configuration found in the store. This is normally not required unless multiple or non-default configurations of the selected module have been checked out -p, --property <property=value> This option allows passing properties to the build process. If the property is already defined, its value is overridden by the value specified in the command-line. To pass multiple properties, use multiple -p options -e, --env <env=value> This option allows passing environment variables to the build process. If the variable is already defined, its value is overridden by the value specified in the command-line. To pass multiple variables, use multiple -e options -t, --target <target-name> Execute the build stopping at the specified target, If not specified the publish target is executed. Allowed targets are: clean, init, checkstyle, compile, test, doc, packager, publish, install --platform <platform-name> Overwrite the local platform (useful for testing or when the local platform is not a valid ETICS platform, but it’s compatible with one). --nobuild Do not perform the build, just print the sequence --continueonerror Do not stop building if an error is found --nodeps Only build the currently specified module (do not build children

INFSOM-RI-026753 2007 © Members of ETICS collaboration 79 / 145 ETICS USER MANUAL

and dependencies) --force Force the build of unmodified modules --cache Enable the property cache. This option allows to build using information cached from a previous etics-checkout command. It’s the default behaviour (no need to specify this option explicitly). This method is faster, but may fail if not all information needed for building is available in the cache --nocache Disable the property cache. If building with the cache fails, this option can be used to regenerate the properties and build sequence. It is safer, but slower

9.2.1. How to Build a Configuration The standard command to build a configuration is:

etics-build <module-name>

This command builds the default configuration of the specified module. The module must be a module within the module structure checked out with the etics-checkout command. The default configuration is the first configuration found in the local metadata store. When run by itself, the command builds the current configuration of the current project. Therefore

etics-build etics-build <current-project>

This assumes that a valid project configuration has been checked out or the command will fail prompting to check out the project. If more than one configuration of the same module exists in the store, for example because a non- default configuration has been checked out (see Chapter 6 for more information about checking out configurations), the -c option can be used to select a specific configuration:

etics-build -c <configuration-name> <module-name>

9.2.2. Build Targets The build targets are defined by the commands available in the Build Commands set associated to a configuration for the current or the default platform. The available targets are described in Section 1.2.4 - Commands and Properties.

The etics-build command executes the target in the following order:

init checkstyle compile test doc packaging publish

stopping after the execution of the target specified with the -t option or after publish if -t is not used. The clean, install and configure targets must be executed separately using the -t option.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 80 / 145 ETICS USER MANUAL

If a target is not specified (i.e. left empty) it is normally simply skipped. However, there are a few special rules:

# If packaging is not defined, the internal ETICS Packager is used. This is the recommended behaviour, unless you really have some special packaging tasks already defined for you code # If the install target is not defined, no packaging can be done with the ETICS Packager, since ETICS relies on your Build Command’s install target to detect which files must be packaged. # If publish is not defined, the internal ETICS Publisher is used. It is however possible to mix the ETICS Publisher with custom publishing actions by using the command ${eticsPublisher} anywhere in your publish target. For example:

publish = my_pre_publish_steps_here.sh; ${eticsPublisher}; my_post_publish_steps_here.sh

More information about the packaging and publish targets are given in following sections.

9.2.3. Forcing Build Execution The standard behaviour of the etics-build command is to execute any given action once and then cache information on disk that prevents executing again the same action if nothing in the code has changed. In addition, the caching mechanism is aware of the target chaining, so if the publish target is executed first, then any attempt to run explicitly the other targets will report that there is nothing to do, since they have been already executed. This allows speeding up the operation of restarting a failed build, since all successfully executed steps will be skipped. However in certain occasions it may be necessary to rerun the same action again, for example because something has changed in a place outside ETICS control or because the same action has to be repeated with different options. In this case, the build can be force using the --force option:

etics-build --force -c <configuration-name> <module-name>

9.3. The ETICS Packaging System The ETICS Client comes with a built-in packaging system able to build distribution packages on several supported platforms in different formats1 (e.g. tarballs, RPMS, debs, MSIs). In order to use the ETICS Packager the install target has to be defined, since it is used by the packager to identify the content of the package. In order to activate the ETICS Packager simply leave the packaging target undefined. To disable the Packager and if no custom packaging action is needed, the packaging target must be set to an ‘empty’ command, like the shell echo command for example. By default the ETICS Packager tries to create binary tarballs and RPMS, debs or MSIs (depending on the working platform1) by using information collected during the build, such as run-time dependencies. The generated packages have the following naming conventions:

Binary tarball: <module-name>-x.y.x-r.tar.gz

1 In the current version the supported packaging format are tarballs, RPMS and debs.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 81 / 145 ETICS USER MANUAL

Source tarball: <module-name>-x.y.x-r.src.tar.gz Binary RPMS: <module-name>-x.y.x-r...rpm Source RPMS: <module-name>-x.y.x-r.src.rpm Binary debs: <module-name>_x.y.x-r._.deb Source dsc: <module-name>_x.y.x-r.dsc

where distribution is a standard OS distribution name like slc3, rhel4, fc4, ubuntu7, etc and architecture is a cpu architecture name like i386, i686, x86_64, noarch/all, etc. A number of options and built-in properties are available to customise the Packager behaviour. The possible options are listed in the following table:

Option Description --createsource Create source tarball and source RPMS in addition to binary ones (this slows down the build due to extra compilation steps --createdebug Create debug RPMS in addition to binary ones (this slows down the build due to the extra compilation steps)1 --nodistname Do not append the distribution name to the RPM revision (it has no effect if --userspec is used) --usetimestamp Append a timestamp to the RPM revision number (it has no effect if --userspec is used) --versioneddeps Use version information when setting package dependencies in RPMS (ex: etics-client >= 1.0.0). Cannot be used together with –strictversioneddeps --strictversioneddeps Use strict version information when setting package dependencies in RPMS (ex: etics-client = 1.0.0). Cannot be used together with –versioneddeps --userspec Use a user-defined spec file for RPM generation. If this option is used all other packaging options are ignored. is relative to the module source location --usercontrol Use a user-defined control file for deb generation. If this option is used other packaging options are ignored. is relative to the module source location --userchangelog Use a user-defined changelog file for the deb. If this option is not used, a default changelog file is created containing a single version info entry. is relative to the module source location --userrules Use a user-defined rules for deb generation. is relative to the module source location Table 17: etics-build command packaging options

1 Not supported in the current release

INFSOM-RI-026753 2007 © Members of ETICS collaboration 82 / 145 ETICS USER MANUAL

In addition to the command-line options, the following built-in properties can be specified either as properties attached to the configuration or as command-line properties (a complete list of all built-in ETICS properties can be found for reference in Appendix B at the end of the User Guide):

Property name Default value if not Type Description specified package.prefix /opt/module-name String The default installation prefix for the generated RPMS package.buildarch Local platform/distribution String It can be used to specifier (slc3, rhel4, override the default noarch, etc) specifier. It is necessary to set it to ‘noarch’ to generate ‘noarch’ RPMS and ‘all’ debs package.priority optional String The value of the Priority field in the deb control file package.section devel String The value of the Section field in the deb control file package.group Unknown String The value of the Group tag in the RPM spec file package.packager ETICS String The value of the Packager tag in the RPM spec file and the Maintainer field in the deb control file package.provides Autodetected String The list of provides entries for the RPM spec file (overrides the autodetected list) package.requires Autodetected String The list of Requires entries for the RPM spec file (overrides the autodetected list) package.depends Autodetected String The list of Depends entries for the deb control file (overrides the autodetected list) package.predepends String The list of Pre- Depends entries for the deb control file package.obsoletes String The list of Obsoletes

INFSOM-RI-026753 2007 © Members of ETICS collaboration 83 / 145 ETICS USER MANUAL

entries for the RPM spec file package.replaces String The list of Replaces entries for the deb control file package.conflicts String The list of Conflicts entries for the RPM spec file and deb control file package.recommends String The list of Recommends entries for the deb control file package.suggests String The list of Suggests entries for the deb control file package.enhances String The list of Enhances entries for the deb control file package.autoreqprov Yes String Can be ‘yes’ or ‘no’. If it is set to yes, rpm will try to automatically detect requires and provides from the packaged files and libs, otherwise will only use specified entries package.userspec String Use a user-defined spec file for RPM generation. If this option is used all other packaging options are ignored. The value is a file path relative to the module source location package.usercontrol String Use a user-defined control file for deb generation. If this option is used other packaging options are ignored. The value is a file path relative to the module source location package.userchangelog String Use a user-defined changelog file for the

INFSOM-RI-026753 2007 © Members of ETICS collaboration 84 / 145 ETICS USER MANUAL

deb. If this option is not used, a default changelog file is created containing a single version info entry. The value is a file path relative to the module source location package.userrules String Use a user-defined rules for deb generation. The value is a file path relative to the module source location package.configFiles String A comma-separated list of files to be flagged as configuration files in the RPM spec file and in the deb conffiles file package.configFilesNorepl String A comma-separated ace list of files to be flagged as configuration files in the RPM spec file and in the deb conffiles file. In the case of the RPM spec file, the entries are flagged as %config(noreplace) package.suidFiles = <file- String Comma-separated list path[,file-path]> of files to be installed with permission mask 4555 package.fileRights = <file- String Comma-separated list path@xxxx[,file- of files to be installed path@xxxx]> with permission mask xxxx (the mask can be different for each file) package.tgz.name String If the tarball name in the ETICS repository doesn’t follow the standard ETICS conventions, this property can be used to specify a custom name. It only applies

INFSOM-RI-026753 2007 © Members of ETICS collaboration 85 / 145 ETICS USER MANUAL

when downloading packages, not when creating packages package.deb.name String If the deb name in the ETICS repository doesn’t follow the standard ETICS conventions, this property can be used to specify a custom name. It only applies when downloading packages, not when creating packages package.rpm.name String If the RPM name in the ETICS repository doesn’t follow the standard ETICS conventions, this property can be used to specify a custom name. It only applies when downloading packages, not when creating packages package.msi.name String If the msi name in the ETICS repository doesn’t follow the standard ETICS conventions, this property can be used to specify a custom name. It only applies when downloading packages, not when creating packages Table 18: packaging properties

9.3.1. How to Pass Installation Instructions to the Packager As previously seen, it is necessary to pass installation instructions to the Packager using the install target in order to create packages. However, the installation command by itself is not enough, since typical installation procedures tend to install the software in locations that are not writable for non- privileged users. The ETICS Packager needs to have a way of redirecting the installation to a location of its choice. This is typically done by most installation procedures by specifying explicitly the installation prefix. Since there are various conventions about how to pass this parameter depending on which installation method is used (autotools, ant, python distutils, etc), ETICS defines the following generic conventions: whatever method is used to pass the prefix, the value of the prefix must be set in the

INFSOM-RI-026753 2007 © Members of ETICS collaboration 86 / 145 ETICS USER MANUAL

commands as ${prefix}. The ETICS Packager will then resolve this value to an internal location of its own. For example, using standard Autotools practices, this becomes:

init = ./configure compile = make install = make install --prefix=${prefix}

or using python distutils

compile = python setup.py build install = python setup.py --prefix=${prefix}

and so on. Of course the same convention applies to completely custom installation scripts as long as it is possible to pass the installation root in some way, for example:

install = ./my_installation_script --install_location=${prefix}

9.4. The ETICS Publisher The ETICS Client comes with pre-defined publishing conventions that allow harvesting of build and test artefacts (e.g. packages, logs, metrics, test reports) from the various modules into a common location. The ETICS Publisher uses the following conventions:

# All checkout and build logs are stored in the reports directory of the workspace. The main log file is called:

build-status.xml

and contains summary checkout, build and test information for each module in the current build or test run. Detailed logs of the checkout, build and test operations of each modules are saved in files of the format:

[CHECKOUT|BUILD|TEST]-<module-name>-<configuration-name>.log

# All packages are stored in the directory

dist

The packages are organised in a tree that mirrors the structure of the ETICS software repository:

<project-name>/<module-name>///<package-name>

The packages are by default taken from the following locations in each module root:

(S)RPMS: <module-root>/RPMS

INFSOM-RI-026753 2007 © Members of ETICS collaboration 87 / 145 ETICS USER MANUAL

(src.)tar.gz: <module-root>/tgz dsc: <module-root>/debs deb: <module-root>/debs

The location where the publisher looks for packages can be changed by using the following built-in packager properties: Table 19: publishing properties Property name Default value if not Type Description specified package.tgzLocation ${location}/tgz string The default location where generated tarballs are stored in each module at the end of a build and where the Publisher looks for them package.SRPMSLocation ${location}/RPMS string The default location where generated source RPMS are stored in each module at the end of a build and where the Publisher looks for them package.RPMSLocation ${location}/RPMS string The default location where generated binary RPMS are stored in each module at the end of a build and where the Publisher looks for them package.SDEBSLocation ${location}/debs string The default location where generated source dsc packages are stored in each module at the end of a build and where the Publisher looks for them package.DEBSLocation ${location}/debs string The default location where generated binary deb packages are stored in each module at the end of a build and where the Publisher looks for

INFSOM-RI-026753 2007 © Members of ETICS collaboration 88 / 145 ETICS USER MANUAL

them

9.5. Build Reports The build reports are organised in a hierarchical tree with a top index.html page in the reports directory that shows summary information for the build and links to the detailed build reports, metrics, tests, etc. The exact same HTML report is available both for local and remote build and test. For more information about the reporting functionality refer to Chapter 11.2.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 89 / 145 ETICS USER MANUAL

10. Testing Configurations

10.1. Overview Previous chapters described how to get a project into a workspace, how to browse its structure; edit it by creating, modifying or deleting modules, configurations and related commands, properties and dependencies. We also covered how to build configurations to produce build artefacts. Once the information is registered in the ETICS system, we can use the ETICS client to execute tests, calculate metrics and generate reports. This chapter will describe the commands used to perform these operations and how to control the way tests are performed. Testing configurations is generally used to perform system tests, as oppose to unit tests performed during the build procedure, as described in Chapter 9 - Building Configurations, as part of the test target. Each configuration can have Test Commands that include the commands that will be executed during a test procedure. As for the build procedure, before executing tests, a checkout is required. Therefore, in the following section, it is assumed that a project has been inserted in the workspace and that a suitable configuration has been checked out.

10.2. The etics-test Command The ETICS command to perform builds is called etics-test. The syntax of the command is:

etics-test [options] [<module-name>]

The command has several options that allow controlling how tests are executed. The following table shows the most common options, while more advanced options will be described in the following sections: Table 20: Options for etics-test command Option Description -h, --help Show the usage instructions -c, --config <configuration-name> Define a specific configuration to be used instead of the default one, where default is the first configuration found in the store. This is normally not required unless multiple or non-default configurations of the selected module have been checked out -p, --property <property=value> This option allows passing properties to the test process. If the property is already defined, its value is overridden by the value specified in the command-line. To pass multiple properties, use multiple -p options -e, --env <env=value> This option allows passing environment variables to the test process. If the variable is already defined, its value is overridden by the value specified in the command-line. To pass multiple variables, use multiple -e options -t, --target <target-name> Execute the test stopping at the specified target. If not specified the publish target is executed. Allowed targets are: clean, init, test --platform <platform-name> Overwrite the local platform (useful for testing or when the local platform is not a valid ETICS platform, but it’s compatible with one).

INFSOM-RI-026753 2007 © Members of ETICS collaboration 90 / 145 ETICS USER MANUAL

--notest Do not perform the actual test --continueonerror Do not stop testing if an error is found --nodeps Only test the currently specified module (do not test children and dependencies) --verbose Print verbose messages --version Return the current client version number.

10.2.1. How to Test a Configuration The standard command to test a configuration is:

etics-test <module-name>

This command tests the default configuration of the specified module. The module must be a module within the module structure checked out with the etics-checkout command. The default configuration is the first configuration found in the local metadata store. When run by itself, the command tests the current configuration of the current project. Therefore

etics-test etics-test <current-project>

This assumes that a valid project configuration has been checked out or the command will fail prompting to check out the project. If more than one configuration of the same module exists in the store, for example because a non- default configuration has been checked out (see Chapter 6 for more information about checking out configurations), the -c option can be used to select a specific configuration:

etics-test -c <configuration-name> <module-name>

10.2.2. Test Targets The test targets are defined by the commands available in the Test Commands set associated to a configuration for the current or the default platform. The available commands have been described Section 1.2.4: Table 21: Test targets Command name Description Mandatory clean Command to clean No init Command to perform initialisation operations before No compiling (e.g. deployment and configuration of services) test Command to execute the test No

The etics-test command executes the target in the following order:

init test

INFSOM-RI-026753 2007 © Members of ETICS collaboration 91 / 145 ETICS USER MANUAL

stopping after the execution of the target specified with the -t option or after test if -t is not used. The clean target must be executed separately using the -t option.

If a target command is not specified in the set it is normally simply skipped.

10.3. Test Reports The test reports are organised in a hierarchical tree with a top index.html page in the reports directory that shows summary information for the test and links to the detailed test reports, metrics, etc. The exact same HTML report is available both for local and remote build and test. For more information about the reporting functionality refer to Chapter 11.2 (“Submitting Builds and Tests Using the ”). The ETICS Publisher is also invoked during a test procedure, following a similar process as described in section 9.4 - The ETICS Publisher, but without the packaging.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 92 / 145 ETICS USER MANUAL

# SUBMITTING REMOTE BUILDS AND TESTS TO THE ETICS SYSTEM This chapter describes two ways of submitting builds and tests. The user can either use the Build and Test Web Application (WA hereafter), or the command-line client.

11.1. Submitting Builds and Tests Using the Web Application Besides modelling capabilities, the WA provides the user with remote build and test submission facilities. Similarly to in CLI-based client, the WA provides the user with the ability to submit remote builds and tests to a large number of host types and platforms. Following the execution of a remote build and test, a report is automatically generated and registered. If a build is issued, artefacts can be published to the ETICS repository. The ETICS remote builds and tests implementation leverage the Metronome infrastructure. Remote build and test can be submitted through the WA once a specific configuration has been selected.

Figure 40 - Build Submission Page Remote submission can also be triggered for more than one platform at the same time by selecting multiple entries in the ‘host selection’ list.

11.2. Submitting Builds and Tests Using the Command-Line Client The ETICS command to submit build and test is called etics-submit. The syntax of the command is:

etics-submit [options] [<module-name>]

It supports two operations: build that submits a remote build job, and test that submits a remote test job.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 93 / 145 ETICS USER MANUAL

The following table shows the options available for submitting a remote build or test: Table 223: Options for remote build and test Property name Default value if not Type Description specified --platforms Current local platform Comma separate The list of platforms list of platform where to build and test strings (do not use blanks or quotes) --project Current project if defined Define the name of the project to get for the build and test -- project-config Define a specific project configuration for the metadata --fromsource When possible, check out source code instead of downloading binaries. --frombinary When possible, download binaries instead of checking out source code. This implies that packages will be built (this is the default if not option is specified). --fromsourceonly Check out source code only. This means that all the configurations, including dependencies have to be available in source code form or the operation will fail. --frombinaryonly Checkout binaries only. This means that all the configurations, including dependencies have to be available in binary form, except the modules specified to the command. --volatile Use a named volatile repository to look for packages. If a package

INFSOM-RI-026753 2007 © Members of ETICS collaboration 94 / 145 ETICS USER MANUAL

is not found in the volatile repository, it is searched for in the permanent repository, before giving up unless --volatileonly is used. This option cannot be used together with -- defaultvolatile. --defaultvolatile Use the default volatile repository to look for packages. If a package is not found in the volatile repository, it is searched for in the permanent repository, before giving up unless --volatileonly is used. This is equivalent to -- volatile=default. This option cannot be used together with -- volatile. --volatileonly Use only the specified volatile repository to look for packages. If a package is not found in the volatile repository, it is not searched for in the permanent repository. --register Register the build packages and reports in the permanent ETICS repository (it requires permissions, ask your project administrator). Packages are always stored in a volatile repository, even if this option is not used. --register-volatile Register the build packages and reports in the named volatile repository instead of

INFSOM-RI-026753 2007 © Members of ETICS collaboration 95 / 145 ETICS USER MANUAL

the default one. If this option is not used the packages are stored in the default volatile repository. --runasroot Run the remote build and test as super user (e.g. root on linux/unix). --freeze Freeze the remote node after execution of the build and test. The parameter must be in the following format: 'm' for minutes 'h' for hours.

WARNING: If this option is combined with --runasroot the machine will not be available for further submissions which might starve the resource pool. --private-resource Run the remote build <private-resource-name> and test on a private resource. In order for the match making to succeed, at least one machine must have been register with the key <private-resource- name>. --requirements Specify custom requirements for the match making of the remote resources. -c, --config Define a specific <configuration-name> configuration to be used instead of the default one, where default is the first configuration found in the store. This is normally not required

INFSOM-RI-026753 2007 © Members of ETICS collaboration 96 / 145 ETICS USER MANUAL

unless multiple or non- default configurations of the selected module have been checked out -p, --property This option allows <property=value> passing properties to the test/build process. If the property is already defined, its value is overridden by the value specified in the command-line. To pass multiple properties, use multiple -p options -e, --env <env=value> This option allows passing environment variables to the test /build process. If the variable is already defined, its value is overridden by the value specified in the command-line. To pass multiple variables, use multiple -e options -t, --target <target-name> Execute the test/build stopping at the specified target. If not specified the publish target is executed. Allowed targets are: clean, init, test --notest Do not perform the actual test --continueonerror Do not stop testing/build if an error is found --verbose Print verbose messages --version Return the current client version number. --nobuild Do not perform the build, just print the sequence --force Force the build of

INFSOM-RI-026753 2007 © Members of ETICS collaboration 97 / 145 ETICS USER MANUAL

unmodified modules --urlname <report-name> Specify a custom string to be used in the URL for the published build/test reports. --shallowbindeps When checking out using the –frombinary or –frombinaryonly options, dependencies of binary packages are not checked out. --runtimedeps When checking out configurations, dependencies of run- time dependencies are also processed. It is useful to make sure that the final list of packages contains everything needed to deploy the components. --createsource Create source tarball and RPMS --nodistname Do not append the distribution name to the RPM revision. It does not have effect if --userspec is used. --usertimestamp Append a timestamp to the RPM revision number. It does not have effect if -- userspec is used. --versioneddeps Use version information when setting package dependencies in RPMS (e.g., etics- client >= 1.0.0). Cannot be together with -- strictversioneddeps --strictversioneddeps Use strict version information when setting package dependencies in

INFSOM-RI-026753 2007 © Members of ETICS collaboration 98 / 145 ETICS USER MANUAL

RPMS (e.g., etics- client = 1.0.0). Cannot be used together with –versioneddeps --userspec Use a user-defined spec file. The spec file must be placed in /project. If this option is used all other packaging options are ignored.

The URL for the published build/test reports takes the form:

http://<etics-servername>/rundir//reports

The standard URLs will be created even if the <report-name> is specified.

11.2.1. How to submit a remote build job The standard command to submit a remote build job:

etics-submit build <module-name>

11.2.2. How to submit a remote test job The standard command to submit a remote test job:

etics-submit test <module-name>

INFSOM-RI-026753 2007 © Members of ETICS collaboration 99 / 145 ETICS USER MANUAL

# ACCOUNT INFORMATION The ETICS command to retrieve and show account information is called etics-log. The syntax of the command is:

etics-log [options] [<module-name>]

The following table shows the options available for retrieving and showing account information: Table 234: Options for account information Property name Default value if not Type Description specified --project Current project if defined Define the name of the project to get for the build and test. --dn Specify the dn information of user’certificate. --mydn Select the certificate that belongs to the user running the command. --operation <operation- ‘’ Specify the type of type> operation (add, modify, remove) --component | -s, -- Specify the type of a subsystem moduleName. Mandatory without having run etics-get- project --startdate <start-date> Specify the lower bound of the search range. Optional parameter. Set the date and time when the log information will start. If it is not set, there is not any lower-limit value. --enddate <end-date> Specify the lower bound of the search range. Optional parameter. Set the date and time when the log information will be stopped. If it is not set, there is not any upper- limit value.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 100 / 145 ETICS USER MANUAL

--limit Specify the maximum number of returned entries. -c, --config Define a specific <configuration-name> configuration. --verbose Print verbose messages

The <start-date> parameter must be in one of the following formats: the absolutre start time: ‘dd/mm/yyyy hh:mm’ assuming hh_mm to be 00:00 ‘dd/mm/yyyy’

The <end-date> parameter must be in one of the following two formats: to stop after 123 minutes from <start-date> ‘+123’ to stop after 321 hours from <start-date> ‘+321’ or the absolute stop time: ‘dd/mm/yyyy hh:mm’ assuming hh:mm to be 23:59: ‘dd/mm/yyyy’

INFSOM-RI-026753 2007 © Members of ETICS collaboration 101 / 145 ETICS USER MANUAL

# REPOSITORY As outcome of remote build and test submissions, the ETICS system produces different types of artefacts: packages generated during the build process, build and test reports, and metrics collected by the plug-in framework (see Chapter 16). The ETICS Repository is the standard location where all the software artefacts are stored and archived. In addition to the artefacts generated by the ETICS system, third party packages (e.g. externals) are also available as dependencies to build software. The repository can be accessed and used in different ways: # The ETICS client communicates with the repository to get the packages during the checkout phase of a build or test procedure. Following the successful execution of a remote build, the ETICS Build and Test Web Service can store in the repository all the built packages (for local builds the built packages are only stored in the local workspace). # The user communicates with the repository via the Repository web application to download the packages built during previous remote builds, to analyse the reports generated, or the metrics collected. # The ETICS team adds to the repository as externals1 the third party packages requested by the ETICS user community. The repository is composed of two main parts: a set of volatile storage areas and a permanent storage area for the registration of packages together with related reports and metrics. These two parts have different purposes. 13.1. Volatile Storage AREAS Volatile storage areas are temporary storage spaces that users can allocate on the repository for their private use. Volatile storage areas are used as locations where to find all packages, related reports and metrics that have been generated during the remote build operations on the ETICS infrastructure. These areas can be used by developers to check whether the generated packages are correct or not, to analyse the reports of builds or tests for debugging purposes, or to check the metrics collected by the client. They can also be used as an exchange point for certification. Unlike the Registration storage area, packages, reports and metrics that are registered in the Volatile storage area, overwrite existing ones if previously stored. The overwritten packages and reports are still available with other download locations to let the developer analyse the differences between consecutive versions but the artefacts considered by the Build and Test system are only the latest ones submitted (the old packages and reports are stored with a time stamp of when they are overwritten). By default all artefacts are registered in a default volatile area called default, unless a specific volatile area is specified by the user. If the user specifies a different volatile area, all generated packages will be registered in that area. New storage areas will be automatically created as specified by the user. Issuing the following command, the module <module-name> will be remotely built and by default the packages will be registered in the default volatile area: etics-submit build <module-name> Since every remote-build artefact will be registered by default in the default area, artefacts already in this storage area may be unexpectedly overwritten. To avoid this, the repository provides custom volatile areas, where users can specify the name of the volatile area where the artefacts are to be registered. The following command executes a remote-build and stores the artefacts in the specified volatile area: etics-submit build --register-volatile <area-name> <module-name>

1 The externals project in ETICS contains all third party modules that are not necessarily built using ETICS, but used as dependencies by other projects.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 102 / 145 ETICS USER MANUAL

If <area-name> is not present in the repository, it will be created as part of the artefact registration. Be aware however that the volatile storage area is scratched on a regular basis, according to the available space on our disk server. Once a volatile storage area is present in the repository, it is possible to set it as the location where to get the binaries during the checkout operation. If the package is not present in that area, by default the client will look for the package in the permanent repository. It is important to understand that only the latest version of stored packages is available in volatile areas for the checkout operation. The following command specifies a specific volatile area for checkout: etics-checkout --volatile <area-name> <module-name> for a remote build: etics-submit build --remote-volatile <area-name> <module-name> or for a remote test: etics-submit test --remote-volatile <area-name> <module-name> 13.2. Registered Storage Area The registration storage area is the official location where to store released packages together with their related reports and metrics. The registered storage is the location to chose in order to save permanently packages (e.g. when a package can be made public, final version of a configuration). Packages stored in the permanent storage area cannot be overwritten with other packages. Build reports and their metrics are also stored together with their packages. In other words, if the build doesn’t generate any package or it generates only packages that are already present in the registered area, the report and the metrics will not be saved in the registered part pf the repository. Test reports and their metrics are on the other hand always registered. This area is set by default as the location where binaries are downloaded from during the checkout operations. To register the artefacts in the permanent storage area of the repository, use the --register option. The user must also have Developer role on the corresponding configuration. etics-submit build --register <component-name> Note that by default the etics-submit command doesn’t register artefacts. 13.3. Repository Web Application The ETICS repository provides a web application to browse both Volatile and Registered storage areas of the repository. It provides an easy way to access generated reports, packages and metrics. This web application is included in the ETICS Portal as panel at the following URL: https://etics.cern.ch/eticsPortal (panel Repository) The Repository web application is composed of two parts: a tree on the left showing the hierarchical structure of the repository and a panel on the right showing information on the selected node of the tree.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 103 / 145 ETICS USER MANUAL

Figure 41: Repository web application The Tree Panel On the left of the Repository web application, a tree panel representing the hierarchical structure of the repository is available to access all artefacts. On the top of the tree, a tool bar is available with three buttons to refresh the currently selected node of the tree and to resize the panel moving the split left or right.

Figure 42: Tree tool bar The top of the tree is divided in Registered and Volatile nodes. The first node is the permanent storage area, the second node includes a list of volatiles areas created by users and the default volatile area.

Figure 43: Registered and Volatile repository nodes Inside each area a list of project is present. For each project, it is possible to access the information in two ways: by packages or by reports. Browsing a project by packages allows the user to focus on the outcome of the build or test and choose a particular version of a module selecting version, platform and package. Once a package is identified,

INFSOM-RI-026753 2007 © Members of ETICS collaboration 104 / 145 ETICS USER MANUAL

it is possible to download it, to check its contents if it is one of the supported formats, to see the related build information to verify where the package comes from and who has generated it, or finally to view the associated report to check in details the generation process and the environment on which the package has been built. This view of the repository addresses the need of downloading specific versions of software also verifying their origin (usually used by certification and integration teams, and by end users). Browsing a project by reports instead, provides a way to access the same information focusing on the generation process and not on the final outcome of a submission. Selecting a configuration and a platform allows users to browse all builds and tests performed on the system and to view all reports that have been generated during these submissions. Checking the reports, it is possible to see what packages have been generated and to download them. This view of the repository addresses the need to keep track of every submission and check their results (usually used by developers and testers).

Figure 44: Browsing a project by packages or by reports As said before, in the volatile areas or the repository, not only the latest packages, but also the versions that have been overwritten are stored. The old versions are renamed appending the time stamp of when they have been created providing the history of a package. These packages are available until the cleaning mechanism of the repository deletes them after a certain amount of time or after a certain number of consecutive builds.

Figure 45: Browsing latest and overwritten packages Reports are shown ordered by date starting from the latest. Two types of reports are displayed with different icons: the main reports and the module reports.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 105 / 145 ETICS USER MANUAL

The main report is shown where the selected configuration was directly built or tested. This means that the user actually submitted that configuration for a build or test. For these configurations, the report shown is the global report on which all information of the submission are shown in a summary and all the other sections are available through links from the main page. This report is identified with a full page icon. The module report is shown if the selected configuration was not directly submitted by the user, but was triggered as a direct or indirect dependency or child of a configuration submitted by the user. For these configurations, the report shown is the log of the execution of that module instead of the whole report of the submission. This report is identified with a half page icon. Using links, it is always possible to jump from a module report to the summary, or to drill down from a summary to module reports. Generally, when browsing a platform of a certain configuration, a list of reports will be available. Some of them will be main reports, because the user (typically the developer) submitted a build or a test on that specific configuration, some other will be module reports because that configuration was triggered by another configuration through dependency or child relationships. In both cases it is possible to keep track of the build or test information.

Figure 46: Browsing main and module reports Statistics and Latest Panel Every summary panel shown on the right side of the Repository web application contains some statistics and latest artefact tables where it is possible to find information about the contents of that particular branch of the repository and the latest reports and packages submitted for that branch. The statistics are refreshed once per day by the first request of that day. Every latest table contains a refresh button to re-query the repository for latest artefacts.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 106 / 145 ETICS USER MANUAL

Figure 47: Summary panel Clicking on one of the latest report opens a pop-up with the selected report. The user can also download one of the latest packages by clicking on the corresponding entry.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 107 / 145 ETICS USER MANUAL

# ANALYSING BUILD AND TEST REPORTS During the build and test process, locally and remotely, the ETICS client assembles a rich report including information such as which commands were used to execute the procedure, high-level information regarding which module and configuration are being built and/or tested, logs generated during the process and reports generated by the plug-in system (see Chapter 15.2: Administration with the Command-Line Client). These build and test reports are accessible via the Report Browser web application (see section 13.3 “Repository Web Application” for details) or via the CLI commands etics-get-report and etics-get-rundir.

14.1. Standard Reports Each build and test procedure generates a rich report which can be viewed locally in the workspace ‘reports’ directory. Further, this report is also available for remote build and test submissions. Each top level report contains the following information: * Module Summary: Information on the module that has been built or tested, including the name of the project, module name, description and version, release, vendor and licence type. * Execution Summary: Information on the build process, including final results of the build and test, number and percentage of successfully built components, start time, end time and duration of the process, configuration used, tag on the VCS, path of the VCS root and the platform on which the process has been executed. * ETICS Commands: The list of ETICS commands executed on the worker node. * Environment Properties: The list of environment variables used to execute the build and test.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 108 / 145 ETICS USER MANUAL

Figure 48: Summary build and test page

On the top left part of the report summary, a side menu is present. From here are accessible the logs, the package lists report, metrics pages, and if available test reports and custom reports. By clicking on the Logs link, the user can access the logs section. The logs page shows summary information of the configuration built or tested and the list of all its sub-configurations and dependencies – i.e. these entries are ordered, following the same order used during the build and test procedure, with the requested module last, since all its children and dependencies are executed first. From the list it is possible to see the component name, the configuration name, the last build time and the result of the build.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 109 / 145 ETICS USER MANUAL

Figure 49: List of built and tested modules

Clicking on any module displayed in the module list (see Figure 49), detailed logs are available. There, a detailed page (see Figure 50) is available containing the logs generated during the checkout or download operations, then logs generated during the build and finally logs generated during the test operations.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 110 / 145 ETICS USER MANUAL

Figure 50: Detailed log

To access the dependency resolution, a Dependencies link is present. The dependency resolution is composed of two lists: * Dependencies: a list of dependencies for the current module. * Used by: a list of the components that depend on the current module.

For each entry, the scope of the dependency is visible: B are build type dependencies, R are runtime dependencies, BR are build and run time dependencies.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 111 / 145 ETICS USER MANUAL

Figure 51: Used-by and dependencies page

The Package List report provides an overview of the different packages created or used during a build as build or run-time dependencies. The report by default shows all packages, but the user can also select specific package types (such as rpms or tar.gz files). The report displays the package file name, the description, the package type and a status flag. If the package is available in the repository the displayed flag is ‘Ok’, otherwise the flag is ‘Not found’. The package file names have links to download the corresponding files from the repository. Additionally, a link at the top of the report provides a downloadable text file of the package list that can be used directly to download all packages using for example standard wget commands (Figure 52).

INFSOM-RI-026753 2007 © Members of ETICS collaboration 112 / 145 ETICS USER MANUAL

Figure 52: Package List report

The last section of the build and test reports is the metrics page. Clicking in the summary page on the Metrics link displays the list of high-level metrics (see Figure 53).

14.2. The etics-get-report command The command etics-get-report downloads the logs section of the report browser for remote build and test reports and saves it in plain-text format. Its syntax is:

etics-get-report [options] []

Where is the Submission ID returned by the etics-submit command (ignored if --dn option is set).

The following table shows the options available for the command: Table 24: Options for etics-get-report Option name Default value if not specified Description --dn If both and --dn are Save the reports for all the jobs submitted by omitted, all reports for jobs of the specified user. If this option is specified, the user running the command value is ignored. are saved --diff <other-gid> No comparison will take place Save also the reports for the job with id <other-gid> and compares them with the reports of , in a way similar to the diff command. Ignored if --dn is also specified. -l, --limit No limit When getting the reports for all of a user's jobs, save only the last jobs

INFSOM-RI-026753 2007 © Members of ETICS collaboration 113 / 145 ETICS USER MANUAL

submitted. Ignored when is specified and --dn is not specified. --verbose Print verbose messages

The plain-text reports are saved in the remote/reports directory of the workspace, one text file for every platform the job was submitted on. The report for a platform won't be saved if the job is still running or it wasn't submitted (i.e. etics-status command reports the status as Running or Not found for that job)

14.3. The etics-get-rundir command The command etics-get-rundir saves locally all the rundirs generated by a remote submission. A rundir is a directory created on the build and test system web server, containing all the files needed to execute a submission on a single platform and the files generated by the job itself (including the report browser pages). The command syntax is:

etics-get-report [options] []

Where is the Submission ID returned by the etics-submit command (ignored if --dn option is set).

The following table shows the options available for the command: Table 25: Options for etics-get-rundir Option name Default value if not specified Description --dn If both and --dn are Save the rundirs for all the jobs submitted omitted, all rundirs for jobs of by the specified user. If this option is the user running the command specified, value is ignored. are saved -l, --limit No limit When getting the rundirs for all of a user's jobs, save only the last jobs submitted. Ignored when is specified and --dn is not specified. --verbose Print verbose messages

The rundirs are saved in the remote/rundirs directory of the workspace, one folder for every platform the job was submitted on. If the command is run multiple times for the same submission, only files added/changed since the last run will be downloaded. If the job is still running or it wasn't submitted (i.e. etics-status command reports the status as Running or Not found for that job) the command will issue a warning and download available files.

14.4. Specific reports Depending on the type of build and test procedure, specific reports are generated. Most of these specific pages will be generated by plugins, either provided out-of-the-box by ETICS or custom plugins provided by users. For more details on plugins and the framework they run under, refer to Chapter 15.2 (“Administration with the Command-Line Client”).

INFSOM-RI-026753 2007 © Members of ETICS collaboration 114 / 145 ETICS USER MANUAL

Single Lines Of Code Count report The SLOCCount stands for Single Lines Of Code Count. The metrics summary page (see Figure 53) displays the high-level metrics for a build and test procedure, including for standard builds the SLOCCount. This metrics counts the number of lines of code, organised by language. The metrics list includes all the language present in the source code with the number of lines of code calculated for each.

Figure 53: Metrics summary page

Clicking on Metrics details, a detailed graph is displayed (see Figure 54) with the metrics evaluated for every component part of the current report.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 115 / 145 ETICS USER MANUAL

Figure 54: SLOC count metrics detailed page

INFSOM-RI-026753 2007 © Members of ETICS collaboration 116 / 145 ETICS USER MANUAL

Test and Custom reports Test reports, either for unit or system tests, if generated, can be accessed from the report summary page. The same is true for custom reports. A set of simple naming conventions are used to influence the generation of the summary page. If a file named index-tests.html is present in the reports directory of the workspace, an extra link “Testing” is added to list of links at the top left of the summary page.

Figure 55: Summary report with testing and custom links

Similarly, if a file called index-custom.html is available in the reports directory, a supplementary link called “Custom output” is added to the summary report. Examples of both above mention cases can be seen in Figure 55.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 117 / 145 ETICS USER MANUAL

Figure 56: Example of detailed test report

Figure 56 shows an example of test report automatically generated by ETICS. Setting the profile configuration parameter to ‘python, ETICS automatically executed all PyUnit1 test present in the module and generates this report.

If for example a testsuite generates its own HTML custom report (see Figure 57), as described above, the report can be accessed from the summary page via the “Custom output” link.

1 Similar support is been added for Java.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 118 / 145 ETICS USER MANUAL

Figure 57: Example of custom report

This extensibility mechanism allows users to extend the generation of build and test reports, thus providing further information to facilitate the analysis of a build and test procedure.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 119 / 145 ETICS USER MANUAL

# USER AND MODULE ADMINISTRATION Administration of users and modules can be performed using a dedicated web application or the command-line client. This chapter describes how to use both tools to perform administration tasks.

15.1. The ETICS Administration Application The ETICS Administration application is a web-based application dedicated to manage projects and users in the ETICS system. This application is restricted to user with the ETICS system administrator and ETICS module administrator roles. The ETICS system administrators may perform all available operations on any project, subsystem, component, configuration, role or user. The ETICS module administrator can only administer project, subsystem, module or configurations for which he/she has this role on. In particular this means that he/she: # cannot edit users or add projects; # cannot add/remove role to elements outside his/her scope. For instance, a module administrator of a single subsystem cannot assign roles to other subsystems in the same project; # for every user, can add/remove any role to elements within his/her scope. For instance, a module administrator of a given subsystem can assign the role “Module Administrator” to any user on any component or configuration in this subsystem (including the subsystem itself). The same user can be an ETICS module administrator for several components.

The following preconditions have to be satisfied before running the ETICS Administration application: # A valid user certificate is installed in a web-browser. # The user has the role ETICS system administrator and/or ETICS module administrator. For details on setting-up certificates in your browser, refer to section 3.3 (“Enabling Security”). To start the application, open the main page in a web browser – e.g.: https://etics.cern.ch/eticsPortal (panel Administration)

15.1.1. Layout The application screen is divided into 3 parts (see Figure 58 below): # The ETICS banner (top). # The menu (left side of the screen). Functions selected in the menu are displayed in the workspace area. This menu only shows the functions available for the current user – for instance the option “Add New Project” is only visible for the ETICS system administrator (not the ETICS module administrators). # The workspace (right side of the screen). This area displays the function selected from the menu. This is the only part of the application that changes content during the interaction, all next screenshots in this section will include only the workspace content.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 120 / 145 ETICS USER MANUAL

Figure 58: Administration web application

15.1.2. Manage projects List projects To list information related to projects, select option “List Projects” from the menu.

Figure 59: Project list

The project list is filtered depending on the user role: # All projects for the ETICS system administrator. # Only related projects for the ETICS module administrator. For instance module administrator of subsystem “build-system” from “ETICS System” will see only one project – “ETICS System”.

Add a new project (system administrators only) Only users with the role ETICS system administrators have access to this feature. To add a new project to the ETICS system, select the option “Add New Project” from the menu.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 121 / 145 ETICS USER MANUAL

Figure 60: Add new project form

The following parameters can be specified: Table 26: New project parameters Parameters Description Logo URL HTTP address to a graphics file with project logo (any public graphic file recognized by the web-browser, like gif, jpg, png, etc.). Name Project name. DisplayName Display name (more descriptive than name). Description Project description. Repository Binary repository root. VCSRoot VCS server address. Vendor Vendor name. Homepage URL Project home page.

Only the field “Name” is required, all others are optional.

15.1.3. Manage users List users To list users registered in the ETICS system, select the option “List / Edit Users” from the menu.

Figure 61: User list

The following data is shown for every listed user:

INFSOM-RI-026753 2007 © Members of ETICS collaboration 122 / 145 ETICS USER MANUAL

# User name, e-mail address, internal user ID. # Certificate name. # Activation status (active/inactive), link to view user’s permissions, link to edit user data (only when the application user is the ETICS system administrator).

The user list is filtered depending on the application user role: # All users for the ETICS system administrator. # Only active users for the ETICS module administrator.

Edit user (system administrators only) Only users with the role ETICS system administrators have access to this feature. To edit information regarding registered ETICS user: # Select the option “List / Edit Users” from the menu. # Locate the user to edit (web-browser search functionality can be used). # Select the link “Edit User”.

The following user data can be modified: first name, last name, e-mail address, certificate name, activation status.

Figure 62: Edit user form

15.1.4. Manage user permissions Permissions in the ETICS system are relations between users, ETICS modules/configurations and roles. Permissions are propagated top-down in the ETICS hierarchy. This means that a user having permission A for an element X automatically has the same permission for all children of element X. For instance, if user A has the role “Module Administrator” for a given subsystem in a given project, this means that user A has the same role for all configurations, components and components configurations children of this subsystem. However, as permissions propagate top-down, user A does not inherit the same role for other subsystems in the project.

There is a privileged role called administrator with an ETICS system-wide scope. Users having ETICS system administrator role are refer to as the ETICS system administrators.

User permissions can be managed only by the ETICS system administrator or the ETICS module administrator. The ETICS system administrator can manage permissions of all registered users for

INFSOM-RI-026753 2007 © Members of ETICS collaboration 123 / 145 ETICS USER MANUAL

all ETICS elements. The ETICS module administrator can manage permissions of all registered users but only in the scope of the ETICS element(s) he/she administrates. For instance, if user X is an ETICS module administrator of a given subsystem in a given project, then he/she can only manage the permissions related to ETICS elements within this subsystem (including the subsystem itself).

List roles To list all available roles predefined in the ETICS system, select the option “List Roles” from the menu. This list is static and cannot be modified.

Figure 63: Role list

Add user permission To add a new role to a user, select the option “Add User Permissions” from the menu.

Figure 64: Add user permissions form

The following parameters have to be specified: Table 27: New user permission parameters Parameters Description ETICS element One of project, subsystem, component or configuration. The permission is assigned to the lowest selected element. For instance after selecting a project and a subsystem, the permission will be assigned to the subsystem (not the project). The ETICS system administrator may specify any element.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 124 / 145 ETICS USER MANUAL

The ETICS module administrator only sees elements within his/her scope. User Any registered ETICS user. Roles Any ETICS role. Several roles might be assigned at the same time (at least one).

List user permissions The ETICS system administrator can view all the permissions. The ETICS module administrator sees only permissions related to his/her scope. Permissions can be viewed by user, project or role. To list the permissions of a selected user: # Select the option “List / Edit User” from the menu. # Navigate to a user (web-browser search functionality can be used). # Select the option “Permissions” next to the user data.

Figure 65: How to list user permissions

List project related permissions To list permissions of a selected user by project: # Select the option “List Projects” from the menu. # Navigate to a project (web-browser search functionality can be used). # Select the option “List user permissions for this project” at the bottom of the project info box.

Figure 66: How to list project related permissions

List permissions for a given role To list permissions of a selected user for a given role: # Select the option “List Roles” from the menu. # Navigate to a role. # Select the option “List users with this role” next to the role name.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 125 / 145 ETICS USER MANUAL

Figure 67: How to list permissions for a given role Remove user permission To remove user permissions, first list permissions as described in the above section, find the permission you want to remove and then select the option “Delete”. Note that permissions are removed immediately, without asking for a confirmation. Example of removing a role from user:

Figure 68: How to remove user permissions

15.2. Administration with the Command-Line Client This section describes in how to register users and assign roles to registered users for specific modules and configurations.

15.2.1. Manage users How to edit a user The ETICS command to edit a user is called etics-user. The syntax of the command is

etics-user [] <dn-value>

The following table shows the options: Table 28: Options for etics-user command Option Description -h, --help Show the usage instructions --fname Specify the first name of the user. It is mandatory with the operation add. --lname Specify the last name of the user. It is mandatory with the operation add. --email Specify the email address of the user. It is mandatory with the operation add.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 126 / 145 ETICS USER MANUAL

--dn Specify the dn information of the user certificate. It is mandatory to change the dn value during the operation modify.

<dn-value> represents the value of the distinguished name.

How to add a user The command to create a user is:

etics-user add --fname --fname --email <dn-value>

How to modify a user The command to modify a user is:

etics-user modify --fname <dn-value>

If you need to change the current dn, you can run the following command:

etics-user modify --dn <new-dn-value> <dn-value>

How to remove a user The command to remove a user is:

etics-user remove <dn-value>

15.2.2. Manage user permissions As mentioned earlier, permissions in the ETICS system are relations between users, ETICS modules/configurations and roles.

How to assign a role The ETICS command to edit a user is called etics-role. The syntax of the command is

etics-role [options] [] <role-value>

The following table shows the options:

Option Description -h, --help Show the usage instructions -m,--module <module-name> Specify the module name -c,--configuration <coniguration-name> Specify the configuration name. In this case it is mandatory to specify also --module, -m option --dn Specify the dn information of the user certificate

represents the type of role handled in the ETICS infrastructure. It can be one of the following values:

INFSOM-RI-026753 2007 © Members of ETICS collaboration 127 / 145 ETICS USER MANUAL

- Administrator - Developer - Guest - Integrator - ModuleAdministrator - ReleaseManager - Tester See section 1.3 (“Authentication, Authorization and Roles”) for a description of the different roles supported by ETICS.

How to add a role The command to add a role to a user is:

etics-role add -m <module-name> --dn <dn-value> <role-value> etics-role add -m <module-name> -c <configuration-name> --dn <dn- value> <role-value>

How to remove a role The command to remove a role from a user is:

etics-role remove -m <module-name> --dn <dn-value> <role-value> etics-role remove -m <module-name> -c <configuration-name> --dn <dn- value> <role-value>

INFSOM-RI-026753 2007 © Members of ETICS collaboration 128 / 145 ETICS USER MANUAL

# CLIENT PLUGIN FRAMEWORK This section provides an overview of the client-side plugin framework users can use to extend their build and test procedures. A recipe is also provided for users wanting to create their own plugins.

16.1. Overview The plugin framework provides the ability for common actions to be performed during build and test execution, without putting the burden on the user for selecting which build actions should take place and when. In other words, the framework provides a clean separation between specific build and test execution, analysis and metrics collection. This section provides a high-level description of the plugin framework architecture and guiding principals.

The reason for exposing the framework to users is that the framework also provides the ability for the ETICS client to be extended dynamically, without having to modify the core client functionality. Further, ETICS users can leverage the extensibility, such that they can better reuse investments in home grown and/or commercial tools and techniques valuable to them during build and test.

In order to even better leverage the work performed by existing plugins, the plugin framework also allow plugins to contribute to other plugins, without having the plugin being contributed to needing intimate knowledge from the contributing plugins. For example, a plugin calculating code coverage during execution can contribute to a testing plugin, without the testing plugin requiring intimate knowledge about how code coverage is calculated.

The goal of the plugin framework can be summarised as follows: * Provide an extendible framework for common actions to be performed during build and test execution * Provide natural separation between specific build and test execution, analysis and metrics collection * Provide open interface for users to register their own plugins to be executed during build and test * Provide raw data for build and test and repository services

To guide the definition of the framework, here are examples of actions that could map to plugins, to be executed by the framework: * Static analysis – Single line of code count (SLOC) – Cyclomatic complexity – Depth of inheritance – IPv6 compliance – Check style * Dynamic analysis – Code coverage – Memory leaks * Test execution – Java: JUnit – C++: CPPUNIT (CPPTEST)

INFSOM-RI-026753 2007 © Members of ETICS collaboration 129 / 145 ETICS USER MANUAL

– Python: PyUnit – Script/Executable

The guiding principal in the examples above is that none of these actions related to the specifics of the modules on which these actions are performed.

Figure 69 describes the high-level framework architecture.

Figure 69 - Plugin Framework High-level architecture

The framework is implemented as an extension to the ETICS command-line client. In this respect the framework is totally transparent to the user. The framework also leverages the property processing already performed by the client. Raw data can be generated and should be placed in the reports directory in order for these generated outputs being available after the remote execution.

When a command is executed, the plugin framework first loads and registers all plugins. The registration process requires for each plugin to provide information regarding the profile for which they should be activated, as well as which ETICS target of which command they apply to (i.e. VCS Command, Build Command and/or Test Command). In other words, the following combination dictates the execution of each plugin: profile+command+target.

To add an extra level of flexibility, wildcards can be used by plugins during the registration. For example, the generic SystemCallPlugin responsible for executing as a system call the string value of each command might register for: profiles=“*”, commands=“*” and target=“*”.

The next section provides an overview of the specification of plugins.

16.2. Plugin specification In order to qualify as a valid plugin, plugin implementations must comply with a few basic rules. The plugin must be provided in the form of a Python module or package. The module must be dropped in

INFSOM-RI-026753 2007 © Members of ETICS collaboration 130 / 145 ETICS USER MANUAL

the plugins directory. If a plugin is implemented as a Python module (e.g. applicable for very simple plugins), it must following this naming convention: * “*plugin.py” * “plugin*.py” On the other hand, if a plugin is implemented as Python package, the package directory must follow this naming convention: * “*plugin” * “plugin*” Note: these conventions is case insensitive.

The module or the init.py module in the case of a package must implement the method getPluginInstance, which returns an instance of the plugin.

Python doesn’t support natively the concept of interface, like Java or C#. Instead, each plugin must provide a class that implements the following methods: * register: defines which profile, command and target it registers for. A helper class is available to the plugin for implementing this easily. * contribute: defines which plugin it contributes to. A helper class is available to the plugin for implementing this easily. * init: initialise the plugin and its environment * execute: execute the plugin * stopExecute: stop the execution of the plugin * publish: harvest and publish to a known location the results of the execution * finalise: executed once at the end of the processing, gives a chance to the plugin to clean its resources and perform any final actions, if any.

The first two methods are called as part of the registration process, in two distinct passes, which guarantees that all plugins have being registered before contributions can be made.

During each target execution (e.g. init, checkstyle, compile, etc in the case of the BuildCommand), the following methods are called for the plugins matching the current processing state: init, execute, stopExecute and publish. Each plugin is expected to call its contributors in each method. A helper class is provided by the plugin framework to easily execute the contributors.

The plugins are also provided with an instance of the PropertyManager class (providing full access to the property space of the current module being executed) and the BuildStatus class (providing access to the build and test status XML document).

Examples of plugin implementations are provided in Appendix A.

INFSOM-RI-026753 2007 © Members of ETICS collaboration 131 / 145 ETICS USER MANUAL

A. PLUGIN SAMPLES The following samples show examples of plugins. The samples have been edited for clarity and are not complete. You can find a complete version in the ETICS CVS repository. Specific links are provided for each sample.

A.1. SLOCCOUNT PLUGIN This plugin uses David Wheeler SLOCCount program to calculate SLOCS for a build configuration. It also generates a custom report that is included as part of the metrics overall report. An example of the output of this plugin can be seen in section 14.2. This plugin is provided as a package, which means that a init.py module is present and both files are located in a directory called SloccountPlugin.

The complete plugin26 can be found in the ETICS CVS repository.

SloccountPlugin.py:

#!/usr/bin/env python import sys import os

import log4py

import util import shutil import processUtil import Store

import PluginBase import PluginManager from PluginExecutionError import PluginExecutionError

logger = log4py.Logger().get_instance()

26 http://etics.cvs.cern.ch/cgi-bin/etics.cgi/org.etics.plugins.sloccount/

INFSOM-RI-026753 2007 © Members of ETICS collaboration 132 / 145 ETICS USER MANUAL

class SloccountPlugin(PluginBase):

def __init__(self,verbose=False): super(SloccountPlugin,self).__init__(verbose) if self.verbose: print 'Instantiating SloccountPlugin' self.pluginManager = None

...

def register(self,pluginManager): # This plugin should run if no other plugins do profiles = ['*'] targets = { 'buildCommands':['precheckstyle'] } definition = PluginDefinition(profiles,targets) self.pluginManager = pluginManager

...

def execute(self,profile=None,command=None,target=None,targetString=None,propertyManager=None,buildStatus=None,cmds={},**kw): if propertyManager.properties.has_key('nocheckstyle'): if propertyManager.properties['nocheckstyle']: return True

if not os.getenv('HOME'): os.environ['HOME'] = propertyManager['workspaceDir']

self.outdata = None self.errdata = None if self.verbose: print 'Executing SloccountPlugin' process = processUtil.ProcessUtil() process.nooutput = not self.verbose

cmd = "sloccount %s" % propertyManager['moduleDir'] print ' [%s]: %s' % (target,cmd) try: self.outdata, self.errdata, err = process.systemCall(cmd) except RuntimeError, ex: if ex.args[1].find('SLOC total is zero') = -1: self.outdata = ex.args[1] return True

INFSOM-RI-026753 2007 © Members of ETICS collaboration 133 / 145 ETICS USER MANUAL

else: logger.error(str(ex)) raise PluginExecutionError(str(ex)) if err = 0: logger.error(str(ex)) raise PluginExecutionError((outdata,errdata,err))

return True

def publish(self,profile=None,command=None,target=None,targetString=None,propertyManager=None,buildStatus=None,**kw): wasSuccessful = True if self.verbose: print 'Publishing' if propertyManager = None: if propertyManager.properties.has_key('nocheckstyle'): if propertyManager.properties['nocheckstyle']: return True else: return wasSuccessful

try:

...

except KeyboardInterrupt: raise except Exception, ex: logger.error(str(ex)) print ex raise PluginExecutionError(ex) return wasSuccessful

def finalise(self,profile=None,command=None,target=None,targetString=None,propertyManager=None,buildStatus=None,**kw): wasSuccessful = True if self.verbose: print 'Finalising Sloccount plugin' if propertyManager.properties.has_key('nocheckstyle'): if propertyManager.properties['nocheckstyle']: return True

...

if self.verbose: print 'Generating sloccount report' metrics = buildStatus.CreateMetrics('SLOCCount')

INFSOM-RI-026753 2007 © Members of ETICS collaboration 134 / 145 ETICS USER MANUAL

metrics.detailshtmllink = 'sloccount/index.html' for lang in self.totalLangSLOCs: metrics.values[lang] = self.totalLangSLOCs[lang] metrics.value = self.totalSLOCs buildStatus.setOverallMetrics(metrics)

return wasSuccessful

init.py:

#!/usr/bin/env python

import SloccountPlugin

def getPluginInstance(): return SloccountPlugin()

A.2. PYUNIT AND PYCOVERAGE PLUGINS This example features a standard plugin “PyUnit” and a contributing plugin “PyCoverage”. The first plugin registers for a given set of commands, targets and profile, while the second plugin contributes the first one by name. In order for the plugin being contributed to accept a contribution, it must do a little more work, in order to define when in its execution it invites contributions.

The complete PyUnit27 and PyCoverage28 plugins and can be found in the ETICS CVS repository.

PyUnitPlugin.py:

27 http://etics.cvs.cern.ch/cgi-bin/etics.cgi/org.etics.build-system.plugin-framework/src/plugins/PyUnitPlugin.py 28 http://etics.cvs.cern.ch/cgi-bin/etics.cgi/org.etics.build-system.plugin-framework/src/plugins/PyCoveragePlugin

INFSOM-RI-026753 2007 © Members of ETICS collaboration 135 / 145 ETICS USER MANUAL

#!/usr/bin/env python

import sys import os import traceback

import log4py

import PluginBase import PluginManager from PluginManager import PluginExecutionError import testtools.TestManager as TestManager import xunittest

logger = log4py.Logger().get_instance()

def getPluginInstance(): return PyUnitPlugin()

class PyUnitPlugin(PluginBase):

def init__(self,verbose=True): super(PyUnitPlugin,self).__init__(verbose) if self.verbose: print 'Instantiating PyUnitPlugin' self.pluginManager = None self.pluginName = self.__module

def register(self,pluginManager): # Which profile this plugin works with profiles = ['pyunit'] targets = {'buildCommands':['test'],'testCommands':['test']} definition = PluginDefinition(profiles,targets) self.pluginManager = pluginManager self.pluginManager.register(self.__module__,definition)

def init(self,profile=None,command=None,target=None,targetString='',propertyManager=None,**kw): if propertyManager = None: if propertyManager.properties.has_key('notest'): if propertyManager.properties['notest']: return True

INFSOM-RI-026753 2007 © Members of ETICS collaboration 136 / 145 ETICS USER MANUAL

kw[self.__module__] = self return self.pluginManager.initContributions(self,profile,command,target,targetString,propertyManager,**kw)

def execute(self,profile=None,command=None,target=None,targetString='',propertyManager=None,**kw):

wasSuccessful = True if propertyManager.properties.has_key('notest'): if propertyManager.properties['notest']: print " Property 'notest' set, skipping tests" return wasSuccessful

...

# Execute any contribution made by other plugins to this one self.pluginManager.executeContributions(self,profile,command,target,targetString,propertyManager,**kw)

...

return wasSuccessful

def stopExecute(self,profile=None,command=None,target=None,targetString=None,propertyManager=None,**kw): if propertyManager.properties.has_key('notest'): if propertyManager.properties['notest']: return True return self.pluginManager.stopExecuteContributions(self,profile,command,target,targetString,propertyManager)

def publish(self,profile=None,command=None,target=None,targetString=None,propertyManager=None,**kw): if self.verbose: print 'Generating test report' if propertyManager.properties.has_key('notest'): if propertyManager.properties['notest']: return True oldDir = self._changeDir(propertyManager) wasSuccessful = False try: generator = ReportGenerator() wasSuccessful = generator.generateReport() finally: kw[self.__module__] = self self.pluginManager.publishContributions(self,profile,command,target, targetString,propertyManager) os.chdir(oldDir) return wasSuccessful

INFSOM-RI-026753 2007 © Members of ETICS collaboration 137 / 145 ETICS USER MANUAL

PyCoveragePlugin.py:

#!/usr/bin/env python ''' Copyright (c) Members of the ETICS Collaboration. # http://www.eu-etics.org

File: PyCoveragePlugin.py

Authors: Marc-Elian Begin <Marc-Elian.Begin@cern.ch>

Version info: $Id: PyCoveragePlugin.py,v 1.13 2007/01/26 11:22:48 mbegin Exp $ '''

import sys import os import traceback

import log4py import coverage

import PluginBase import PluginManager import testtools.TestManager as TestManager

logger = log4py.Logger().get_instance()

def getPluginInstance(): return PyCoveragePlugin()

class PyCoveragePlugin(PluginBase):

def __init__(self,verbose=False): super(PyCoveragePlugin,self).__init__(verbose) if self.verbose: print 'Instantiating PyCoveragePlugin' self.pluginManager = None

INFSOM-RI-026753 2007 © Members of ETICS collaboration 138 / 145 ETICS USER MANUAL

def register(self,pluginManager): self.pluginManager = pluginManager self.pluginManager.register(self.__module__)

def contribute(self,pluginManager): self.pluginManager = pluginManager self.pluginManager.contribute(self.__module__,'plugins.PyUnitPlugin')

def init(self,profile=None,command=None,target=None,targetString=None,propertyManager=None,buildStatus=None,**kw): if self.verbose: print 'Initialising coverage for plugin: %s' % kw['plugins.PyUnitPlugin'].__module__ return True

def execute(self,profile=None,command=None,target=None,targetString=None,propertyManager=None,buildStatus=None,**kw): if self.verbose: print 'Calculating coverage' if propertyManager = None: if propertyManager.properties.has_key('nocoverage'): if propertyManager.properties['nocoverage']: if self.verbose: print "Property 'nocoverage' set, skipping coverage calculation" return True coverage.start() return True

def stopExecution(self,profile=None,command=None,target='',targetString=None,propertyManager=None,buildStatus=None,**kw): if self.verbose: print 'Stopping calculating coverage' if propertyManager = None: if propertyManager.properties.has_key('nocoverage'): if propertyManager.properties['nocoverage']: return True coverage.stop() return True

def publish(self,profile=None,command=None,target='',targetString=None,propertyManager=None,buildStatus=None,**kw): if self.verbose: print 'Generating coverage report' srcDir = os.path.join('..','..','src') if propertyManager = None: if propertyManager.properties.has_key('nocoverage'): if propertyManager.properties['nocoverage']: if self.verbose: print "Property 'nocoverage' set, skipping coverage calculation" return True if propertyManager.properties.has_key('srcDir'): srcDir = propertyManager.properties['srcDir'] print ' Calculating coverage for all Python modules in directory structure:'

INFSOM-RI-026753 2007 © Members of ETICS collaboration 139 / 145 ETICS USER MANUAL

print ' %s' % srcDir, '(this may take several minutes)' modules = [] try: for n,m in sys.modules.items(): try: if not 'etics' in m.__file__ or 'externals' in m.__file__: continue except AttributeError: continue if self.verbose: print 'Calculating coverage for module: %s' % n try: coverage.analysis(m.__file__) modules.append(m) except (coverage.CoverageException,SyntaxError), ex: if self.verbose: print 'Error analysing module %s' % m.__file__ if self.verbose: print 'Generating coverage report for all loaded modules' metrics = coverage.report(modules,ignore_errors=1, show_missing=0,metrics=buildStatus.CreateMetrics('Coverage')) buildStatus.setOverallMetrics(metrics) except KeyboardInterrupt: raise except Exception, ex: print >> sys.stderr, 'Error occured calculating coverage, see log for details' error = '\n'.join(traceback.format_exception(ex.__class__.__name__,ex,sys.exc_info()[2])) if self.verbose: print >> sys.stderr, error logger.error(error) return False return True

INFSOM-RI-026753 2007 © Members of ETICS collaboration 140 / 145 ETICS USER MANUAL

B. PROPERTIES The following table lists all the properties that have a special semantic for the ETICS CLI. This table also specifies if the different properties are automatically created by the system, or can be defined by users, with the corresponding semantic.

Property Name Description Default Value build.root Relative to the root directory of the module, the directory from where to perform the build distDir Directory in which built artefacts are Dist located eticsHome Directory of the root installation of The value of the the ETICS ETICS_HOME environment variable if it is set, otherwise ‘etics’, assuming that the client was installed in the workspace gcc.version Version of the GCC compiler Locally detected libtool.build-prefix String to be used for relocation of /opt/ libtool *.la files. By default the client looks for the string ‘/opt/’ and replaces it with the local installation path in the ETICS workspace. If the package has a different prefix, this property must be set accordingly location Directory pointing to the location of The binaries path the binaries of the configuration package.forceSource If this property is set to True, the Undefined configuration is built from source irrespective of what command-line options have been used package.preserve.libtool Setting this property to True prevents False the client from stripping the libtool *.la files when creating binary packages. Use with care since it may cause the packages not to be relocatable or even usable package.userspec Location (relative to the module Undefined root) of a user-defined spec file for creating RPMs package.tgzLocation The default location (relative to the tgz module root) where generated

INFSOM-RI-026753 2007 © Members of ETICS collaboration 141 / 145 ETICS USER MANUAL

tarballs are stored in each module at the end of a build and where the Publisher looks for them package.SDEBSLocation The default location (relative to the debs module root) where generated source dsc are stored in each module at the end of a build and where the Publisher looks for them package.DEBSLocation The default location (relative to the debs module root) where generated binary debs are stored in each module at the end of a build and where the Publisher looks for them package.prefix The default installation prefix for the /opt/ generated RPMS package.buildarch It can be used to override the default The local platform name specifier. It is necessary to set it to ‘noarch’ to generate ‘noarch’ RPMS and ‘all’ debs package.provides The list of Provides entries for the Undefined RPM spec file (overrides the autodetected list) or deb control file (for virtual packages) package.requires The list of Requires entries for the Undefined RPM spec file (overrides the autodetected list) package.depends The list of Depends entries for the Undefined deb control file (overrides the autodetected list) package.predepends The list of Pre-Depends entries for Undefined the deb control file package.obsoletes The list of Obsoletes entries for the Undefined RPM spec file package.replaces The list of Replaces entries for the Undefined deb control file package.conflicts The list of Conflicts entries for the Undefined RPM spec file and the deb control file package.suggests The list of Suggests entries for the Undefined deb control file package.recommends The list of Recommends entries for Undefined the deb control file

INFSOM-RI-026753 2007 © Members of ETICS collaboration 142 / 145 ETICS USER MANUAL

package.enhances The list of Enhances entries for the Undefined deb control file package.autoreqprov Can be ‘yes’ or ‘no’. If it is set to Yes yes, rpm will try to automatically detect requires and provides from the packaged files and libs, otherwise will only use specified entries (from the ETICS packager or user defined package.provides/package.requires properties) packageName Name to be given to generated packages platformName Name of the local platform Locally detected profile Profile used to steer the selection of Undefined plugins python.version Version of the local Python Locally detected installation reportsDir Location of the reports directory /reports repositoryDir Location of the repository directory /repository src.location If a given module is checked-out The location of the source from source, this property points to files the root of the module stageDir Directory where files are staged /stage test.root Relative to the root directory of the module, the directory from where tests should be executed workspaceDir Location of the workspace package.createsource Instructs the client to create source False packages as well as binary packages (like the --createsource option) package.nodistname Do not insert distribution name in Undefined the package file name (e.g. slc4) package.usetimestamp Append timestamp to package file Undefined name package.versioneddeps Use version constraints in spec file False Requires directives (>=) package.strictversioning Use strinct version constraints in False spec file Requires directives (=) (must be used together with package.versioneddeps) package.description String to be used as package

INFSOM-RI-026753 2007 © Members of ETICS collaboration 143 / 145 ETICS USER MANUAL

description in the spec file package.summary String to be used as package package.xxx.name If the generated binary file with Undefined extension xxx has a name not following the standard conventions, setting this property will allow the client to use the actual name when creating package list reports package.sourcefile Location of a source tarball to be Undefined used to create files lists for the spec file (setting this property overrides using the client generated source tarball) package.binfile Location of a binary tarball to be Undefined used to create files lists for the spec file (setting this property overrides using the client generated binary tarball) package.customtag Custom tag (string) to append to the Undefined generated package file name package.group Value of the RPM spec file Group ‘Unknown’ tag package.packager Value of the spec file Packager tag ‘ETICS’ and deb control file Maintainer tag package.define Comma-separated list of additional Undefined %define directives for the RPM spec file. Each entry is used to add a directive like:

%define package.priority Value of the Priority entry in the deb optional control file Package.section Value of the Section entry in the deb devel control file package.configFiles Comma-separated list of files to be Undefined prefixed with the %config directive in the package spec file or added to the deb conffiles file. The entries are the full file path relative to the module root package.configFilesNoreplace Comma-separated list of files to be Undefined prefixed with the

INFSOM-RI-026753 2007 © Members of ETICS collaboration 144 / 145 ETICS USER MANUAL

%config(noreplace) directive in the package spec file or added to the deb conffiles file. The entries are the full file path relative to the module root Table 29: Special properties

INFSOM-RI-026753 2007 © Members of ETICS collaboration 145 / 145

-- LorenzoDini - 06 Jun 2008