---

Authors: Antonio Retico(SA1), Maite Barroso(SA1), Stephen Newhouse(EGEE Project Director)
Last Content Review: 20-Nov-2009
Expiration: 30-Apr-2010 (after this date please contact the author(s))

---

Pilot and Experimental services: general lines

Scope and goals of this document

The aim of this document is to provide a generic overview of pilot and experimental services management in the EGI infrastructure.

This reference is addressed in particular to organisers and coordinators of pilot and experimental services .

Definition

The Pilot and Experimental Services are services set up on demand in order to let user interact with a new/changed MW product/feature in production context. They have the following characteristics:

• Finite duration in time
• Measurable Objectives (success criteria)
• Pilot Coordination in EGI (EGI.org or NGI)
• Operated by a task-force according to the Pilot Service Operations Process

The set-up of Pilot and Experimental Services is regarded as part of the overall middleware development and release process in the context of the EGI infrastructure as described in the EGI BluePrint [1] and exemplified in the picture below

The distinction between Pilot and Experimental services in EGI is based on the different level of maturity of the software at the start-up. As this doesn't cause much difference from the operational perspective, in the following document we will refer to both Experimental and Pilot services as "pilots" stressing on the distinction only if relevant

Due to its characteristic a Pilot service can be regarded as a simple project

• Input : a certified version of MW introducing a new product/feature
• Output : a version of the MW which is recognized by the users to have met all the defined success criteria
• Added value : focused testing of functional and operational use cases. Parallel observation of non-functional behavior by developers, MU and OU is possible.

Pilot Service Operations Process

Ownership

This process was developed within the EGEE-III project by the SA1 activity, which retain ownership. At the end of the EGEE-III project the ownership will be transferred to EGI.eu

Players

• The EGI.eu Middleware Unit (MU)
• The EGI.eu Operations Unit (OU)
• The Product Team
• The *National Grid Initiative (NGI) *
• The Specialised Support Centres (SSC)
• The Site : A production site that commits with EGI to run and support services needed for the Pilot throughout its duration

Definitions available from the document "EGI Managing the Software Process" [2] .

Specifically the work is distributed as follows:

• Coordination and documentation: EGI.eu OU or NGI
• Infrastructure (HW and site support): NGI
• MW Support: Product teams
• Use Cases: SSC and EGI.eu OU (e.g. for monitoring and operations use cases)
• Feedback to EMI: EGI MU

Software and Repository

• Initial installation:
  • [ not applicable for Experimental Services ] based on certified MW versions (e.g. MW taken from MU’s ‘beta’ repository).
  • In particular the software entering the Pilot must be:
    • deployable by sites using standard methods
    • proved not to disturb in any way the pre-existing functionality of other/related services
• Upgrades : subsequent MW updates may bypass the standard certification process (e.g. by using dedicated repositories) and be applied to the pilot service through shortcut releases.
• Final Version : It would be highly desirable that the release process used in EGI allowed a default method to track the changes done in the pilot in order to make sure that the version deployed in the pilot installation is reproducible at any time. In absence of this it is responsibility of the Pilot Coordinator to assure that all the updates are duly tracked using procedures to be defined case-by-case within the pilot task force. As an operational workaround to verify that the version released as output of the pilot matches with what was really deployed we consider the possibility of leaving the pilot alive for the time needed to the MU to cross-check its conformity to the versions certified.

Workflow

• Request
  • Pilot services can be requested by many subjects (e.g EGI.org, users (SSC), NGIs) or bodies (e.g. MCB) and even by UMD developers
  • The decision to set-up a pilot service is made by EGI.org (as coordinator of the operational resources EGI.org OU must be involved in the decision). An essential requirement for starting-up a pilot is interest and commitment to use the pilot from at least a user community. Otherwise the request will be rejected.
• Kickstart (ruled by EGI.org OU)
  • Set up of the initial task force (to be approved by the relevant users and NGI)
  • Appointment of a Pilot Coordinator (must be in EGI)
• Operations (ruled by Pilot Coordinator)
  • Definitions of objectives and timelines (objectives to be set obligatorily together with users)
  • Set-up and run
    • Problems reported through standard channels (e.g. GGUS and Savannah tickets)
    • If there are critical issues (e.g. big unexpected functional flaws), decision to be taken by the pilot coordinator and the task force to continue or not.
    • Check-points to be run periodically to follow-up actions and to decide on the continuation/interruption of the pilot
    • Documentation and reports
• Closure
  • Wrap-up of results
  • Communication of results to EGI.org
  • Release of resources

Related Documents

1. EGI Blueprint - EGI_DS project
2. EGI: Managing the Software Process - Steven Newhouse et al.