M1: Defects and enhancement requests in None state.

Description: upon submission a defect or an enhancement request has state None. It is moved to Accepted after the team responsible for the affected product has done the assessment, which should not last more than two weeks.

The metric shows the number of defects and enhancement requests (total and by Category), submitted by more than two weeks and still in state None, on a weekly basis, considering previous week's submissions.

The starting situation is quite bad: at Friday, Jan 15th 2010, 706 entries out of 1859 for all gLite are in state None.

As of Monday Feb 1st there are 529 Open gLite bugs in state "None". 22 of these have been in another state in their lifetime. These are important to note as they have traversed some other states and returned to "None".

Monday March 15th... 439 bugs in state "None" (23 have been in another previous state)

Friday May 21st... 262 open bugs in state "None".

Target: 0

M2: number of defects with Detection Area "Production" vs "Development" vs general testing activities

Description: the metric shows the number of defects by Detection Area on a weekly basis: Production, Development and general testing activities.

Ideally defects should be discovered as early as possible during the lifetime of a product.

Target: minimize the number of defects found in Production.

Fraction of bugs found in development
Fraction of bugs found in production
Fraction of bugs found in testing

The following plot shows the total number of bugs PER MONTH opened by Testing, Development and Production activities and the outline, the total opened. It can be seen that in earlier project phases this was not well-organized.

Non-cumulative per month

M3: Defects found by users vs defects found by insiders

Description: the metric considers the following two numbers:

- defects found in Production by users (A); - defects found in Production by gLite insiders (B).

The metric shows the ratio A / (A+B) on a weekly basis, considering previous week's submissions.

gLite insiders are people involved in gLite development, either as developers, testers (including alpha and beta testing), packagers.

Target: minimize the ratio.

Fraction of bugs submitted by users

M4: Defects submitted via GGUS

Description: final users of gLite (either applications or operations) are technically allowed to submit an entry directly to Savannah when they experience a problem in their activity. This however makes it more difficult to provide user-face metrics about, for example, the time required to solve an incident. For this reason, following the recommendations of the project reviewers, an incident experienced in production by a final user should always be reported via GGUS and submitted to Savannah only after first- and second-level support have evaluated it and concluded that it is actually caused by a software defect. In that case the GGUS ticket and the Savannah entry are properly linked together via the "Related issue" and "GGUS Reference URL" fields respectively.

This metric considers the following two numbers:

- defects found in Production by a user and submitted directly to Savannah;

- defects found in Production by a user, submitted firts to GGUS and eventually to Savannah.

The metric shows the ratio A / (A+B) on a weekly basis, considering previous week's submissions.

Target: minimize the ratio.

Fraction of production GGUS bugs

M5: Defects and enhancement requests by priority

Description: defects and enhancement requests are classified in four priority levels: Emergency, High, Medium, Low (for more details see https://edms.cern.ch/document/1019911).

This metric shows the distribution of defects and enhancement requests by priority.

Target: minimize the defects with Emergency and High priority.

M6: Non-cumulative GGUS bugs

Description: This shows the number of bugs found in production submitted per 30 day period for gLite providers (those directly or closely involved in producing gLite), gLite users (with and without GGUS links).

Bugs found in production from providers and users

Target: For bugs found in production there should be none from gLite providers and all bugs from users should have a GGUS link as they should all come through GGUS!

The query used to produce the vectors for this picture is here.

Of particular note is the selection to find the gLite "providers" which is subsequently vetoed to obtain the bugs submitted by users.

The selection to find gLite bugs has been updated to reflect the new categories that have been lately created.

-- JohnWhite - 01-Feb-2010