Deliverable Review Form

Identification of the deliverable or milestone
Project: EMI Deliverable or milestone identifier: D4.3.3
Title: DSA2.3.3 - Periodic QA and QC Reports Doc. identifier: EMI-DXXX-CDSREF-Title-vx.x
Author(s): SA2* Due date: 30/04/2012

Identification of the reviewer
Name: A. Ceccanti Affiliation: INFN EMI Activity/External project or Institute: SA1

Review date mm/dd/yyyy
Author(s) revision date mm/dd/yyyy
Reviewer acceptance date mm/dd/yyyy

General comments

The deliverable in general looks good.

I think that of the negative statements should be turned into positive, as an increase in the quality of the releases is observed and testified by the data. The improvement is also observed outside EMI. A link to the results of EGI QC, which show that no failures have been observed in the latest released could be included in this document as a further proof of the positive quality trend.

The deliverable could elaborate a bit more in how SA2 plans to improve the communication with PTs (some points are already given in the QC conclusions sections). The idea of having "campaigns" focused at improving a specific quality aspect is a very good one.

As objected in the document, QC should give some more background to some quality metrics. In particular, for what concerns unit testing, it should be clearly mentioned that the low coverage is not expected to increase much during the course of the project, as test-driven development will likely be employed only for newly developed functionality. Indeed QC should be really severe on this with newly developed code, while having a different treshold for legacy code that will not be refactored and for which regression testing coverage is the most important quality metric.

I am a bit perplexed about the SLOC measurements as well. While it is true that EMI had an objective of reducing the maintained LOC, it's also true that new features can be provided only by increasing the LOC of EMI components. For this reason, the metric should take into account the agreed EMI harmonization objective and should count components that have already been removed from the EMI software stack (SLCS). It should also be explained that some reduction in code will likely be obtained by year three, with the adoption of the CANL. How this reduction can be effectively measured is a good challenge for SA2.

More detailed comments in the attached document.

25/05 Andrea I have included the comments that were not addressed.

Of these comments there are those that I consider important and would like to be addressed (or at least answered):

- Acknowledge in the text that the work on the policies included active contributions and good collaboration from JRA1 and SA1 representatives.

- In Section include references to EMI 1 QC reports (when QC was in SA1 and JRA1) to put things in context

- Fix titolo asse in figure 9 and 10 with the right axis title

- Clarify how the packaging of the RfCs regression coverage by product is calculated, e.g. If a product has 10 rfcs, 9 with associated test and one without does it score as a failure in reporting? Or am I just worrying too much?

- Does it make sense to include the full security assessment in this report? The detailed description of the assessed components?

Additional recommendations (not affecting the document content, e.g. recommendation for future work)

Detailed comments on the content

Note 1: The reviewers must list here any observation they want to track explicitly and that require interaction with the authors
Alternatively all changes must be listed in the document itself using Word change tracking features (if you use Word)
Note 2: These comments have to be explicitly addressed by the authors and the action taken must be clearly described

N Page Section Observations and Replies Is Addressed?
1 xx x.y Sequence of comments and replies separated by twiki signature and date    

Any other modification, spelling or grammatical corrections, etc must be done directly in the document using tracked changes or similar mechanisms that allows the authors to identify which correction is suggested.

-- AndreaCeccanti - 24-May-2012

Edit | Attach | Watch | Print version | History: r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r3 - 2012-05-26 - unknown
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    EMI All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback