Deliverable Review Form

Identification of the deliverable or milestone
Project: EMI Deliverable or milestone identifier: DNA1.1
Title: Project Quality Assurance and Progress Monitoring Plan Doc. identifier: EMI-DNA1.1-CDSREF-Project_QA_Plan-v0_5
Author(s): Alberto Di Meglio (NA1, CERN), Florida Estrella (NA1, CERN) Due date: PM1 (May 2010)

Identification of the reviewer
Name: Francesco Giacomini Affiliation: INFN EMI Activity: SA1

Review date 07/20/2010
Author(s) revision date 10/08/2010
Second review date 2010-08-13
Reviewer acceptance date mm/dd/yyyy

Trivial fixes have been included directly in the document with change tracking.

General comments

The document is satisfactory. The remarks in the table below suggest some improvements.

Additional recommendations (not affecting the document content, e.g. recommendation for future work)


Detailed comments on the content

Note 1: The reviewers must list here any observation they want to track explicitly and that require interaction with the authors
Note 2: These comments have to be explicitly addressed by the authors and the action taken must be clearly described

Page Section Observations Reply Is Addressed?
1 7 1.4 The text describing the amendment procedure will be very similar, if not identical, for all deliverables. Why don’t simply reference a document with this description? Probably it would be this document in fact. Well, not necessarily. Different documents maay have different amendment procedures. For example, this QA plan explicitly references the QA Manager as responsible for the amendments and this doesn't make sense for other documents. The Technical Plans may require a more complex procedure with involvement of external parties. And so on.  
In EGEE3 we used to write:
Amendments, comments and suggestions should be sent to the authors. The procedures documented in the EGEE “Document Management Procedure” will be followed:
and I've never seen anything different. My guess is that authors will simply cut&paste from previous deliverables, probably even from EGEE3, so I keep suggesting that providing some boilerplate for the common case would be useful. Since this text can be included in the template, maybe with a note saying that one is free to change it if the procedure for that specific document ought to be different, this document can in fact ignore the issue.
2 8 1.5 The acronym VV is never used Removed DONE
3 10 3.1 Is "progress factor" a technical term? is so, it should be explained. No, bad phrasing for 'performance indicators'. Replaced with that. DONE
4 11 4 The second paragraph is truncated after "and". Added missing table references DONE
5 11 4 Second row in table 2, about Project planning and reporting: which validation method and responsibility apply to which task? Made correspondence between task, method and responsibility explicit DONE
6 12 4.1.1 CVS is still used at CERN, not only SVN. Added DONE
7 14 4.1.4 It would be useful to have template for risk management. Ok, will be added to TWiki DONE
8 14 4.2.1 "Submission - 30" is not immediately clear. Better something like: "30 days before submission date". Modified DONE
9 14 4.2.1 The TOC of a deliverable has to be validated by the task leader and WP leader in advance. It can simply be the WP leader to send it to the PEB, without the need to involve the PO. Slighly modified to take this into account. However, the new phrasing leaves open the issue of how long the WP leader validation takes before they send it to the PEB. It is assumed that it is a very brief delay DONE
10 15 4.2.1 Item 4), about the twiki-based review form: In practice there is no difference wrt the procedure followed in EGEE with dialog forms, apart that editing in twiki is a greater pain. Twiki is a better option only if it allows more interactivity between reviewers and the author, possibly all on the same page. Otherwise the dialog form is a better option, since the reviewer can work offline and just send the form at the end of the his/her review. Moreover the dates in the dialog form, e.g. the Review Date, should be dd/mm/yyyy or, even better, yyyy-mm-dd. The choice of TWiki was to allow greater flexibility and interactivity and track the comments close to the documents. However, I'm not very happy myself with this, personally I would only use Word change tracking, using the log tables at the beginning of the document for the comments. I'm open to suggestions, but I would leave them for the next revision of this QA Plan  
Comments in the document don't allow much iterativity either (just imagine this discussion managed with comments). TWiki can be fine, but it cannot just emulate the dialog form, especially if one has to respect the tabular format as I'm doing right now; it has to be used for example as suggested in the "hints on good style" (bullet on how to deal with a discussion). And you cannot use the log tables in the document for all the comments! Just imagine writing all this into that! I'm reappraising the EGEE dialog forms... We can certainly continue the discussion not in the scope of this document, but what should the reviewers of the next deliverables do?    
11 15 4.2.1 A note on how internal reviewers are chosen would be useful. The following has been added: "The reviewers have to be chosen taking into account the skills and technical knowledge required to understand and critically assess the document within the context of EMI. Possibly they should be chosen from a different WP than the one issuing the deliverable, with the exception of JRA1 where the extent and diversity of skills may allow for independent reviews within the same WP. Whenever appropriate or necessary, external reviewers should also be considered, especially when the deliverable concerns topics of relevance in the relationships of EMI with other projects or communities."  
That sentence is certainly worth specifying, but I was rather referring to the procedure to choose reviewers, e.g. something like: the author suggests the WPs that should provide a review and the leaders of those WPs then identifies them.    
12 15 4.2.2 A link to the template for the milestone report would be useful. Added at point 1 of the procedure DONE
13 15 4.2.2 What if the milestone is not achieved? probably the PEB should discuss and define appropriate actions, including change of plans. The following has been added at the end of the section: "If a milestone is not achieved within the expected deadline, the PD must foresee appropriate time for discussion during the regular PEB meeting to identify the issue and define appropriate corrective actions. Changes of plan can also be considered in case the milestone is not relevant anymore due to changed technical conditions. In case a relevant milestone is not achieved even after reasonable corrective actions have been defined and proposed, the PD must escalate the issue to the CB and discuss possible interventions on the Partner(s) responsible for the milestone." DONE
14 16 4.2.3 How should a WP contribution to the QR/PR be provided? should the document be uploaded to twiki? to CDS? sent as an attachment? other? Right. I forgot this. Added in the procedure that it should be uploaded to TWiki DONE
15 17 4.2.3 First item: "WP leaders provides contributions 20 days before the end of the period." I suppose it's after the end of the period. No, this is the procedure for the Periodic Management Reports. the contributions should describe the work done in the full previous year and have to be put together in the full official report for the EC. The entire process has to start before the end of the period, since the report has to be sent to the EC 10 working days before the Review and at the latest 45 days after the and of the period. DONE
16 17 4.2.3 End of the section, why specify here that PD and WP leaders provide presentations? should they already be attached to the PR? No. Why? The presentations for the review and the PR are different things. But I agree that this sentence makes little sense here. I just removed it, since the review process is described elsewhere DONE
17 17 4.2.4 If/when CDS supports reviews, will our review procedures change? if that's a possibility, maybe mention it. I thought of that, however, it is still not clear whether this is indeed the case. it seems to offer this feature, but not in a way that we may actually fond useful. In any case, I've added a sentence about this. DONE
18 18 4.2.5 Who decides the authors of a publication, so that they can be all acknowledged? Good question. I'm not sure I have an answer. Should the PEB or PTB be involved or should this be left to the Partners? It may be difficult to enforce specific authors for papers concerning the detailed work of individual PTs. The PEB and/or PTB could decide in case of general overview papers about EMI  
Whatever, but what is in the document now ("All authors have to be acknowledged") is not satisfactory.    
19 18 4.2.5 Include the exact wording on how to acknowledge the EC funding contribution. Added DONE
20 19 4.3 Provide more details on how documents should be versioned, not just "The version of the documents must be increased at every revision". Added detailed description at the end of the section  
Good attempt, but there is something wrong at least with the first item:
The version number is composed of two (2) numbers and has the format x.y. The three numbers are called major version (x), minor version (y) and revision number (z) respectively.
(emphasis mine)
21 21 5.1 KSA1.1 should be reviewed/removed. See discussions on User Support in the PEB. Can it be redefined? If we remove it, we do not have any metric about the number of incidents. We should tack how many incidents we receive. We should probably replace Service Desk with a more appropriate term, but leave the KPI DONE
22 28 6.3 Aren't three rehearsals a bit too much? In my experience, they are not enough. Take into account that the first is to get initial drafts and discuss about the principles, the second is to work on the details and the third is to properly rehearse timing and smoothness of delivery DONE
23 30 7 Why are deliverable review dialog forms protected? how to do this in twiki? Indeed, no need to restrict them. In any case, TWiki has a protection mechanism based on login names, which is used for the Quarterly Reports for example DONE
24 30 7 There are probably missing links corresponding to "EU Quarterly and Periodical Report" and "EU review report". The assumption was that CDS was going to be in place by now. For the moment I've added links to the TWiki pages DONE
25 30 7 It would be nice if or equivalent page would be generated automatically. Yes, but how? I will provide a template, but I don't see any way of creating the content automatically. The WP leaders have to provide numbers and NA1 fill the TWiki pages manually DONE
26 31 8.1 No ODS/OpenOffice for spreadsheets?. Added DONE
27 32 8.6 About EVO: mention that we have already a room (or whatever is called) and how to access and use it (or a link thereto). Added. There is no direct link to the communities, as far as I know, but I've explained where to find it DONE
Edit | Attach | Watch | Print version | History: r9 | r7 < r6 < r5 < r4 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r5 - 2010-08-13 - FrancescoGiacomini
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    EMI All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback