Deliverable Review Form

Identification of the deliverable or milestone
Project: EMI Deliverable or milestone identifier: DNA1.1
Title: Project Quality Assurance and Progress Monitoring Plan Doc. identifier: EMI-DNA1.1-CDSREF-Project_QA_Plan-v0_5
Author(s): Alberto Di Meglio (NA1, CERN), Florida Estrella (NA1, CERN) Due date: PM1 (May 2010)

Identification of the reviewer
Name: Francesco Giacomini Affiliation: INFN EMI Activity: SA1

Review date 20/07/2010
Author(s) revision date 10/08/2010
Second review date 13/08/2010
Author(s) second revision date 17/08/2010
Reviewer acceptance date 24/08/2010

Trivial fixes have been included directly in the document with change tracking.

General comments

The document is satisfactory. The remarks in the table below suggest some improvements.

Additional recommendations (not affecting the document content, e.g. recommendation for future work)

None.

Detailed comments on the content

Note 1: The reviewers must list here any observation they want to track explicitly and that require interaction with the authors
Note 2: These comments have to be explicitly addressed by the authors and the action taken must be clearly described

Page Section Observations and replies Is Addressed?
1 7 1.4 The text describing the amendment procedure will be very similar, if not identical, for all deliverables. Why don’t simply reference a document with this description? Probably it would be this document in fact.
-- FrancescoGiacomini - 20-Jul-2010

Well, not necessarily. Different documents may have different amendment procedures. For example, this QA plan explicitly references the QA Manager as responsible for the amendments and this doesn't make sense for other documents. The Technical Plans may require a more complex procedure with involvement of external parties. And so on.
-- AlbertoDiMeglio - 08-Aug-2010

In EGEE3 we used to write:
Amendments, comments and suggestions should be sent to the authors. The procedures documented in the EGEE “Document Management Procedure” will be followed: http://project-EGEE-III-na1-qa.web.cern.ch/project-EGEE-III-NA1-QA/EGEE-III/Procedures/DocManagmtProcedure/DocMngmt.htm
and I've never seen anything different. My guess is that authors will simply cut&paste from previous deliverables, probably even from EGEE3, so I keep suggesting that providing some boilerplate for the common case would be useful. Since this text can be included in the template, maybe with a note saying that one is free to change it if the procedure for that specific document ought to be different, this document can in fact ignore the issue.
-- FrancescoGiacomini - 13-Aug-2010

I've made the paragraph more generic in the template and added a note that the text can be adapted if needed
-- AlbertoDiMeglio - 17-Aug-2010
DONE
2 8 1.5 The acronym VV is never used
-- FrancescoGiacomini - 20-Jul-2010

Removed
-- AlbertoDiMeglio - 08-Aug-2010
DONE
3 10 3.1 Is "progress factor" a technical term? is so, it should be explained.
-- FrancescoGiacomini - 20-Jul-2010

No, bad phrasing for 'performance indicators'. Replaced with that.
-- AlbertoDiMeglio - 08-Aug-2010
DONE
4 11 4 The second paragraph is truncated after "and".
-- FrancescoGiacomini - 20-Jul-2010

Added missing table references
-- AlbertoDiMeglio - 08-Aug-2010
DONE
5 11 4 Second row in table 2, about Project planning and reporting: which validation method and responsibility apply to which task?
-- FrancescoGiacomini - 20-Jul-2010

Made correspondence between task, method and responsibility explicit
-- AlbertoDiMeglio - 08-Aug-2010
DONE
6 12 4.1.1 CVS is still used at CERN, not only SVN.
-- FrancescoGiacomini - 20-Jul-2010

Added
-- AlbertoDiMeglio - 08-Aug-2010
DONE
7 14 4.1.4 It would be useful to have template for risk management.
-- FrancescoGiacomini - 20-Jul-2010

Ok, will be added to TWiki
-- AlbertoDiMeglio - 08-Aug-2010
DONE
8 14 4.2.1 "Submission - 30" is not immediately clear. Better something like: "30 days before submission date"
-- FrancescoGiacomini - 20-Jul-2010

Modified
-- AlbertoDiMeglio - 08-Aug-2010
DONE
9 14 4.2.1 The TOC of a deliverable has to be validated by the task leader and WP leader in advance. It can simply be the WP leader to send it to the PEB, without the need to involve the PO
-- FrancescoGiacomini - 20-Jul-2010

Slighly modified to take this into account. However, the new phrasing leaves open the issue of how long the WP leader validation takes before they send it to the PEB. It is assumed that it is a very brief delay
-- AlbertoDiMeglio - 08-Aug-2010
DONE
10 15 4.2.1 Item 4), about the twiki-based review form: In practice there is no difference wrt the procedure followed in EGEE with dialog forms, apart that editing in twiki is a greater pain. Twiki is a better option only if it allows more interactivity between reviewers and the author, possibly all on the same page. Otherwise the dialog form is a better option, since the reviewer can work offline and just send the form at the end of the his/her review. Moreover the dates in the dialog form, e.g. the Review Date, should be dd/mm/yyyy or, even better, yyyy-mm-dd
-- FrancescoGiacomini - 20-Jul-2010

The choice of TWiki was to allow greater flexibility and interactivity and track the comments close to the documents. However, I'm not very happy myself with this, personally I would only use Word change tracking, using the log tables at the beginning of the document for the comments. I'm open to suggestions, but I would leave them for the next revision of this QA Plan
-- AlbertoDiMeglio - 08-Aug-2010

Comments in the document don't allow much iterativity either (just imagine this discussion managed with comments). TWiki can be fine, but it cannot just emulate the dialog form, especially if one has to respect the tabular format as I'm doing right now; it has to be used for example as suggested in the "hints on good style" (bullet on how to deal with a discussion). And you cannot use the log tables in the document for all the comments! Just imagine writing all this into that! I'm reappraising the EGEE dialog forms... We can certainly continue the discussion not in the scope of this document, but what should the reviewers of the next deliverables do?
-- FrancescoGiacomini - 13-Aug-2010

Although TWiki is not the most comfortable tool for editing, it allows some degree of interactivity as this dialogue form shows. The detailed comments table can be used to discuss each topic as needed. It is recommended to use the TWiki style guidelines and use TWiki signatures for each comment
-- AlbertoDiMeglio - 17-Aug-2010
DONE
11 15 4.2.1 A note on how internal reviewers are chosen would be useful
-- FrancescoGiacomini - 20-Jul-2010

The following has been added: "The reviewers have to be chosen taking into account the skills and technical knowledge required to understand and critically assess the document within the context of EMI. Possibly they should be chosen from a different WP than the one issuing the deliverable, with the exception of JRA1 where the extent and diversity of skills may allow for independent reviews within the same WP. Whenever appropriate or necessary, external reviewers should also be considered, especially when the deliverable concerns topics of relevance in the relationships of EMI with other projects or communities."
-- AlbertoDiMeglio - 08-Aug-2010

That sentence is certainly worth specifying, but I was rather referring to the procedure to choose reviewers, e.g. something like: the author suggests the WPs that should provide a review and the leaders of those WPs then identifies them.
-- FrancescoGiacomini - 13-Aug-2010

The following note has been added:
"The Work Package leader is responsible to coordinate the process of selecting appropriate reviewers for the deliverables. The WP leader must contact other WP leaders within EMI or knowledgeable persons outside EMI and ask for the names of possible reviewers. The WP leader is requested to make sure that reviewers are selected for each deliverable at least three months before the deliverable due date"
-- AlbertoDiMeglio - 17-Aug-2010
DONE
12 15 4.2.2 A link to the template for the milestone report would be useful
-- FrancescoGiacomini - 20-Jul-2010

Added at point 1 of the procedure
-- AlbertoDiMeglio - 08-Aug-2010
DONE
13 15 4.2.2 What if the milestone is not achieved? probably the PEB should discuss and define appropriate actions, including change of plans
-- FrancescoGiacomini - 20-Jul-2010

The following has been added at the end of the section: "If a milestone is not achieved within the expected deadline, the PD must foresee appropriate time for discussion during the regular PEB meeting to identify the issue and define appropriate corrective actions. Changes of plan can also be considered in case the milestone is not relevant anymore due to changed technical conditions. In case a relevant milestone is not achieved even after reasonable corrective actions have been defined and proposed, the PD must escalate the issue to the CB and discuss possible interventions on the Partner(s) responsible for the milestone."
-- AlbertoDiMeglio - 08-Aug-2010
DONE
14 16 4.2.3 How should a WP contribution to the QR/PR be provided? Should the document be uploaded to twiki? to CDS? sent as an attachment? other?
-- FrancescoGiacomini - 20-Jul-2010

Right. I forgot this. Added in the procedure that it should be uploaded to TWiki
-- AlbertoDiMeglio - 08-Aug-2010
DONE
15 17 4.2.3 First item: "WP leaders provides contributions 20 days before the end of the period." I suppose it's after the end of the period
-- FrancescoGiacomini - 20-Jul-2010

No, this is the procedure for the Periodic Management Reports. the contributions should describe the work done in the full previous year and have to be put together in the full official report for the EC. The entire process has to start before the end of the period, since the report has to be sent to the EC 10 working days before the Review and at the latest 45 days after the and of the period.
-- AlbertoDiMeglio - 08-Aug-2010
DONE
16 17 4.2.3 End of the section, why specify here that PD and WP leaders provide presentations? Should they already be attached to the PR?
-- FrancescoGiacomini - 20-Jul-2010

No. Why? The presentations for the review and the PR are different things. But I agree that this sentence makes little sense here. I just removed it, since the review process is described elsewhere
-- AlbertoDiMeglio - 08-Aug-2010
DONE
17 17 4.2.4 If/when CDS supports reviews, will our review procedures change? if that's a possibility, maybe mention it
-- FrancescoGiacomini - 20-Jul-2010

I thought of that, however, it is still not clear whether this is indeed the case. it seems to offer this feature, but not in a way that we may actually fond useful. In any case, I've added a sentence about this
-- AlbertoDiMeglio - 08-Aug-2010
DONE
18 18 4.2.5 Who decides the authors of a publication, so that they can be all acknowledged?
-- FrancescoGiacomini - 20-Jul-2010

Good question. I'm not sure I have an answer. Should the PEB or PTB be involved or should this be left to the Partners? It may be difficult to enforce specific authors for papers concerning the detailed work of individual PTs. The PEB and/or PTB could decide in case of general overview papers about EMI
-- AlbertoDiMeglio - 08-Aug-2010

Whatever, but what is in the document now ("All authors have to be acknowledged") is not satisfactory.
-- FrancescoGiacomini - 13-Aug-2010

I've tried to clarify this issue in the text. The main point is that the 'authors' to be acknowledge are actually the 'contributors' having produced some of the work described in the paper. in addition, all paper describing relevant EMI work have to be communicated to the PEB for verification and dissemination
-- AlbertoDiMeglio - 17-Aug-2010
DONE
19 18 4.2.5 Include the exact wording on how to acknowledge the EC funding contribution
-- FrancescoGiacomini - 20-Jul-2010

Added
-- AlbertoDiMeglio - 08-Aug-2010
DONE
20 19 4.3 Provide more details on how documents should be versioned, not just "The version of the documents must be increased at every revision"
-- FrancescoGiacomini - 20-Jul-2010

Added detailed description at the end of the section
-- AlbertoDiMeglio - 08-Aug-2010

Good attempt, but there is something wrong at least with the first item:
The version number is composed of two (2) numbers and has the format x.y. The three numbers are called major version (x), minor version (y) and revision number (z) respectively.
(emphasis mine)
-- FrancescoGiacomini - 13-Aug-2010

Fixed the inconsistency. The version number should be three, although z can be omitted if z=0
-- AlbertoDiMeglio - 17-Aug-2010
DONE
21 21 5.1 KSA1.1 should be reviewed/removed. See discussions on User Support in the PEB
-- FrancescoGiacomini - 20-Jul-2010

Can it be redefined? If we remove it, we do not have any metric about the number of incidents. We should tack how many incidents we receive. We should probably replace Service Desk with a more appropriate term, but leave the KPI
-- AlbertoDiMeglio - 08-Aug-2010
DONE
22 28 6.3 Aren't three rehearsals a bit too much?
-- FrancescoGiacomini - 20-Jul-2010

In my experience, they are not enough. Take into account that the first is to get initial drafts and discuss about the principles, the second is to work on the details and the third is to properly rehearse timing and smoothness of delivery
-- AlbertoDiMeglio - 08-Aug-2010
DONE
23 30 7 Why are deliverable review dialog forms protected? how to do this in twiki?
-- FrancescoGiacomini - 20-Jul-2010

Indeed, no need to restrict them. In any case, TWiki has a protection mechanism based on login names, which is used for the Quarterly Reports for example
-- AlbertoDiMeglio - 08-Aug-2010
DONE
24 30 7 There are probably missing links corresponding to "EU Quarterly and Periodical Report" and "EU review report"
-- FrancescoGiacomini - 20-Jul-2010

The assumption was that CDS was going to be in place by now. For the moment I've added links to the TWiki pages
-- AlbertoDiMeglio - 08-Aug-2010
DONE
25 30 7 It would be nice if https://twiki.cern.ch/twiki/bin/view/EMI/EmiKPIs or equivalent page would be generated automatically
-- FrancescoGiacomini - 20-Jul-2010

Yes, but how? I will provide a template, but I don't see any way of creating the content automatically. The WP leaders have to provide numbers and NA1 fill the TWiki pages manually
-- AlbertoDiMeglio - 08-Aug-2010
DONE
26 31 8.1 No ODS/OpenOffice for spreadsheets?
-- FrancescoGiacomini - 20-Jul-2010

Added
-- AlbertoDiMeglio - 08-Aug-2010
DONE
27 32 8.6 About EVO: mention that we have already a room (or whatever is called) and how to access and use it (or a link thereto)
-- FrancescoGiacomini - 20-Jul-2010

Added. There is no direct link to the communities, as far as I know, but I've explained where to find it
-- AlbertoDiMeglio - 08-Aug-2010
DONE
Edit | Attach | Watch | Print version | History: r9 < r8 < r7 < r6 < r5 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r9 - 2010-08-24 - AlbertoDiMeglio
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    EMI All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback