ReviewDNA11FG < EMI

EMI Web>EmiDocuments>EmiDeliverables>DeliverableDNA11>ReviewDNA11FG (2010-08-24, AlbertoDiMeglio) (raw view)
---+ Deliverable Review Form

| *Identification of the deliverable or milestone* ||
| Project:	EMI | Deliverable or milestone identifier:	DNA1.1 |
| Title: Project Quality Assurance and Progress Monitoring Plan | Doc. identifier: [[https://twiki.cern.ch/twiki/pub/EMI/DeliverableDNA11/EMI-DNA1.1-CDSREF-Project_QA_Plan-v0_5.doc][EMI-DNA1.1-CDSREF-Project_QA_Plan-v0_5]] |
| Author(s): Alberto Di Meglio (NA1, CERN), Florida Estrella (NA1, CERN) | Due date: PM1 (May 2010) |

| *Identification of the reviewer* |||
| Name: Francesco Giacomini | Affiliation: INFN | EMI Activity: SA1 |

| Review date | 20/07/2010 |
| Author(s) revision date | 10/08/2010 |
| Second review date | 13/08/2010 |
| Author(s) second revision date | 17/08/2010 |
| Reviewer acceptance date | 24/08/2010 |

Trivial fixes have been included directly in the [[https://twiki.cern.ch/twiki/pub/EMI/ReviewDNA11FG/EMI-DNA1.1-CDSREF-Project_QA_Plan-v0_5-fg1.doc][document]] with change tracking.

---++ General comments

The document is satisfactory. The remarks in the table below suggest some improvements.

---++ Additional recommendations (not affecting the document content, e.g.  recommendation for future work)

None.

---++ Detailed comments on the content

*Note 1*: The reviewers must list here any observation they want to track explicitly and that require interaction with the authors<br/>
*Note 2*: These comments have to be explicitly addressed by the authors and the action taken must be clearly described

| N° | Page | Section | Observations and replies | Is Addressed? |
| 1 | 7 | 1.4 |\
The text describing the amendment procedure will be very similar, if not identical, for all deliverables. Why  don’t simply reference a document with this description? Probably it would be this document in fact.%BR%\
-- Main.FrancescoGiacomini - 20-Jul-2010%BR%\
%BR%\
Well, not necessarily. Different documents may have different amendment procedures. For example, this QA plan explicitly references the QA Manager as responsible for the amendments and this doesn't make sense for other documents. The Technical Plans may require a more complex procedure with involvement of external parties. And so on.%BR%\
-- Main.AlbertoDiMeglio - 08-Aug-2010%BR%\
%BR%\
In EGEE3 we used to write:%BR%\
Amendments, comments and suggestions should be sent to the authors. The procedures documented in the EGEE “Document Management Procedure” will be followed: http://project-EGEE-III-na1-qa.web.cern.ch/project-EGEE-III-NA1-QA/EGEE-III/Procedures/DocManagmtProcedure/DocMngmt.htm%BR%\
and I've never seen anything different. My guess is that authors will simply cut&paste from previous deliverables, probably even from EGEE3, so I keep suggesting that providing some boilerplate for the common case would be useful. Since this text can be included in the template, maybe with a note saying that one is free to change it if the procedure for that specific document ought to be different, this document can in fact ignore the issue.%BR% \
-- Main.FrancescoGiacomini - 13-Aug-2010%BR%\
%BR%\
I've made the paragraph more generic in the template and added a note that the text can be adapted if needed%BR%\
-- Main.AlbertoDiMeglio - 17-Aug-2010\
| %Y% |
| 2 | 8 | 1.5 |\
The acronym VV is never used%BR%\
-- Main.FrancescoGiacomini - 20-Jul-2010%BR%\
%BR%\
Removed%BR%\
-- Main.AlbertoDiMeglio - 08-Aug-2010\
| %Y% |
| 3 | 10 | 3.1 |\
Is "progress factor" a technical term? is so, it should be explained.%BR%\
-- Main.FrancescoGiacomini - 20-Jul-2010%BR%\
%BR%\
No, bad phrasing for 'performance indicators'. Replaced with that.%BR%\
-- Main.AlbertoDiMeglio - 08-Aug-2010\
| %Y% |
| 4 | 11 | 4 |\
The second paragraph is truncated after "and".%BR%\
-- Main.FrancescoGiacomini - 20-Jul-2010%BR%\
%BR%\
Added missing table references%BR%\
-- Main.AlbertoDiMeglio - 08-Aug-2010\
| %Y% |
| 5 | 11 | 4 |\
Second row in table 2, about Project planning and reporting: which validation method and responsibility apply to which task?%BR%\
-- Main.FrancescoGiacomini - 20-Jul-2010%BR%\
%BR%\
Made correspondence between task, method and responsibility explicit%BR%\
-- Main.AlbertoDiMeglio - 08-Aug-2010\
| %Y% |
| 6 | 12 | 4.1.1 |\
CVS is still used at CERN, not only SVN.%BR%\
-- Main.FrancescoGiacomini - 20-Jul-2010%BR%\
%BR%\
Added%BR%\
-- Main.AlbertoDiMeglio - 08-Aug-2010\
| %Y% |
| 7 | 14 | 4.1.4 |\
It would be useful to have template for risk management.%BR%\
-- Main.FrancescoGiacomini - 20-Jul-2010%BR%\
%BR%\
Ok, will be added to TWiki%BR%\
-- Main.AlbertoDiMeglio - 08-Aug-2010\
| %Y% |
| 8 | 14 | 4.2.1 |\
"Submission - 30" is not immediately clear. Better something like: "30 days before submission date"%BR%\
-- Main.FrancescoGiacomini - 20-Jul-2010%BR%\
%BR%\
Modified%BR%\
-- Main.AlbertoDiMeglio - 08-Aug-2010\
| %Y% |
| 9 | 14 | 4.2.1 |\
The TOC of a deliverable has to be validated by the task leader and WP leader in advance. It can simply be the WP leader to send it to the PEB, without the need to involve the PO%BR%\
-- Main.FrancescoGiacomini - 20-Jul-2010%BR%\
%BR%\
Slighly modified to take this into account. However, the new phrasing leaves open the issue of how long the WP leader validation takes before they send it to the PEB. It is assumed that it is a very brief delay%BR%\
-- Main.AlbertoDiMeglio - 08-Aug-2010\
| %Y% |
| 10 | 15 | 4.2.1 |\
Item 4), about the twiki-based review form: In practice there is no difference wrt the procedure followed in EGEE with dialog forms, apart that editing in twiki is a greater pain. Twiki is a better option only if it allows more interactivity between reviewers and the author, possibly all on the same page. Otherwise the dialog form is a better option, since the reviewer can work offline and just send the form at the end of the his/her review. Moreover the dates in the dialog form, e.g. the Review Date, should be dd/mm/yyyy or, even better, yyyy-mm-dd%BR%\
-- Main.FrancescoGiacomini - 20-Jul-2010%BR%\
%BR%\
The choice of TWiki was to allow greater flexibility and interactivity and track the comments close to the documents. However, I'm not very happy myself with this, personally I would only use Word change tracking, using the log tables at the beginning of the document for the comments. I'm open to suggestions, but I would leave them for the next revision of this QA Plan%BR%\
-- Main.AlbertoDiMeglio - 08-Aug-2010%BR%\
%BR%\
Comments in the document don't allow much iterativity either (just imagine this discussion managed with comments). \
TWiki can be fine, but it cannot just emulate the dialog form, especially if one has to respect the tabular \
format as I'm doing right now; it has to be used for example as suggested in the "hints on good style" (bullet on how to deal with a discussion). \
And you cannot use the log tables in the document for all the comments! Just imagine writing all this into that! \
I'm reappraising the EGEE dialog forms... We can certainly continue the discussion not in the scope of this document, but what should the reviewers of the next deliverables do?%BR% \
-- Main.FrancescoGiacomini - 13-Aug-2010%BR%\
%BR%\
Although TWiki is not the most comfortable tool for editing, it allows some degree of interactivity as this dialogue form shows. The detailed comments table can  be used to discuss each topic as needed. It is recommended to use the TWiki style guidelines and use TWiki signatures for each comment%BR%\
-- Main.AlbertoDiMeglio - 17-Aug-2010\
| %Y% |
| 11 | 15 | 4.2.1 |\
A note on how internal reviewers are chosen would be useful%BR%\
-- Main.FrancescoGiacomini - 20-Jul-2010%BR%\
%BR%\
The following has been added: "The reviewers have to be chosen taking into account the skills and technical knowledge required to understand and critically assess the document within the context of EMI. Possibly they should be chosen from a different WP than the one issuing the deliverable, with the exception of JRA1 where the extent and diversity of skills may allow for independent reviews within the same WP. Whenever appropriate or necessary, external reviewers should also be considered, especially when the deliverable concerns topics of relevance in the relationships of EMI with other projects or communities."%BR%\
-- Main.AlbertoDiMeglio - 08-Aug-2010%BR%\
%BR%\
That sentence is certainly worth specifying, but I was rather referring to the procedure to choose reviewers, e.g. something like: the author suggests the WPs that should provide a review and the leaders of those WPs then identifies them.%BR% \
-- Main.FrancescoGiacomini - 13-Aug-2010%BR%\
%BR%\
The following note has been added:%BR%\
"The Work Package leader is responsible to coordinate the process of selecting appropriate reviewers for the deliverables. The WP leader must contact other WP leaders within EMI or knowledgeable persons outside EMI and ask for the names of possible reviewers. The WP leader is requested to make sure that reviewers are selected for each deliverable at least three months before the deliverable due date"%BR%\
-- Main.AlbertoDiMeglio - 17-Aug-2010\
| %Y% |
| 12 | 15 | 4.2.2 |\
A link to the template for the milestone report would be useful%BR%\
-- Main.FrancescoGiacomini - 20-Jul-2010%BR%\
%BR%\
Added at point 1 of the procedure%BR%\
-- Main.AlbertoDiMeglio - 08-Aug-2010\
| %Y% |
| 13 | 15 | 4.2.2 |\
What if the milestone is not achieved? probably the PEB should discuss and define appropriate actions, including change of plans%BR%\
-- Main.FrancescoGiacomini - 20-Jul-2010%BR%\
%BR%\
The following has been added at the end of the section: "If a milestone is not achieved within the expected deadline, the PD must foresee appropriate time for discussion during the regular PEB meeting to identify the issue and define appropriate corrective actions. Changes of plan can also be considered in case the milestone is not relevant anymore due to changed technical conditions. In case a relevant milestone is not achieved even after reasonable corrective actions have been defined and proposed, the PD must escalate the issue to the CB and discuss possible interventions on the Partner(s) responsible for the milestone."%BR%\
-- Main.AlbertoDiMeglio - 08-Aug-2010\
| %Y% |
| 14 | 16 | 4.2.3 |\
How should a WP contribution to the QR/PR be provided? Should the document be uploaded to twiki? to CDS? sent as an attachment? other?%BR%\
-- Main.FrancescoGiacomini - 20-Jul-2010%BR%\
%BR%\
Right. I forgot this. Added in the procedure that it should be uploaded to TWiki%BR%\
-- Main.AlbertoDiMeglio - 08-Aug-2010\
| %Y% |
| 15 | 17 | 4.2.3 |\
First item: "WP leaders provides contributions 20 days *before* the end of the period." I suppose it's *after* the end of the period%BR%\
-- Main.FrancescoGiacomini - 20-Jul-2010%BR%\
%BR%\
No, this is the procedure for the Periodic Management Reports. the contributions should describe the work done in the full previous year and have to be put together in the full official report for the EC. The entire process has to start before the end of the period, since the report has to be sent to the EC 10 working days before the Review and at the latest 45 days after the and of the period.%BR%\
-- Main.AlbertoDiMeglio - 08-Aug-2010\
| %Y% |
| 16 | 17 | 4.2.3 |\
End of the section, why specify here that PD and WP leaders provide presentations? Should they already be attached to the PR?%BR%\
-- Main.FrancescoGiacomini - 20-Jul-2010%BR%\
%BR%\
No. Why? The presentations for the review and the PR are different things. But I agree that this sentence makes little sense here. I just removed it, since the review process is described elsewhere%BR%\
-- Main.AlbertoDiMeglio - 08-Aug-2010\
| %Y% |
| 17 | 17 | 4.2.4 |\
If/when CDS supports reviews, will our review procedures change? if that's a possibility, maybe mention it%BR%\
-- Main.FrancescoGiacomini - 20-Jul-2010%BR%\
%BR%\
I thought of that, however, it is still not clear whether this is indeed the case. it seems to offer this feature, but not in a way that we may actually fond useful. In any case, I've added a sentence about this%BR%\
-- Main.AlbertoDiMeglio - 08-Aug-2010\
| %Y% |
| 18 | 18 | 4.2.5 |\
Who decides the authors of a publication, so that they can be all acknowledged?%BR%\
-- Main.FrancescoGiacomini - 20-Jul-2010%BR%\
%BR%\
Good question. I'm not sure I have an answer. Should the PEB or PTB be involved or should this be left to the Partners? It may be difficult to enforce specific authors for papers concerning the detailed work of individual PTs. The PEB and/or PTB could decide in case of general overview papers about EMI%BR%\
-- Main.AlbertoDiMeglio - 08-Aug-2010%BR%\
%BR%\
Whatever, but what is in the document now ("All authors have to be acknowledged") is not satisfactory.%BR% \
-- Main.FrancescoGiacomini - 13-Aug-2010%BR%\
%BR%\
I've tried to clarify this issue in the text. The main point is that the 'authors' to be acknowledge are actually the 'contributors' having produced some of the work described in the paper. in addition, all paper describing relevant EMI work have to be communicated to the PEB for verification and dissemination%BR%\
-- Main.AlbertoDiMeglio - 17-Aug-2010\
| %Y% |
| 19 | 18 | 4.2.5 |\
Include the exact wording on how to acknowledge the EC funding contribution%BR%\
-- Main.FrancescoGiacomini - 20-Jul-2010%BR%\
%BR%\
Added%BR%\
-- Main.AlbertoDiMeglio - 08-Aug-2010\
| %Y% |
| 20 | 19 | 4.3 |\
Provide more details on how documents should be versioned, not just "The version of the documents must be increased at every revision"%BR%\
-- Main.FrancescoGiacomini - 20-Jul-2010%BR%\
%BR%\
Added detailed description at the end of the section%BR%\
-- Main.AlbertoDiMeglio - 08-Aug-2010%BR%\
%BR%\
Good attempt, but there is something wrong at least with the first item:%BR%\
The version number is composed of *two* (2) numbers and has the format x.y. The *three* numbers are called major version (x), minor version (y) and revision number (z) respectively.%BR%\
(emphasis mine)%BR% \
-- Main.FrancescoGiacomini - 13-Aug-2010%BR%\
%BR%\
Fixed the inconsistency. The version number should be three, although z can be omitted if z=0%BR%\
-- Main.AlbertoDiMeglio - 17-Aug-2010\
| %Y% |
| 21 | 21 | 5.1 |\
KSA1.1 should be reviewed/removed. See discussions on User Support in the PEB%BR%\
-- Main.FrancescoGiacomini - 20-Jul-2010%BR%\
%BR%\
Can it be redefined? If we remove it, we do not have any metric about the number of incidents. We should tack how many incidents we receive. We should probably replace Service Desk with a more appropriate term, but leave the KPI%BR%\
-- Main.AlbertoDiMeglio - 08-Aug-2010\
| %Y% |
| 22 | 28 | 6.3 |\
Aren't three rehearsals a bit too much?%BR%\
-- Main.FrancescoGiacomini - 20-Jul-2010%BR%\
%BR%\
In my experience, they are not enough. Take into account that the first is to get initial drafts and discuss about the principles, the second is to work on the details and the third is to properly rehearse timing and smoothness of delivery%BR%\
-- Main.AlbertoDiMeglio - 08-Aug-2010\
| %Y% |
| 23 | 30 | 7 |\
Why are deliverable review dialog forms protected? how to do this in twiki?%BR%\
-- Main.FrancescoGiacomini - 20-Jul-2010%BR%\
%BR%\
Indeed, no need to restrict them. In any case, TWiki has a protection mechanism based on login names, which is used for the Quarterly Reports for example%BR%\
-- Main.AlbertoDiMeglio - 08-Aug-2010\
| %Y% |
| 24 | 30 | 7 |\
There are probably missing links corresponding to "EU Quarterly and Periodical Report" and "EU review report"%BR%\
-- Main.FrancescoGiacomini - 20-Jul-2010%BR%\
%BR%\
The assumption was that CDS was going to be in place by now. For the moment I've added links to the TWiki pages%BR%\
-- Main.AlbertoDiMeglio - 08-Aug-2010\
| %Y% |
| 25 | 30 | 7 |\
It would be nice if https://twiki.cern.ch/twiki/bin/view/EMI/EmiKPIs or equivalent page would be generated automatically%BR%\
-- Main.FrancescoGiacomini - 20-Jul-2010%BR%\
%BR%\
Yes, but how? I will provide a template, but I don't see any way of creating the content automatically. The WP leaders have to provide numbers and NA1 fill the TWiki pages manually%BR%\
-- Main.AlbertoDiMeglio - 08-Aug-2010\
| %Y% |
| 26 | 31 | 8.1 |\
No ODS/OpenOffice for spreadsheets?%BR%\
-- Main.FrancescoGiacomini - 20-Jul-2010%BR%\
%BR%\
Added%BR%\
-- Main.AlbertoDiMeglio - 08-Aug-2010\
| %Y% |
| 27 | 32 | 8.6 |\
About EVO: mention that we have already a room (or whatever is called) and how to access and use it (or a link thereto)%BR%\
-- Main.FrancescoGiacomini - 20-Jul-2010%BR%\
%BR%\
Added. There is no direct link to the communities, as far as I know, but I've explained where to find it%BR%\
-- Main.AlbertoDiMeglio - 08-Aug-2010\
| %Y% |
Attachments
Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
doc	EMI-DNA1.1-CDSREF-Project_QA_Plan-v0_5-fg1.doc	r1	manage	330.5 K	2010-07-20 - 15:15	FrancescoGiacomini
Topic revision: r9 - 2010-08-24 - AlbertoDiMeglio
Public webs
- Cern Search
- TWiki Search
- Google Search
EMI All webs
Copyright &© 2008-2023 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback