JRA1 Deliverable Review Form

Identification of the deliverable or milestone
Project: EMI Deliverable or milestone identifier: D3.4.1
Title: DSA1.4.1 - Annual Maintenance and Support Report Doc. identifier: EMI-DXXX-CDSREF-Title-vx.x
Author(s): F. Giacomini Due date: __

Identification of the reviewer
Name: Jedrzej Rybicki Affiliation: JUELICH EMI Activity/External project or Institute: JRA1

Review date mm/dd/yyyy
Author(s) revision date mm/dd/yyyy
Reviewer acceptance date mm/dd/yyyy

Attach the reviewed document to the deliverable page, put here a link

* ToC Comments (28/04/11) * ToC Comments (28/04/11)

280411 ToC Comments:

The document has a good, clear structure. Only few small
comments/suggestions on my part.


There is some discrepancy between the abstract and the document
structure. The abstract states:
"This report contains a consolidated view of the results of all SA1
tasks with particular focus on the compliance with the established
processes and procedures and the implementation of the Service Level
Agreements with major customers."

however:
a) SLAs are not "visible" in the TOC? It might be a good idea to reflect
the main objective in the TOC as well.
b) SA1 is composed of 4 tasks (software maintenance, release management,
user support, and quality control), thus it might be a good idea to
devote a single section to each of the tasks. For now, the task: "TSA1.4
 Quality Control" does not have its own section but rather exists as
subsections in other tasks. Hence the provided view of the SA1 task
results is not "consolidated", and leaves an impression that TSA1.4 is
not included.


User Support:
The section structure could be improved. E.g. it is hard to grasp what
"User support plan" might be, is it something like a user support
strategy? Generally the word plan is a little bit misleading as this
deliverable is a report not a plan.


One of the SA1 Objective is:
"[to] Establish the EMI User Support (Service Desk) function and
integrate it with the overall EGI, PRACE and VRCs user support channels "
Thus it would be conceivable to describe user support efforts in each of
the support to mention each of the major customers also in TOC (again:
customers are explicitly mentioned in the abstract)


Minors:
-Section numbers make reviews much easier (and are actually part of the
EMI template)
-Inconsistent capital letters usage in TOC: "User *s*upport Plan",
"Document *a*mendment *p*rocedure"
-"Table of contents" appears two times on the page (one should be removed)
-Identifier (in header) should be DSA1.4... not DSA1.1 I guess?
-Page layout: Identifier is too long? But it is a general EMI problem
which should be addressed,
-low resolution EMI logo is used (current template uses higher
resolution/better quality, could you update as well?)
-title in header is not set

General comments

EMI-0 was treated as an exercise release for the developers and to test the whole EMI releasing cycle. The DSA141 report leaves an impression that the opportunity of using this release to exercise the Maintenance and Support was missed. On many places in the text an excuse of not available "proper EMI release" is used.

EMI-0 could not be used to test the Maintenance and Support procedures, because it was not made available to the users. In practice it was used only as an integration (at build time) exercise.
By "proper EMI release" I mean EMI 1 and its future updates.
-- FrancescoGiacomini - 24-May-2011

In this report, there are many delays listed. I am not naive I know from personal experience that keeping on the schedule sometimes does not work. However, each of this delays should be justified, the lessons learned from the delay should be pointed out, and countermeasures to avoid the delays in the future should be included. IMHO, it might be one of the most important outcomes of EMI: the collective experience of managing such a big software project.

Re-checking the whole document, it seems to me that the only delay that is not justified is for the release of the Security Assessment Plan and I have re-phrased that part anyway (see below). All the rest is justified and the conclusion is that the delays didn't compromise neither the achievement of the main target (i.e. the release of EMI 1) nor the maintenance and support activities for the existing software in production.
-- FrancescoGiacomini - 24-May-2011

Page 14, point 4: how EMI0 could be delayed by problems of adopting Debian packaging policies and trying to keep conform with Fedora packaging guidelines? Neither of this was actually implemented (also in EMI1 not). According to my knowledge, there are numerous of EMI packages which are not conform with Fedora policies (as the rpmlint report in ETICS show). The dependencies management, especially for Java projects, is not applied. Debian policies, AFAIK, are planed for EMI 2.

I've listed more than just the packaging policies as a cause of delay, including the stricter management of external dependencies and communication problems with PTs. Moreover conformance with the packaging policies is not "all or nothing"; many things have been done, e.g. the adoption of the FHS for the installation paths. Taking gLite as an example (which I know better), this required a lot of changes. About Java, just choosing between the OpenJDK or the SunJDK took lengthy discussion on the EMT. Finally, policies are not only about packaging, I've left just "policies".
-- FrancescoGiacomini - 24-May-2011

The sections concluding each chapter ("Issues and suggested improvements") seldom fulfils its job with regard to suggested improvements (e.g. Section 4.4). This issue is correlated to the "delay" problem. On one hand you have a lot of delays on the other very few improvements are proposed... so can we expect more delays for EMI year 2 and 3? You should also provide plans on implementing the improvements (for the sections where improvements are listed) otherwise it is only a whishfull thinking.

In the "Issues and Suggested Improvements" sections and in the Conclusions I clearly state that no changes in the plans are required for the future. The fact that EMI 1 has come out (almost) on time proves that all delays have been already absorbed during the first year. It's true that 4.4 looks like wishful thinking and I'll replace that with two statements: one is about the release management plan (no changes foreseen), the other is a recommendation to the PEB to follow more closely the release phases, since the cycle involves at least JRA1. (to be checked with Alberto Di Meglio)
-- FrancescoGiacomini - 24-May-2011

Page 18: the whole interaction between generic SU and specific SUs with regard to issues which are not clearly assignable, yields a possibility (or a danger) of fuzzy responsibility. As the generic SU might not be able to assess the technical challenges of the issue, and has to rely on the information provided by the specific SU. Which again might provide incomplete information in order to avoid the additional work. Furthermore for issues affecting multiple specific SUs some way of coordinating the work should be established.

Ok. The EMT is the first place where these issues should be dealt with. I'll state it explicitly.
-- FrancescoGiacomini - 24-May-2011

Security Assessments of selected EMI components. This issue is not explained. I don't know what is meant here. This should be extended. Furthermore "selected components", who will select them? when, what are the criteria. I think it is an important issue (as the security usually is).

Right. Some of this is explained in the Security Assessment Plan itself and I forgot to put a reference. I have also re-phrased the paragraph in order to tell the positive aspects of the plan, without mentioning the delay. The plan is not an official milestone or deliverable and the effort for this activity is very low.
-- FrancescoGiacomini - 24-May-2011

Section 7: Plans for further SLA activities (if they are not confidential) should be included. IMHO SLAs remain one of the major objectives of EMI.

True, but this is something followed by NA1. SA1 (and JRA1) just apply SLAs signed by the project office.
-- FrancescoGiacomini - 24-May-2011

All references to WLCG and LHC should be removed from the document, I cannot see the reason for treating one "customer" of one middelware solution in a special manner.

We can't ignore the major (by far) customer of EGI. I understand your point and I tried hard to reduce the occurrences of WLCG, but less than two is really difficult. How can I justify the choice of SL5/64bit platform otherwise? Also the prudent policy on maintenance is mainly due to that community.
I'm pretty sure that I'll receive complaints going in the opposite direction.
-- FrancescoGiacomini - 24-May-2011

Maybe there are other users which can be mentioned here? I have no personal problem with WLCG but I have my concerns if by mentioning only that you don't close door for further cooperation with other groups. The take home message should be that EMI should be open for every partner.
-- JedrzejRybicki - 25-May-2011

I agree. I've added a sentence in 5.4 saying that we will investigate a support relationship with others (PRACE in particular), which is indeed being discussed.
-- FrancescoGiacomini - 27-May-2011

Additional recommendations (not affecting the document content, e.g. recommendation for future work)

Page 11, point 4: "By extension, the same approach was adopted, where it made sense, for the major distributions already existing when EMI started. Consequently the ARC 0.8 and the gLite 3.2 distributions will be fully supported until October 2011" UC is also a major distribution and existed when EMI started, why it is not included here?

ARC and gLite were just examples, the easiest to cite without being too verbose. Anyway, I'll remove the references to ARC 0.8 and gLite 3.2 and make the sentence more generic.
-- FrancescoGiacomini - 24-May-2011

Page 13: "There is some overlapping, but also many differences. For this reason the corresponding processes and procedures are maintained as they are and managed according to the existing organization. This approach offers the best guarantee against the risk of causing problems in the support of the software already in production during a significant discontinuity in the organization of how the Grid infrastructure is managed;" I have my problems with getting the head around this argumentation. So on this place you use the old, existing solutions not to cause problems. Later (for other aspects like build) common solutions and "harmonization" of the processes across the original distribution is sought after. Although it could be also rejected with the same argument of "best guarantee". Secondly, one could argue that by preserving the "independent" solutions you increase the effort of maintaining and releasing common software components. Since one of the technical objectives of EMI is to harmonize, one can expected that the issue will become increasingly important. I am not sure if it is a right place to rise the issue (as the decision is already made) but the attentive readers might also note the inconsequence, so maybe you should provide more background info here.

Changing the procedures used to managed the software already in production is not feasible. Taking gLite as an example (only because I know it better), it is installed in hundreds of sites. What do you think it would happen, for example, if an update of a package all of a sudden installed files in /usr instead of /opt/glite? That's the sort of change that you do only in a major release of the distribution and even in that case there are complaints.
Clearly the harmonization effort is an investment and all investments cost at the beginning. In our case the cost is the temporary application of different procedures, but the cost will go away completely one year from now, when support and maintenance for (versions of) software components coming from before EMI will stop.
-- FrancescoGiacomini - 24-May-2011

Page 15: "The results obtained at PM6 were shown and discussed at the first EMI All-Hands meeting in November 2010.", if it was part of your activity at least a summary of the discussion with important outcomes could be included.

The problem was project-wide, but I have added a sentence, underlying the communication problems, which I really think were at the heart of most of our problems and delays so far.
-- FrancescoGiacomini - 24-May-2011

The header of Section 5.3 is copied from 4.3, maybe you can rephrase it a little (only for the quality of the presentation not the content).

And the header of 4.3 is copied from 3.3 smile I know, it's not nice, but I prefer uniformity.
-- FrancescoGiacomini - 24-May-2011

The lists in the document are quite strange. Single items start with lower-case and ends with semicolon. Never seen such a solution. Either full sentence starting with a capital letter and ending with a period or lower-case and colon at the end.

The style probably comes from EGEE guidelines. I've changed all items starting with a capital letter and ending with a period.
-- FrancescoGiacomini - 24-May-2011

Detailed comments on the content

Note 1: The reviewers must list here any observation they want to track explicitly and that require interaction with the authors
Alternatively all changes must be listed in the document itself using Word change tracking features (if you use Word)
Note 2: These comments have to be explicitly addressed by the authors and the action taken must be clearly described

N Page Section Observations and Replies Is Addressed?
1     Despite the fact that the document date is 18/05 I found places stating that no EMI release is available (which is not true). For instance page 8 stays that you are preparing the releases and few days of delay are envisioned.

The date of the document is April 30th and for me that's the time of writing. I would like to have a clear statement from the project office that that's not the correct approach before changing. And in that case I need to rethink a few things of course, which I would prefer to avoid.
-- FrancescoGiacomini - 24-May-2011

This one should be clarified with PO than.
-- JedrzejRybicki - 25-May-2011
 
2     All the references to WLCG and LHC should be removed.

See above.
-- FrancescoGiacomini - 24-May-2011
 
3 Page 5   R15, missing comma

Fixed
-- FrancescoGiacomini - 24-May-2011
 
4 Page 8 top "in terms of: maintain the software", maintaining?

Fixed
-- FrancescoGiacomini - 24-May-2011
 
5 Page 8   "infrastructure providers (e.g. EGI) and the software providers (e.g EMI)" I guess i.e. would be better at least in the second place.

EMI is just one of the software providers and I wanted to show that there is a collaboration going on. I have reshuffled the text to better express the concept.
-- FrancescoGiacomini - 24-May-2011
 
6 Page 11   two last points (7,8) are identical.

They are syntactically identical but they refer to two different plan items.
-- FrancescoGiacomini - 24-May-2011
 
7 Page 14 point 8 you could provide the list of the platforms.

Isn't it there? SL5/64 now, with SL6 and some Debian (still to be formally decided) coming soon.
-- FrancescoGiacomini - 24-May-2011
 
8 Page 14 point 10 "responsible for governing", responsible to means something else.

Fixed. Thanks.
-- FrancescoGiacomini - 24-May-2011
 
9 Page 14 point 3 EMI 1 is there!  
10 Page 14 point 4 (EMI0 delayed by Debian)

See above.
-- FrancescoGiacomini - 24-May-2011
 
11 Page 15 Section 4.4 should include suggested improvements

See above.
-- FrancescoGiacomini - 24-May-2011
 
12 Page 18   Interaction between generic SU and specific SUs should be explained more detailed.

See above.
-- FrancescoGiacomini - 24-May-2011
 
13 Page 20   Security Assessments of the selected comments should be explained in more detailed fashion.

See above.
-- FrancescoGiacomini - 24-May-2011
 
14 Page 22 Section 7 should include planned activities on SLA.

See above.
-- FrancescoGiacomini - 24-May-2011
 
15     Lessons learned by SA from EMI0 should be provided

See above.
-- FrancescoGiacomini - 24-May-2011
 
16     Delays should be justified, the lessons learned from the delay should be pointed out, and countermeasures to avoid the delays in the future should be included

See above.
-- FrancescoGiacomini - 24-May-2011
 

Any other modification, spelling or grammatical corrections, etc must be done directly in the document using tracked changes or similar mechanisms that allows the authors to identify which correction is suggested.

-- AlbertoDiMeglio - 12-Jul-2010

-- FloridaEstrella - 28-Apr-2011

-- JedrzejRybicki - 19-May-2011

Edit | Attach | Watch | Print version | History: r8 < r7 < r6 < r5 < r4 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r8 - 2011-05-27 - FrancescoGiacomini
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    EMI All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback