Deliverable Review Form

Identification of the deliverable or milestone
Project: EMI Deliverable or milestone identifier: DSA2.4
Title: Continuos Integration Testbeds Doc. identifier: EMI-DSA24-CDS???-QAReport-v0.5
Author(s): Danilo Dongiovanni Due date: 03.09.2010

Identification of the reviewer
Name: Balazs Konya Affiliation: Lund University EMI Activity/External project or Institute: EMI Technical Director

Review date 15 October 2010
Author(s) revision date mm/dd/yyyy
Reviewer acceptance date 22 October 2010

Attach the reviewed document to the deliverable page, put here a link

General comments

Additional recommendations (not affecting the document content, e.g. recommendation for future work)

Detailed comments on the content

Note for Reviewers: Please put you comments and changes in the Word document

Note 1: The reviewers must list here any observation they want to track explicitly and that require interaction with the authors
Alternatively all changes must be listed in the document itself using Word change tracking features (if you use Word)
Note 2: These comments have to be explicitly addressed by the authors and the action taken must be clearly described

Please find below comments concerning the DSA2.4 Continous Integration and Certification TEstbeds" report. Please please these notes to the appropriate place on the deliverable wiki page.

This review is based on the 31/07/2010 version of the document

1) The long document provides a thorough presentation of requirements and wishes towards EMI testbed. The document mostly presents plans, models, ideas for the future. it is full of future tense statements (testbed will be ..., future plans, testbed model, wishes).

2) Unfortunately, the document tells very little about what is actually available and deployed as an EMI testbed for continuous integration as of PM5. There are links to external wiki pages that the EC reviewers will not check out for sure.

Therefore, hard to separate the reality, the actual status of the testbed from the numerous plans. E.g., Section 4 "Testbed implementation model" is a strange mixture of plans and some present status.

Would it be possible to add a section "Testbed status" and collect there the core info: hardware, op system, deployed software, versions, applied access control, authorized users, etc..?

3) The document is not really clear about the maintenance, operational aspect of the testbed as well. The roles are not clearly separated: hardware provisioning, opsys installation, EMI service deployment and configuration, running the box with the service, granting access, etc.. I'd like to see a dedicated section addressing the Testbed operation with clear description of roles and tasks: maybe a table with the above tasks specifying what is done by SA2.6 personnel, what is the job of the PTs and what is the job of EMI support unit?

4) I find very confusing the continous mixture of testbed vs. testbeds. Are we really talking about several testbeds? Isn't it just one EMI integration testbed which contains services with different versions? Services with newer versions are "continously" deployed as there is a need for them. These new versions are not (necessarily) replacing the older versions, several versions live together in the same testbed.

I find the Testbed classifications of Section 3.3.1 artificial. For me there should be only one EMI integration testbed that should offer several versions of EMI services on a numerous platforms. Example: i'd like to have several versions of the CREAM CE deployed and be able to test my client against all of them.

I don't like the creation of separate testbeds for a) minor release candidates b) major release candidates c) prod versions. What if i want to test a prod version of client against the minor release candidate? For me all these services with different versions should be part of the same testbed.

btw, on page 15 it is declared that "Release candidate versions will be made available and maintained under specific requests" which suggests as if the testbed "be default" will only run the production version of services. I think the EMI integration testbed should really come with several version of the same services.

5) all over the document the separation of the three middleware stacks are emphasised. example: page 17 "Initially, every middleware will have its testbed".

This i don't like at all. EMI should aim for a single testbed where all the ARC/Glite/Unicore/dcache services are deployed and can at least be tested against each other.

Therefore, please remove such statements from the document. EMI testbed should not be split by middlewares, let's not talk about different EMI testbeds for the different stacks.

all in all: the deliverable focuses too much on "planning, future" than reporting the current status of the testbed. The current content of the document would better fit under the "EMI Continous Integration and Certification TEstbeds *plans*" title smile

i'd like to read about: - available hardware as of Today - deployed EMI services (name, version) as of Today - testbed operation and maintenance in practice: who and how installs, configures the services as of Today - testbed usage: how the testbed is used (who can use it, how is it planned to be used, how those the testbed fits into the general testing procedures? - the description of the procedure and average time to deploy a certain version of EMI service. A scenario: the ARC CE PT wants to test its data staging module against particular Dcache version.

a minor thing: on page 15, please remove the " at present moment, EMI month 3" sentence.

Concerning the acceptance of the deliverable: If this deliverable was meant to describe testbed plans and not actual status, then i see no problem accepting the deliverable after some of my comments addressed (e.g. the testbeds vs. a single testbed and maintenance part).

Any other modification, spelling or grammatical corrections, etc must be done directly in the document using tracked changes or similar mechanisms that allows the authors to identify which correction is suggested.

Edit | Attach | Watch | Print version | History: r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r3 - 2010-11-18 - BalazsKonya
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    EMI All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback