Andrea's HEP SSC Comments

Highlights from the Guide for Applicants

Objectives for call INFRA-2010-1.2.3 (Virtual Research Communities)

The main objective is to enable an ever increasing number of users from all science and engineering disciplines to access and effectively use e-Infrastructures and/or to allow them to access and share facilities, software and data. This should lead to more effective scientific collaboration and innovation in Europe. Proposals should address one or more of these objectives:

• Deployment of e-Infrastructures in research communities to enable multidisciplinary collaboration and address their needs
• Deployment of end-to-end e-Infrastructure services and tools in support of virtual organisations to integrate and increase their research capabilities
• Building user-configured virtual research facilities/test-beds by coalition of existing resources from diverse facilities
• Addressing human and socioeconomic factors influencing the creation of sustainable virtual research communities and the maintenance of e-Infrastructure services by communities
• Integrating regional e-Infrastructures and linking them to provide access to resources on an European or global scale

Proposals should have users from academia and industry from one or more scientific communities, computational scientists and e-Infrastructure providers.

Training activities are welcome.

Expected impact

• Increased effectiveness of European research through the broader use of e-Infrastructures
• emergence of international virtual research communities that cannot be achieved by national initiatives alone
• easier development and adoption of standards, common tools, procedures and best practices
• use of e-Infrastructure services and tools by actors from new disciplines
• increased quality and attractiveness of e-Infrastructures

Activities

The proposal must include:

1. Networking activities
2. Service activities
3. Joint research activities

Networking activities

Must foster a culture of cooperation between the participants and the scientific communities.

• joint management of access and distributed resources
• strengthening of virtual research communities
• definition of common standards (interoperability)
• development and maintenance of databases to manage users and infrastructures
• spreading of good practices, consultancy and user training
• studies for new methods and technologies
• promotion of clustering amongst related projects
• coordination with (inter)national related activities
• dissemination
• promotion of long term sustainability

Service activities

Must provide specific research infrastructure related services.

• communication infrastructure, network operation, end-to-end services
• Grid infrastructure support, operation and management, integration, test, certification, VO-specific services
• deployment and quality assurance of middleware
• data and resource management ( secure access, job scheduling, user and application support services)
• vertical integration of different services to support virtual research communities

Joint research activities

They should be innovative and explore new technologies or techniques to better use the infrastructure.

• higher performance methods and protocols, testing of components
• integration of infrastructures into virtual facilities
• innovative solutions for data management
• innovative solutions for communications and introduction of new end-to-end services (e.g. for dynamic allocation of resources or accounting management)
• novel Grid architecture frameworks and technologies or new middleware solutions
• foster adoption by industry
• innovative software solutions to make new communities benefit from computing services.

Miscellaneous requirements

• Do not use colour and do not insert hypertext links (no other documents than the proposal will be read)
• Respect page limits
• The proposal must be in PDF format and be less than 10 MB in size
• Part B must follow exactly the structure described in Annex 4 of the Guide for Applicants. Omitting requested information can be fatal

Eligibility criteria

• The proposal is received before the deadline
• It involves the minimum number of participants (at least 3 from member or associated countries, no 2 of which in the same country)
• It is complete
• It is relevant to the call topic and funding scheme

Evaluation criteria

Scientific/technological quality	Implementation	Impact
Soundness of concept and objectives	Appropriate management structure and procedures	Contribution towards a balanced territorial development and optimal use of the research infrastructures in Europe
Progress beyond end-of-the-art	Quality of the participants	Appropriate measures for dissemination and explotation of results
Quality of the methodology to achieve the objectives, in particular the provision of integrated services	Quality of the consortium	Contribution to socio-economic impacts
Quality of the Networking Activities and their work plan	Appropriate justification and allocation of resources
Quality of the Trans-national Access and/or Services and their work plan (to offer access to the infrastructures and enable users to conduct high-quality research
Quality of the Joint Research Activities and their work plan (to improve the services provided by the infrastructures)

The relevance of a proposal to the call work programme goes under "S/T quality" and the relevance to the call objectives under "Impact".

Each column gets a score between 0 and 5, where threshold is 3 for each column and 10 for the sum. If more proposals have the same total score, the one with the highest "Impact" score wins.

Comments to the draft (25-08-2009)

1. The cover page is not compliant with the Guide. I guess that this is because the "real" draft will be the EGI draft edited by Cal.
2. The numbering, ordering and title of the sections does not match the template. See above.
3. A general description of the project is missing. See above.
4. The S&T objectives should be described and for each one it should be explained which call objective is addressed. Now the logic is reversed, which is what Cal requested but wrong for the final proposal. Do not forget to state objectives in a measurable and verifiable form.
5. The response to the objective "Deployment of e-Infrastructures" is good but could be improved by being a bit more specific when talking about the nature of the shared tools and the specific needs of the mentioned non-HEP disciplines; just a few words, but it cannot be assumed at this point that the reader understands what Ganga and GEANT4 are.
6. The response to the objective "Deployment of end-to-end e-Infrastructure services" mentions only "one of the main goals" which is support for the LHC community, but eventually all goals (meaning all communities) must be mentioned in the project objectives.
7. I am not convinced that the response to the objective "Building user-configured virtual research facilities" can only be N/A. A possibility could be to relate it to work done to develop tools and components used by experiments on top of the standard middleware to improve or expand on functionalities like site monitoring of information system (e.g. the ATLAS "info system", the CMS site readiness) to allow experiments to define their own Grid on top of WLCG.
8. The response to the objective "Addressing human etc." is valid but a bit too generic; it can apply to almost any discipline. I would put much more stress on the aspect of sustainability and helping the VOs to effectively run their own services after an initial period where this responsibility might be largely on the SSC. We should not end up doing the VO's work, unless absolutely necessary and in any case for a limited period of time.
9. The response to the objective "Integrating regional e-Infrastructures" seems more relevant to the 1.2.1.2 call than to this one, and describes the current situation rather than an objective to achieve. The specificity of the SSC objective is not visible.
10. In section 1.2 a few more details on the nature of the collaborations and the initiatives would be useful.
11. There is nothing about the expected progress beyond the state-of-the-art. To be written by Cal?
12. There is nothing about the methodology to achieve the objectives. To be written by Cal?

Service activities

1. The overall strategy of the work plan is missing.
2. The work packages should have a title, not just a number.
3. SSC.1 objectives: I would talk of "processing of experimental data, Monte Carlo simulation" rather than "production data processing".
4. The DESY contribution mentions INFN several times. Their role is too vaguely defined (apart bullet 2).
5. The GSI contribution description is too detailed on the physics side and too vague on the computing side.
6. The INFN contribution is not really described, but it might be ok.
7. The Oslo contribution is too vague.
8. The Prague contribution, again, is too vague.
9. The risk about staffing and contract policies seems to refer to the risk for LHC if the SSC is not funded, rather than a risk for the SSC work plan assuming the SSC is funded. Are you sure it is relevant?
10. The mitigation strategies for the risk of service disruptions could be described more concisely. They describe a status quo rather then improvements in the area. It might be worth to stress the improvements of the procedures as operational experience is collected and how it is planned to minimize the frequency and the impact of disruptions (not all of them are due to bad luck, many are due to human errors!).
11. There is some overlap between the 2nd and 3rd risk: it should be more explicit that the 3rd is related to scalability issues rather than incidents.
12. A risk which should be evaluated is the risk of having a system too complex to operate and support continuously with the available manpower. One of the objectives should actually to improve the efficiency of operations and support with respect to the current situation.

Joint research activities

1. No comments. The work plan is clear and detailed.

Network activities

1. The mitigation strategy for the only risk described does not really describe any mitigation stragegy.

Personal evaluation

S/T quality: 2

The objectives of the project are not clearly described in the first part, nor the user communities involved. There is not enough focus on the progress beyond the state-of-the-art and the project seems to mainly be concerned about the continuation of current activities (which is undoubtely important). In some cases, there is not enough detail on the methodologies to achieve the goals.

The networking activities are adequate to the scope of the project: even if they mostly consist in liaison activities, these are essential to the achievement of the project objectives. More attention to the spreading of good practices and the consultancy aspect would be desirable, though, as these are already an important component of the EIS activities. Writing documentation (like the gLite User Guide or similar documents) should also be mentioned.

The service activities have clear objectives, highly relevant to the call objectives. However, the description of work is totally insufficient. This applies also to the partner contributions. At the very least, there should be a description of the several tools to be developed and maintained and a justification of the numbers of FTE given. A lot could be said about middleware testing, operations support, experiment-level workload and data management, etc. It should also be explained the impact on each LHC experiment and on the other HEP communities covered by the SSC.

The joint research activities are adequate but are limited to those pieces of middleware that EGI does not want to support. There could be a need to develop completely new components, maybe small, but useful. Examples from the past are commands developed in EIS for querying the information system or managing VO tags, test suites, prestaging tools, etc. On the other hand, the task 2 (future data management) is very well motivated.

Implementation: ?

I can only comment on the allocation and justification of resources, which is difficult to evaluate given the lack of details.

Impact: 3

It is difficult not to be biased by the fact that I know that the HEP SSC is extremely important for HEP and CERN in particular. The proposal should be much more explicit in explaining what would be the difference for WLCG between the case without a HEP SSC and the case with it: the former would be a very serious problem, so the impact is huge! One should also stress the importance of helping the communities to build their own expertise in grid matters, which was done for the LHC experiments in the past years (one of the main EIS achievements) and should be done with less experienced communities.

Comments to the draft (20-10-2009)

General note on work package descriptions

When the WP activity involves several SSC, I would follow this structure in the description of work:

1. an initial description common to all SSC
2. for each SSC:
   1. an initial description of the SSC objectives
   2. a list of tasks, properly defined
   3. the partner contributions, with the total FTE per partner

Section 1: Scientific and/or technical quality, relevant to the topics addressed by the call

• Concept explained? Yes
• Main ideas explained? Yes
• S&T objectives described in detail? Not in detail, but section is to be expanded
• Related to call objectives? Yes, to all of them, but not explicitly (but section is to be expanded)

Progress beyond the state-of-the-art

Section missing.

Methodology to achieve the objectives of the project, in particular the provision of integrated services

Section missing.

Networking Activities and associated work plan

• Description of the extend to which the proposel co-ordination mechanisms will foster a culture of cooperation between the participants and enhance the services to the users? No.
• Work plan strategy described? No, section missing.
• Gantt chart present? No
• Work package list? Yes, but person/months missing. EGI.eu listed as partner, but missing from initial table.
• Deliverables listed? No.

Description of NA1

• Table? OK.
• Objectives? Yes.
• Description of work broken down into tasks and role of partners? Yes.
• Brief description of deliverables and month of delivery? Yes.

Description of NA2

• Table? Participant names and person-months missing.
• Objectives? Yes.
• Description of work broken down into tasks and role of partners? Not in a consistent way: for some SSC the role of partners is missing and the tasks are not clearly enumerated and described. Sometimes very detailed FTE information is given, sometimes none is.
  • HEP: too concise, FTE information too detailed, tasks not described, WLCG not defined (can it be taken for granted?)
  • LF: tasks vague and not defined, no role of partners, some tasks seem more SA than NA tasks.
  • CCMST: tasks very vague, no role of partners.
  • GO: text is out of place as it is a description of the SSC, not of its NA2 activities!
  • CS: tasks should be clearly enumerated.
  • PS: messy; role of partners missing.
  • H: missing
• Brief description of deliverables and month of delivery? Yes.

Description of NA3

• Table? Participant names and person-months missing.
• Objectives? Yes.
• Description of work broken down into tasks and role of partners? The general part is confusing: maybe a problem in text formatting? I do not understand what "coordination of support providers" has to do with dissemination or training.
  • HEP: SA.HEP.2 should be NA.HEP.2. The HEP tasks are not properly listed and described. The role of partners is incomplete: who is going to do the training for FAIR? What is iSGTW?
  • LS: the text is confusing. Why to list separately "dissemination goals" and "dissemination"? Besides, there should be a list of tasks, not of goals. There is no description of the tasks. The role of partners is missing.
  • CCMST: text very confusing, tasks are not clearly defined. Role of partners is missing.
  • GO: what is the Autonomic Computing Community? The text is too detailed and fails to communicate in a clear way what are the tasks to be performed. The role of partners is missing.
  • CS: almost ok, but the tasks should be clearly enumerated.
  • PS: almost ok, but the tasks should be clearly enumerated. The role of partners is missing.
  • H: missing.
• Brief description of deliverables and month of delivery? No.

NA: general

• Summary of effort table: No.
• List of milestones: Incomplete, could be compacted
• Pert diagram? No.
• Risk description? No.

Service activities and associated work plan

• Description the extent to which the activities will offer access to state-of-the-art infrastructures, high

quality services, and will enable users to conduct high quality research? No.

• Work plan strategy described? No, section missing.
• Gantt chart present? No
• Work package list? No.
• Deliverables listed? No.

Description of SA1

• Table? Participants information missing
• Objectives? Yes.
• Description of work broken down into tasks and role of partners? Not in a consistent way: for some SSC the role of partners is missing and the tasks are not clearly enumerated and described. Sometimes very detailed FTE information is given, sometimes none is.
• Brief description of deliverables and month of delivery? No.

-- AndreaSciaba - 2009-08-25