TSA 2.3 Kick Off meeting

Attendees: Alberto Aimar (AA), Andres Abad Rodrigez (AAR), Eamonn Kenny (EK), Gianni Pucciani (GP), Lorenzo Dini (LD)

Welcome

  • Record in future on EVO. There was a general consensus to have the session recorded on the phone.

Metrics and Measures

EK: what metrics do we want for each part of the project: JRA1: software code + bug tracking, SA2: Build system + testbed/certification.

  • EK: What ETICS metrics are required by SA1?
  • GP: Use the metrics of ETICS that SA2 define.
  • AA: Two large classifications: Product related and Process related
  • Outcome : Two classifications: Product and Process related metrics

EK: Are there penalties for product teams not conforming to metrics?

  • LD: There are no incurred penalties for the product teams if they don't meet the Q/A requirement.
  • LD: We make a measure of the software and the processes and make in publicly available, to the management, the partners and the users.
  • LD: Multiple components use the same service so there will be competition within product teams.
  • LD: Racking of software will be made publicly available and compared with existing software.
  • AA: The penalty is: the openness to public scrutiny of the results of the metrics/measures.
  • Outcome : No penality, however, there will be public/project wide scrutiny of metrics and competing product teams. The primary purpose of metrics collection is to help product teams to improve their processes, not to penalise them.

Comments regarding Cross-grid, test-beds and certification

  • GP: Crossgrid produced many metrics but how meaningful are they?
  • AA: The number of classes to the number of lines of code for instance is a good example.
  • AA: what testbeds metrics can we use?
  • EK: reliability of the testbed and how often it is up, was one metric provided in Cross-grid.
  • LD: The number of bugs found/caught in certification before the software goes to production.
  • LD: Many build systems that the metrics must work in.
  • Outcome : There is the possibility to produce metrics within the certification process.
  • Action Point : Eamonn to contact members of Cross-grid to obtain access to the project documentation.

EK: how useful is A-QCM?

  • AA: metrics and measures, adding criteria to generate the metric values. Hard to work out the meaningfulness of the metrics.
  • AA: its a starting point from on which to improve. We need more tools.
  • LD: The specification is good for A-QCM but the implementation isn't so good.
  • LD: Sonar is very similar and worth a look.
  • Action point : Look at Sonar in addition to A-QCM

LD: metrics and measurements, there is a distinction

  • LD: measurements against metrics are being confused. The measure itself isn't useful, but the metric obtains must be meaningful.
  • LD: We need a definition of what we call metrics and what we call measurements.
  • LD: measurements are very understandable but its not clear how to improve the code.
  • Outcome : Clearly define the distinction between a metric and a measure.

GP: how do we access the conformance?

  • EK: with regard to portability: compliance and conformance are very different. You must be 100% compliant to POSIX.1 to be conformant.
  • AA: IPv6 compliance is for example important. Compliance is important in some cases.
  • EK: 100% compliance = conformant
  • LD: its an improvement process, we move towards conformance, so the compliance metric is still important.
  • EK: totally agreed.
  • Outcome : No action point, just keep in mind both compliance and conformance for the future

Conformance to subsets of a standard

  • EK: POSIX.1 may be important to one developer but POSIX.1a and POSIX.1b may not be important. We must ascertain which metrics are of interest to the developers. What is relevant to each developer?
  • LD: ETICS will not die, but its not the uniform way to build software or produce metrics.
  • LD: ETICS plugins is just a wrapper for the existing tools. Unicore and dCache will use Maven.
  • AA: we should focus on the tools/metrics below us, not the ETICS plugins.
  • LD: security team have a set of tools, security code tools.
  • EK: Very good, you could move it across from gLite to other middlewares.
  • Action Point : Gianni to look at security plugins at CERN.

Initial tools structure and ISO 9126

  • EK: Define a framework for metrics/measures based on the ISO 9126? Measures move across from one section to another.
  • AA: people are using these ISO standards already to build metrics.
  • EK: There is a huge amount of background reading required to produce a KPI/metrics set that will feed into SA2.4 and eventually be used to define the tools.
  • GP: should we define some objectives for next week. Understand the results of the survey and understand the ISO 9126 standard.
  • EK: we could use ISO 9126 as a framework and then try to include the results of the survey and what already exists in A-QCM. We agree amongst ourselves on what is useful.
  • Action Point : Eamonn will look at ISO standards on the web. TCD has free access to all IEEE publications.

What starting point to take?

  • LD: what is measureable and what we need to have are two different things?
  • EK: After we have a set of metrics we need to take the process to the management level and discuss it so that the developers are involved in the decision making process.
  • AA: Vision of where we want to be is important, but start with metrics defined by category is important. We could then define the top 3 metrics per category. Start with what we have and try to improve from there.
  • AAR: The metrics need to be usable.
  • EK: A-QCM is not transparent about what the metrics are and what they are trying to measure.
  • AA: send a survey to the work package leaders. When the ideas are clear the information should go into a deliverable.
  • AA: disseminate the information to the work packages leaders.
  • EK: concern is that developers stay quiet about their point of views.
  • Action Point : Wait for survey inventory compiled by Gianni before progressing.
  • Action Point : Eamonn will contact members about what metrics they see as being useful to their overall tasks
  • Action Point : Alberto will ensure that the list of members to contact is available this afternoon

GP: who are our users? Do we have to consider EGI/PRACE as the users?

  • AA: EGI want the KPIs and metrics, but must be less strict than ours. We need to know who are the people involved.
  • EK: I've asked for this information. We haven't got it yet. I'll ask again. Our information must be very transparent.
  • GP: Can we generate metrics from GGUS tickets, that we can use? Or just applicable to EGI?
  • EK: training day in GGUS at the EGI council meeting on Monday 7th June.
  • AA: training was just to use GGUS, the question is, can we get the information?
  • GP: We need to know what EGI requires as a user?
  • AA: Our users are EGI. We will use anything we can get from EGI.
  • AA: We should really use our internal bug tracking systems to produce metrics. User issues in EGI that relate to EMI will be propagated down into EMI and therefore can be tracked from without our internal bug tracking systems.
  • Outcome : EGI is the end user of EMI software, but metrics should be produced from internal bug tracking systems

Proposal for metrics and due date for first deliverable

  • AA: information on wiki by end of next week
  • AA: Need information for report for the end of the month.
  • GP: Have seen the one used with Savannah.
  • AA: We need to which and why we are using some certain metrics? What, why and how?
  • AA: that would imply that you have tools
  • EK: valgrind for memory leaks for instance.
  • EK: run checks per programming language. Top 3 checks for one program language may not be the same for others.
  • LD: compare software metrics per language. A-QCM tried to extract the language, but doesn't work well.
  • AA: Ask the work-package leaders, all former developers, what they would see as the most important metrics in there opinion, that would improve productivity.
  • AA: guide them as much as possible.
  • EK: need percentage of languages, what build system used
  • Action Point : We need a draft of metrics for a proposal in one weeks time to go into the deliverable due to be completed in two weeks time
  • Action Point : Alberto will pass e-mail addresses to EK of each area leader to address the issue of what are the most important measurements to turn into metrics
  • Action Point : Eamonn will produce further survey for metrics. Gianni can create a new survey if needs be, otherwise just e-mail questions to each person concerned

Need more effort from others

  • GP: Include people from SA2.5.
  • Action Point : Alberto will contact SA2.5 partners by e-mail with EK in CC (carbon copy).

Action Point Summary

  1. Eamonn to contact members of Cross-grid to obtain access to the project documentation.
  2. Look at Sonar in addition to A-QCM
  3. Gianni to look at security plugins at CERN.
  4. Eamonn will look at ISO standards on the web. TCD has free access to all IEEE publications.
  5. Wait for survey inventory compiled by Gianni before progressing.
  6. Eamonn will contact members about what metrics they see as being useful to their overall tasks
  7. Alberto will ensure that the list of members to contact is available this afternoon
  8. We need a draft of metrics for a proposal in one weeks time to go into the deliverable due to be completed in two weeks time
  9. Alberto will pass e-mail addresses to EK of each area leader to address the issue of what are the most important measurements to turn into metrics
  10. Eamonn will produce further survey for metrics. Gianni can create a new survey if needs be, otherwise just e-mail questions to each person concerned
  11. Alberto will contact SA2.5 partners by e-mail with EK in CC (carbon copy).
Edit | Attach | Watch | Print version | History: r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r3 - 2010-06-11 - unknown
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    EMI All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback