SQA Standards in EMI


In order to define the SQAP for EMI, a set of standards have been used. In the following sections, we present how SQA Standards have been applied in the EMI project.

Some Definitions

  • ISO 8402-94 - Quality Management and Quality Assurance -- Vocabulary
    • Quality: The set of characteristics of an entity that give that entity the ability to satisfy expressed and implicit needs.
    • Quality Assurance: The series of preestablished and systematic activities laid out in the quality system framework that are performed when needed to prove that an entity will meet quality expectations.

ISO/IEC 12207

The Standard ISO/IEC 12207 is about software lifecycle processes. It defines all the tasks required for developing and maintaining software. In the following Matrix, the processes and activities defined in the standard are presented and mapped to what it is currently defined for EMI.

Type Process Activity EMI
Primary Acquisition Initiation NA
Contract Preparation and update
Supplier monitoring
Acceptance and completion
Supply Initiation EU project proposal?
Preparation of response EU project proposal?
Contract Dow?
Planning Technical Area work plans?
Execution and control Project Deliverables?
Review and evaluation EU Project review?
Delivery and completion ?
Development Process implementation Y
System requirements analysis N
System architectural design N
Software requirements analysis N
Software architectural design N
Software detailed design N
Software coding and testing Y
Software integration Y
Software qualification testing Y
System integration Y
System qualification testing Y
Software installation NA
Software acceptance support Y
Operation Process implementation NA (done by EGI)
Operational testing NA
System operation NA
User support Y (3rd level support)
Maintenance Process implementation Y
Problem and modification analysis Y
Modification implementation Y
Maintenance review/acceptance Y
Migration N
Software Retirement N
Supporting Documentation Process implementation Y
Design and Development Y
Production Y
Maintenance Y
Configuration Management Process implementation Y
Configuration identification Y
Configuration control Y
Configuration status accounting Y
Configuration evaluation Y
Release management and delivery Y
Quality Assurance Process implementation Y
Product assurance Y
Process assurance Y
Assurance of quality systems ?
Verification Process implementation Y
Verification Y
Validation Process implementation Y
Validation Y
Joint Review Process implementation Y
Project Management reviews Y
Technical reviews N
Audit Process implementation Y
Audit Y
Problem resolution Process implementation Y
Problem resolution Y
Organisational Management Initiation and scope definition Y?
Planning Y?
Execution and control Y?
Review and evaluation Y?
Closure N
Infrastructure Process implementation Y (ETICS?)
Establishment of the infrastructure Y
Maintenance of the infrastructure Y
Improvement Process establishment N
Process assessment N
Process improvement N
Training Process implementation Y
Training material development Y
Training plan implementation Y

Quality assurance process

The quality assurance process is a process for providing adequate assurance that the software products and processes in the project life cycle conform to their specified requirements and adhere to their established plans. The process consists of the following activities:

  • Process implementation
  • Product assurance
  • Process assurance
  • Assurance of quality systems

Process implementation

  • A quality assurance process tailored to the project shall be established. The objectives of the process are:
    • Assure that the software products and the processes employed for providing those software products comply with their established requirements and adhere to their established plans.
  • The quality assurance process should be coordinated with the following processes:
    • Verification and Validation
    • Joint Review
    • Audit
  • A plan for conducting the quality assurance process activities and tasks shall be developed, documented, implemented and maintained. The plan shall include:
    • Quality standards, methodologies, procedures and tools for performing quality assurance activities.
    • Procedures for contract review.
    • Procedures for identification, collection, filing, maintenance and disposition of quality records.
    • Resources, schedule and responsibilities for conducting the quality assurance activities.
  • Scheduled and on-going quality assurance activities and tasks shall be executed. When problems or non-conformances with contract requirements are detected, they shall be documented and serve as input to Problem Resolution Process. Records of these activities and tasks, their execution, problems, and problem resolutions shall be prepared and maintained.
  • Records of quality assurance activities and tasks shall be made available to the acquirer as specified in the contract.

Product Assurance

  • It shall be assured that all the plans required by the contract are documented, comply with the contract, are mutually consistent and are being executed as required.
  • It shall be assured that the software products and related documentation comply with the contract and adhere to the plans.
  • In preparation for the delivery of the software products, it shall be assured that they have fully satisfied their contractual requirements and are acceptable to the acquirer.

Process Assurance

  • It shall be assured that those software life cycle processes (suppy, development, operation, maintenance, and supporting processes including quality assurance) employed for the project comply with the contract and adhere to the plans.
  • It shall be assured that the internal software engineering practices, development environment, test environment and libraries, comply with the contract.
  • It shall be assured that the acquirer and other parties are provided the required support and cooperation in accordance with the contract, negotiation and plans.
  • It should be assured that software product and process measurements are in accordance with established standards and procedures.
  • It shall be assured that the staff assigned have the skill and knowledge needed to meet the requirements of the project and receive any necessary training.

Assurance of quality systems

  • Additional quality management activities shall be assured in accordance with the clauses of ISO 9001 as specified in the contract.

ISO/IEC 9126

The standard ISO/IEC 9126 is for the evaluation of software quality. The standard is divided into four parts:

  • quality model: it helps to define software product quality.
  • internal metrics: applicable to non-executable software product during its development stages.
  • external metrics: applicable to executable software product during testing and operation stages.
  • quality in use metrics: checks whether the software product meets the needs specified by the user to achieve specified goals with effectiveness, productivity, safety and satisfaction. It is measured in terms of using the software rather than the properties of the software itself.

Quality Model

The quality model for external and internal quality categorises the software quality attributes into six characteristics, which are further subdivided into subcharacteristics:

The capabily of the software product...

Characteristic Subcharacteristic Definition Applicable in EMI?
Functionality   to provide functions which meet stated and implied needs when the software is used under specific conditions
Suitability to provide an appropriate set of functions for specified tasks and user objectives Y
Accuracy to provide the right or agreed results or effects with the needed degree of precission Y
Interoperability to interact with one or more specified systems Y
Security to protect information and data so that unauthorised persons or systems cannot read or modify them and authorised persons or systems are not denied access to them Y
Functionality compliance to adhere to standards, conventions or regulations in laws and similar prescriptions relating to functionality Y
Reliability   to maintain a specified level of performance when used under specified conditions  
Maturity to avoid failure as a result of faults in the software Y
Fault Tolerance to maintain a specified level of performance in cases of software faults or of infringement of its specified interface Y
Recoverability to re-establish a specified level of performance and recover the data directly affected in the case of a failure Y
Reliability compliance to adhere to standards, conventions or regulations in laws and similar prescriptions relating to reliability Y
Usability   to be understood, learned, used and attractive to the user, when used under specified conditions  
Understandability to enable the user to understand whether the software is suitable, and how it can be used for particular tasks and conditions of use Y
Learnability to enable the user to learn its application Y
Operability to enable the user to operate and control it Y
Attractiveness to be attractive to the user Y
Usability compliance to adhere to standards, conventions or regulations in laws and similar prescriptions relating to usability Y
Efficiency to provide appropriate performance, relative to the amount of resources used, under stated conditions  
Time behaviour to provide appropriate response and processing times and throughput rates when performing its function, under stated conditions Y
Resource utilisation to use appropriate amounts and types of resources when the software performs its function under stated conditions Y
Efficiency compliance to adhere to standards, conventions or regulations in laws and similar prescriptions relating to efficiency Y
Maintainability   to be modifed. Modifications may include corrections, improvements or adaptacion of the software to changes in environment, and in requirements and functional specifications  
Analysability to be diagnosed for deficiencies or causes of failures in the software, or for the parts to be modified to be identified Y
Changeability to enable a specific modification to be implemented Y
Stability to avoid unexpected effects from modifications of the software Y
Testability to enable modified software to be validated Y
Maintainability compliance to adhere to standards, conventions or regulations in laws and similar prescriptions relating to maintainability Y
Portability   to be transferred from one environment to another  
Adaptability to be adapted for different specified environments without applying actions or means other than those provided for this purpose for the software considered Y
Installability to be installed in a specified environment Y
co-existence to co-exist with other independent software in a common environment sharing common resources Y
Replaceability to be used in place of another specified software product for the same purpose in the same environment Y
Portability compliance to adhere to standards, conventions or regulations in laws and similar prescriptions relating to portability Y

External metrics

Characteristic Subcharacteristic Metric Definition EMI implementation Comments
Suitability Functional Adequacy How adequate are the evaluated functions? Number of open bugs associated to a technical objective/ Total number of bugs associated to the same technical objective It's difficult to automate this calculation because we don't have any mean to establish a relationship between a bug and a technical objective. Bugs are only related to software components. It's a metric that needs to be calculated manually. A possibility to make it easier is to simplify the metric by calculating only the number of major or critical bugs. Since normally the number is not very high, the exercise of associating them with a technical objective wouldn't be very long.
Edit | Attach | Watch | Print version | History: r10 | r7 < r6 < r5 < r4 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r5 - 2011-03-25 - MariaALANDESPRADILLO
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    EMI All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback