Review of Software Quality Attributes and Characteristics

Relevance

Relevance: The document is relevant to the field of software engineering, and focuses on the notion of quality in Research software. It mentions the complexities in defining software quality due to the various perspectives that can be taken, and aims to propose a list of Quality Characteristics and most important, an extensive list of Quality Attributes that can be related to good software.

Scope

The group’s report analyses how the topic has been addressed in relevant works in the literature and how the Software Quality characteristics can be classified as attributes. It starts with a definition of Research Software as “software used and/or generated in a research context, including and not limited to scientific, non-scientific, commercial, academic and non-academic research.”

In Section 1.1, the report details the method used for a survey of the literature. In Section 1.2, the group reviews what are, according to the literature, the Quality Characteristics of good software.

The characteristics can be related to quality from very different points of view (ranging from metrics of the source code to service operability, just as an example). These are classified as Quality Attributes and Metrics in Section 2.

The key notion of Verification and Validation (V&V) is also thoroughly defined.

Main highlights

The Task Force has identified a total 126 Quality Attributes and Metrics, the following categories and their codenames are proposed:

• Source Code Metrics (EOSC-SCMet): 17 metrics. Metrics related to the source code, such as the number of lines of code or the number of assertions, for example.326
• Time and Performance Metrics (EOSC-TMet): 11 metrics. Metrics related to time or periods of time. For example, the number of resolved bugs per period of time.
• Qualitative Attributes (EOSC-Qual): 27 attributes. Qualitative attributes are obtained in general through surveys to, or some manual analysis by software developers, administrators, or users. In general, these are not possible to automate and are in general subjective.
• DevOps – Software Release and Management Attributes (EOSC-SWRelMan): 34 attributes. Largely based on the DevOps methodology, they can be automated for Verification & Validation. Although the possibility of having code reviews in the software development process is a manual step.
• DevOps – Testing Attributes (EOSC-SWTest): 25 attributes. Again based in DevOps, these are related to software testing and can also be automatically verified. For example, whether integration tests are used in the system.
• Service Operability Attributes (EOSC-SrvOps): 12 attributes. They refer to scientific services or platforms in operation. For example, whether the system provides monitoring and accounting services.

These Quality Attributes and Metrics are classified with one or more of the 25 characteristics identified from three sources:

1. The ISO/IEC 25010:2011(E) standard is a norm that proposes two models for the characteristics. The first group is related to the context in which the software product is used, and contains five characteristics. The second group has eight characteristics according to characteristics of the software of the computer system itself, without relying on how it is used.
2. The ISO/IEC/IEEE 24765:2017 standard defines a common vocabulary for systems and software engineering, in a way that it is always applicable to general applications. This standardised list provides a vast quantity of very precise definitions that avoid trivial misinterpretations, and that therefore we apply in this report.
3. The chapter Design Fundamentals of the Microsoft Application Architecture Guide, chapter Quality Attributes.

Key recommendations

Recommendations for Quality Research Software are given in the main deliverable Ensure Software Quality