Ensure Software Quality

Relevance

The document is relevant to the field of software engineering and focuses on the notion of quality in Research software.

Scope

Following up their work document proposing a list of Quality Software Characteristics and Attributes (Deliverable D019), the Task Force extended its research and proposed a classification, a landscaping, and a series of recommendations for Quality Research Software.

Main highlights

In this report the Task Force provides insights on how the quality of Research Software could be improved from different points of view and provides practical recommendations.

The Task Force conducted a systematic and thorough survey of the existing literature according to the keywords, title, and field of the articles, and ended up with a set of relevant articles, leading to a classification into software quality characteristics and software quality attributes and metrics, summarised in Appendix A of the report.

After giving a definition of Research Software, they reviewed the stack upon which software is built, from libraries to complete services and platforms.

To complete the landscape section, they include a discussion about what could be the expectations depending on the type of Research Software and depending on specific layers of the Software stack.

The first classification shows that the requirements for a team to build a complete service are very different from those of individual researchers who build their own scripts to produce results for a publication.

In the second classification, the different contexts where Software could be considered are discussed, for example as a scientific structure or as a domain-specific tool.

The classification on characteristics, attributes, and metrics from the survey, along with the landscaping on Research Software, allowed them to propose practical recommendations.

Key recommendations

Given the landscape of different users and scopes of Research Software, the Task Force wrote the recommendations at two different levels. The first group are general recommendations for all types of software including Research Software.

The second group of recommendations are related to user stories and are recommendations depending on the role of the users: a user working in a team, a developer working on an open-source project or a user developing a service or platform.

As a case of special interest, they added a section specifically about software in production stage, discussing the management of its release cycle and quality attributes related to services and platforms in production. They provide specific examples and use cases where the recommendations could be applied.

The recommendations give the perspectives or expectations of developers, users, and service providers with respect to Research Software.

The report mentions metadata of software and the FAIR principles and its relation with quality. It shows examples of existing tools and initiatives to include metadata for software with a focus specifically on the FAIR4RS of the Research Software Working Group, and it maps most of the FAIR4RS principles with the quality attributes proposed.

Not all quality criteria are equally applicable to different types of Research Software or in any context, and the recommendations also take into account this diversity, completed with actual use case examples.