Measuring societal impact of standards

Society faces existential challenges, such as global warming, demographic change and digital innovations. Standards help solving problems and reaping opportunities. They have long been recognised as important, from technological, business, and economic perspectives, but clear evidence about their societal impact is still missing. Against this background, the XXM Partners commissioned this project to get a clearer view of how standards impact society. The project was carried out by researchers from Eindhoven University of Technology and Rotterdam School of Management, Erasmus University between November 2021 and November 2022. We conducted a pre-study of 48 standards, reviewed academic literature on standards’ impact and its measurement. We carried out eight in-depth case studies of particular standards with 86 expert interviews, relevant documents, and other sources. In doing so, we reached all five goals that were agreed in the project contract: Goal 1: Overview over dimensions of standards impacts Societal impact has many dimensions, which relate, e.g., to public health, education, or the environment. Our literature review identified the 17 United Nations Sustainable Development Goals (SDGs) as the most suitable framework for our project. The SDGs are goals, agreed upon in global consensus, on which society should focus and improve. A growing global movement has formed around them to address societal challenges. In a pre-study, we conducted desk research to explore potential impacts of 48 standards. They cover the breadth of subject areas, and include standards with societal, environmental, technical and/or business purposes. We find that these standards can be linked to all SDGs. Often one standard affects multiple SDGs, going beyond intended impacts. Goal 2: Overview over scientific literature on standards’ impacts Standards are important for society. This is widely recognised in academic literature. However, there is relatively limited research on how they impact society. Most of this limited research focuses on two standards: ISO 9001 (quality management systems) and ISO 14001 (environmental management systems). Our review reveals a research focus on business/economic (both standards) and environmental (ISO 14001) impacts. Some evidence exists on non-environmental societal impacts (e.g., workplace safety), but this remains limited. We extended our review to other fields, especially addressing best practice for impact measurement. We compared multiple approaches, selected the Logic Model for understanding impact, and adapted it to standards. This model helps understand impact by tracing it in five steps: (1) inputs (including standards), (2) activities for standard implementation, (3) outputs (a situation reflecting what a standard prescribes), (4) outcomes (changes for stakeholders), and eventually (5) long-term impact. Goal 3: Evidence of standards’ societal impacts – empirical research Under the project team’s guidance, master students carried out eight in-depth studies in Norway (two studies), Sweden (two studies), Finland (two studies), Denmark and Austria. Standards were selected in consultation with the XXM Partners. They cover a broad range of important areas, e.g., greenhouse gas (GHG) emissions reporting (ISO 14064-1), clinical trials for medical devices (ISO 14155), and information security management (ISO 27001). Our studies led to findings about (1) specific societal changes caused by the investigated standards, (2) what drives standards’ impacts in general, and (3) measuring standards’ impact (see Goal 4 below). Findings about standards’ specific societal changes Two standards in our study are particularly successful in creating positive impact: • EN 16516 (emissions of construction products into indoor air) promotes the availability of low-emission construction products, thereby contributing to healthy indoor air. • ISO 14155 (clinical investigations for medical devices) contributes to safer clinical trials and availability of innovative medical devices for patients. In the other cases, we observe strong potential for positive societal impact. However, this is not achieved, e.g., due to standards not being implemented at large scale, not meeting potential users’ expectations, and/or competing with other standards. In two cases (ISO 14064-1, GHG emissions accounting; ISO 14044, life-cycle assessment), foreseeable abuse of the standard may even provide opportunities for greenwashing. All investigated standards have broader (potential) impacts on the SDGs than identified by ISO. E.g., ISO 14155 does not only contribute to SDG 3 (“Good Health and Wellbeing”), but also to SDG 9 (“Industry, Innovation and Infrastructure”) and the institutional aspect of SDG 16 (“Peace, Justice and Strong Institutions”). Findings about drivers behind standards’ impacts: importance of their ecosystems All investigated standards are deeply embedded in their ecosystems. A remarkable finding concerns the large extent, to which (mis)alignment with this context drives their impact. This also makes it challenging to isolate the standard’s impact from that of its entire ecosystem. Where we observed positive impact, this was largely due to a standard serving a clear purpose in its ecosystem (e.g., meeting market needs, supporting certification, alignment with regulatory requirements). In the cases where potentials for positive impact have not yet been realised, our data show how this is driven by misaligned ecosystems and standards. We have recommendations for how to address this (see Goal 5 below) Goal 4: Applied methodology for demonstrating societal impacts In the course of the project, we developed a methodology for measuring standard’s societal impacts. It consists of a six-step process, which is based on four essential frameworks/tools: 1. Stakeholder analysis, 2. A checklist for analysing a standard’s ecosystem, 3. The logic model for standards, 4. The SDGs as a framework for classifying impact. We recommend this methodology for systematically studying further standards’ impacts. Goal 5: Recommendations to National Standard Bodies (NSBs) Our results have clear implications for the XXM Partners, ISO/IEC, CEN/CENELEC, and their members. Based on these implications, we provide recommendations on (1) communicating standards’ impacts to policymakers and other stakeholders, and (2) enhancing standards’ positive impact. Section 9.5 (p.102) specifies these recommendations in detail. Communicating standards’ impact • To policymakers: Emphasise standards’ functions as essential instruments for change in their respective ecosystems. • To other stakeholders: Focus on standards’ contributions to the SDGs, while acknowledging that the full potential is not yet achieved. • Ensure standards’ inclusion in frameworks for measuring societal impact: Engage in dialogue with the impact measurement community. Ensure that increasingly influential approaches to measuring societal impact include standards. This would support clearer communication about standards’ impacts and contribute to frameworks’ accuracy. Enhancing standards’ positive impact • Already address societal impacts during the standardisation process. We provide two tools, which may be used for this. • Improve standards’ relevance and quality in line with their ecosystems’ expectations (positioning standards relative to competing and/or complementary standards, clarity of requirements, quality of translations, withdrawing unused standards). • Need for future research in six directions: (1) Studies at the level of standard families and/or ecosystems. (2) Large-scale replication. (3) Understanding standard competition. (4) Reflecting ongoing developments in impact measurement. (5) Effects of societal change on standards’ role in society. (6) Starting from the “grand- societal challenges”. Conclusion Our work makes a novel contribution to knowledge about standards’ impacts. By doing so, we offer concrete insights, which the XXM Partners can use in communicating the benefits of standards. We also identify key factors affecting standards’ societal impacts, and offer tools and methods that are relevant for practitioners and researchers alike. The XXM Partners can use these insights in their work to further maximise standards’ value to society, and limit any potential negative effects.