Search

Background. In Production Systems Engineering (PSE) models, which describe plants, represent different views on several engineering disciplines (such as mechanical, electrical and software engineering) and may contain up to 10,000s of instance elements, such as concepts, attributes and relationships. Validating these models requires an integrated multi-model view and the domain expertise of human experts related to individual views. Unfortunately, the heterogeneity of disciplines, tools, and data formats makes it hard to provide a technology-independent multi-model view. Aim. In this paper, we aim at improving Multi-Model Reviewing (MMR) capabilities of domain experts based on selected model visualisation methods and mechanisms. Method. We (a) derive requirements for graph-based visualisation to facilitate reviewing multi-disciplinary models; (b) introduce the MMR approach to visualise engineering models for review as hierarchical and linked structures; (c) design an MMR software prototype; and (d) evaluate the prototype based on tasks derived from real-world PSE use cases. For evaluation purposes we compare capabilities of the MMR prototype and a text-based model editor. Results. The MMR prototype enabled performing the evaluation tasks in most cases considerable faster than the standard text-based model editor. Conclusion. The promising results of the MMR approach in the evaluation context warrant empirical studies with a wider range of domain experts and use cases on the usability and usefulness of the MMR approach in practice.

Empirical software engineering (ESE) focuses on gathering evidence through measurements and experiments involving humans and software systems (software products, processes, and resources). While empirical studies often include considerable human effort for study planning, execution, and data analysis, human computation (HC) methods, such as crowdsourcing, are increasingly used to address human input intensive tasks in software engineering and beyond. Therefore, in this chapter, we explore the use of HC techniques to support ESE experiments. We address researchers from both research communities and provide (1) introductory notions into both fields, (2) an analysis of ESE experiment requirements and HC capabilities that could match those, and (3) a concrete example of an ESE experiment that compares the effects of using HC in software inspection with respect to a traditional inspection process preformed using pen and paper. Our focus is on software inspection for detecting defects in software engineering models (namely, extended entity relationship models). This chapter will enable ESE researchers to apply HC in their work and HC researchers to explore ESE as a new application area to further improve their methods and tools.

In the parallel engineering of large and long-running automation systems, such as Production Systems Engineering (PSE) projects, engineering teams with different backgrounds work in a so-called Round-Trip Engineering (RTE) process to iteratively enrich and refine their engineering artifacts, and need to exchange data efficiently to prevent the divergence of local engineering models. Unfortunately, the heterogeneity of local engineering artifacts and data, coming from several engineering disciplines, makes it hard to integrate the discipline-specific views on the data for efficient synchronization.

Context and background: In Production Systems Engineering (PSE), software and systems testing are success-critical along the production automation life cycle to identify defects early and efficiently. Although test automation concepts enable continuous integration and tests during engineering and maintenance, tool chains are often hardwired, less flexible, and inefficient. Thus, there is a need for more flexible tool chains to support verification and validation of control code variants. Objective: In this book chapter, we (a) describe a flexible Test Automation Framework (TAF) to enable continuous integration and tests and (b) provide an adapted maintenance process to enable efficient verification and validation of control code variants. Method: We build on best practices from Software Engineering and Software Testing to establish a flexible TAF based on Behavior-Driven Testing. We use the Abstract Syntax Tree (AST) as foundation for human-based verification and validation. We developed an initial prototype derived from industry partners and used an Industry 4.0 Testbed for evaluation. Results and conclusion: First results of the prototype implementation with selected testing tools showed the capability of the TAF concept for supporting flexible configurations of testing tool chains. The AST concept can support the human-based verification and validation of control code variants.

Context. In Production Systems Engineering (PSE), experts of different domains collaborate in loosely coupled engineering processes to collectively plan and develop an Automation System (AS). Due to limited collaboration capabilities of discipline-specific tools, engineering knowledge is often lost and needs to be recovered manually with considerable effort. Information backflow is often limited due to incompatible artifacts and engineering models. Goal. Main goal is to establish a hybrid process combined with an improved data model to overcome initial limitations of an existing engineering process. Method. In a case study at a large-scale automation engineering organization, we investigate challenges and requirements for a hybrid Engineering Process in PSE. For efficient knowledge exchange, we build on a Product, Process, and Resource (PPR) concept that aims at bridging the gap between engineering disciplines, project phases, and artifacts. Results. The proposed hybrid process improved knowledge exchange and backflows and allows experts to maintain planning processes within their scope. Conclusions. Although the PPR concept was found useful, initial effort is needed for analyzing processes and data for PPR concept implementation.

Context and Background. Software testing and test automation are important activities in software development where frequent requirements changes and the fast delivery of software increments are supported by traditional and agile development processes. Test reports are often used as “proof of evidence” for executed software tests. However, the practical impact of test reports, such as decision making and quality assessment, requires structured information which might not be available in sufficient quality. Goal. In this paper we (a) report on needs of test reports of different stakeholders at a large-scale Austrian logistics organization, (b) develop candidate improvement actions based on the state of the practice, and (c) conceptually evaluate selected improvement actions. Method. We used surveys and interviews to elicit needs and expected capabilities for test reporting and developed candidate improvement. We used expert discussions prioritize improvement actions in the organization context for further implementation. Results. Based on 23 recommended improvement actions, 14 were initially selected for implementation. Most of these accepted improvement action focus on regular test status reports and visualization aspects of test reports. Conclusion. Although test reporting is systematically applied in development processes, there is still some potential to improve test reports to gain (additional) benefits for related stakeholder.

Flexible manufacturing systems, as a vision of Industry 4.0, depend on the collaboration of domain experts coming from different engineering disciplines. These experts often depend on (interdisciplinary) results from previous engineering phases and require an explicit representation of knowledge on relationships between products and production systems. However, production systems engineering organizations, which are set in a multidisciplinary environment, rather than focusing on process analysis and improvement options ranging over multiple disciplines, focus mostly on one particular discipline and neglect collaborations between several workgroups. In this chapter, we investigate requirements for the product/ion (i.e., product and production process)-aware analysis of engineering processes to improve the engineering process across workgroups. We, therefore, consider the following three aspects: (1) engineering process analysis methods; (2) artifact and data modeling approaches, from business informatics and from production systems engineering; and (3) persistent representation of product/ion-aware engineering knowledge and data. We extend existing work on business process analysis methods and BPMN 2.0 to address their limited capabilities for product/ion-aware process analysis. We evaluate the resulting contributions in a case study with domain experts from a large production system engineering company. We conclude that an improved product/ion-aware knowledge representation facilitates traceable design decisions as foundation for better quality assurance in the engineering process.

In Production Systems Engineering (PSE), many projects conceptually follow the plan of traditional waterfall processes with sequential process steps and limited security activities, while engineers actually work in parallel and distributed groups following a Round-Trip-Engineering (RTE) process. Unfortunately, the applied RTE process in PSE is coarse-grained, i.e., often data are exchanged via E-Mail and integrated seldom and inefficiently as the RTE process is not well supported by methods and tools that facilitate efficient and secure data exchange. Thus, there is a need for frequent synchronization in a secure way to enable engineers building on a stable and baseline of engineering data. We build on Scrum, as an established agile engineering process, and security best practices to support flexible and secure RTE processes. In this paper, we introduce and initially evaluate an efficient and secure RTE process for PSE, augmented with agile practices, and discuss the identification and mitigation of security concerns and risks. First results show that the augmented RTE process can provide strong benefits from agile practices for the collaboration of engineers in PSE environments. Security practices can be added but need to be balanced well regarding sufficient mitigation of security risks and extra effort for engineers to ensure an overall benefit to both engineers and the management.

Context. Expert sourcing is a novel approach to support quality assurance: it relies on methods and tooling from crowdsourcing research to split model quality assurance tasks and parallelize task execution across several expert users. Typical quality assurance tasks focus on checking an inspection object, e.g., a model, towards a reference document, e.g., a requirements specification, that is considered to be correct. For example, given a text-based system description and a corresponding model such as an Extended Entity Relationship (EER) diagram, experts are guided towards inspecting the model based on so-called expected model elements (EMEs). EMEs are entities, attributes and relations that appear in text and are reflected by the corresponding model. In common inspection tasks, EMEs are not explicitly expressed but implicitly available via textual descriptions. Thus, a main improvement is to make EMEs explicit by using crowdsourcing mechanisms to drive model quality assurance among experts. Objective and Method. In this paper, we investigate the effectiveness of identifying the EMEs through expert sourcing. To that end, we perform a feasibility study in which we compare EMEs identified through expert sourcing with EMEs provided by a task owner who has a deep knowledge of the entire system specification text. Conclusions. Results of the data analysis show that the effectiveness of the crowdsourcing-style EME acquisition is influenced by the complexity of these EMEs: entity EMEs can be harvested with high recall and precision, but the lexical and semantic variations of attribute EMEs hamper their automatic aggregation and reaching consensus (these EMEs are harvested with high precisions but limited recall). Based on these lessons learned we propose a new task design for expert sourcing EMEs.

The application of software process models in industry includes traditional processes, agile processes, and process variants that aim at balancing traditional and agile with focus on specific industry needs. To investigate the characteristics of such hybrid software and system development approaches that combine agile and traditional approaches the HELENA project was initiated. HELENA is based on a large international survey. Based on the first HELENA survey, conducted in 2016, in 2017 a second round of surveys has been launched. This paper focuses on initial results and discussions of the data from Austria where 22 persons participated. Results showed a good balance of small and medium enterprises and large organizations. Iterative development processes and Scrum are widely spread in these organizations where traditional approaches are often combined with some agile practices.

In multi-disciplinary engineering (MDE) projects, information models play an important role as inputs to and outputs of engineering processes. In MDE projects, engineers collaborate from various disciplines, such as mechanical, electrical, and software engineering. These disciplines use general-purpose and domain-specific models in their engineering context. Important challenges include model synchronization and model quality assurance (MQA) that are covered insufficiently in current MDE practices. This chapter focuses on the needs and approaches for MQA in MDE environments. We address the following two research questions (RQs): The first RQ focuses on investigating needs and expected capabilities that are required for a systematic review process that focuses on changes in MDE design models (RQ-MQA1). The second RQ focuses on how to extend a standard modeling language for MDE, such as the AutomationML, to address needs for storing process-relevant attributes in the context of quality assurance and review process support (RQ-MQA2). This chapter presents concepts and an initial evaluation of MQA approaches in the context of selected MDE processes, i.e., the addition, change, or removal of a component in an engineering discipline and an impact analysis on the integrated plant model. Main results are that (a) an adapted review process helps to systematically drive model reviews for MDE and (b) the standardized language description of AutomationML can be extended with process-related attributes that are useful for quality assurance and reviewing.

The application of best-practice software inspection processes for early defect detection requires considerable human effort. Crowdsourcing approaches can support inspection activities (a) by distributing inspection effort among a group of human experts and (b) by increasing inspection control. Thus, the application of crowdsourcing techniques aims at making inspection processes more effective and efficient. In this paper, we present a crowdsourcing-supported model inspection (CSI) process and investigate its defect detection effectiveness and efficiency when inspecting an Extended Entity Relationship (EER) model. The CSI process uses so-called Expected Model Elements (EMEs) to guide CSI inspectors during defect detection. We conducted a controlled experiment on defect detection effectiveness, efficiency, and false positives. While CSI effectiveness and efficiency is lower for CSI inspectors, the number of false positives decreases. However, CSI was found promising for increasing the control of defect detection and supports the inspection of large-scale engineering models.

This chapter introduces key concepts of the Industrie 4.0 vision, focusing on variability issues in traditional and cyber-physical production systems (CPPS) and their engineering processes. Four usage scenarios illustrate key challenges of system engineers and managers in the transition from traditional to CPPS engineering environments. We derive needs for semantic support from the usage scenarios as a foundation for evaluating solution approaches and discuss Semantic Web capabilities to address the identified multidisciplinary engineering needs. We compare the strengths and limitations of Semantic Web capabilities to alternative solution approaches in practice. Semantic Web technologies seem to be a very good match for addressing the aspects of heterogeneity in engineering due to their capability to integrate data intelligently and flexibly on a large scale. Engineers and managers from engineering domains can use the scenarios to select and adopt appropriate Semantic Web solutions in their own settings.

In Multi-Disciplinary Engineering (MDE) environments, engineers coming from different disciplines have to collaborate. Typically, individual engineers apply isolated tools with heterogeneous data models and strong limitations for collaboration and data exchange. Thus, projects become more error-prone and risky. Although Quality Assurance (QA) methods help to improve individual engineering artifacts, results and experiences from previous activities remain unused. This paper describes a Collective Intelligence-Based Quality Assurance (CI-Based QA) approach that combines two established QA approaches, i.e., (Software) Inspection and the Failure Mode and Effect Analysis (FMEA), supported by a Collective Intelligence System (CIS) to improve engineering artifacts and processes based on reusable experience. CIS can help to bridge the gap between inspection and FMEA by collecting and exchanging previously isolated knowledge and experience. The conceptual evaluation with industry partners showed promising results of reusing experience and improving quality assurance performance as foundation for engineering process improvement.

Many software projects fail due to problems in requirements engineering (RE). The goal of this paper is analyzing a specific and relevant RE problem in detail: incomplete/hidden requirements. We replicated a global family of RE surveys with representatives of software organizations in Austria and Brazil. We used the data to (a) characterize the criticality of the selected RE problem, and to (b) analyze the reported main causes and mitigation actions. Based on the analysis, we discuss how to prevent the problem. The survey includes 14 different organizations in Austria and 74 in Brazil, including small, medium and large sized companies, conducting both, plan-driven and agile development processes. Respondents from both countries cited the incomplete/hidden requirements problem as one of the most critical RE problems. We identified and graphically represented the main causes and documented solution options to address these causes. Further, we compiled a list of reported mitigation actions. From a practical point of view, this paper provides further insights into common causes of incomplete/hidden requirements and on how to prevent this problem., in Proceedings of the Software Quality Days, 2015

[Context] In Automation Systems Engineering ASE Environments, engineers coming from different disciplines, have to collaborate. Individual engineers, e.g., from electrical, mechanical, or software domains, apply domain-specific tools and related data models that hinder efficient collaboration due to limited capabilities for interaction and data exchange on technical and semantic level. Manual activities are required to synchronize planning data from different disciplines and can raise additional risks caused by defects and/or changes that cannot be identified efficiently. [Objective] Main objective is to improve a engineering processes by providing efficient data exchange mechanism and to support b defect detection performance in ASE environments. [Method] Software inspections SI are commonly used by engineers in Software Engineering SE by applying well-defined approaches to systematically identify defects early in the development process. In this paper we adapt the traditional SI process for application in ASE environments and provide a software tool to support frequent synchronization and focused reviews. We evaluate and discuss the adapted process in an industry context. [Results] Main results were that the adapted process and the software tool can be useful in the application context in order to identify defects early, increase overall product quality, and improve engineering processes in the ASE domain. [Conclusion] The proposed adapted inspection approach showed promising results to improve ASE projects.

Developing large systems engineering projects require combined efforts of various engineering disciplines. Each engineering group uses specific engineering tools and data model concepts representing interfaces to other disciplines. However, individual concepts lack in completeness and include strong limitations regarding interoperability and data exchange capabilities. Thus, highly heterogeneous data models cause semantic gaps that hinder efficient collaboration between various disciplines. The design of an integration solution within a systematic engineering process typically requires re-modelling of the common data model (used for mapping individual local tool data models) to enable efficient data integration. However, designing and implementing integration approaches include continuously collecting new knowledge on the related application domains, in our case automation systems engineering projects, and integration capability that meet requirements of related domains. In this paper we report on a sequence of different architectural designs for an efficient and effective integration solution that lead to a similar and stable data model design for application in the automation systems domain. By means of iterative prototyping, candidates for modelling styles were tested for feasibility in context of industry use cases. In addition we applied an adjusted Architecture Tradeoff Analysis Method (ATAM) to assess the resulting final architecture variant.

Requirements managers aim at keeping the set of requirements consistent and up to date throughout the project by conducting the following tasks: requirements categorization, requirements conflict analysis, and requirements tracing. However, the manual conduct of these tasks takes significant effort and is error-prone. In this paper we propose to use semantic technology as foundation for automating the requirements management tasks and introduce the ontology-based reporting approach OntRep. We evaluate the effectiveness and effort the OntRep approach based on a real-world industrial empirical study with professional Austrian IT project managers. Major results were that OntRep provides reasonable capabilities for the automated categorization of requirements, was when compared to a manual approach considerably more effective to identify conflicts, and produced less false positives with similar effort.