Search

Few mobile robot developers already test their software on simulated robots in virtual environments or sceneries. However, the majority still shy away from simulation-based test campaigns because it remains challenging to specify and execute suitable testing scenarios, that is, models of the environment and the robots' tasks. Through developer interviews, we identified that managing the enormous variability of testing scenarios is a major barrier to the application of simulation-based testing in robotics. Furthermore, traditional CAD or 3Dmodelling tools such as SolidWorks, 3ds Max, or Blender are not suitable for specifying sceneries that vary significantly and serve different testing objectives. For some testing campaigns, it is required that the scenery replicates the dynamic (e.g., opening doors) and static features of real-world environments, whereas for others, simplified scenery is sufficient. Similarly, the task and mission specifications used for simulation-based testing range from simple point-to-point navigation tasks to more elaborate tasks that require advanced deliberation and decision-making. We propose the concept of composable and executable scenarios and associated tooling to support developers in specifying, reusing, and executing scenarios for the simulation-based testing of robotic systems. Our approach differs from traditional approaches in that it offers a means of creating scenarios that allow the addition of new semantics (e.g., dynamic elements such as doors or varying task specifications) to existing models without altering them. Thus, we can systematically construct richer scenarios that remain manageable. We evaluated our approach in a small simulation-based testing campaign, with scenarios defined around the navigation stack of a mobile robot. The scenarios gradually increased in complexity, composing new features into the scenery of previous scenarios. Our evaluation demonstrated how our approach can facilitate the reuse of models and revealed the presence of errors in the configuration of the publicly available navigation stack of our SUT, which had gone unnoticed despite its frequent use. [ABSTRACT FROM AUTHOR]

Few mobile robot developers already test their software on simulated robots in virtual environments or sceneries. However, the majority still shy away from simulation-based test campaigns because it remains challenging to specify and execute suitable testing scenarios, that is, models of the environment and the robots’ tasks. Through developer interviews, we identified that managing the enormous variability of testing scenarios is a major barrier to the application of simulation-based testing in robotics. Furthermore, traditional CAD or 3D-modelling tools such as SolidWorks, 3ds Max, or Blender are not suitable for specifying sceneries that vary significantly and serve different testing objectives. For some testing campaigns, it is required that the scenery replicates the dynamic (e.g., opening doors) and static features of real-world environments, whereas for others, simplified scenery is sufficient. Similarly, the task and mission specifications used for simulation-based testing range from simple point-to-point navigation tasks to more elaborate tasks that require advanced deliberation and decision-making. We propose the concept of composable and executable scenarios and associated tooling to support developers in specifying, reusing, and executing scenarios for the simulation-based testing of robotic systems. Our approach differs from traditional approaches in that it offers a means of creating scenarios that allow the addition of new semantics (e.g., dynamic elements such as doors or varying task specifications) to existing models without altering them. Thus, we can systematically construct richer scenarios that remain manageable. We evaluated our approach in a small simulation-based testing campaign, with scenarios defined around the navigation stack of a mobile robot. The scenarios gradually increased in complexity, composing new features into the scenery of previous scenarios. Our evaluation demonstrated how our approach can facilitate the reuse of models and revealed the presence of errors in the configuration of the publicly available navigation stack of our SUT, which had gone unnoticed despite its frequent use.

Copyright of Automatisierungstechnik is the property of De Gruyter and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Highly automated vehicles are increasingly gaining the public’s attention. To achieve broad acceptance for the deployment of such vehicles, it is necessary to ensure their functionality and safety. One approach that has become popular in research is the scenario-based approach. However, manual testing of such complex systems is impractical and time-consuming. Using simulations to run and evaluate such scenarios appears to be the most viable approach. This, in turn, raises new challenges, especially in modeling the scenarios to be tested simulatively and incorporating the system under test as part of these. Since existing solutions do not solve these challenges satisfactorily—due to the strict separation of scenario and simulation model, among other reasons—this work addresses the need for a standardized, holistic, and extensible approach for modeling traffic scenarios to be executed simulatively. Requirements for such an approach are identified with focus on its application in simulation- and scenario-based verification and validation. Based on these, a model-based multi-layered approach is proposed. The foundations of this are then implemented utilizing a Meta Object Facility based heavyweight extension of the Unified Modeling Language metamodel. The resulting metamodel is used to demonstrate the applicability of the proposed approach by modeling a maritime traffic scenario.

Highly automated vehicles are increasingly gaining the public's attention. To achieve broad acceptance for the deployment of such vehicles, it is necessary to ensure their functionality and safety. One approach that has become popular in research is the scenario-based approach. However, manual testing of such complex systems is impractical and time-consuming. Using simulations to run and evaluate such scenarios appears to be the most viable approach. This, in turn, raises new challenges, especially in modeling the scenarios to be tested simulatively and incorporating the system under test as part of these. Since existing solutions do not solve these challenges satisfactorily—due to the strict separation of scenario and simulation model, among other reasons—this work addresses the need for a standardized, holistic, and extensible approach for modeling traffic scenarios to be executed simulatively. Requirements for such an approach are identified with focus on its application in simulation- and scenario-based verification and validation. Based on these, a model-based multi-layered approach is proposed. The foundations of this are then implemented utilizing a Meta Object Facility based heavyweight extension of the Unified Modeling Language metamodel. The resulting metamodel is used to demonstrate the applicability of the proposed approach by modeling a maritime traffic scenario. [ABSTRACT FROM AUTHOR]