Metamorphic Relations for Testing Automated and Autonomous Driving Systems and Simulation Platforms

Advanced Driver Assistance Systems (ADASs) and Autonomous Vehicles (AVs) are becoming increasingly popular. These technologies have the potential to revolutionise transportation, but they also face a number of challenges in terms of safety. Simulation testing —– the use of simulations for testing –— is essential when developing ADAS and autonomous driving (AD) systems. It plays a vital role in augmenting and speeding up real-world testing. It is reproducible, scalable and reduces development costs. However, a challenge for simulation testing can occur when determining whether or not the SUT or the simulator correctly produces the intended behaviors. The effectiveness of simulation testing depends on the simulation platform’s quality: if the simulation platform is unreliable, the simulation findings could be inaccurate and, therefore, useless. The oracle problem is a circumstance where it can be too difficult (or expensive) to verify or validate the simulation platforms due to a lack of appropriate test oracles or ways to establish the validity of the program output or behavior. The use of testing standards and protocols is essential for designing and developing autonomous systems. However, a lack of common testing mechanisms and strategies makes it challenging for testers and industries to achieve their targets. One of the popular methods to alleviate the oracle problem is Metamorphic Testing. When applying MT, the identification of Metamorphic Relations is critical. This thesis aims to provide a simulation-based-MT approach by identifying a series of MRs to implement testing standards and protocols. The empirical studies were conducted on the Verification and Validation (V&V) of three popular driving simulators for ADAS development (MATLAB & Simulink, 51Sim-one Cloud, and esmini) and examining their embedded ADAS systems. This research conducts case studies using existing ADAS testing standards, such as the European New Car Assessment Programme (Euro NCAP), and maj