Using Sensor Errors to Define Autonomous System Situational Awareness

Autonomous uncrewed aerial systems (UASs) are expected to operate without a human being in or on the loop. As there will not be a human to interpret the environment, the autonomous UAS is expected to make aeronautical decisions based on situational awareness (SA). This is a limiting factor in the field of truly autonomous systems. Before we can field systems that function without human oversight, we need methods to evaluate whether the SA of that system has been established. This study uses a hypothetical scenario and subject matter expert (SME) opinion to establish a quantifiable metric for SA within an established United States Department of Defense recognized modeling and simulation (M&S) environment. Within this environment, it is assumed that all errors within the UAS sensor suite are known. Through the M&S environment, we were able to vary six separate error variables, with three unique values to provide a total of 729 different data points to be analyzed in our attempt to develop predictive equations. Each data point was evaluated 2,000 times, which gave us a dataset consisting of over 1.4 million individual simulations. From the dataset, we developed linear and nonlinear statistical models to define a point where the SA formed by the UAS is no longer valid for making a sound aeronautical decision. We developed objective measures (inequalities) for the subjective end (SA) through both linear and nonlinear analyses. The M&S environment may not be a direct duplication of reality; however, the results of this study may influence how future autonomous UASs are fielded. This study demonstrates that if a point can be defined where an UAS possesses sufficient SA, decision-makers (Subject Matter Experts in this case) would permit the UAS to make decisions currently reserved for fully qualified human operators.