Explainable Metamodels for ATM Performance Assessment

Fast-time simulation constitutes a well-known and long-established technique within the Air Traffic Management (ATM) community. However, it is often the case that simulation input and output spaces are underutilized, limiting the full understandability, transparency, and interpretability of the obtained results. In this paper, we propose a methodology that combines simulation metamodeling and SHapley Additive exPlanations (SHAP) values, aimed at uncovering the intricate hidden relationships among the input and output variables of a simulated ATM system in a rather practical way. Whereas metamodeling provides explicit functional approximations mimicking the behavior of the simulators, the SHAP-based analysis delivers a systematic framework for improving their explainability. We illustrate our approach using a state-of-the-art ATM simulator across two case studies in which two delay-centered performance metrics are analyzed. The results show that the proposed methodology can effectively make simulation and its results more explainable, facilitating the interpretation of the obtained emergent behavior, and additionally opening new opportunities towards novel performance assessment processes within the ATM research field.