Mission Engineering and Design using Real-Time Strategy Games: An Explainable-AI Approach

In this paper, we adapt computational design approaches, widely used by the engineering design community, to address the unique challenges associated with mission design using RTS games. Specifically, we present a modeling approach that combines experimental design techniques, meta-modeling using convolutional neural networks (CNNs), uncertainty quantification, and explainable AI (XAI). We illustrate the approach using an open-source real-time strategy (RTS) game called microRTS. The modeling approach consists of microRTS player agents (bots), design of experiments that arranges games between identical agents with asymmetric initial conditions, and an AI infused layer comprising CNNs, XAI, and uncertainty analysis through Monte Carlo Dropout Network analysis that allows analysis of game balance. A sample balanced game and corresponding predictions and SHapley Additive exPlanations (SHAP) are presented in this study. Three additional perturbations were introduced to this balanced gameplay and the observations about important features of the game using SHAP are presented. Our results show that this analysis can successfully predict probability of win for self-play microRTS games, as well as capture uncertainty in predictions that can be used to guide additional data collection to improve the model, or refine the game balance measure.