Improving Air-to-Air Combat Behavior through Transparent Machine Learning

Training simulations, especially those for tactical training, require properly behaving computer generated forces (CGFs) in the opponent role for an effective training experience. Traditionally, the behavior of such CGFs is controlled through scripts. There are two main problems with the use of scripts for controlling the behavior of CGFs: (1) building an effective script requires expert knowledge, which is costly, and (2) costs further increase with the number of ‘learning events’ in a scenario (e.g. a new opponent tactic). Machine learning techniques may offer a solution to these two problems, by automatically generating, evaluating and improving CGF behavior. In this paper we describe an application of the dynamic scripting technique to the generation of CGF behavior for training simulations. Dynamic scripting is a machine learning technique that searches for effective scripts by combining rules from a rule base with predefined behavior rules. Although dynamic scripting was initially developed for artificial intelligence (AI) in commercial video games, its computational and functional qualities are also desirable in military training simulations. Among other qualities, dynamic scripting generates behavior in a transparent manner. Also, dynamic scripting’s learning method is robust: a minimum level of effectiveness is guaranteed through the use of domain knowledge in the initial rule base. In our research, we investigate the application of dynamic scripting for generating behaviors of multiple cooperating aircraft in air-to-air combat. Coordination in multi-agent systems remains a non-trivial problem. We enabled explicit team coordination through communication between team members. This coordination method was tested in an air combat simulation experiment, and compared against a baseline that consisted of a similar dynamic scripting setup, without explicit coordination. In terms of combat performance, the team using the explicit team coordination was 20% more effective than the baseline. Finally, the paper will discuss the application of dynamic scripting in a practical setting.