Search

BACKGROUND: Fighter pilots’ Team Situation Awareness (TSA) has been studied from the perspective of TSA accuracy, which represents how closely the pilots’ collective knowledge is aligned with the real world. When TSA accuracy is low, the pilots can have similarly or dissimilarly inaccurate SA. The concept of TSA similarity represents the similarity of team members’ collective knowledge. This paper investigates how TSA accuracy and similarity of F/A-18 pilots are associated with performance.METHOD: Data were extracted from simulated air combat missions. Performance and TSA were investigated in 58 engagements. The accuracy and similarity of pilots’ SA were elicited and performance was evaluated. TSA accuracy and similarity were analyzed with respect to the flights’ performance, and the independent variables were events in which the flights initiated engagements with enemy aircraft versus events in which the flights were engaged by enemy aircraft.RESULTS: With the mentioned events as the main effect, there were statistically significant differences at all levels of TSA accuracy and similarity. With performance as the main effect, there were also significant differences at all levels of TSA accuracy and similarity. TSA accuracy and similarity were superior in offensive engagements and when engagements were successful.DISCUSSION: The main contribution of this paper is the extension of the concept of TSA similarity to air combat: both TSA similarity and accuracy were higher when the flight was engaging the enemy aircraft, compared to situations when the flight itself was being engaged. The results also suggest that low TSA accuracy and similarity have a statistically significantly negative impact on the flights’ performance.Mansikka H, Harris D, Virtanen K. Accuracy and similarity of team situation awareness in simulated air combat. Aerosp Med Hum Perform. 2023; 94(6):429–436.

BACKGROUND: Normative performance (NP) describes the pilots’ adherence to tactics, techniques, and procedures (TTPs). Until now, there has not been a global NP measurement technique for beyond visual range (BVR) air combat, and the methodology and technology related to the evaluation of NP have fallen behind the pace of the overall technical progress of distributed mission operations (DMO) training.METHODS: Platform-independent core air combat tasks were identified. The execution of these tasks is directed with TTPs. BVR air combat missions were flown in a DMO simulator system and the design NP was varied between missions. Observers viewed debriefs of these missions and attempted to identify TTP-regulated air combat tasks. Once identified, they scored the pilots’ NP in those tasks. The scoring was based on the level of TTP adherence and the impact a nonadherence had on the mission accomplishment.RESULTS: All observers were able to identify most of the TTP-regulated air combat tasks. There was a strong positive correlation between the observed and design NP scores. The overall Kappa indicated a fair agreement between the observers. The percentage of observers’ NP assessments which agreed with the design NP varied from 49.60 to 85.28% in different air combat missions. On average, 73.96% of the observers’ NP scores agreed with the design NP scores.CONCLUSIONS: Observers were able to accurately identify TTP-regulated tasks and score NP of these tasks during an air combat debrief. There was a moderate agreement between the observers’ NP scores.Mansikka H, Virtanen K, Mäkinen L, Harris D. Normative performance measurement in simulated air combat. Aerosp Med Hum Perform. 2021; 92(11):908–912.

In air combat, a traditional way of evaluating team’s taskwork performance is to measure its performance output. However, it provides a narrow view about the team’s performance and potentially miss...

This paper provides new insight into the theoretical principle and the practical applicability of the ranked nodes method (RNM) that is used to construct conditional probability tables (CPTs) for Bayesian networks (BNs) by expert elicitation. RNM is designed for specific types of discrete random variables called ranked nodes that are common in real-world applications of BNs. Despite its active use in recent years, there remains ambiguity about the exact theoretical basis of RNM which can hamper its effective employment. In addition, there are a lack of studies about the general ability of CPTs generated with RNM to represent probabilistic relationships in real-world applications. In this paper, it is shown how the generation of probabilities with RNM is underpinned by a regression model of continuous random variables. Then, it is experimentally determined that in typical applications of RNM, one can generate in a matter of seconds CPTs whose elements reflect well probabilities given by the underlying regression model. Another experiment discovers that CPTs generated with RNM provide a good average fit to a large portion of various real-world CPTs investigated. This confirms the usefulness of RNM in practical applications. The results of the experiment also indicate that choices made by the user of RNM can considerably impact the ability of a generated CPT to represent a given probabilistic relationship. This paper then provides practical advice on the efficient use of RNM with regard to the user-controlled features explored in the experiment.

In this paper, the use of the live (L), virtual (V), and constructive (C) simulation framework introduced in Part 1 of this two-part study is demonstrated in the testing and evaluation of air combat tactics, techniques, and procedures (TTP). Each TTP consists of rules that describe how aircraft pilots coordinate their actions to achieve goals in air combat. In the demonstration, the initial rules are defined by subject matter experts (SMEs). These rules are refined iteratively in separate C-, V-, and L-simulation stages. In the C-stage, an operationally used C-simulation model is used to provide optimal rules with respect to the probabilities of survival ( Ps) and kill ( Pk) of aircraft without considering human–machine interaction (HMI). In the V-stage, fighter squadrons’ V-simulators and SMEs’ assessment are used to modify these rules by evaluating their applicability with Pk and Ps, as well as HMI measures regarding pilots’ situation awareness, mental workload, and TTP rule adherence. In the L-stage, qualified fighter pilots fly F/A-18C aircraft in a real-life environment. Based on SMEs’ assessment, the TTP rules refined in the C- and L-stages result in acceptable Pk, Ps, and HMI measures in the L-stage. As such, the demonstration highlights the utility of the LVC framework.

This paper advances live (L), virtual (V), and constructive (C) simulation methodologies by introducing a new LVC simulation framework for the development of air combat tactics, techniques, and procedures (TTP). In the framework, TTP is developed iteratively in separate C-, V-, and L-simulation stages. This allows the utilization of the strengths of each simulation class while avoiding the challenges of pure LVC simulations. The C-stage provides the optimal TTP with respect to the probabilities of survival ( Ps) and kill ( Pk) of aircraft without considering the human–machine interaction (HMI). In the V-stage, the optimal TTP is modified by assessing its applicability with Pk and Ps, as well as HMI measures regarding pilots’ situation awareness, mental workload, and TTP adherence. In the L-stage, real aircraft are used to evaluate whether the developed TTP leads to acceptable Pk, Ps, and HMI measures in a real-life environment. The iterative nature of the framework enables that V- or L-stages can reveal flaws of the TTP and an inadequate TTP can be returned to the C- or V-stage for revision. This paper is Part 1 of a two-part study. Part 2 demonstrates the use of the framework with operationally used C- and V-simulators as well as real F/A-18C aircraft and pilots.

The purpose of this study was to demonstrate how the inclusion of a tactical task goal awareness measure complemented mental workload and performance measures in a simulated air combat mission. It was hypothesized that the evaluation of the tactical task goal awareness could provide additional information concerning the cognitive demands a pilot is exposed to during a complex air combat task. A test setting was developed to test this hypothesis in a virtual flight training device. To highlight the impact of task complexity, high performance aircraft pilots’ heart rate (HR), interbeat-interval (IBI), and performance in two simple flying tasks were first compared. Then, a similar comparison, complemented with the tactical task goal awareness measure, was made with two complex flying tasks. It was found that when the pilot's awareness of the tactical goals was low, a combination of low performance and low mental workload occurred. It was concluded that when the pilots’ performance is evaluated on a complex air combat task, the awareness of the tactical goals, performance, and mental workload should be studied together as the pilot's awareness can explain some of HR/IBI responses that could be otherwise misinterpreted. More generally, mental workload, performance, and task goal awareness should all be considered when the operator's performance in any complex human–machine system is assessed.

Publisher Copyright: © 2021 The Authors This paper advances the use of the ranked nodes method (RNM) to portray probabilistic relationships of continuous quantities in Bayesian networks (BNs). In RNM, continuous quantities are represented by ranked nodes with discrete ordinal scales. The probabilistic relationships of the nodes are quantified in conditional probability tables (CPTs) generated with expert-elicited parameters. When ranked nodes are formed by discretizing continuous scales, ignorance about the functioning of RNM can lead to discretizations that make the generation of sensible CPTs impossible. While a guideline exists on this matter, it is limited by a requirement to define an equal number of ordinal states for all the nodes. This paper presents two novel discretization approaches that consider the functioning of RNM and allow the nodes to have non-equal numbers of ordinal states. In the first one, called the “static discretization approach”, the nodes can be given any desired discretizations that stay unchanged during the use of the BN. In the second one, called the “dynamic discretization approach”, the discretizations are algorithmically updated during the use of the BN to help manage the sizes of the generated CPTs. Both approaches are based on the original idea that, besides the RNM parameters, the nodes probabilistic relationship is defined by initial RNM-compatible discretizations elicited from the domain expert. Overall, the new approaches offer an easier and more versatile way of using RNM to depict the probabilistic relationships of continuous quantities. In doing so, they also facilitate the effective and diverse use of BNs in decision support systems.

Abstract. The aim of this study was to investigate the relationship between the flight-related core competencies for professional airline pilots and to structuralize them as components in a team performance framework. To achieve this, the core competency scores from a total of 2,560 OPC (Operator Proficiency Check) missions were analyzed. A principal component analysis (PCA) of pilots’ performance scores across the different competencies was conducted. Four principal components were extracted and a path analysis model was constructed on the basis of these factors. The path analysis utilizing the core competencies extracted adopted an input–process–output’ (IPO) model of team performance related directly to the activities on the flight deck. The results of the PCA and the path analysis strongly supported the proposed IPO model.

The essential skills underlying crew resource management (CRM) are described in the competency frameworks published by the International Civil Aviation Organization and International Air Transport ...

The optimal use of a surveillance radar system requires proper understanding about the system behavior in different configurations, modes, and operating conditions. This paper proposes a machine learning framework for producing and validating the performance model of the surveillance radar system. The framework consists of an optimization method for the parameterization of a radar model and a machine learning method for the modeling of a tracker. Optimization and machine learning is based on the satellite navigation data of cooperative aircraft and corresponding track data from the surveillance system. The aim is to learn the system performance in a wide range of operating conditions using the extensive measurement history and then to predict the present performance with high accuracy at specified locations in the airspace. The feasibility of the proposed framework is assessed using real data. acceptedVersion

The objective of the radar performance prediction is to compute quantities of interest concerning the ability of the radar to observe its surroundings. Besides the properties of the radar system, the performance is affected by the target, whose radar cross section (RCS) is one of the predominant factors. The performance prediction is usually performed in relation to the target RCS characterized by a constant value or a particular statistical distribution. Such representations generalize real-life complex targets rendering them unsuitable for some objectives since the RCS is significantly influenced by the target aspect angle and is inherently stochastic by nature. Thus, a more dynamic description may be valuable e.g. for analyzing the radar performance on a flight path of interest. We propose representing the RCS with a histogram that includes such dynamic properties and is suitable for considering the target in different ways for performance prediction: in a more general manner or dependent on its aspect angle. We consider the case of traditional RCS with low spatial resolution and demonstrate the proposed approach through the probability of detection computed for a generic surveillance radar. acceptedVersion

This paper introduces the use of Bayesian networks (BNs) as an exploratory metamodelling tool for supporting simulation studies conducted with stochastic simulation models containing multiple inputs and outputs. BN metamodels combine simulation data with available expert knowledge into a non-parametric description of the joint probability distribution of discrete random variables representing simulation inputs and outputs. The distributions of the inputs are determined based on expert knowledge and/or a real-world data source while the conditional distributions of the outputs are estimated from the simulation data. The exploratory use of the BN metamodels is an iterative process including the construction and validation of the BNs and allowing various analyses dealing with the dependencies among the inputs and the outputs, input uncertainty, and inverse reasoning. The results of these analyses are applied to guide and aid the utilization and interpretation of the simulation model under consideration. In addition, the analyses are used for studying the behaviour of the simulated system. The exploratory use is illustrated with an example involving a simulated queue.

Fighter pilots' heart rate (HR), heart rate variation (HRV) and performance during instrument approaches were examined. The subjects were required to fly instrument approaches in a high-fidelity simulator under various levels of task demand. The task demand was manipulated by increasing the load on the subjects by reducing the range at which they commenced the approach. HR and the time domain components of HRV were used as measures of pilot mental workload (PMWL). The findings of this study indicate that HR and HRV are sensitive to varying task demands. HR and HRV were able to distinguish the level of PMWL after which the subjects were no longer able to cope with the increasing task demands and their instrument landing system performance fell to a sub-standard level. The major finding was the HR/HRV's ability to differentiate the sub-standard performance approaches from the high-performance approaches. Practitioner Summary: This paper examined if HR and HRV were sensitive to varying task demands in a fighter aviation environment and if these measures were related to variations in pilot's performance.

This paper presents a new simulated annealing (SA) algorithm for noisy multiobjective optimization with continuous decision variables. A novel feature of the algorithm in the context of SA is that the performance of a candidate solution is determined by estimating the probabilities that the candidate is dominated by the current non-dominated solutions. The sum of these probabilities provides a scalar performance measure that is used to determine the acceptance of the candidate as the current solution and whether the candidate is inserted into the non-dominated set. The second novel feature of the algorithm is the technique utilized for generating candidate solutions. Empirical probability distributions for sampling the new values of the decision variables are constructed on the basis of the values of the variables in the current non-dominated set. Thus, the information contained by the non-dominated set is utilized to improve the quality of the generated candidates, whereas this information is ignored in the existing multiobjective SA algorithms. The proposed algorithm is compared with a reference state-of-the-art evolutionary algorithm as well as two other SA algorithms in numerical experiments involving 16 problems from commonly applied test suites. The proposed algorithm performs as good or better compared with the reference algorithms in majority of the experiments and therefore represents a promising solution method for noisy multiobjective optimization problems. Copyright © 2012 John Wiley & Sons, Ltd.

This paper presents a novel approach to simulation metamodeling using dynamic Bayesian networks (DBNs) in the context of discrete event simulation. A DBN is a probabilistic model that represents the joint distribution of a sequence of random variables and enables the efficient calculation of their marginal and conditional distributions. In this paper, the construction of a DBN based on simulation data and its utilization in simulation analyses are presented. The DBN metamodel allows the study of the time evolution of simulation by tracking the probability distribution of the simulation state over the duration of the simulation. This feature is unprecedented among existing simulation metamodels. The DBN metamodel also enables effective what-if analysis which reveals the conditional evolution of the simulation. In such an analysis, the simulation state at a given time is fixed and the probability distributions representing the state at other time instants are updated. Simulation parameters can be included in the DBN metamodel as external random variables. Then, the DBN offers a way to study the effects of parameter values and their uncertainty on the evolution of the simulation. The accuracy of the analyses allowed by DBNs is studied by constructing appropriate confidence intervals. These analyses could be conducted based on raw simulation data but the use of DBNs reduces the duration of repetitive analyses and is expedited by available Bayesian network software. The construction and analysis capabilities of DBN metamodels are illustrated with two example simulation studies.

We used discrete-event simulation to model the maintenance of fighter aircraft and improve maintenance-related decision making within the Finnish Air Force. We implemented the simulation model as a stand-alone tool that maintenance designers could use independently. The model has helped the designers to study the impact of maintenance resources, policies, and operating conditions on aircraft availability. It has also enabled the Finnish Air Force to advance the operational capability of its aircraft fleet. We designed the model to simulate both normal and conflict operating conditions. The main challenge of the project was the scarcity and confidentiality of data about the fighter aircraft, their maintenance, and various operational scenarios, especially during conflict situations.

The paper introduces a receding horizon control scheme for obtaining near-optimal controls in a feedback form for an aircraft trying to avoid a closing air-to-air missile. The vehicles are modeled as point-masses. Rotation kinematics of the aircraft are taken into account by limiting the pitch and roll rates as well as the angular accelerations of the angle of attack and the bank angle. The missile utilizes proportional navigation and it has a boost-sustain propulsion system. In the proposed scheme, the optimal controls of the aircraft over a short planning horizon are solved on-line by the direct shooting method at each decision instant. Thereafter, the state of the system is updated by using only the first controls in the sequence, and the process is repeated. The performance measure defining the objective of the aircraft can be chosen freely. In this paper, six performance measures consisting of the capture time, closing velocity, miss distance, gimbal angle, tracking rate, and control effort of the missile are considered. The quality of the receding horizon solutions computed by the scheme is validated by comparing them to the off-line computed optimal open-loop solutions.

This paper formulates a support time game arising in one-on-one air combat with medium range air-to-air missiles. The game model provides game optimal support times of the missiles that can receive target information from the launching aircraft for selectable support times. The payoffs of the game are formulated as aweighted sumof the probabilities of hit to the adversary and own survival. Under suitable simplifying assumptions, a Nash equilibrium of the game can be computed by an iterative search involving a series of optimal control problems. For practical situations, an approximate real time computation scheme is introduced. The constructed model and the scheme are illustrated by numerical examples.

Thepaperdescribesamultistageinfluencediagramgameformodelingthemaneuveringdecisionsofpilotsinoneon-one air combat. The game graphically describes the elements of the decision process, contains a model for the dynamics of the aircraft, and takes into account the pilots’ preferences under conditions of uncertainty. The pilots’ game optimal control sequences with respect to their preference models are obtained by solving the influence diagram game with a moving horizon control approach. In this approach, the time horizon of the original game is truncated, and a feedback Nash equilibrium of the dynamic game lasting only a limited planning horizon is determined and implemented at each decision stage. To demonstrate the influence diagram game and its aspects, exampleswithapointmassaircraftmodelarecomputedandanalyzed.Thegamemodelpresentedinthepaperoffers a new way to analyze optimal air combat maneuvering and to develop an automated decision making system for selecting combat maneuvers in air combat simulators.

The paper presents an approach toward the off-line computation of preference optimal flight paths against given air combat maneuvers. The approach consists of a multistage influence diagram modeling the pilot’s sequential maneuvering decisions and a solution procedure that utilizes nonlinear programming. The influence diagram graphically describes the elements of the decision process, contains a point-mass model for the dynamics of an aircraft, and takes into account the decision maker’s preferences under conditions of uncertainty. Optimal trajectories with respect to the given preference model are obtained by converting the multistage influence diagram into a discrete-time dynamic optimization problem that is solved with nonlinear programming. The initial estimate for the decision variables of the problem is generated by solving a set of myopic single stage influence diagrams that anticipate the future state of the aircraft only a short planning horizon ahead. The presented solution procedure is illustrated by a numerical example.

We simulate and analyze pilot decision making in one-on-one air combat by using an ine uence diagram. Unlike most of the existing approaches, an ine uence diagram graphically describes the factors of a decision process and explicitly handles the decision maker’ s preferences under conditions of uncertainty. In the pilot decision model, the possible combat situations related to each maneuver alternative are associated with a probability and a utility. Ine uence diagram analysis produces a probability distribution of the overall utility that represents the successfulness of a maneuver and gives information to make rational maneuvering decisions. Sensitivity analysis determines the impacts of different factors on the outcome of the maneuvering decision. The effects of sensor information that will reduce the uncertainty of the model are evaluated with Bayesian reasoning. The model can be utilized in the analysis of a single decision situation or as an automated decision-making system that selects combat maneuvers in air combat simulators.

An adaptive controller for the avoidance of an air combat missile utilizing a guidance law unknown to the target is introduced. The controls of the aircraft are determined at discrete instants using a receding horizon control scheme providing near-optimal feedback controls of the aircraft against a closing missile. The controls are optimized with respect to the expected final distance between the vehicles. In the controller, the target's belief in the guidance law of the missile is represented as a discrete probability distribution over a set of guidance laws. As the missile closes on the aircraft, the probability distribution is updated on the basis of the realized vehicle trajectories using Bayesian reasoning. The controller is demonstrated with numerical examples.

An approach towards automated solution of aircraft trajectory optimization problems is introduced and applied to an interactive MS-Windows software called 'Visual Interactive Aircraft Trajectory Optimization' (VIATO). The software solves, e.g., minimum time trajectories to a fixed or to a moving target. It consists of an automated optimization routine and a graphical user interface. The software is easy-to-use and thus its user does not need to know optimal control theory and mathematical modeling. Most mathematical models of flight vehicles are structurally similar and differ only in parameters. In VIATO software the equation of motion and the state and the control constraints are fixed in advance and different aircraft types are represented as a set of parameters. Specifying also objective functions of the optimal control problems the explicit formulation of optimal control problems is totally avoided. Reliable convergence of solution methods is achieved by replacing the original infinite dimensional problem with a finite dimensional approximation. The approximation is solved using nonlinear programming.

Modern fighter aircraft are equipped with a data link system for relaying encrypted messages containing information on the states of friendly or hostile aircraft within a team of friendly aircraft. During a many-versus-many air combat, only the most important and essential information must be transmitted in order to achieve the best possible situation awareness of pilots. In this paper, we introduce a prioritization approach for solving the team optimal signaling strategy referring to the best sequence of relayed information with respect to the overall goals of the friendly pilots. The approach is based on a value function that captures the preferences of the pilots and prioritizes the existing information. Uncertainty about the states of the aircraft as well as about incomplete preference statements is incorporated into the prioritization by utilizing interval analysis. We also describe a software that is designed for simulating the evolution of team members' information and trajectories in an air-combat game. The prioritization approach is illustrated by numerical examples

We apply an influence diagram to model pilot decision making in one versus one air combat. An influence diagram graphically describes the factors of a decision process under uncertainty and takes into account all available information to provide logical decisions. In the constructed model, the possible combat situations related to each maneuver alternative are associated with a probability and a utility. Influence diagram analysis produces probability distributions of the overall utility that are used in selecting the preferred maneuver. Sensitivity analysis determines the impacts of different factors on the outcome of the maneuvering decision. The effects of sensor information that will reduce the uncertainty of the model, are evaluated using Bayesian reasoning. The model can be utilized in the analysis of a single decision situation or as an automated decision making system that selects combat maneuvers in air combat simulators.