Search

BACKGROUND: Fighter pilots’ Team Situation Awareness (TSA) has been studied from the perspective of TSA accuracy, which represents how closely the pilots’ collective knowledge is aligned with the real world. When TSA accuracy is low, the pilots can have similarly or dissimilarly inaccurate SA. The concept of TSA similarity represents the similarity of team members’ collective knowledge. This paper investigates how TSA accuracy and similarity of F/A-18 pilots are associated with performance.METHOD: Data were extracted from simulated air combat missions. Performance and TSA were investigated in 58 engagements. The accuracy and similarity of pilots’ SA were elicited and performance was evaluated. TSA accuracy and similarity were analyzed with respect to the flights’ performance, and the independent variables were events in which the flights initiated engagements with enemy aircraft versus events in which the flights were engaged by enemy aircraft.RESULTS: With the mentioned events as the main effect, there were statistically significant differences at all levels of TSA accuracy and similarity. With performance as the main effect, there were also significant differences at all levels of TSA accuracy and similarity. TSA accuracy and similarity were superior in offensive engagements and when engagements were successful.DISCUSSION: The main contribution of this paper is the extension of the concept of TSA similarity to air combat: both TSA similarity and accuracy were higher when the flight was engaging the enemy aircraft, compared to situations when the flight itself was being engaged. The results also suggest that low TSA accuracy and similarity have a statistically significantly negative impact on the flights’ performance.Mansikka H, Harris D, Virtanen K. Accuracy and similarity of team situation awareness in simulated air combat. Aerosp Med Hum Perform. 2023; 94(6):429–436.

BACKGROUND: Normative performance (NP) describes the pilots’ adherence to tactics, techniques, and procedures (TTPs). Until now, there has not been a global NP measurement technique for beyond visual range (BVR) air combat, and the methodology and technology related to the evaluation of NP have fallen behind the pace of the overall technical progress of distributed mission operations (DMO) training.METHODS: Platform-independent core air combat tasks were identified. The execution of these tasks is directed with TTPs. BVR air combat missions were flown in a DMO simulator system and the design NP was varied between missions. Observers viewed debriefs of these missions and attempted to identify TTP-regulated air combat tasks. Once identified, they scored the pilots’ NP in those tasks. The scoring was based on the level of TTP adherence and the impact a nonadherence had on the mission accomplishment.RESULTS: All observers were able to identify most of the TTP-regulated air combat tasks. There was a strong positive correlation between the observed and design NP scores. The overall Kappa indicated a fair agreement between the observers. The percentage of observers’ NP assessments which agreed with the design NP varied from 49.60 to 85.28% in different air combat missions. On average, 73.96% of the observers’ NP scores agreed with the design NP scores.CONCLUSIONS: Observers were able to accurately identify TTP-regulated tasks and score NP of these tasks during an air combat debrief. There was a moderate agreement between the observers’ NP scores.Mansikka H, Virtanen K, Mäkinen L, Harris D. Normative performance measurement in simulated air combat. Aerosp Med Hum Perform. 2021; 92(11):908–912.

In air combat, a traditional way of evaluating team’s taskwork performance is to measure its performance output. However, it provides a narrow view about the team’s performance and potentially miss...

Purpose The purpose of this paper is to propose and test a multi-criteria decision analysis (MCDA) approach based on an additive value function (AVF) to select the most economically advantageous tender under European Union public procurement regulations. Design/methodology/approach A case study in which the AVF tender evaluation model is constructed by the procurement personnel and the results of the original, real-life public procurement evaluation model are compared to those discovered by the MCDA approach. Findings The AVF model captures the preferences of the procurement authority in a more reliable and transparent manner than commonly used evaluation models based on scoring formulas. Practical implications While commonly used in public procurement, relative scoring formulas can neither present the preferences of a procurement unit accurately nor do they enable bidders to draft bids according to these preferences. The proposed MCDA approach can achieve both. Originality/value The contribution of this paper is threefold. First, the successful construction of the AVF model with procurement personnel is introduced. Second, the model is used in an actual, real-life case. Third, a thoughtful comparison of features, structures and results of the AVF model and the evaluation model using scoring formulas is presented.

Publisher Copyright: © 2021 A new technique for the assessment of Team Situation Awareness (TSA) accuracy based upon post task Critical Decision Method structured interviews was developed and tested using 39 combat-ready F/A-18 pilots. Pilots undertook a number of simulated air combat scenarios, flying in flights of four aircraft against a formation of enemy aircraft. Results showed a strong curvilinear relationship where high TSA accuracy resulted in higher performance in some areas of air combat, measured with friendly losses and kills. There were diminishing returns in performance as TSA accuracy increased. This may explain why previous studies on air combat have found relatively weak relationships between situation awareness and performance where the relationship has been assumed to be linear.

In this paper, the use of the live (L), virtual (V), and constructive (C) simulation framework introduced in Part 1 of this two-part study is demonstrated in the testing and evaluation of air combat tactics, techniques, and procedures (TTP). Each TTP consists of rules that describe how aircraft pilots coordinate their actions to achieve goals in air combat. In the demonstration, the initial rules are defined by subject matter experts (SMEs). These rules are refined iteratively in separate C-, V-, and L-simulation stages. In the C-stage, an operationally used C-simulation model is used to provide optimal rules with respect to the probabilities of survival ( Ps) and kill ( Pk) of aircraft without considering human–machine interaction (HMI). In the V-stage, fighter squadrons’ V-simulators and SMEs’ assessment are used to modify these rules by evaluating their applicability with Pk and Ps, as well as HMI measures regarding pilots’ situation awareness, mental workload, and TTP rule adherence. In the L-stage, qualified fighter pilots fly F/A-18C aircraft in a real-life environment. Based on SMEs’ assessment, the TTP rules refined in the C- and L-stages result in acceptable Pk, Ps, and HMI measures in the L-stage. As such, the demonstration highlights the utility of the LVC framework.

This paper advances live (L), virtual (V), and constructive (C) simulation methodologies by introducing a new LVC simulation framework for the development of air combat tactics, techniques, and procedures (TTP). In the framework, TTP is developed iteratively in separate C-, V-, and L-simulation stages. This allows the utilization of the strengths of each simulation class while avoiding the challenges of pure LVC simulations. The C-stage provides the optimal TTP with respect to the probabilities of survival ( Ps) and kill ( Pk) of aircraft without considering the human–machine interaction (HMI). In the V-stage, the optimal TTP is modified by assessing its applicability with Pk and Ps, as well as HMI measures regarding pilots’ situation awareness, mental workload, and TTP adherence. In the L-stage, real aircraft are used to evaluate whether the developed TTP leads to acceptable Pk, Ps, and HMI measures in a real-life environment. The iterative nature of the framework enables that V- or L-stages can reveal flaws of the TTP and an inadequate TTP can be returned to the C- or V-stage for revision. This paper is Part 1 of a two-part study. Part 2 demonstrates the use of the framework with operationally used C- and V-simulators as well as real F/A-18C aircraft and pilots.

The purpose of this study was to demonstrate how the inclusion of a tactical task goal awareness measure complemented mental workload and performance measures in a simulated air combat mission. It was hypothesized that the evaluation of the tactical task goal awareness could provide additional information concerning the cognitive demands a pilot is exposed to during a complex air combat task. A test setting was developed to test this hypothesis in a virtual flight training device. To highlight the impact of task complexity, high performance aircraft pilots’ heart rate (HR), interbeat-interval (IBI), and performance in two simple flying tasks were first compared. Then, a similar comparison, complemented with the tactical task goal awareness measure, was made with two complex flying tasks. It was found that when the pilot's awareness of the tactical goals was low, a combination of low performance and low mental workload occurred. It was concluded that when the pilots’ performance is evaluated on a complex air combat task, the awareness of the tactical goals, performance, and mental workload should be studied together as the pilot's awareness can explain some of HR/IBI responses that could be otherwise misinterpreted. More generally, mental workload, performance, and task goal awareness should all be considered when the operator's performance in any complex human–machine system is assessed.

This paper advances the theoretical foundations and the methodology of spatial decision analysis in which multi-attribute consequences of decision alternatives vary over a spatial region. First, we introduce necessary and sufficient conditions for representing the decision maker’s preferences among such decision alternatives with an additive spatial value function. This new axiomatization allows for the representation of preferences when the spatial region consists of an infinite number of locations, which is often the case in practical applications. Moreover, we show that spatial value functions suggested in the existing literature can be interpreted as special cases of our additive spatial value function. Second, motivated by the high effort required to elicit preferences in spatial decision problems, we develop a method for utilizing the additive spatial value function with incomplete preference information about spatial weights describing the importance of locations and attribute weights. This method provides defensible decision recommendations through the use of dominance concepts and decision rules. The applicability of the developed value function and analysis method is illustrated with a real-life application in air defense planning.

Abstract. The aim of this study was to investigate the relationship between the flight-related core competencies for professional airline pilots and to structuralize them as components in a team performance framework. To achieve this, the core competency scores from a total of 2,560 OPC (Operator Proficiency Check) missions were analyzed. A principal component analysis (PCA) of pilots’ performance scores across the different competencies was conducted. Four principal components were extracted and a path analysis model was constructed on the basis of these factors. The path analysis utilizing the core competencies extracted adopted an input–process–output’ (IPO) model of team performance related directly to the activities on the flight deck. The results of the PCA and the path analysis strongly supported the proposed IPO model.

Games with incomplete preferences are normal-form games where the preferences of the players are defined as partial orders over the outcomes of the game. We define rationality in these games as follows. A rational player forms a set-valued belief of possible strategies selected by the opponent(s) and selects a strategy that is not dominated with respect to this belief. Here, we say a strategy is dominated with respect to the set-valued belief if the player has another strategy that would yield a better outcome according to the player's preference relation, no matter which strategy combination the opponent(s) play among those contained in the belief. We define rationalizable strategies as the logical implication of common knowledge of this rationality. We show that the sets of rationalizable strategies are the maximal mutually nondominated sets, i.e., the maximal sets that contain no dominated strategies with respect to each other. We show that no new rationalizable strategies appear when additional preference information is included. We consider multicriteria games as a special case of games with incomplete preferences and introduce a way of representing incomplete preference information in multicriteria games by sets of feasible weights of the criteria.

This paper introduces a novel approach for controlling the exterior ballistic properties of spin-stabilized bullets by optimizing their internal mass distributions. Specifically, the properties of interest are the bullets’ stability characteristics that are examined through dynamic and gyroscopic stability parameters. New analytical expressions for aerodynamic quantities are also derived to address the compressibility of air. These expressions are utilized in a bullet model that enables efficient computation of the stability parameters for a given mass distribution. The bullet model is used in the formulation of nonlinear optimization problems that provide optimal mass distributions with respect to given goals, i.e., desired stability characteristics. The bullet types investigated in this paper are a long range bullet and a limited range training bullet. In the optimization of the mass distribution of the long range bullet, the goal is that the bullet stays stable for as long as possible. The mass distribution of the training bullet is optimized such that the bullet is stable at launch but becomes unstable shortly afterwards. The global optimal solutions obtained with the new approach fulfill the desired stability characteristics better than currently used uniformly filled bullets. Overall, the optimization approach reveals a new goal focused philosophy for bullet design compared to current trial and error design practices. Keywords: Aerodynamics, Bullet, Exterior ballistics, Mass properties, Optimization, Stability

The essential skills underlying crew resource management (CRM) are described in the competency frameworks published by the International Civil Aviation Organization and International Air Transport ...

The utilization of dynamic Bayesian networks (DBNs) in simulation metamodeling enables the investigation of the time evolution of state variables of a simulation model. DBN metamodels have previously described the changes in the probability distribution of the simulation state by using a time slice structure in which the state variables are described at common time instants. In this paper, the novel approach to the determination of the time slice structure is introduced. It enables the selection of time instants of the DBN separately for each state variable. In this way, a more accurate metamodel representing multiple time scales of the variables is achieved. Furthermore, the construction is streamlined by presenting a dynamic programming algorithm for determining the key time instants for individual variables. The construction and use of the DBN metamodels are illustrated by an example problem dealing with the simulated operation of an air base.

The sensitivity of NASA-TLX scale, modified Cooper-Harper (MCH) scale and the mean inter-beat interval (IBI) of successive heart beats, as measures of pilot mental workload (MWL), were evaluated in a flight training device (FTD). Operational F/A-18C pilots flew instrument approaches with varying task loads. Pilots' performance, subjective MWL ratings and IBI were measured. Based on the pilots' performance, three performance categories were formed; high-, medium- and low-performance. Values of the subjective rating scales and IBI were compared between categories. It was found that all measures were able to differentiate most task conditions and there was a strong, positive correlation between NASA-TLX and MCH scale. An explicit link between IBI, NASA-TLX, MCH and performance was demonstrated. While NASA-TLX, MCH and IBI have all been previously used to measure MWL, this study is the first one to investigate their association in a modern FTD, using a realistic flying mission and operational pilots. Practitioner summary: NASA-TLX scale, MCH scale and the IBI were evaluated in a flight training device. All measures were able to differentiate most task conditions and there was a positive correlation between NASA-TLX and MCH scale. An explicit link between IBI, NASA-TLX, MCH and performance was demonstrated. Abbreviations: ANOVA: Analysis of Variance; ECG: Electrocardiograph; F/A: fighter/attack; ft: feet; FTD: flight training device; G: Gravity; km: kilometer; m: meter; m/s: meters per second; MWL: mental workload; MCH: modified cooper-harper; NASA-TLX: NASA Task Load Index; NM: Nautical Mile; NN: normal-to-normal; IBI: inter-beat interval; ILS: Instrument Landing System; RR: R-Wave to R-Wave; SD: standard deviation; TTP: tactics, techniques and procedures; WTSAT: Weapon Tactics and Situation Awareness Trainer.

Increased task demand will increase the pilot mental workload (PMWL). When PMWL is increased, mental overload may occur resulting in degraded performance. During pilots' instrument flight rules (IFR) proficiency test, PMWL is typically not measured. Therefore, little is known about workload during the proficiency test and pilots' potential to cope with higher task demands than those experienced during the test. In this study, fighter pilots' performance and PMWL was measured during a real IFR proficiency test in an F/A-18 simulator. PMWL was measured using heart rate (HR) and heart rate variation (HRV). Performance was rated using Finnish Air Force's official rating scales. Results indicated that HR and HRV differentiate varying task demands in situations where variations in performance are insignificant. It was concluded that during a proficiency test, PMWL should be measured together with the task performance measurement.

The optimal use of a surveillance radar system requires proper understanding about the system behavior in different configurations, modes, and operating conditions. This paper proposes a machine learning framework for producing and validating the performance model of the surveillance radar system. The framework consists of an optimization method for the parameterization of a radar model and a machine learning method for the modeling of a tracker. Optimization and machine learning is based on the satellite navigation data of cooperative aircraft and corresponding track data from the surveillance system. The aim is to learn the system performance in a wide range of operating conditions using the extensive measurement history and then to predict the present performance with high accuracy at specified locations in the airspace. The feasibility of the proposed framework is assessed using real data. acceptedVersion

The objective of the radar performance prediction is to compute quantities of interest concerning the ability of the radar to observe its surroundings. Besides the properties of the radar system, the performance is affected by the target, whose radar cross section (RCS) is one of the predominant factors. The performance prediction is usually performed in relation to the target RCS characterized by a constant value or a particular statistical distribution. Such representations generalize real-life complex targets rendering them unsuitable for some objectives since the RCS is significantly influenced by the target aspect angle and is inherently stochastic by nature. Thus, a more dynamic description may be valuable e.g. for analyzing the radar performance on a flight path of interest. We propose representing the RCS with a histogram that includes such dynamic properties and is suitable for considering the target in different ways for performance prediction: in a more general manner or dependent on its aspect angle. We consider the case of traditional RCS with low spatial resolution and demonstrate the proposed approach through the probability of detection computed for a generic surveillance radar. acceptedVersion

Fighter pilots' heart rate (HR), heart rate variation (HRV) and performance during instrument approaches were examined. The subjects were required to fly instrument approaches in a high-fidelity simulator under various levels of task demand. The task demand was manipulated by increasing the load on the subjects by reducing the range at which they commenced the approach. HR and the time domain components of HRV were used as measures of pilot mental workload (PMWL). The findings of this study indicate that HR and HRV are sensitive to varying task demands. HR and HRV were able to distinguish the level of PMWL after which the subjects were no longer able to cope with the increasing task demands and their instrument landing system performance fell to a sub-standard level. The major finding was the HR/HRV's ability to differentiate the sub-standard performance approaches from the high-performance approaches. Practitioner Summary: This paper examined if HR and HRV were sensitive to varying task demands in a fighter aviation environment and if these measures were related to variations in pilot's performance.