Search

This dissertation research will cover lessons learned from the three-year, iterative design and evaluation of TECUMSA (Tasking and Execution of Collaborative Unmanned and Manned Systems with Autonomy). TECUMSA is a graphical user interface and autonomous tool suite that enables a single operator (e.g., an Air Mission Commander) to team with autonomous capabilities (e.g., route planning, aircraft task allocation) to effectively command and control multiple manned and unmanned aircraft in a contested battlespace. The user/AMC was responsible for accomplishing a series of reconnaissance, surveillance, and threat neutralization tasks in a hostile and dynamic simulated battlespace. The main challenges in this problem space are cognitive bandwidth of operators (e.g., maintaining situation awareness, allocating attention flexibly across multiple aircraft), and their ability to coordinate and collaborate with subordinate autonomous agents. The main objective of this research was therefore determining what control mechanisms offered the TECUMSA operator stability and reliability of control. Two formal system evaluations will be discussed, where a total of 15 Army aviators used TECUMSA to complete multiple hours of simulated air assault operations in a synthetic task environment. This research explored distributed supervisory control, where the operator distributed authority to automation for continuous manual control tasks using Play Calling (i.e., directability). The following research will also cover observations from the system evaluations highlighting interface features that afforded the user the ability to observe, perceive, and understand the state of the world relative to their goals and intentions (i.e., observability). One of the major themes in this dissertation is the importance of observability and directability as design principles, and the implications they have for both user interface design and human-autonomy teaming.

Can humans discriminate whether strings of events (e.g., shooting success in basketball) were generated by a random or constrained process (e.g., hot and cold streaks)? Conventional wisdom suggests that humans are not good at this discrimination. Following from Cooper, Hammack, Lemasters, and Flach (2014), a series of Monte Carlo simulations and an empirical experiment examined the abilities of both humans and statistical tests (Wald-Wolfowitz Runs Test and 1/f) to detect specific constraints that are representative of plausible factors that might influence the performance of athletes (e.g., learning, non-stationary task constraints). Using a performance/success dependent learning constraint that was calibrated to reflect shooting percentages representative of shooting in NBA games, we found that the conventional null hypothesis tests were unable to detect this constraint as being significantly different from random. Interestingly however, the analysis of human performance showed that people were able to make this discrimination reliably better than chance. Hence, people may also be able to detect patterned/constrained processes in a real-world setting (e.g., streaks in basketball performance), thus supporting the belief in the hot hand.