Search

We describe the all-engine-out landing of Air Transat Flight 236 in the Azores Islands (August 24, 2001) and use certain aspects of that accident to motivate a conceptual framework for the organization and display of information in complex human-interactive systems. Four hours into the flight, the aircraft experienced unusual oil indications. Two hours later, a fuel system failure led to a full-blown emergency that was not evident to the crew until it was too late. Although all relevant data to avoid the emergency were available to the aircraft computer systems, the design choices made about what to display and how to display it kept the pilots “in the dark.” The framework proposed here consists of six levels, beginning from the extraction of data from physical signals, abstracting from raw data to form visual representations on the user interface, and finally integrating high-level elements and information structures. We illustrate how the framework can be used to analyze some of the shortcomings in current display design, and we discuss some principles of information organization and formal analysis of task logic that might help to improve design. Finally, we sketch a design for a helicopter engine display based on these principles.

Dramatic advances have been made in intelligent systems from 1980 to 2010. The Web has created an environment in which some of the basic problems identified in the 1970s are emerging in new, often more concrete and practical forms. Correspondingly, the opportunities for space exploration to benefit from intelligent systems are evolving in unexpected directions, but with certain invariants in regard to knowledge representation and process modeling. This special issue covers semantic technologies, including natural language processing, sophisticated on-board fault management, human-robotic interaction, and agent-based simulation architectures.

How far can teaching methods go to enhance learning? Optimal methods of teaching have been considered in research on supervised and unsupervised learning. Locally optimal methods are usually hybrids of teaching and self-directed approaches. The costs and benefits of specific methods have been shown to depend on the structure of the learning task, the learners, the teachers, and the environment.

A formal, model-based approach is proposed for the development and evaluation of the sequences of actions specified in procedures. The approach employs methodologies developed within the discipline of discrete-event and hybrid systems control. We demonstrate the proposed approach through an evaluation of a procedure for handling an irregular engine-start on board a modern commercial aircraft.

This article surveys and discusses human interaction with automated control systems that employ modes. We focus our discussion on the features of the control system that lead to mode confusion and error. We first provide working definitions of the term mode and discuss key constructs that contribute to mode error, such as mode ambiguity and user expectations. Following this, we discuss human interaction with automated control systems in general and cockpit automation in particular. To provide a formal description of human interaction with such control systems, we introduce a modeling language, Statecharts, which is used by system engineers to specify complex control systems. We use the Statecharts language to describe the 3 types of modes that are commonly found in modem control systems: interface, functional, and supervisory modes. Examples from cockpit automation are used to illustrate each type of mode. The article concludes with a brief discussion of the links between the mode constructs and formal re...

Canonical correlation analysis is a type of multivariate linear statistical analysis, first described by Hotelling (1935), which is used in a wide range of disciplines to analyze the relationships between multiple independent and multiple dependent variables. We argue that canonical correlation analysis is the method of choice for use with many kinds of datasets encountered in human factors research, including field-study data, part-task and full-mission simulation data, and flight-recorder data. Although canonical correlation analysis is documented in standard textbooks and is available in many statistical computing packages, there are some technical and interpretive problems which prevent its routine use by human factors practitioners. These include problems of computation, interpretation, statistical significance, and treatment of discrete variables. In this paper we discuss these problems and suggest solutions to them. We illustrate the problems and their solutions based on our experience in using canonical correlation in the analysis of a field study of crew-automation interaction in commercial aviation.

Contents: Robert J. Mislevy, Introduction. J.R. Frederiksen, B. White, Intelligent Tutors as Intelligent Testers. J.R. Anderson, Analysis of Student Performance With the LISP Tutor. D.E. Kieras, The Role of Cognitive Simulation Models in the Development of Advanced Training and Testing Systems. A. Collins, Reformulating Testing to Measure Learning and Thinking: Comments on Chapters 1, 2, and 3. C.S. Gadd, H.E. Pople, Jr., Evidence From Internal Medicine Teaching Rounds of the Multiple Roles of Diagnosis in the Transmission and Testing of Medical Expertise. R.S. Siegler, J. Campbell, Diagnosing Individual Differences in Strategy Choice Procedures. J.C. Campione, A.L. Brown, Guided Learning and Transfer: Implications for Approaches to Assessment. S.P. Gott, The Assisted Learning of Strategic Skills: Comments on Chapters 5, 6, and 7. J.A. Reggia, C.L. D'Autrechy, Parsimonious Covering Theory in Cognitive Diagnosis and Adaptive Instruction. P. Langley, J. Wogulis, S. Ohlsson, Rules and Principles in Cognitive Diagnosis. S. Ohlsson, Trace Analysis and Spatial Reasoning: An Example of Intensive Cognitive Diagnosis and Its Implications for Testing. J. Wesley Regian, W. Schneider, Assessment Procedures for Predicting and Optimizing Skill Acquisition After Extensive Practice. A. Lesgold, S. Lajoie, D. Logan, G. Eggan, Applying Cognitive Task Analysis and Research Methods to Assessment. C.H. Frederiksen, A. Breuleux, Monitoring Cognitive Processing in Semantically Complex Domains. J.M. Orasanu, Diagnostic Approaches to Learning: Measuring What, How, and How Much: Comments on Chapters 12, 13, and 14. S. Embretson, Diagnostic Testing by Measuring Learning Processes: Psychometric Considerations for Dynamic Testing. S.P. Marshall, Generating Good Items for Diagnostic Tests. K.K. Tatsuoka, Toward an Integration of Item-Response Theory and Cognitive Error Diagnosis. R.L. Linn, Diagnostic Testing: Comments on Chapters 16, 17, and 18.

The on-going transformation from the current US Air Traffic System (ATS) to the Next Generation Air Traffic System (NextGen) will force the introduction of new automated systems and most likely will cause automation to migrate from ground to air. This will yield new function allocations between humans and automation and therefore change the roles and responsibilities in the ATS. Yet, safety in NextGen is required to be at least as good as in the current system. We therefore need techniques to evaluate the safety of the interactions between humans and automation. We think that current human factor studies and simulation-based techniques will fall short in front of the ATS complexity, and that we need to add more automated techniques to simulations, such as model checking, which offers exhaustive coverage of the non-deterministic behaviors in nominal and off-nominal scenarios. In this work, we present a verification approach based both on simulations and on model checking for evaluating the roles and responsibilities of humans and automation. Models are created using Brahms (a multi-agent framework) and we show that the traditional Brahms simulations can be integrated with automated exploration techniques based on model checking, thus offering a complete exploration of the behavioral space of the scenario. Our formal analysis supports the notion of beliefs and probabilities to reason about human behavior. We demonstrate the technique with the Uberlingen accident since it exemplifies authority problems when receiving conflicting advices from human and automated systems.

We describe a formal, model-based, approach for the analysis and design of human-automation interaction. The approach is guided by two primary objectives: first, that the interaction be correct in the sense that it will enable the user to operate the system reliably; second, that the interaction be as simple as possible while maintaining all the required operational flexibility. With these objectives we first discuss a methodology for verification of user-interface correctness, thereby addressing the issue of operational reliability, and then present an algorithmic approach for designing simple and succinct interfaces.

Most usability quality measures are performance-based and center on detection of interface problems. This study focused on the customer's subjective assessment of ease of use for Ameritech telephony products relative to other consumer products that they use every day. This assessment was accomplished through an empirical study that measured customer attitudes and opinions about their experiences using 12 Ameritech telephony products and 16 other consumer products. The results demonstrate where these telephony products sit relative to other consumer products in customers' perceptions of usability. In addition, a Multidimensional Scaling Analysis of the data revealed three separate dimensions that underlie people's perceptions of usability: automaticity of use, value to the customer, and self-evident operation.

Mode confusion is increasingly becoming a significant contributor to accidents and incidents involving highly automated airliners; in the last seven years there have been four airline accidents in which mode problems were present. This paper attempts to provide some initial observations about modes and how pilots use them. The authors define the terms “mode“,“mode transitions“,“mode configurations“, and propose a framework for describing and classifying modes. Preliminary results from a field study that documented mode usage in “Glass Cockpit” aircraft are presented. The data were collected during 30 flights onboard Boeing 757/767 type aircraft. Summary of the data depicts the paths that pilots use in transitioning from one mode to another. Analysis of the data suggest that these mode transitions are influenced by changes in aircraft altitude as well as by two factors in the operational environment: the type of air traffic control facility supervising the flight, and the type of instruction (clearance) issued.

One of the founders of the fields of artificial intelligence and cognitive science, Allen Newell, died on July 19, 1992 in Pittsburgh, Pennsylvania. He was the U. A. and Helen Whitaker University Professor of Computer Science at Carnegie Mellon University. He obtained his Ph.D. in Industrial Administration at Carnegie Institute of Technology in 1957 and was associated with the Rand Corporation from 1950 to 1961. He received numerous awards including that for Distinguished Scientific Contribution from the American Psychological Association and the A. M. Turing Award (with H. A. Simon) from the Association for Computing Machinery. His extensive publications include the books Human Problem Solving (with Herbert A. Simon, Prentice-Hall, 1972) and The Psychology of Human Computer Interaction (with Stuart Card and Thomas Moran, Erlbaum, 1983). Colleen Seifert is an Assistant Professor of Psychology at the University of Michigan. Her primary research interests are knowledge representation and case-based reasoning. Michael Shafto is Director of Research, Flight Human Factors Branch, NASA-Ames Research Center. His primary research interests are human factors and applied cognitive science.

Formal models are required to analyze the behavior of human-automation systems. This requirement follows from the number and variety of factors, and of interactions among factors, in such systems. The level of complexity of current and planned human-automation systems defies intuitive analysis. Fortunately, recent advances in the theory and practice of modeling have demonstrated the low cost and substantial benefits of formal analyses of real problems in human-automation interaction. This paper briefly notes some of the recent advances in this domain.

In this chapter we provide a Brunswikian perspective on human interaction with everyday technologies such as traffic lights, automotive devices (e.g., warning systems), and also advanced technologies such as flight control systems in modern airliners. We apply this perspective toward suggesting a framework for evaluating interface designs and for ultimately improving the usability, robustness, and effectiveness of a range of interactive technologies.

In this poster we present several methods for abstracting data into information and then integrating and organizing it for the purpose of display. We use an example from analysis of pilot-automation interaction to illustrate some of the underlying concepts.

In a study of crew interaction with the automatic flight control system of the Boeing 757/767 aircraft, we observed 60 flights and recorded every change in the aircraft control modes, as well as every observable change in the operational environment. To quantify the relationships between the state of the operating environment and pilots' actions and responses, we used canonical correlation because of its unique suitability for finding multiple patterns in large datasets. Traditionally, the results of canonical correlation analysis are presented by means of numerical tables, which are not conducive to recognizing multidimensional patterns in the data. We created a sun-ray-like diagram (which we call a heliograph) to present the multiple patterns that exist in the data by employing Alexander's theory of centers. The theory describes 15 heuristic properties that help create wholeness in a design, and can be extended to the problem of information abstraction and integration as well as packing of large amounts of data for visualization.

We describe an approach to evaluating algorithmic and human performance in directing UAV-based surveillance. Its key elements are a decision-theoretic framework for measuring the utility of a surveillance schedule and an evaluation testbed consisting of 243 scenarios covering a well-defined space of possible missions. We apply this approach to two example UAV-based surveillance methods, an algorithm called 2-opt and a human-directed approach, then compare them to identify general strengths and weaknesses of each method.

Computer simulation could be used to reduce the cost of designing human-machine systems, just as it is currently used in the design process for inanimate systems such as electronic circuits. However, past efforts have met with limited success due to difficulties modeling the human components of these systems. We have constructed a software framework and methodology for modeling human performance, APEX, that addresses several of these difficulties. This paper describes a methodology for using APEX to evaluate designs in complex, dynamic task environments; we then illustrate this process using an example from the domain of air traffic control.

Chow's book provides a thorough analysis of the confusing array of issues surrounding conventional tests of statistical significance. This book should be required reading for behavioral and social scientists. Chow concludes that the null-hypothesis significance-testing procedure (NHSTP) plays a limited, but necessary, role in the experimental sciences. Another possibility is that – owing in part to its metaphorical underpinnings and convoluted logic – the NHSTP is declining in importance in those few sciences in which it ever played a role.

The psychological validity of Fillmore's (1971b) theory of case grammar was tested in two experiments. Experiment I used a concept-identification task to see if subjects could learn to recognize case-relations of nouns. Different rates of learning were found for four cases: Agent was easiest to learn, followed by Experiencer, Instrument, and Object. Scaling of confusion data revealed a two-feature cognitive structure for the “psychological” case system: living versus nonliving and active versus passive. Experiment II used a running recognition task to see if the predicate structure of verbs is a salient factor in encoding and long-term storage of sentences. Confusability of sentences was primarily a function of the similarity between the predicate structures of their main verbs. Three models for the access and comparison of sentential information were proposed and tested against the data. The best-fitting model assumed similarity of encoded sentences in memory to be a function of the similarity between the predicate structures of their main verbs, rather than surface or “logical” grammatical functions of their nouns.

What characterizes students who become involved in political campaigns and what effects does their campaigning have on them? During a 2-week election recess, about one-third of the students at Princeton University chose to campaign. Those students who did so were liberal rather than radical in their political orientation. Those who participated seemed predisposed to campaign because they thought campaigning was an effective way of bringing about the changes that they sought. Campaigners were more likely than their fellow students to have engaged in political activity before. Such efforts may be evidence of their beliefs that political activity causes change, or might have been the forming experience for those beliefs. If the candidate for whom the student campaigned won the election, the student by and large strengthened or maintained his original attitudes about the efficacy of campaigning. If his candidate lost, the student became more pessimistic about the efficacy of campaigners whose candidates won also changed in an internal direction on a personal-control subscale of Rotter's (1966) internal-external scale; those whose candidates lost tended to change in an external direction.

Hughes (1984) has reported that the magnitude of the cue-validity effect in luminance detection is unaffected by target luminance. In three experiments, we explored the possible basis of this counterintuitive finding. The experiments focused on the design of the Hughes study, in which target luminance was treated as a between-blocks variable. Our results reveal that when target luminance is varied randomly within trial blocks, the cue-validity effect grows with declining target luminance. The difference between our findings and those of Hughes is interpreted in terms of cue-utilization strategies, which may adapt to target luminance when luminance remains invariant within trial blocks. Several alternative conceptions of the nature and locus of the cuevalidity effect in luminance detection are considered in light of these results.