Search

This research investigated the effects of early judgment on (1) the handling of new information, some of which confirmed and some of which contradicted the early judgment, and (2) the selection of hypothesis-testing indicators. The context was situation assessment by Army intelligence analysts during an evolving battlefield scenario. Unaided analysts typically ignored or underweighted contradictory evidence; their confidence in their early judgment tended to rise. A second group was given a brief tutorial on common decision biases and graphic displays that fostered awareness of uncertainty; in this group the tendencies were reduced (but not eliminated), and one-half of the group reversed their judgment at least once. A third group selected indicators; however, in the face of balanced feedback, their confidence remained constant rather than rising. The findings support the extension of confirmation bias theories to trained personnel performing realistic tasks. In addition, the results suggest that when decision makes the indicators they believe to be important, they pay more attention to contradictory evidence than when they are the passive recipients of new information.

Tactical decision making occurs in environments characterized by high uncertainty, high-valued outcomes, and time stress. The critical relation of decision outcomes to uncertain events pushes decision makers to resolution of uncertainty. Time stress, however, works in the opposite direction, precluding ideal planning efforts. It is thus crucial to systematically characterize the impacts of uncertainty and time stress on tactical decision making to facilitate the design of decision aids that will be effective in the environments described. The present study employs a theoretical framework that extends Kahneman and Tversky's prospect theory (1979) to goal-directed decisions such as those involved in battlefield tactical decisions. The framework is used to provide a basis for characterizing the critical issues involved in such environments and to design decisions aiding concepts based on studies employing the framework. Four studies were conducts. The first study provided participants with choices involving gains and losses in goal-directed tactical and nontactical situations. The purpose was to examine preferences among risky options and to validate the extended theory as an appropriate characterization of the tactical decision making process. The results strongly confirmed the theoretical formulation. The second study involved a more rigorous test of the framework using a more realistic tactical problem that provided both for a test of the framework and development of concepts for aiding decisions under time stressed conditions. Tradeoffs between effort allocated to uncertainty reduction and option generation were examined in detail, and aid concepts were elicited. Study two results were used to design aiding concepts that were further refined and tested in study three to yield a final set of aiding concepts to be tested in study four.

Army intelligence analysts were given a realistic battlefield scenario and asked to make preliminary decisions about most likely enemy avenue of approach, and their confidence level. Subsequently, they were asked to reconsider their decisions in the light of updated intelligence reports containing some items which confirmed and some which contradicted their early decisions. Three such updating judgments were requested. Finally, they were asked to rate each information item in terms of the degree to which it supported or contradicted their hypothesis.

This report presents the results of Phase 2 research on decision making in an evolving situation. As in Phase 1, the problem context was situation assessment by trained Army intelligence analysts working in pairs. Participants were given an initial battlefield scenario and asked to determine the enemy's most likely avenue of approach and to give their level of confidence; subsequently they were asked to reconsider their decisions after receiving each of three updated intelligence reports that contained some items that confirmed and some that did not confirm their early hypothesis. Finally, the participants were asked to rate each information item in terms of the degree to which it supported or contradicted their hypotheses. (AN)

This paper describes a recent experiment conducted with Patriot air defense officers and using the Patriot air defense simulators at Fort Bliss, Texas. The experimenters found that, under certain conditions, the participants made different identification judgments and took different engagement actions depending on the sequence in which the same information was presented to them. This finding was consistent with theoretical predictions regarding how operators process information, and the hypothesis that their processing approach (or Heuristic) would result in biased judgments under certain conditions. Future research is directed toward investigating whether display modification can remove the observed judgmental bias. Generally, this experiment demonstrates the applied implications of basic research investigating human information processing, and the importance of understanding cognitive processes when developing computer systems. (KAR) P. 3

This project studies the effects of uncertainty and time stress on the decision making of experienced Army officers playing the role of a division operations officer analyzing courses of action in a tactical scenario. Thirteen lieutenant colonels (LTC) were presented with a defensive scenario (a Fulda Gap scenario) and asked to analyze three courses of action and make a final recommendation. After being given time to familiarize themselves with the scenario, seven of the participants were given 45 minutes for their analyses (time-stress condition), after which participants made their final recommendations. Data were collected on the information participants used to perform their analyses. No-stress participants compared with time-stress ones spent more time on information that was relevant to resolving uncertainty and providing a big picture of their mission, used more analysis methods to arrive at a recommendation, and tended to recommend less conservative courses of action. These results suggest that under conditions of uncertainty and time stress, aiding concepts are needed that supplement the breadth of analysis and deliver information more efficiently so that decision makers will be able to use all relevant information.

Research indicates that humans use heuristics to make inferences and that, depending on task characteristics, these heuristics can lead to inconsistencies and errors in judgment-that is, cognitive biases. Most of this research has been performed with university students performing tasks, requiring logical thinking but not expertise in a particular substantive area. Our concern is in determining whether heuristics can lead to cognitive biases among experienced personnel performing their substantive task. In particular, we examined whether information order and response mode could affect the judgements of Army air defense operators. A within-subject factorial experiment was performed in December, 1989, with 63 Army air defense operators. Information order and response mode interacted to affect the Army air defense operators' judgments. When information was presented sequentially and a probability estimate was obtained after each piece of information, participants gave different probability estimates of whether an unknown aircraft was friendly or hostile, depending on the order with which the same information was presented. These results support the predictions of the Hogarth-Einhorn belief updating model.

This report summarizes the results of a project designed to develop methods to elicit and represent expert knowledge. Previous work in knowledge elicitation/engineering has been largely applications-driven, with expert knowledge force-fit into predefined knowledge frameworks. The goals of this project were to gain insight into how experts structure knowledge, to develop and test a set of integrated knowledge-elicitation techniques driven by this understanding of expert knowledge representation and use, and to gain insight into the nature of expertise and how it develops. Knowledge-elicitation sessions were conducted with some fifty experts in order of battle analysis, situation assessment, target development, and collection management at Forts Bragg, Carson, Lewis, Hood, and Huachuca. Results indicate that experts use a diverse range of knowledge structures depending on their functional area, level of experience, unit, and other variables. These structures are used integratively rather than singly. As a result, we propose an Integrated Knowledge Structure (INKS) framework for expert knowledge representation.

This report summarizes work testing the usefulness of neural network models for measuring and predicting Army unit performance in settings such as the National Training Center. A back-propagation neural network model was developed and trained from wargaming simulation. Results suggest that such a model can train to recognize unit success and failure in simulated engagements. Further work will require access to clean, large, data sets, such as those available from SIMNET. In addition, an expert-based preprocessor is suggested as a useful approach to implementing a model.

Managing an organization as large and complex as the U.S. Army requires leaders of the highest caliber. Previous research has shown that leaders at the executive level, corresponding in the Army to three-star general officers and above, think and plan in ways that are qualitatively different from officers at lower levels. This report describes the development of a theory of the cognitive structures that support executive-level functioning. A key theoretical tenet is that the primary discriminator of executives is how their knowledge is organized and accessed. Researchers reviewed literature and performed an observational study to document the difference in executive and nonexecutive problem solving. Researchers developed a theory of the cognitive underpinnings of executive performance and constructed a set of recommendations for executive development on the basis of the theory.

Advances in automated image understanding technology are essential to our ability to exploit today's sophisticated imagery capabilities to support battlefield intelligence requirements. This report describes the application of a unique inference framework, Non-Monotonic Probabilist, to the problem of achieving consensus among modules, each of which supports a different part of the image understanding problem. Non-Monotonic Probabilist combines symbolic default

Logical and mathematical issues of developing, pooling and reconciling multiple models of a given uncertain variable have been addressed. A number of technical papers have been published in refereed journals. Keywords: Abstracts, Bibliographies.

This research note presents a framework for knowledge elicitation and representation based on theories of how experts themselves represent knowledge. Much previous work on knowledge elicitation has been guided by requirements of the systems for which the knowledge is to be encoded. Two methodologies are presented: an interpretive method, based on the premise that experts organize knowledge in a top down fashion; and a generative method, based on the premise that they organize knowledge in a bottom up way. The interpretive representation of expert knowledge integrates scripts, object frames, and mental models. The generative representation integrates production rules, semantic nets, and mental models. Knowledge elicitation techniques for each method are selected and tailored according to knowledge requirements of these representations. Both methods were tested between subjects on ten situation development specialists and ten order of battle specialists in the Army Intelligence domain. Knowledge models were constructed and evaluated for each of the methods, and the intelligence specialties. Results show that experts use a variety of knowledge structures in processing information and reasoning, and that both representations are necessary to capture expert knowledge adequately. Keywords: Information theory, Military intelligence.

This research investigated how users interact with an expert system in which underlying values change as a function of the situation or of the planning time horizon. The problem context was the prioritization of tactical air strike targets, and Air Force targeteers were the experimental subjects. Their task was to explain why the system made the recommendations it did. The expert system was simulated in storyboard form. It was found that users who were given a good conceptual model of the expert system, in the form of a brief summary of its step-by-step processes, performed better than those whose model of the system was relatively poor. Users whose displays were relatively user-oriented (geographic, top-down, and simplified) did not consistently differ in performance from users whose displays were relatively aid-oriented (data-intensive matrices). However, the aid-oriented displays seem to encourage users to make more frequent reference to three tables (available to all subjects) containing important information about the expert assessments on which the aid's algorithms operated. Users would typically generate their own solutions, using criteria and heuristics that often differed from those used by the aid, and question the expert system solution. Keywords: Training devices.