Search

Relational data representations have become an increasingly important topic due to the recent proliferation of network datasets (e.g., social, biological, information networks) and a corresponding increase in the application of Statistical Relational Learning (SRL) algorithms to these domains. In this article, we examine and categorize techniques for transforming graph-based relational data to improve SRL algorithms. In particular, appropriate transformations of the nodes, links, and/or features of the data can dramatically affect the capabilities and results of SRL algorithms. We introduce an intuitive taxonomy for data representation transformations in relational domains that incorporates link transformation and node transformation as symmetric representation tasks. More specifically the transformation tasks for both nodes and links include (i) predicting their existence, (ii) predicting their label or type, (iii) estimating their weight or importance, and (iv) systematically constructing their relevant features. We motivate our taxonomy through detailed examples and use it to survey competing approaches for each of these tasks. We also discuss general conditions for transforming links, nodes, and features. Finally, we highlight challenges that remain to be addressed., Journal of Articial Intelligence Research, v45 p363-441, 2012. The original document contains color images.

Previous research examining how people resume a task following an interruption has focused primarily on pure memory processes. In this paper, we focus on the perceptual processes underlying task resumption and show that spatial memory guides task resumption. In Experiment 1, fixation patterns suggest participants were able to resume remarkably close to where they were in the task prior to interruption. In Experiment 2, a spatial interruption disrupted resumption performance more than a non-spatial interrupting task. Together, these results implicate spatial memory as a mechanism for resumption., Prepared in collaboration with George Mason University, Fairfax, VA.

Most theories of graph comprehension posit the existence of a graph schema to account for people's prior knowledge of how to understand different graph types. The graph schema is, however, a purely theoretical construct: there are no empirical studies that have explicitly examined the nature of the graph schema. We sought to determine whether graph schemas are based on perceptual features or on a common invariant structure shared between certain graphs. The process of activating the graph schema was isolated as participants responded to graphs presented in pure and mixed blocks. Any differences in reaction time between the blocks could be attributed to loading the appropriate schema. Results from a series of experiments using five types of graphs suggest graph schemas are based on the graphical framework, a common invariant structure among certain types of graphs. These results provide insight into the comprehension of novel graphs., Prepared in collaboration with George Mason University, Fairfax, VA. ONR grant no. N00014-03-WX3-0001.

Task analytic theories of graph comprehension account for the perceptual and conceptual processes required to extract specific information from graphs. Comparatively, the processes underlying information integration have received less attention. We propose a new framework for information integration that highlights visual integration and cognitive integration. During visual integration, pattern recognition processes are used to form visual clusters of information; these visual clusters are then used to reason about the graph during cognitive integration. In three experiments the processes required to extract specific information and to integrate information were examined by collecting verbal protocol and eye movement data. Results supported the task analytic theories for specific information extraction and the processes of visual and cognitive integration for integrative questions. Further, the integrative processes scaled up as graph complexity increased, highlighting the importance of these processes for integration in more complex graphs. Finally, based on this framework, design principles to improve both visual and cognitive integration are described., Prepared in collaboration with George Mason University, Fairfax, VA.

We conducted an experiment to investigate whether spatial processing is used in graph comprehension tasks. Using an interference paradigm, we demonstrate that a graph task interfered more with performance on a spatial memory task than on a visual "non-spatial" memory task. Reaction times showed there was no speed-accuracy tradeoff. We conclude that it was the spatial nature of the graph task that caused the additional interference in the spatial memory task. We propose that current theories of graph comprehension should be expanded to include a spatial processing component., Presented at the Annual Conference of the Cognitive Science Society (29th), CogSci 2007, Nashville, TN on 1-4 Aug 2007 and published in proceedings of the same, p1563-1568.; ISBN 978-0-9768318-3-9.

Theories accounting for the process of primary task resumption following an interruption have focused on the suspension and retrieval of a specific goal. The ability to recall the spatial location of where in the task one was prior to being interrupted may also be important. We show that being able to maintain a spatial representation of the primary task facilitates task resumption. Participants were interrupted by an instant message window that either partially or fully occluded the primary task interface. Reaction time measures show that participants were faster at resuming in the partial occlusion condition. In addition, eye track data suggest that participants were more accurate at returning to where they left off, suggesting that they were able to maintain a spatial representation of the task and use this information to resume more quickly., Presented at the Annual Meeting of the Human Factors and Ergonomics Society (51st), held in Baltimore, MD, on 1-5 Oct 2007. Prepared in collaboration with George Mason University, Fairfax, VA. The original document contains color images.

Research has shown that different types of interruptions can affect their disruptiveness. However, it is unclear how different features of the interrupting task determine its disruptive effects. Specifically, some theories predict that the difficulty of an interruption does not contribute to the disruptive effects of that interruption alone. Disruptive effects can be mediated by the extent to which the interrupting task interferes with the ability to rehearse during the interruption. In this experiment participants performed a single primary task with three interruptions of different difficulty. We found that interruptions were more disruptive when the task minimized the participant's ability to rehearse (as measured by the number of mental operators required to perform the task) and not just when they were more difficult. These results suggest that the ability to rehearse during an interruption is critical in facilitating resumption of a primary task., Presented at the Annual Meeting of the Human Factors and Ergonomics Society (51st), held in Baltimore, MD, on 1-5 Oct 2007. Prepared in collaboration with George Mason University.

How do problem solvers represent visual-spatial information in complex problem solving tasks? This paper explores the predictions of symbolic computation, embodied problem solving and a neurocomputational theory for what factors influence internal representation choices. Across two studies, data are collected from experts and novices in three different, complex visual-spatial problem-solving domains (weather forecasting, submarine target motion analysis, and fMRI data analysis). Internal spatial representations are coded from spontaneous gestures made during cued-recall summaries of problem solving activities. Analyses of domain differences, expertise differences, and changes over time with problem solving suggest that neurocomputational constraints play a larger role than the nature of the visual input or the nature of the underlying real world being examined through problem solving, especially for expert problem solvers. The particular neurocomptuational feature that was found to drive internal representation choice is the required spatial precision of the main goals of problem solving., Prepared in collaboration with University of Pittsburgh, grant no's. N00014-02-1-0113 and N00014-03-1-0061.

The process of resuming an interrupted task has been understood by task level goals. Recent empirical evidence has implicated spatial memory as a component of the resumption process suggesting that spatial level representations are important as well. We collected eye track data in an interruptions paradigm to examine the perceptual processes involved in resumption. Four models were created to illustrate the importance of the role of spatial representations and further, to demonstrate how the task level and spatial representations can be integrated., Presented at the International Conference on Cognitive Modeling (8th), held in Ann Arbor, MI, on 27-29 Jul 2007. Prepared in collaboration with George Mason University, Fairfax, VA. The original document contains color images.

Our topic in this chapter is not so much what happens when experts have to work "out of context," but how cognitive engineering might help weather forecasters, in particular, remain within familiar decision-making spaces by improving on their display technology. Most weather forecasters get data, charts, and satellite images from Internet sources. In this chapter, we discuss some of what we know about how weather forecasters use information technology to display and support the interpretation of complex meteorological visualizations. Based on notions of human-centered computing (HCC), we offer some suggestions on how to improve the visualizations and tools., Prepared in collaboration with the Institute for Human and Machine Cognition, Pensacola, FL. Published in R. R. Hoffamn, Experise Out of Context, p337-358, Lawrence Erlbaum Associates, 2007.

Good team members seem to have the ability to simulate what others on the team will do in different situations. Team researchers have long studied what makes an effective team. Their methodology has been to examine how high and low performing teams accomplish team-related tasks. They have suggested that a good team-member has three knowledge components (Cannon-Bowers, Salas, & Converse, 1993): (1) Knowledge of own capabilities [meta-knowledge], (2) Knowledge of the task, and (3) Knowledge about the capabilities of their teammates. Most researchers have suggested that these three components are deeply inter-related; without any one of these, a person is not a good team member. However, without a computational theory, these claims can be difficult to examine empirically. The focus of this paper is the third component, the cognitive modeling done of a teammate's cognitive processes. This shared understanding of teammates frequently called a shared mental model (Mathieu, Heffner, Goodwin, Salas, & Cannon-Bowers, 2000). We start with the premise that humans use themselves as an initial model of their teammate, and then refine it as the team (and individuals within the team) gains experience. Our primary research goal is to create a computational theory of teamwork by modeling the individuals within the team so that we can eventually build plausible robots for teamwork and human-robot interaction., Presented at the International Conference on Cognitive Modeling (8th) held in Ann Arbor, MI on 27-29 July 2007. Published in the Proceedings of the International Conference on Cognitive Modeling (8th), p253-254, 2007.

The characteristics of long-term, symbolic learning were investigated using Soar and ACT-R models of a task to rearrange blocks into specific configurations. Long sequences of problems were run collecting data to answer fundamental questions about long-term, symbolic learning. The questions were whether symbolic learning continues indefinitely, how learned knowledge is used, and whether performance degrades over the long term. It was found that in both systems symbolic learning eventually stopped, ACT-R produced three observable phases of learning, and both Soar and ACT-R suffer from the utility problem of degraded performance with continuous on-line learning.

This paper outlines work that was completed to assist the warfighter during the critical mission planning process. This was accomplished by delivering current weather data from Tactical Environmental Data Services (TEDServices), an API used to request meterological, oceanographic, and environmental information, through the Extensible Tactical C4I Framework (XTCF), which is a prototype extensible data management framework implemented in Java. It includes discussion of relevant technologies, such as XML and JMS., The original document contains color images.

How do children learn how to play hide and seek? At ages 3-4, children do not typically have perspective taking ability, so their hiding ability should be extremely limited. The authors show through a case study that a 3-1/2-year-old child can, in fact, play a credible game of hide and seek, even though she does not seem to have perspective taking ability. They propose that children are able to learn how to play hide and seek by learning the features and relations of objects (e.g., containment, under) and use that information to play a credible game of hide and seek. They model this hypothesis within the ACT-R cognitive architecture and put the model on a robot, which is able to mimic the child's hiding behavior. They also take the "hiding" model and use it as the basis for a "seeking" model. They suggest that using the same representations and procedures that a person uses allows better interaction between the human and robotic system.

We present two key components of a principled method for constructing modular, heterogeneous swarms. First, we generalize a well-known technique for representing swarm behaviors to extend the power of multiagent systems by specializing agents and their interactions. Second, a novel graph-based method is introduced for designing swarm-based behaviors for multiagent teams. This method includes engineer-provided knowledge through explicit design decisions pertaining to specialization, heterogeneity, and modularity. We show the representational power of our generalized representation can be used to evolve a solution to a challenging multiagent resource protection problem. We also construct a modular design by hand, resulting in a scalable and intuitive heterogeneous solution for the resource protection problem., Prepared in collaboration with the Department of Computer Science, University of Wyoming, Laramie, WY. Presented at the International Conference on Parallel Problem Solving from Nature (9th), PPSN IX, held Reykjavik, Iceland, 9-13 Sep 2006. Published in proceedings of the same, Lecture Notes in Computer Science, v4193, p741-750, 2006; ISBN 3-540-3899-0.

For the last few years, our lab has been attempting to build robots that are similar to humans in a variety of ways. Our goal has been to build systems that think and act like a person rather than look like a person since the state of the art is not sufficient for a robot to look (even superficially) like a human person. We believe that there are at least two reasons to build robots that think and act like a human. First, how an artificial system acts has a profound effect on how people act toward the system. Second, how an artificial system thinks has a profound effect on how people interact with the system., Published in Cognition and Multi-Agent Interaction: From Cognitive Modeling to Social Simulation as Chapter 10, p252-278, 2006. ISBN 0521839645 (Hardcover).

Current theories of gesture production all suggest that spatial working memory is a critical component of iconic gesture production. However, none of the models has a selection mechanism for what aspect of spatial working memory is gestured. We explored how expert and journeyman scientists gestured while discussing their work. Participants were most likely to make iconic gestures about change over time (spatial transformations), less likely to gesture about spatial relations and locations (geometric relations), and far less likely to gesture about the magnitude of spatial entities. We also found that experts were especially likely to have a high degree of association between iconic gestures and spatial transformations. These results show that different features of spatial language are gestured about at different rates. We suggest that current gesture production models need to he expanded to include selection mechanisms to account for these differences., Prepared in cooperation with George Mason University, the University of Pittsburgh and the Naval Undersea Warfare Center. Published in Spatial Cognition and Computation, v6, n1, p1-29, 2006.

The authors examine the effects of environmental cues on being interrupted while performing a task. They conducted an experiment in which participants, after an interruption, received either a blatant environmental cue of their previous action (a red arrow), a subtle environmental cue of their previous action (a cursor that was placed in the same location as their previous action), or no environmental cue at all. The authors found that participants in the blatant condition resumed their task faster than participants in the other two conditions. Furthermore, a subtle environmental cue was no better than no cue at all. The results support their model of memory for goals., Presented at the Human Factors and Ergonomics Society Annual Meeting (49th) held in Orlando, FL on 26-30 Sep 2005. Project order number 55-8122-05. Prepared in cooperation with Michigan State University, East Lansing, MI. The original document contains color images.

The effective use of humanoid robots in space will depend upon the efficacy of interaction between humans and robots. The key to achieving this interaction is to provide the robot with sufficient skills for natural communication with humans so that humans can interact with the robot almost as though it were another human. This paper describes the design, implementation, and capabilities of a robotic system architecture for a robot which can be used (at some level) to collaborate with a human. The capabilities required of the robot include voice recognition, natural language understanding, gesture recognition, spatial reasoning, and cognitive modeling with perspective-taking. These represent a small subset of potential capabilities humans utilize with one another in collaborating to perform a task in a complex environment, and barely scratches the surface of capabilities that one might want to build into an intelligent, collaborative robot. The capabilities described here have been successfully implemented and demonstrated on several mobile robotic platforms (Sofge, et al., 2004), and the authors are now porting them to Robonaut. They also are extending the capabilities of the cognitive architectures (both ACT-R/S and Polyscheme) and their perspective-taking cognitive models. Future work will focus on enhancing the cognitive models through expanded rule sets and cognitively plausible behaviors and reasoning mechanisms, and adding learning capabilities to the models so that the robots may be able to acquire new knowledge and skills through interaction with humans and while performing tasks., Published in Proceedings of the 16th IFAC Symposium on Automatic Control in Aerospace, 2004. Prepared in collaboration with the University of Missouri-Columbia, Electrical and Computer Engineering Department, Columbia, MO. The original document contains color images.

In an attempt to solve as much of the AAAI Robot Challenge as possible, five research institutions representing academia, industry and government, integrated their research on a pair of robots named GRACE and GEORGE. This paper describes the second year effort by the GRACE team, the various techniques each participant brought to GRACE, and the integration effort itself., Prepared in collaboration with Carnegie Mellon University; Northwestern University; Metrica, Inc.; and Swarthmore College. The original document contains color images.

This study examined how graph readers extract specific information, integrate information, and make inferences from choropleth graphs. The authors present a hierarchical framework of graph comprehension that suggests how graph readers extract different types of information. The framework suggests that the cognitive operations required to extract different types of information build in a hierarchical fashion as the complexity of the type of extraction increases. Empirical data gathered in their laboratory is reviewed in support of the hierarchical framework, and implications of the work are discussed., Presented at the Human Factors and Ergonomics Society Annual Meeting (48th) held in New Orleans, LA on 20-24 Sep 2004. Prepared in cooperation with George Mason University, Fairfax, VA. The original document contains color images.

Although there is considerable evidence that humans use the same mechanisms for linguistic and nonlinguistic cognition, the thesis of linguistic modularity will remain plausible so long as well-established formal properties of syntax remain unexplained in terms of domain-general cognitive mechanisms. This paper presents several dualities between the formal structure of syntax and cognitive structures used to represent the physical world. These dualities are used to construct a cognitive model of syntactic parsing that uses only the mechanisms required for infant physical reasoning. The model demonstrates how a formal syntactic constraint, the c-command condition on binding, can be explained by a cognitive process used in physical reasoning. Several consequences for language development and the doctrine of linguistic modularity are considered., Presented at the Annual Conference of the Cognitive Science Society (26th), CogSci 2004, Chicago, IL on 5-7 Aug 2004 and published in proceedings of the same.

Although it is apparent that people are able to make inferences from graphs, it is presently unclear how they do so, even from simple graphs. Current theories of graph comprehension are largely silent about the processes by which such inferences are made ( e.g., Freedman and Shah, 2002; Pinker, 1990). The authors propose that people use spatial reasoning, in the form of spatial transformations (Trafton, Trickett, and Mintz, in press), to answer inferential questions. Spatial transformations are cognitive operations that a person performs on internal or external visualizations, such as graphs. They occur when people must mentally create or delete something (e.g., a line) on the image to facilitate problem solving, and may be related to hypothetical drawing (Shimojima and Fukaya, 2003). This paper investigates the use of spatial transformations when people need to make inferences from graphs., Prepared in cooperation with George Mason University, Department of Psychology, Fairfax, VA.

One of the great challenges of putting humanoid robots into space is developing cognitive capabilities for the robots with an interface that allows human astronauts to collaborate with the robots as naturally and efficiently as they would with other astronauts. In this joint effort with NASA and the entire Robonaut team we are integrating natural language and gesture understanding, spatial reasoning incorporating such features as human-robot perspective taking, and cognitive model-based understanding to achieve this high level of human-robot interaction., Prepared in cooperation with NASA. Prepared in collaboration with University of Missouri, Columbia, MO. Sponsored in part by DARPA. The original document contains color images.

In conversation, people often use spatial relationships to describe their environment, e.g., "There is a desk in front of me and a doorway behind it", and to issue directives, e.g., "Go around the desk and through the doorway." In our research, we have been investigating the use of spatial relationships to establish a natural communication mechanism between people and robots, in particular, for novice users. In this paper, the work on robot spatial relationships is combined with a multi-modal robot interface. We show how linguistic spatial descriptions and other spatial information can be extracted from an evidence grid map and how this information can be used in a natural, human-robot dialog. Examples using spatial language are included for both robot-to-human feedback and also human-to-robot commands. We also discuss some linguistic consequences in the semantic representations of spatial and locative information based on this work., The original document contains color images. Prepared in collaboration with University of Missouri-Columbia, Columbia, MD 65211. Pub. in IEEE Transactions on SMC, Part C, Special Issue on Human-Robot Interaction, July 2002. Sponsored in part by DARPA and NRL.

In their research on intuitive means for humans and intelligent, mobile robots to collaborate, the authors use a multimodal interface that supports speech and gestural inputs. As a preliminary step to evaluate their approach and to identify practical areas for future work, they conducted a Wizard-of-Oz pilot study with five participants who each collaborated with a robot on a search task in a separate room. The goal was to find a sign in the robot's environment with the word "FOO" printed on it. Using a subset of their multimodal interface, participants were told to direct the collaboration. As their subordinate, the robot would understand their utterances and gestures, and recognize objects and structures in the search space. Participants conversed with the robot through a wireless microphone and headphone and, for gestural input, used a touch screen displaying alternative views of the robot's environment to indicate locations and objects., Prepared in collaboration with the University of Missouri-Columbia, Computer Engineering and Computer Science Department, Columbia, MO. Sponsored in part by DARPA. The original document contains color images.

Agents provide a flexible and scalable method of integrating artificial intelligence techniques on a single cohesive distributed computing system. We have designed and implemented an agent-based interface for autonomous control, and for providing web-based information retrieval, for a dynamically autonomous mobile robot. The robot implements and integrates a variety of artificial intelligence techniques including a multimodal interface that allows natural language understanding, gesture interpretation, simultaneous localization and mapbuilding, object identification and spatial reasoning. The agent-based interface augments these capabilities by providing a method of controlling the robot via the CoABS grid, and by providing the means for the robot/operator to request information available through the grid or through the web., Sponsored in part by DARPA (MIPR 02-M733-00). The original document contains color images.

This paper describes an approach to continuous coevolution of form (the morphology) and function (the control behavior) for autonomous vehicles. This study focuses on coevolution of the characteristics such as beam width and range of individual sensors in the sensor suite, and the reactive strategies for collision-free navigation for an autonomous micro air vehicle. The results of the evolution of the system in a fixed simulation model were compared to case-based anytime learning (also called continuous and embedded learning) where the simulation model was updated over time to better match changes in the environment., Presented at the Congress on Evolutionary Computation (CEC-2003) held in Canberra, Australia on 8-12 Dec 2003. Published in the Proceedings of the Congress on Evolutionary Computation (CEC-2003). The original document contains color images.

The authors are designing and implementing a multi-modal interface to a team of dynamically autonomous robots. For this interface, they have elected to use natural language and gesture. Gestures can be either natural gestures perceived by a vision system installed on the robot, or they can be made by using a stylus on a Personal Digital Assistant. In this paper,they describe the integrated modes of input and one of the theoretical constructs that they use to facilitate cooperation and collaboration among members of a team of robots. An integrated context and dialog processing component that incorporates knowledge of spatial relations enables cooperative activity between the multiple agents, both human and robotic., Prepared in collaboration with the University of Missouri-Columbia, Computer Engineering and Computer Science Department, Columbia, MO and ITT Industries, Alexandria, VA. Sponsored in part by the Office of Naval Research. The original document contains color images.

In conversation, people often use spatial relationships to describe their environment, e.g., "There is a desk in front of me and a doorway behind it", and to issue directives, e.g., "Go around the desk and through the doorway." In our research, we have been investigating the use of spatial relationships to establish a natural communication mechanism between people and robots, in particular, for novice users. In this paper, the work on robot spatial relationships is combined with a multi-modal robot interface developed at the Naval Research Lab. We show how linguistic spatial descriptions and other spatial information can be extracted from an evidence grid map and how this information can be used in a natural, human-robot dialog., Prepared in collaboration with University of Missouri, Columbia, MO. Sponsored in part by Office of Naval Research (ONR). The original document contains color images.

The visual occlusion technique has received considerable attention in recent years as a method for measuring the interruptible aspects of in-vehicle information system (IVIS) task performance. Because the visual occlusion technique lacks a loading task during "occluded" periods, an alternate method was adopted to provide increased sensitivity to the attentional costs of interruptions on IVIS-style task performance. Participants alternated between performing a VCR programming task and a simple tracking task. Results indicate that it does matter at which point the VCR task is interrupted in terms of time to resume the VCR task. Specifically, the resumption time, or lag, was lowest right before beginning a new task stage such as entering the show end-time, or when performing a repetitive scrolling task. The results suggest that it might be appropriate to include measures of resumption lag when testing the interruptability of IVIS-style tasks., Presented at the Human Factors and Ergonomics Society Annual Meeting (46th) held in Baltimore, MD on 29 Sep-4 Oct 2002. The original document contains color images.

How do experienced users extract information from a complex visualization? The authors examine this question by performing an experiment. They presented experienced weather forecasters with visualizations that did not show the needed information explicitly and examined their eye movements. They replicated Carpenter and Shah (1998) when the information was explicitly available on the visualization. However, when the information was not explicitly available, they found that forecasters used spatial reasoning in the form of spatial transformations to extract the needed information. They also found a strong image component for constructing meteorological information., Prepared in cooperation with San Diego State University, San Diego, CA and George Mason University, Fairfax, VA. MIPR-N0001400WX40002.

Goal-directed cognition is often discussed in terms of specialized memory structures like the "goal stack." The "goal-activation" model presented here analyzes goal-directed cognition in terms of the general memory constructs of activation and associative priming. The model embodies three predictive constraints: (1) the interference level, which arises from residual memory for old goals; (2) the strengthening constraint, which makes predictions about time to encode a new goal; and (3) the priming constraint, which makes predictions about the role of cues in retrieving pending goals. These constraints are formulated algebraically and tested through simulation of latency and error data from the Tower of Hanoi, a means-ends puzzle that depends heavily on suspension and resumption of goals. Implications of the model for understanding intention superiority, post-completion error, and effects of task interruption are discussed., Pub. in Cognitive Science, v26, p39-83, 2002. Prepared in cooperation with Michigan State University, East Lansing, MI. Sponsored in part by the Office of Naval Research (ONR), Arlington, VA. Portions of this work were presented at the meeting of the Cognitive Science Society (21st) held in Vancouver, Canada, in Aug 1999.

The purpose of this paper is to introduce a hybrid cognitive-reactive system, which integrates a machine- learning algorithm (SAMUEL, an evolutionary algorithm-based rule-learning system) with a computational cognitive model (written in ACT-R). In this system, the learning algorithm handles reactive aspects of the task and provides an adaptation mechanism, while the cognitive model handles cognitive aspects of the task and ensures the realism of the behavior. In this study, the controller architecture is used to implement a controller for a team of micro-air vehicles performing reconnaissance and surveillance., Presented at the IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2002, Lausanne, Switzerland on 30 Sep-4 Oct 2002 and published in proceedings of the same. Sponsored in part by DARPA.

This paper discusses approaches to cooperative coevolution of form and function for autonomous vehicles, specifically evolving morphology and control for an autonomous micro air vehicle (MAV). The evolution of a sensor suite with minimal size, weight, and power requirements, and reactive strategies for collision-free navigation for the simulated MAV is described. Results are presented for several different coevolutionary approaches to evolution of form and function (single- and multiple-species models) and for two different control architectures (a rulebase controller based on the SAMUEL learning system and a neural network controller implemented and evolved using ECkit)., Presented at the NASA/DoD Conference on Evolvable Hardware, EH-2002, Washington, DC, 15-18 Jul 2002 and published in the proceedings of the same. The original document contains color images.

CONCLUSIONS: 1. By using "context predicates" we track actions occurring during a dialog to determine which goals (event and locative) have been achieved or attained and which have not. 2. By tracking "context predicates" we can determine what actions need to be acted upon next; i.e., predicates in the stack that have not been completed. 3. "Locative" expressions, e.g. "there," give us a kind of handle in command and control applications to attempt error correction when locative goals are being discussed. 4. By interleaving complex dialog with natural and mechanical gestures, we hope to achieve dynamic autonomy and an integrated multi-modal interface., The original document contains color images.

Computer Generated Forces (CGF), to be effective training tools, must exhibit robust, challenging, and realistic behaviors. CGF tasks usually have both cognitive and reactive aspects to them. The reactivity has to co-exist with (higher-level) cognitive activities like planning and strategy assessment. The overall purpose of this research is to merge a machine-learning algorithm (SAMUEL, an evolutionary algorithm-based rule learning system) with a cognitive model (ACT-R) into a system where the learning algorithm handles the reactive aspects of the task and provides an adaptation mechanism, and where the behavior's realism is constrained by the cognitive model. Such a system would learn through experience and would be able to adapt to changes in adversaries' strategies and capabilities to present human opponents with more exciting, varied, yet realistic training situations. This preliminary work presents an initial examination of the effects of the changes in task reactivity on human and SAMUEL control abilities., Prepared in cooperation with ITT Industries, AES Division, Alexandria, VA.

While the tone of this paper is informal and tongue-in-cheek, we believe we raise two important issues in robotics and multi-modal interface research; namely, how crucial integration of multiple modes of communication are for adjustable autonomy, which in turn is crucial for having dinner with R2D2. Furthermore, we discuss how our multimodal interface to autonomous robots addresses these issues by tracking goals, allowing for both natural and mechanical modes of input, and how our robotic system adjusts itself to ensure that goals are achieved, despite interruptions., Sponsored in part by Office of Naval Research (ONR). Published by American Association for Artificial Intelligence (AAAI), 2000. The original document contains color images.

We are designing and implementing a multi-modal interface to an autonomous robot. For this interface, we have elected to use natural language and gesture. Gestures can be either natural gestures perceived by a vision system installed on the robot, or they can be made by using a stylus on a Personal Digital Assistant. In this paper we describe how we are attempting to provide a seamless integration of the various modes of input to provide a multi-modal interface that humans can manipulate as they desire. The interface will allow the user to choose whatever mode or combination of modes seems appropriate for interactions with the robot. The human user, therefore, does not have to be limited to any one mode of interaction, but can freely choose whatever mode is most comfortable or natural., Sponsored in part by Office of Naval Research (ONR). The original document contains color images.

This paper addresses issues of co-evolution of form and function for autonomous vehicles specifically evolving morphology and control for an autonomous micro air vehicle (MAV). The evolution of an optimal minimum sensor suite and reactive strategies for navigation and collision avoidance for the simulated MAV is described. The details of the implementation of the simulated aircraft, the environment, and the two cooperating genetic algorithm-based systems, SAMUEL and Genesis, used for evolution, are presented, as are preliminary results., Presented at the Genetic and Evlutionary Computafion Conference, GECCO-2000, Workshop on Evolution of Sensors in Nature, Hardware, and Simulation, Las Vegas, NV on 8 Jul 2000

This report provides abstracts for technical publications produced during 1998 by personnel of the Information Technology Division (ITD), one of the largest research and development collectives at the Naval Research Laboratory (NRL). The abstracts are organized into sections that represent the six branches within ITD: the Navy Center for Applied Research in Artificial Intelligence, Communication Systems, the Center for High Assurance Computer Systems, Transmission Technology, Advanced Information Technology, and the Center for Computational Science. Within each section. a list of branch papers published in 1997 has been included; abstracts for these papers may be found in prior-year editions of this report. Information on obtaining a copy of one or more of the abstracted or listed publications is also provided.

In this paper, we combine two frameworks in the context of an important application. The first framework, called "artificial physics," is described in detail in a companion paper by Spears and Gordon. The purpose of artificial physics is the distributed spatial control of large collections of mobile physical agents. The agents can be composed into geometric patterns (e.g., to act as a sensing grid) by having them sense and respond to local artificial forces that are motivated by natural physics laws. The purpose of the second framework is global monitoring of the agent formations developed with artificial physics. Using only limited global information, the monitor checks that the desired geometric pattern emerges over time as expected. If there is a problem, the global monitor steers the agents to self-repair. Our combined approach of local control through artificial physics, global monitoring, and "steering" for self-repair is implemented and tested on a problem where multiple agents form a hexagonal lattice pattern., Sponsored in part by Army Research Office Contracts nos. ARO DAAG55-98-1-0393 and DAAG55-98-1-0466 and by National Science Foundation Grant no.CCR-9619910. Prepared in collaboration with the Department of Computer and Information Science, University of Pennsylvania, Philadelphia, PA. Presented at the IEEE International Conference on Information, Intelligence, and Systems, Bethesda, MD, on 31 Oct-3 Nov 1999 and published in the proceedings of the same, p681-688, 1999. ISBN 0-7695-0446-9.

Intelligent mobile robots that interact with humans must exhibit adjustable autonomy; that is, the ability to dynamically adjust the level of self-sufficiency of an agent depending on the situation. When intelligent robots require close interactions with humans, they will require modes of communication that enhance the ability for humans to communicate naturally and that allow greater interaction, as well as adapt as a team member or sole agent in achieving various goals. Our previous work examined the use of multiple modes of communication, specifically natural language and gestures, to disambiguate the communication between a human and a robot. In this paper, we propose using context predicates to keep track of various goals during human-robot interactions. These context predicates allow the robot to maintain multiple goals, each with possibly different levels of required autonomy. They permit direct human interruption of the robot, while allowing the robot to smoothly return to a high level of autonomy.

This report provides abstracts for technical publications produced during 1997 by personnel of the Information Technology Division (ITD), one of the largest research and development collectives at the Naval Research Laboratory. The abstracts are organized into sections that represent the six branches with ITD: (1) the Navy Center for Applied Research in Artificial Intelligence, (2) Communication Systems, (3) the Center for High Assurance Computer Systems, (4) Transmission Technology, (5) Advanced Information Technology, and (6) the Center for Computational Science. Within each section, a list of branch papers published in 1995 and 1996 has also been included. Abstracts for these papers may be found in prior year editions of this report. Information on obtaining a copy of one or more of the abstracted or listed publications is provided as well.

NAC is a testbed for experimenting with concepts of retrieval and indexing in Case-Based Reasoning (CBR). The paper describes similarity functions and decision functions used for retrieval as well as methods for re-indexing and case organization. The paper also describes methods for weighting attributes, analyzing their dependence and evaluating the importance index of a single stored case. Two examples of how to use NAC are provided. Methods employed for retrieval and indexing are based on mathematically sound techniques developed in classification, clustering and decision analysis. NAC includes basic functions for specifying similarity, normalizing data and evaluation. Retrieval is done using both nonparametric (e.g., nearest neighbor) and parametric (i.e., Bayesian) statistical procedures with weighted attributes. New indices for cases are generated using clustering methods. Cases are re-organizing using the new indices. NAC also allows the user to test the predictive accuracy of retrieval methods and the quality of indices generated in re-indexing. NAC includes adaptive functions for enhancing the performance of retrieval and indexing. Important functions for adaptation include weighting and selecting attributes, learning dependency relationships and calculating typicality for each stored case. The NAC environment was designed so that additional techniques and metrics can easily be added.

Several commercial case-based reasoning (CBR) shells now use decision trees to index cases, including Remind (Cognitive Systems, Inc.), Kate (AcknoSoft), and The Easy Reasoner (The Haley Enterprise). These trees serve to expedite case retrieval and to generate comprehensible explanations of case retrieval behavior. Unfortunately, induced trees are often large and complex, reducing their explanatory power. To combat this problem, some commercial systems contain an option for simplifying decision trees. However, while many methods for simplifying decision trees exist, they have not been systematically compared and most have not been applied to case retrieval. This report builds on our previous survey and initial empirical comparison of tree simplification procedures. In this report, we compare them on a specific, challenging task that is the focus of an existing CBR effort. We examine which tree simplification procedures are useful for this task and suggest which ones should be included in a commercial CBR tool.

Eucalyptus and InterLACE are two projects integrating spoken natural language interaction with the already-developed graphical user interfaces of currently existing military simulation systems. We take this approach (rather than developing new, fully integrated systems from scratch) to facilitate the development of Navy demonstrations of NL technology applied to their own tools, as well as to explore the relative strengths and weaknesses of graphical vs. NL interfaces and how the two can be used in combination to yield more powerful interaction techniques. The natural language processing capability of the two systems is described in an accompanying Proceedings technical note (Wauchope et al. 1997); here we go into a bit more detail on the issues of multimodal integration with graphical interfaces.

This report provides abstracts for technical publications produced during 1995 by personnel of the Information Technology Division (ITD), one of the largest research and development collectives at the Naval Research Laboratory. The abstracts are organized into sections that represent the six branches with ITD: the Navy Center for Applied Research in Artificial Intelligence, Communications Systems, the Center for High Assurance Computer Systems, Transmission Technology, Advanced Information Technology, and the Center for Computational Science. Within each section, a list of branch papers published in 1993 and 1994 has also been included; abstracts for these papers may be found in prior-year editions of this report. Information on obtaining a copy of one or more of the abstracted orlisted publications is provided as well., Prepared in collaboration with Sabre Systems, Inc., Warminster, PA.

InterFIS is a natural language interface to the troubleshooting module of the Fault Isolation Shell (FIS), which is an expert system development tool for the diagnosis of failures in analog electronics equipment. The main functions of this FIS module are as follows: (1) to compute the probability that a particular fault hypothesis is correct after one or more tests have been performed on a particular piece of electronics equipment, and (2) to recommend the next best test based on information supplied by the diagnostician during a testing session. The original interface to FIS was standard keyboard input, where the appropriate abbreviations for all commands were displayed on the screen in a large list grouped by function. A simple graphic interface also was developed, where the user invoked the commands by clicking on screen buttons labeled according to their functional grouping. Later a natural language interface, InterFIS, was added. InterFIS is a natural language understanding interface that accepts typed English commands as input. The PROTEUS chart parser performs a syntactic analysis, producing an application-independent syntactic representation of the input sentence. This intermediate representation is mapped to domain-specific verb models by the semantic interpreter PFQAS and then converted to FIS commands by the command translator COIN. The main drawback to this interface is that typing English sentences is slow and requires the use of both hands. This report discusses the addition of speech recognition capabilities to InterFIS. Because of the limitations of today's speech recognition technology, the addition of this capability affects the structure and flexibility of the interface. The report describes the speech recognition module, and provides a brief evaluation of its performance.