Search

Cognitive architectures are computational theories of reasoning the human mind engages in as it processes facts and experiences. A cognitive architecture uses declarative and procedural knowledge to represent mental constructs that are involved in decision making. Employing a model of behavioral and perceptual constraints derived from a set of one or more scenarios, the architecture reasons about the most likely consequence(s) of a sequence of events. Reasoning of any complexity and depth involving computational processes, however, is often opaque and challenging to comprehend. Arguably, for decision makers who may need to evaluate or question the results of autonomous reasoning, it would be useful to be able to inspect the steps involved in an interactive, graphical format. When a chain of evidence and constraint-based decision points can be visualized, it becomes easier to explore both how and why a scenario of interest will likely unfold in a particular way., in SPIE Visualization and Data Analysis, Burlingame, CA, 4-6 Feb 2013.

Economic globalization and the needs of the intelligence community have brought ma-chine translation into the forefront. There are not enough skilled human translators to meet the growing demand for high quality transla-tions or good enough translations that suf-fice only to enable understanding. Much research has been done in creating transla-tion systems to aid human translators and to evaluate the output of these systems. Metrics for the latter have primarily focused on im-proving the overall quality of entire test sets but not on gauging the understanding of in-dividual sentences or paragraphs. Therefore, we have focused on developing a theory of translation effectiveness by isolating a set of translation variables and measuring their ef-fects on the comprehension of translations. In the following study, we focus on investi-gating how certain linguistic permutations, omissions, and insertions affect the under-standing of translated texts., Published in the Proceedings of the NAACL HLT 2012 Workshop on Predicting and Improving Text Readability for Target reader populations (PITR2012); Montreal, Canada: NAACL.

Prepared in collaboration with National Virtual Translation Center. Presented at Machine Translation (MT) Summit XII, Government User Session, held 26-30 Aug 2009 in Ottawa, Canada.

How should a robot represent and reason about spatial information when it needs to collaborate effectively with a human? The form of spatial representation that is useful for robot navigation may not be useful in higher-level reasoning or working with humans as a team member. To explore this question, we have extended previous work on how children and robots learn to play hide and seek to a human-robot team covertly approaching a moving target. We used the cognitive modeling system, ACT-R, with an added spatial module to support the robot's spatial reasoning. The robot interacted with a team member through voice, gestures, and movement during the team's covert approach of a moving target. This paper describes the new robotic system and its integration of metric, symbolic, and cognitive layers of spatial representation and reasoning for its individual and team behavior., Published in the Twenty-Second Conference on Artificial Intelligence (AAAI-07), held in Vancouver, Canada on 22-26 July 2007.

For the last few years, our lab has been attempting to build robots that are similar to humans in a variety of ways. Our goal has been to build systems that think and act like a person rather than look like a person since the state of the art is not sufficient for a robot to look (even superficially) like a human person. We believe that there are at least two reasons to build robots that think and act like a human. First, how an artificial system acts has a profound effect on how people act toward the system. Second, how an artificial system thinks has a profound effect on how people interact with the system., Published in Cognition and Multi-Agent Interaction: From Cognitive Modeling to Social Simulation as Chapter 10, p252-278, 2006. ISBN 0521839645 (Hardcover).

M%nmnoid robots sllch as Robona%t offer many opporftinities for advancing the llse of robots in complex enviromnents sllch as space, and for developn%nt of more effective interfaces for h%nmns to interact with then%. Once a s%fficiently high level of interaction between robots and h%nmns is achieved, the operation of and interaction with these robots will becon%e less of an additional b%rden for the h%nmns, and n%ore of a collaboration to achieve the ob\ectives of the task-at-hand. In this paper we describe om plans to endow Robona%t with cognitive capabilities which will sllppoIt collaboration between h%nmn astronallts and Robona%t. We bllild llpon ollr experience in naniral l%%%%1%%% %nderstanding, ge sflire recognition, spatial reasoning and cognitive n%odeling in achieving fl%ese goals., Pub. in International Journal of Humanoid Robotics, v2 n2 p181-201, 2005. The original document contains color images.

The effective use of humanoid robots in space will depend upon the efficacy of interaction between humans and robots. The key to achieving this interaction is to provide the robot with sufficient skills for natural communication with humans so that humans can interact with the robot almost as though it were another human. This paper describes the design, implementation, and capabilities of a robotic system architecture for a robot which can be used (at some level) to collaborate with a human. The capabilities required of the robot include voice recognition, natural language understanding, gesture recognition, spatial reasoning, and cognitive modeling with perspective-taking. These represent a small subset of potential capabilities humans utilize with one another in collaborating to perform a task in a complex environment, and barely scratches the surface of capabilities that one might want to build into an intelligent, collaborative robot. The capabilities described here have been successfully implemented and demonstrated on several mobile robotic platforms (Sofge, et al., 2004), and the authors are now porting them to Robonaut. They also are extending the capabilities of the cognitive architectures (both ACT-R/S and Polyscheme) and their perspective-taking cognitive models. Future work will focus on enhancing the cognitive models through expanded rule sets and cognitively plausible behaviors and reasoning mechanisms, and adding learning capabilities to the models so that the robots may be able to acquire new knowledge and skills through interaction with humans and while performing tasks., Published in Proceedings of the 16th IFAC Symposium on Automatic Control in Aerospace, 2004. Prepared in collaboration with the University of Missouri-Columbia, Electrical and Computer Engineering Department, Columbia, MO. The original document contains color images.

This study investigates how frames of reference are chosen in a dynamic navigational task. Participants issued verbal instructions to an animated robot and were provided with one of three views for navigating the animated robot around a virtual world. The different views included a flat two-dimensional (2D) North-up map, a three-dimensional (3D) robot's eye view of the world, and a 3D view from behind the robot (3D-Camera) in which depth cues were manipulated. Our results show people adopt an egocentric frame of reference when depth cues are salient and an exocentric reference frame when depth cues are absent. The results suggest the absence or presence of depth cues is a critical component in choosing a reference frame. We discuss the extension of Bryant and Tversky's (1999) theoretical framework to a dynamic environment, such as navigation., Presented at Human Factors and Ergonomics Society Annual Meeting (48th) held in New Orleans, LA on 20-24 Sep 2004. Pub. in the Proceedings of the Human Factors and Ergonomics Society Annual Meeting (48th), 2004. The original document contains color images.

One of the great challenges of putting humanoid robots into space is developing cognitive capabilities for the robots with an interface that allows human astronauts to collaborate with the robots as naturally and efficiently as they would with other astronauts. In this joint effort with NASA and the entire Robonaut team we are integrating natural language and gesture understanding, spatial reasoning incorporating such features as human-robot perspective taking, and cognitive model-based understanding to achieve this high level of human-robot interaction., Prepared in cooperation with NASA. Prepared in collaboration with University of Missouri, Columbia, MO. Sponsored in part by DARPA. The original document contains color images.

In conversation, people often use spatial relationships to describe their environment, e.g., "There is a desk in front of me and a doorway behind it", and to issue directives, e.g., "Go around the desk and through the doorway." In our research, we have been investigating the use of spatial relationships to establish a natural communication mechanism between people and robots, in particular, for novice users. In this paper, the work on robot spatial relationships is combined with a multi-modal robot interface. We show how linguistic spatial descriptions and other spatial information can be extracted from an evidence grid map and how this information can be used in a natural, human-robot dialog. Examples using spatial language are included for both robot-to-human feedback and also human-to-robot commands. We also discuss some linguistic consequences in the semantic representations of spatial and locative information based on this work., The original document contains color images. Prepared in collaboration with University of Missouri-Columbia, Columbia, MD 65211. Pub. in IEEE Transactions on SMC, Part C, Special Issue on Human-Robot Interaction, July 2002. Sponsored in part by DARPA and NRL.

In their research on intuitive means for humans and intelligent, mobile robots to collaborate, the authors use a multimodal interface that supports speech and gestural inputs. As a preliminary step to evaluate their approach and to identify practical areas for future work, they conducted a Wizard-of-Oz pilot study with five participants who each collaborated with a robot on a search task in a separate room. The goal was to find a sign in the robot's environment with the word "FOO" printed on it. Using a subset of their multimodal interface, participants were told to direct the collaboration. As their subordinate, the robot would understand their utterances and gestures, and recognize objects and structures in the search space. Participants conversed with the robot through a wireless microphone and headphone and, for gestural input, used a touch screen displaying alternative views of the robot's environment to indicate locations and objects., Prepared in collaboration with the University of Missouri-Columbia, Computer Engineering and Computer Science Department, Columbia, MO. Sponsored in part by DARPA. The original document contains color images.

Agents provide a flexible and scalable method of integrating artificial intelligence techniques on a single cohesive distributed computing system. We have designed and implemented an agent-based interface for autonomous control, and for providing web-based information retrieval, for a dynamically autonomous mobile robot. The robot implements and integrates a variety of artificial intelligence techniques including a multimodal interface that allows natural language understanding, gesture interpretation, simultaneous localization and mapbuilding, object identification and spatial reasoning. The agent-based interface augments these capabilities by providing a method of controlling the robot via the CoABS grid, and by providing the means for the robot/operator to request information available through the grid or through the web., Sponsored in part by DARPA (MIPR 02-M733-00). The original document contains color images.

One of the challenges in implementing a dynamically autonomous mobile robot is achieving a truly human-centric multimodal interface so that human operators can interact with the robot as naturally as they would with another human. Multiple artificial intelligence techniques may be integrated in a distributed computing system through use of an agent-based architecture. In this effort we utilize an agent-based architecture to achieve a multimodal human-centric interface for controlling a dynamically autonomous mobile robot. Capabilities provided by the architecture include natural language understanding, gesture understanding, localization and mapping for accurate navigation, and use of sensors and maps for spatial reasoning., The original document contains color images.

In July and August 2002, five research groups -- Carnegie Mellon University, Northwestern University, Swarthmore College, Metrica, Inc., and the Naval Research Laboratory -- collaborated and integrated their various robotic systems and interfaces to attempt The Robot Challenge held at the AAAI 2002 annual conference in Edmonton, Alberta. The goal of this year's Robot Challenge was to have a robot dropped off at the conference site entrance; negotiate its way at the site, using queries and interactions with humans and visual cues from signs, to the conference registration area; register for the conference; and then give a talk. Issues regarding human/robot interaction and interfaces, navigation, mobility, vision, to name but a few relevant technologies to achieve such a task, were put to the test. In this report we, the team from the Naval Research Laboratory, will focus on our portion of The Robot Challenge. We will discuss some lessons learned from collaborating and integrating our system with our research collaborators, as well as discuss what actually transpired -- what worked and what failed -- during the robot's interactions with conference attendees in achieving goals. We will also discuss some of the informal findings and observations collected at the conference during the interaction and navigation of the robot to complete its various goals., Sponsored in part by Office of Naval Research (N00014-01-WX-40001) and by DARPA (MIPR 02-L697-00). The original document contains color images.

In conversation, people often use spatial relationships to describe their environment, e.g., "There is a desk in front of me and a doorway behind it", and to issue directives, e.g., "Go around the desk and through the doorway." In our research, we have been investigating the use of spatial relationships to establish a natural communication mechanism between people and robots, in particular, for novice users. In this paper, the work on robot spatial relationships is combined with a multi-modal robot interface developed at the Naval Research Lab. We show how linguistic spatial descriptions and other spatial information can be extracted from an evidence grid map and how this information can be used in a natural, human-robot dialog., Prepared in collaboration with University of Missouri, Columbia, MO. Sponsored in part by Office of Naval Research (ONR). The original document contains color images.

No one claims that people must interact with machines in the same way that they interact with other humans. Certainly, people do not carry on conversations with their toasters in the morning, unless they have a serious problem. However, the situation becomes a bit more complex when we begin to build and interact with machines or robots that either look like humans or have functionalities and capabilities. Then, people well might interact with their humanlike machines in ways that mimic human-human communication. For example, if a robot has a face, a human might interact with it similarly to how humans interact with other creatures with faces. Specifically, a human might talk to it, gesture to it, smile at it, and so on. If a human interacts with a computer or a machine that understands spoken commands, the human might converse with the machine, expecting it to have competence in spoken language. In our research on a multimodal interface to mobile robots, we have assumed a model of communication and interaction that, in a sense, mimics how people communicate. Our interface therefore incorporates both natural language understanding and gesture recognition as communication modes. We limited the interface to these two models to simplify integrating them in the interface and to make our research more tractable. We believe that with an integrated system, the user is less concerned with how to communicate (which interactive mode to employ for a task) and is therefore free to concentrate on the tasks and goals at hand. Because we integrate all our system's components, users can choose any combination of our interface's modalities. The onus is on our interface to integrate the input, process it, and produce the desired results., The original document contains color images.

CONCLUSIONS: 1. By using "context predicates" we track actions occurring during a dialog to determine which goals (event and locative) have been achieved or attained and which have not. 2. By tracking "context predicates" we can determine what actions need to be acted upon next; i.e., predicates in the stack that have not been completed. 3. "Locative" expressions, e.g. "there," give us a kind of handle in command and control applications to attempt error correction when locative goals are being discussed. 4. By interleaving complex dialog with natural and mechanical gestures, we hope to achieve dynamic autonomy and an integrated multi-modal interface., The original document contains color images.

While the tone of this paper is informal and tongue-in-cheek, we believe we raise two important issues in robotics and multi-modal interface research; namely, how crucial integration of multiple modes of communication are for adjustable autonomy, which in turn is crucial for having dinner with R2D2. Furthermore, we discuss how our multimodal interface to autonomous robots addresses these issues by tracking goals, allowing for both natural and mechanical modes of input, and how our robotic system adjusts itself to ensure that goals are achieved, despite interruptions., Sponsored in part by Office of Naval Research (ONR). Published by American Association for Artificial Intelligence (AAAI), 2000. The original document contains color images.

We are designing and implementing a multi-modal interface to an autonomous robot. For this interface, we have elected to use natural language and gesture. Gestures can be either natural gestures perceived by a vision system installed on the robot, or they can be made by using a stylus on a Personal Digital Assistant. In this paper we describe how we are attempting to provide a seamless integration of the various modes of input to provide a multi-modal interface that humans can manipulate as they desire. The interface will allow the user to choose whatever mode or combination of modes seems appropriate for interactions with the robot. The human user, therefore, does not have to be limited to any one mode of interaction, but can freely choose whatever mode is most comfortable or natural., Sponsored in part by Office of Naval Research (ONR). The original document contains color images.

Unmanned vehicles, such as mobile robots, must exhibit adjustable autonomy. They must be able to be self-sufficient when the situation warrants; however, as they interact with each other and with humans, they must exhibit an ability to dynamically adjust their independence or dependence as co-operative agents attempting to achieve some goal. This is what we mean by adjustable autonomy. We have been investigating various modes of communication that enhance a robot's capability to work interactively with other robots and with humans. Specifically, we have been investigating how natural language and gesture can provide a user-friendly interface to mobile robots. We have extended this initial work to include semantic and pragmatic procedures that allow humans and robots to act co-operatively, based on whether or not goals have been achieved by the various agents in the interaction. By processing commands that are either spoken or initiated by clicking buttons on a Personal Digital Assistant and by gesturing either naturally or symbolically, we are tracking the various goals in the interaction, the agent involved in the interaction, and whether or not the goal has been achieved. The various agents involved in achieving the goals are each aware of their own and others' goals and what goals have been stated or accomplished so that eventually any member of the group, be it a robot or a human, if necessary, can interact with the other members to achieve the stated goals of a mission., The original document contains color images.

Intelligent mobile robots that interact with humans must be able to exhibit adjustable autonomy, that is the ability to dynamically adjust the level of autonomy of an agent depending on the situation. When intelligent robots require close interactions with humans, they will require modes of communication that enhance the ability for humans to communicate naturally and that allow greater interaction. Our previous work examined the use of multiple modes of communication, specifically natural language and gestures, to disambiguate the communication between a human and a robot. In this paper, we propose using context predicates to keep track of various goals during human-robot interactions. These context predicates allow the robot to maintain multiple goals, each with possibly different levels of required autonomy. They permit direct human interruption of the robot, while allowing the robot to smoothly return to a high level of autonomy.