Search

This paper addresses the role of conversational memory in Embodied Conversational Agents (ECAs). It describes an investigation into developing such a memory architecture and integrating it into an ECA. ECAs are virtual agents whose purpose is to engage in conversations with human users, typically through natural language speech. While several works in the literature seek to produce viable ECA dialog architectures, only a few authors have addressed the episodic memory architectures in conversational agents and their role in enhancing their intelligence. In this work, we propose, implement, and test a unified episodic memory architecture for ECAs. We describe a process that determines the prevalent contexts in the conversations obtained from the interactions. The process presented demonstrates the use of multiple techniques to extract and store relevant snippets from long conversations, most of whose contents are unremarkable and need not be remembered. The mechanisms used to store, retrieve, and recall episodes from previous conversations are presented and discussed. Finally, we test our episodic memory architecture to assess its effectiveness. The results indicate moderate success in some aspects of the memory-enhanced ECAs, as well as some work still to be done in other aspects.

This article describes and evaluates an approach to create and/or improve tactical agents through direct human interaction in real time through a force-feedback haptic device. This concept takes advantage of a force-feedback joystick to enhance motor skill and decision-making transfer from the human to the agent in real time. Haptic devices have been shown to have high bandwidth and sensitivity. Experiments are described for this new approach, named Instructional Learning. It is used both as a way to build agents from scratch as well as to improve and/or correct agents built through other means. The approach is evaluated through experiments that involve three applications of increasing complexity – chasing a fleer (Chaser), shepherding a flock of sheep into a pen (Sheep), and driving a virtual automobile (Car) through a simulated road network. The results indicate that in some instances, instructional learning can successfully create agents under some circumstances. However, instructional learning failed to build and/or improve agents in other instances. The Instructional Learning approach, the experiments, and their results are described and extensively discussed.

This article describes research to build an embodied conversational agent (ECA) as an interface to a question-and-answer (Q/A) system about a National Science Foundation (NSF) program. We call this ECA the LifeLike Avatar, and it can interact with its users in spoken natural language to answer general as well as specific questions about specific topics. In an idealized case, the LifeLike Avatar could conceivably provide a user with a level of interaction such that he or she would not be certain as to whether he or she is talking to the actual person via video teleconference. This could be considered a (vastly) extended version of the seminal Turing test. Although passing such a test is still far off, our work moves the science in that direction. The Uncanny Valley notwithstanding, applications of such lifelike interfaces could include those where specific instructors/caregivers could be represented as stand-ins for the actual person in situations where personal representation is important. Possible areas that come to mind that might benefit from these lifelike ECAs include health-care support for elderly/disabled patients in extended home care, education/training, and knowledge preservation. Another more personal application would be to posthumously preserve elements of the persona of a loved one by family members. We apply this approach to a Q/A system for knowledge preservation and dissemination, where the specific individual who had this knowledge was to retire from the US National Science Foundation. The system is described in detail, and evaluations were performed to determine how well the system was perceived by users.

This paper describes an interactive museum exhibit featuring an avatar of Alan Turing that informs museum visitors about artificial intelligence and Turing's seminal Turing Test for machine intelligence. The objective of the exhibit is to engage and motivate visiting children in the hope of sparking an interest in them about computer science and artificial intelligence, and cause them to consider pursuing future studies and/or careers in these fields. The exhibit interacts with the visitors, allowing them to participate in a simplified version of Turing's test that is brief and informal to suit the limitations of a five-minute exhibit. In this exhibit, the visitor (targeted towards middle school age children) invokes an avatar of his/her own choice, and acts to endow it with human-like qualities (voice, brain, eyesight and hearing). Then, the visitor engages the avatar in a (brief) question-and-answer session to determine whether the visitor thinks that he/she is interacting with a real human on a video conference or with an avatar. We consider this interaction to be an extension of the original Turing Test because, unlike Turing's original that used text via a teletype, this version features a graphical embodiment of an agent with which one can converse in spoken natural language. This extension serves to make passing the Turing Test more difficult, as now the avatar must not only communicate like a human, but also look, sound and act the part. It also makes the exhibit visual, dynamic and interesting to the visitors. Evaluations were performed with museum visitors, both in backrooms with prototypes as well as on the museum floor with the final version. The formative and summative evaluations performed indicated overall success in engaging the museum visitors and increasing their interest in computer science. More specifically, the formative testing, mostly done in quiet back rooms with selected test subjects, indicated that on the important questions about enjoyment of exhibit and increased interest in computer science by the test subjects, their self-reported Likert scale responses (1 being negative and 5 being positive) increased from 3.16 in the first evaluation to 4.38 in the third one for increased interest in CS. Likewise for the question about exhibit enjoyment (from 3.92 to 4.56). The summative evaluation, done through unobtrusive observation of exhibit use on museum floor, indicated that almost 74% of the parties that initiated the exhibit were either highly or moderately engaged by the exhibit. However, there was one major negative finding, namely the overly long duration of the exhibit, which may have caused premature abandonment of the exhibit in several cases during the summative evaluation. These tests and their results are presented and discussed in detail in this paper. The exhibit has been on permanent display at the Orlando (FL) Science Center since June 2014 and has received a strongly positive response from visitors since that time.

This paper presents a review of research works from the last several years in automated story generation systems. These systems are categorised into interactive story generation systems and non-Interactive story generation systems. Interactive systems are those that collaborate with a user/author during the process of creating and/or executing the story. The extent of user interaction varies across systems but remains an integral part of the creation and/or the unfolding of the story. Non-Interactive systems concentrate on complete automation of the creative process involved in narrative generation to create diverse and interesting stories. Interactive story generators specifically designed for video game narratives are reviewed as a separate sub-class of interactive story generation systems. Also reviewed are the methods used for evaluation of story generation systems as a way to explore the possibility of having standard methods of evaluation within the research community. The paper includes a discussion of trends and directions of the research discipline. [ABSTRACT FROM AUTHOR]

This paper explores the issues faced in creating a system that can learn tactical human behavior merely by observing a human perform the behavior in a simulation. More specifically, this paper describes a technique based on fuzzy ARTMAP (FAM) neural networks to discover the criteria that cause a transition between contexts during a strategic game simulation. The approach depends on existing context templates that can identify the high-level action of the human, given a description of the situation along with his action. The learning task then becomes the identification and representation of the context sequence executed by the human. In this paper, we present the FAM/Template-based Interpretation Learning Engine (FAMTILE). This system seeks to achieve this learning task by constructing rules that govern the context transitions made by the human. To evaluate FAMTILE, six test scenarios were developed to achieve three distinct evaluation goals: 1) to assess the learning capabilities of FAM; 2) to evaluate the ability of FAMTILE to correctly predict human and context selections, given an observation; and 3) more fundamentally, to create a model of the human's behavior that can perform the high-level task at a comparable level of proficiency. Index Terms--Context-Based Reasoning (CxBR), fuzzy ARTMAP (FAM), learning from observation, neural network, poker, template-based interpretation (TBI).

Human experts employed in validation exercises for knowledge-based systems (KBSs) often have limited time and availability. Furthermore, they often have different opinions from each other as well as from themselves over time. We address this situation by introducing the use of validation knowledge used in prior validation exercises for the same KBS. We present a validation knowledge base (VKB) that is the collective best experience of several human experts. The VKB is constructed and maintained across various validation exercises, and its primary benefits are given as follows: 1) more reliable validation results by incorporating external knowledge and 2) decrease of the experts' workload. We also present the concept of validation expert software agents (VESAs), which represent a particular expert's knowledge. VESA is a software agent corresponding to a specific human expert. It models the validation knowledge and behavior of its human counterpart by analyzing similarities with the responses of other experts. After a learning period, it can be used to temporarily substitute for its corresponding human expert. We also describe experiments with a small prototype system to evaluate the usefulness of these concepts. Index Terms--Knowledge-based systems (KBSs), systems, validation knowledge, validation of artificial intelligence (AI) systems.

It is widely accepted that the difficulty and expense involved in acquiring the knowledge behind tactical behaviors has been one limiting factor in the development of simulated agents representing adversaries and teammates in military and game simulations. Several researchers have addressed this problem with varying degrees of success. The problem mostly lies in the fact that tactical knowledge is difficult to elicit and represent through interactive sessions between the model developer and the subject matter expert. This paper describes a novel approach that employs genetic programming in conjunction with context-based reasoning to evolve tactical agents based upon automatic observation of a human performing a mission on a simulator. In this paper, we describe the process used to carry out the learning. A prototype was built to demonstrate feasibility and it is described herein. The prototype was rigorously and extensively tested. The evolved agents exhibited good fidelity to the observed human performance, as well as the capacity to generalize from it. Index Terms--Context-based reasoning, human behavioral modeling, genetic programming, simulation.

This paper presents and evaluates a method for automatically generating a narrative that includes initial input from the listener, without forcing him/her to become an actual character in the story. More importantly, the story can be modified by the listener during the course of the story being narrated. This latter feature works without pre-defining alternative paths, thereby permitting the changes to take place at any time throughout the course of the story. This system, called the Campfire Storytelling System, employs a contextual approach to narrative generation called the Cooperating Context Method (CCM). Campfire STS was assessed for its functionality and its user acceptance. Thirty-eight human test subjects evaluated stories generated by Campfire STS. They responded to a survey stating their opinions about the generated stories, their entertainment value for adults, their creativity, and other similar issues. Furthermore, 20 of these subjects who self-reported non-trivial experience with children were additionally asked to evaluate the generated stories with respect to how much children would enjoy them. The results of these evaluations suggest that Campfire STS is able to create interesting and novel stories, as well as change them seamlessly in mid-stream while still maintaining interesting and entertaining stories. [ABSTRACT FROM AUTHOR]

This paper introduces, describes, and evaluates real-time models of affective states of individual learners interacting with Intelligent Tutoring Systems. Computer-based instructors, like human instructors, should use affective information for adapting instruction. This requires an accurate representation of individual learner state during tutoring; however, models described in the literature are generalised and constructed offline. Such total population models have faced validation difficulty with individuals, while individualised models have had difficulties with offline creation and online use. The simultaneous creation and utilisation of an individualised model from sensor-based physiological measurements presents an attractive alternative. We present and evaluate approaches for building affective models during the tutoring session which address the difficulties present in real-time data streams. Additionally, this work examines the impact of occasional direct user query on model quality. The results indicate that individualised real-time model construction is comparable to offline equivalents, yet can be successfully applied in tutoring settings. [ABSTRACT FROM PUBLISHER]

This article describes a knowledge preservation and re-use tool designed to capture the knowledge of a specific individual at the US National Science Foundation, for later retrieval by successors after his retirement. The system is designed in a Q&A format, where it is sufficiently intelligent to ask for clarifying questions. The primary objective was to create a system that would result in acceptance of the system by the users. The domain of interest to be preserved and re-used was programmatic knowledge about the NSF Industry/University Collaborative Research Centers (I/UCRC) Program, and more specifically, the knowledge of its long-time director, Dr. Alex Schwarzkopf. The system is called AskAlex and it uses a trio of techniques to accomplish its objectives. Contextual graphs (CxG) are used as the basic knowledge representation structure. CxG's are assisted by a search engine and an ontology of terms to help find the proper contextual graph that can best answer the question being asked. Evaluations with users and potential users generally confirm our selection and provided some guidance for improvements in the system. [ABSTRACT FROM AUTHOR]

This paper describes research that addresses the problem of dialog management from a strong, context-centric approach. We further present a quantitative method of measuring the importance of contextual cues when dealing with speech-based human-computer interactions. It is generally accepted that using context in conjunction with a human input, such as spoken speech, enhances a machine's understanding of the user's intent as a means to pinpoint an adequate reaction. For this work, however, we present a context-centric approach in which the use of context is the primary basis for understanding and not merely an auxiliary process. We employ an embodied conversation agent that facilitates the seamless engagement of a speech-based information-deployment entity by its human end user. This dialog manager emphasizes the use of context to drive its mixed-initiative discourse model. A typical, modern automatic speech recognizer (ASR) was incorporated to handle the speech-to-text translations. As is the nature of these ASR systems, the recognition rate is consistently less than perfect, thus emphasizing the need for contextual assistance. The dialog system was encapsulated into a speech-based embodied conversation agent platform for prototyping and testing purposes. Experiments were performed to evaluate the robustness of its performance, namely through measures of naturalness and usefulness, with respect to the emphasized use of context. The contribution of this work is to provide empirical evidence of the importance of conversational context in speech-based human-computer interaction using a field-tested context-centric dialog manager. [ABSTRACT FROM AUTHOR]

This paper describes an investigation into creating agents that can learn how to perform a task by observing an expert, then seamlessly turn around and teach the same task to a less proficient person. These agents are taught through observation of expert performance and thereafter refined through unsupervised practice of the task, all on a simulated environment. A less proficient human is subsequently taught by the now-trained agent through a third approach--coaching, executed through a haptic device. This approach addresses tasks that involve complex psychomotor skills. A machine-learning algorithm called PIGEON is used to teach the agents. A prototype is built and then tested on a task involving the manipulation of a crane to move large container boxes in a simulated shipyard. Two evaluations were performed--a proficiency test and a learning rate test. These tests were designed to determine whether this approach improves the human learning more than self-experimentation by the human. While the test results do not conclusively show that our approach provides improvement over self-learning, some positive aspects of the results suggest great potential for this approach. [ABSTRACT FROM PUBLISHER]

This paper describes a two-phase approach for automating the agent-building process when the agent is to perform tactical tasks. The research is inspired by how humans learn—first by observation of a teacher's performance and then by practicing the performance themselves. The objectives of this approach are to produce a high-performing agent that 1) approaches or exceeds the proficiency of a human and 2) does so in a human-like manner. We accomplish these objectives by combining observational learning with experiential learning. These processes are executed sequentially, with the former creating a competent but somewhat limited human-like model from scratch, and the latter improving its performance without significantly eroding its human-like qualities. The process is described in detail, and test results confirming our hypothesis are described. [ABSTRACT FROM PUBLISHER]

This paper describes an extension to a context-driven agent representation paradigm that facilitates modeling collaborative tactical behaviors for simulations of team games or military missions. Called collaborative context-based reasoning, it emphasizes communication among the collaborating agents and carries it out by exchanging their currently active context when feasible. CCxBR is founded on the concepts defined in joint intention theory (JIT). The research described here presents an architecture that incorporates JIT in a contextual framework. The ability to facilitate communication among the collaborating agents by exchanging information about active contexts resembles the ability of humans to agree on a tactic in midstream and predict the behavior of their collaborators. This allows a CCxBR agent to invoke the actions involved in the tactic in the pursuit of a common goal. The paper describes several prototypes built to evaluate the CCxBR approach and the experiments executed to determine its effectiveness. The results of the experiments and the conclusions reached are discussed. © 2010 Wiley Periodicals, Inc. [ABSTRACT FROM AUTHOR]

This article describes an investigation into the feasibility of using contextual reasoning to monitor and supervise the collaborative work of several knowledge workers working together on a project. Managing large and complex projects is a difficult task that requires situational awareness by the project manager to be able to be proactive when possible and to react correctly in the presence of events. In complex projects, effective oversight of the project personnel and the progress of the project are essential in ensuring that project objectives are met. This is especially true of projects that require contributions from various experts, whose interaction may be limited to a Web-based collaborative tool. Such oversight is typically the job of a project manager who is tasked with avoiding cost overruns, shipment delays, and ensuring product effectiveness. We utilize context-based reasoning and contextual graphs as the tools of choice for implementing an agent that emulates the function of a competent project manager. We use rocket design and manufacture as the domain to evaluate our technique. We use a public domain rocket design software package developed by the National Aeronautics and Space Administration as a guide to the domain. The article describes the investigation, its results, and the related works in a collaborative design project. [ABSTRACT FROM AUTHOR]

We describe an investigation into how to automate after-action review (AAR) to provide non-trivial, individual feedback to trainees in a military training context. On a high-level basis, our approach is to monitor the actions of the trainee(s) and compare them with those of software agents (called expert agents) whose behavior represents that of an expert-level performer. By identifying and logging discrepancies between the trainee and the expert agent, a measure of valuable feedback can be given to the trainee to whom the expert agent was assigned to ‘shadow’. The comparisons are made in two dimensions concurrently: the physical dimension and the tactical dimension. In a physical comparison, the trainee is compared with the physical location of the expert agent. In the tactical comparison, the context of the agent is compared with that of the trainee. If the latter comparison agrees, then it can be said that the trainee is employing the same tactics as the agent. The context of the trainee is un-intrusively inferred through a novel process called context agents. A prototype is built and tested with data from an instrumented live exercise. Results indicate that the procedure has significant promise to provide much-needed automation in AAR. [ABSTRACT FROM PUBLISHER]

This paper formally describes the context-based reasoning (CxBR) paradigm. CxBR can be used to represent tactical human behavior in simulations or in the real world. In problem solving, the context can be said to inherently contain much knowledge about the situation in which the problem is to be solved and/or the environment in which it must be solved. This paper discusses some of the issues involved in a context-driven representation of human behavior and introduces a formal description of CxBR. © 2008 Wiley Periodicals, Inc. [ABSTRACT FROM AUTHOR]

Military training relies upon the after-action review (AAR) to provide feedback and instruction to the trainee. After-action review can refer to any number of activities including verbal feedback, review of audio and video recording, and playback of the training session. Training can be directed at an individual or group of individuals. The effectiveness of the review is dependent upon timely and accurate expert assessment. Traditionally, this assessment is made by a human expert; however, the need for this type of expertise far exceeds its availability. Modern intelligent computer systems cannot completely replace human expertise, but there are certainly places where an intelligent system can aid and add to the after-action review process. This paper examines the current state of the research and development in this area. In addition, some recent research in intelligent systems is reviewed for the possibility of use in the automation of military AAR. [ABSTRACT FROM PUBLISHER]