Search

This paper investigates the feasibility of extracting clickstream data from network traces. Clickstream data capture the user Web browsing behavior and require the user's consent for the acquisition of this data directly from the browser. Extracting this data from network traces will provide another source of data for research as well as enable forensic Web analytics.

With the countless number of existing websites alongside the virtually unrestricted growth of the World Wide Web, the Web has no boundaries. As a result, there is an increasing need to automatically categorize and classify web sites into genres in order to improve the personalization of search results. This paper will offer conceptual suggestions on how online images can be used to predict the genre of the website that they are found on, as well as the process for detecting and identifying certain specific images for genre categorization.

In heterogeneous multi-agent systems, where human and non-human agents coexist, intelligent proxy agents can help smooth out fundamental differences. In this context, delegating the coordination role to proxy agents can improve the overall outcome of a task at the expense of human cognitive overload due to switching subtasks. Stability and commitment are characteristics of human teamwork, but must not prevent the detection of better opportunities. In addition, coordination proxy agents must be trained from examples as a single agent, but must interact with multiple agents. We apply machine learning techniques to the task of learning team preferences from mixed-initiative interactions and compare the outcome results of different simulated user patterns. This chapter introduces a novel approach for the adjustable autonomy of coordination proxies based on the reinforcement learning of abstract actions. In conclusion, some consequences of the symbiotic relationship that such an approach suggests are discussed.

SSTR operations are becoming a priority for the Department of Defense (DoD) and are being given the same stature as the more traditional combat operations. The near-term goal of SSTR is to provide the local populace with security, restore essential services, and meet humanitarian needs, while in the longer term ensure a stable infrastructure. Large scale disasters are an example where SSTR operations can provide value. In such scenarios, local governments and non-governmental institutions are under great stress in order to respond in a timely manner to provide basic relief to the affected communities, and there may be outburst of aid from the local groups. All of this can lead to a less than desirable outcome if poorly coordinated! The use of unmanned aerial vehicles (UAVs) to support intelligence, surveillance and reconnaissance (ISR) is becoming increasingly important. These assets can enable the collection of needed information for the execution of a given set of tasks. In large scale operations, however, the ability for the UAVs to self-coordinate may be needed as it will be difficult for human operators to effectively control large teams of UAVs. This paper will begin by introducing some of the key aspects of multiagent coordination, with a focus on the operational challenges with regard to SSTR such as disaster management response as well as UAV coordination. We will then discuss the coordination challenges and gaps in order to motivate an adaptive, multiagent based approach to coordination as well as additional opportunities for research. We will conclude with a brief summary

Two important enabling but evolving technologies supporting future DoD network-centric systems at the tactical edge are, mobile ad hoc networking (MANET) and Multi-Agent Systems (MAS). Despite their value in enabling more autonomous network system operation scenarios, open research and engineering questions remain regarding robust interoperation, standardization, and design of these two technologies. We describe recent research and development that is helping to better understand crosslayer design issues within both MAS and MANET. We describe the problem area and the open software components developed to support our research. We summarize recent modeling and simulation advances in a mixed MAS and MANET scenario environment. MANET multicast approaches for interagent communications are discussed and described. Some early analysis of MAS performance is presented using a variety of interagent MANET communication models. The behavior of MAS autonomous cooperative teamwork and role allocation within disruptive and dynamic MANET environments is examined. We conclude by outlining open issues and areas of further work.

A number of technologies are evolving that will help formulate more adaptive and robust network architectures intended to operate in dynamic, mobile environments. One technology area, mobile ad hoc networking (MANET) enables self-organizing, multi-hop heterogeneous network routing services and organization. Such technology is important in future DoD networking, especially in the forward edge of the battlespace where self-organizing, robust networking is needed. A second technology area, multi-agent systems (MAS) can enable autonomous, team-based problem solving under varying environmental conditions. Previous work done in MAS has assumed relatively benign wired network behavior and inter-agent communications characteristics that may not be well supported in MANET environments. In addition, the resource costs associated with performing inter-agent communications have a more profound impact in a mobile wireless environment. The combined operation of these technology areas, including cross-layer design considerations, has largely been unexplored to date. This paper describes ongoing research to improve the ability of these technologies to work in concert. An outline of various design and system architecture issues is first presented. We then describe models, agent systems, MANET protocols, and additional components that are being applied in our research. We present an analysis method to measure agent effectiveness and early evaluations of working prototypes within MANET environments. We conclude by outlining some open issues and areas of further work

This paper proposes a new approach to multi-agent systems leveraging from recent advances in networking and reinforcement learning to scale up teamwork based on joint intentions. In this approach, teamwork is subsumed by the coordination of learning agents. The intuition behind this approach is that successful coordination at the global level generates opportunities for teamwork interactions at the local level and vice versa. This unique approach scales up model-based teamwork theory with an adaptive approach to coordination. Some preliminary results are reported using a novel coordination evaluation.

On-policy reinforcement learning provides online adaptation, a characteristic of intelligent systems and lifelong learning. Unlike dynamic programming, an exhaustive sweep of the search space is not necessary for convergence in reinforcement learning with an efficient exploration strategy. For efficient and "believable" online performance, an exploration strategy also has to avoid cycling through previous solutions and know when to stop without getting stuck in a local optimum. This paper addresses the above problem with tabu search (TS) exploration. Several strategies for reinforcement learning are introduced. Experimental results are presented in the game of Go, a deterministic, perfect-information two-player game, and Sarsa learning vector quantization (SLVQ), an on-policy reinforcement learning algorithm.

This paper describes a novel hybrid reinforcement learning algorithm, Sarsa Learning Vector Quantization (SLVQ), that leaves the reinforcement part intact but employs a more effective representation of the policy function using a piecewise constant function based upon "policy prototypes". The prototypes correspond to the pattern classes induced by the Voronoi tessellation generated by self-organizing methods like Learning Vector Quantization (LVQ). The determination of the optimal policy function can be now viewed as a pattern recognition problem in the sense that the assignment of an action to a point in the phase space is similar to the assignment of a pattern class to a point in phase space. The distributed LVQ representation of the policy function automatically generates a piecewise constant tessellation of the state space and yields in a major simplification of the learning task relative to the standard reinforcement learning algorithms for whom a discontinuous table look function has to be learned. The feasibility and comparative advantages of the new algorithm is shown on the cart centering and mountain car problems, two control problems of increased difficulty.

This paper shows that the competitive learning rule found in learning vector quantization (LVQ) serves as a promising function approximator to enable reinforcement learning methods to cope with a large decision search space, defined in terms of difference classes of input patterns, like those found in the game of Go. This paper describes S[arsa]LVQ, a novel reinforcement learning algorithm and shows its feasibility for Go. As the distributed LVQ representation corresponds to a (quantized) codebook of compressed and generalized pattern templates, the state space requirements for online reinforcement methods are significantly reduced, thus decreasing the complexity of the decision space and consequently improving the play performance. As a result of competitive learning, SLVQ can win against heuristic players and starts to level off against stronger opponents such as Wally. SLVQ outperforms S[arsa]Linear when playing against both a heuristic player and Wally. Furthermore, while playing Go, SLVQ learns to stay alive while SLinear fails to do so.

In July and August 2002, five research groups -- Carnegie Mellon University, Northwestern University, Swarthmore College, Metrica, Inc., and the Naval Research Laboratory -- collaborated and integrated their various robotic systems and interfaces to attempt The Robot Challenge held at the AAAI 2002 annual conference in Edmonton, Alberta. The goal of this year's Robot Challenge was to have a robot dropped off at the conference site entrance; negotiate its way at the site, using queries and interactions with humans and visual cues from signs, to the conference registration area; register for the conference; and then give a talk. Issues regarding human/robot interaction and interfaces, navigation, mobility, vision, to name but a few relevant technologies to achieve such a task, were put to the test. In this report we, the team from the Naval Research Laboratory, will focus on our portion of The Robot Challenge. We will discuss some lessons learned from collaborating and integrating our system with our research collaborators, as well as discuss what actually transpired -- what worked and what failed -- during the robot's interactions with conference attendees in achieving goals. We will also discuss some of the informal findings and observations collected at the conference during the interaction and navigation of the robot to complete its various goals.

We are designing and implementing a multi-modal interface to a team of dynamically autonomous robots. For this interface, we have elected to use natural language and gesture. Gestures can be either natural gestures perceived by a vision system installed on the robot, or they can be made by using a stylus on a Personal Digital Assistant. In this paper we describe the integrated modes of input and one of the theoretical constructs that we use to facilitate cooperation and collaboration among members of a team of robots. An integrated context and dialog processing component that incorporates knowledge of spatial relations enables cooperative activity between the multiple agents, both human and robotic.