Communicating with Teams of Cooperative Robots

The authors are designing and implementing a multi-modal interface to a team of dynamically autonomous robots. For this interface, they have elected to use natural language and gesture. Gestures can be either natural gestures perceived by a vision system installed on the robot, or they can be made by using a stylus on a Personal Digital Assistant. In this paper,they describe the integrated modes of input and one of the theoretical constructs that they use to facilitate cooperation and collaboration among members of a team of robots. An integrated context and dialog processing component that incorporates knowledge of spatial relations enables cooperative activity between the multiple agents, both human and robotic.
Prepared in collaboration with the University of Missouri-Columbia, Computer Engineering and Computer Science Department, Columbia, MO and ITT Industries, Alexandria, VA. Sponsored in part by the Office of Naval Research. The original document contains color images.