Finding the FOO: A Pilot Study for a Multimodal Interface

In their research on intuitive means for humans and intelligent, mobile robots to collaborate, the authors use a multimodal interface that supports speech and gestural inputs. As a preliminary step to evaluate their approach and to identify practical areas for future work, they conducted a Wizard-of-Oz pilot study with five participants who each collaborated with a robot on a search task in a separate room. The goal was to find a sign in the robot's environment with the word "FOO" printed on it. Using a subset of their multimodal interface, participants were told to direct the collaboration. As their subordinate, the robot would understand their utterances and gestures, and recognize objects and structures in the search space. Participants conversed with the robot through a wireless microphone and headphone and, for gestural input, used a touch screen displaying alternative views of the robot's environment to indicate locations and objects.
Prepared in collaboration with the University of Missouri-Columbia, Computer Engineering and Computer Science Department, Columbia, MO. Sponsored in part by DARPA. The original document contains color images.