Search

This paper presents AirMouse, a new interaction technique based on finger gestures above the laptop΄s keyboard. At a reasonably low cost, the technique can replace the traditional methods for pointing in two or three dimensions. Moreover, the device-switching time is reduced and no additional surface than the one for the laptop is needed. In a 2D pointing evaluation, a vision-based implementation of the technique is compared with commonly used devices. The same implementation is also compared with the two most commonly used 3D pointing devices. The two user experiments show the benefits of the polyvalent technique: it is easy to learn, intuitive and efficient by providing good performance. In particular, our conducted experiment shows that performance with AirMouse is promising in comparison with a touchpad and with dedicated 3D pointing devices. It shows that AirMouse offers better performance as compared to FlowMouse, a previous solution using fingers above the keyboard. [ABSTRACT FROM AUTHOR]

Since R. Bolt's seminal "Put that there" demonstrator, more and more robust and innovative modalities can be used and empirical work on the usage of multiple modalities is now available for guiding the design of efficient and usable multimodal interfaces. This paper presents a system of patterns for capitalizing and formalizing this design knowledge about multimodal interfaces as patterns. Patterns are used for illustrating our system of patterns. [ABSTRACT FROM AUTHOR]

We present a test-bed platform for the iterative design of multimodal games on a mobile phone or a PDA. While our test-bed platform is general to multimodal systems, in this paper we focus on games on mobile devices, since games are intrinsically multimodal, enriching thus the gaming experience and potential for interaction innovations. As part of a player-centered design approach, our goal is to be able to quickly test various forms of multimodal interaction before beginning with the development of the final multimodal game on mobile devices. We adopt a micro-approach for our game test-bed, focusing on multimodal interaction for generic tasks, including moving of an object or a character. Our test-bed facilitates the efficient exploration of the multimodal design space and enables controlled experiments of the designed interaction in-lab and/or in-situ. [ABSTRACT FROM AUTHOR]

This paper presents a method for automatically testing interactive multimodal systems. The method is based on the Lutess testing environment, originally dedicated to synchronous software specified using the Lustre language. The behaviour of synchronous systems, consisting of cycles starting by reading an external input and ending by issuing an output, is to a certain extent similar to the one of interactive systems. Under this hypothesis, the paper presents our method for automatically testing interactive multimodal systems using the Lutess environment. In particular, we show that automatic test data generation based on different strategies can be carried out. Furthermore, we show how multimodality-related properties can be specified in Lustre and integrated in test oracles. [ABSTRACT FROM AUTHOR]

Interactive systems including multiple interaction devices and surfaces for supporting the collaboration of a group of co-located users are increasingly common in various domains. Nevertheless few collaborative and multimodal interface specification notations are proposed. As a first step towards a notation for specifying a design solution prior to its software design and development, we adopt an empirical approach. We applied and compared four existing notations for collaborative systems by considering a case study, namely, a system for supporting informal co-located collaboration in hospital work. This paper reports the conclusions from this empirical comparison. [ABSTRACT FROM AUTHOR]

The power and versatility of multimodal interfaces results in an increased complexity of the code to be developed. To address this problem of software development, multimodal interfaces therefore make necessary the definition of software development tools that satisfy specific requirements such as the fusion of data from different interaction modalities and the management of multiple processes including support for synchronization and race conditions between distinct interaction modalities. This chapter is a reflection on current software design methods and tools for multimodal interfaces. First we present the PAC-Amodeus software architectural model as a generic conceptual solution for addressing the challenges of data fusion and concurrent processing of data of multimodal interfaces. We then focus on software tools for developing multimodal interfaces. The existing tools dedicated to multimodal interfaces are currently few and limited in scope. Either they address a specific technical problem including the fusion mechanism, the composition of several devices and mutual disambiguation, or they are dedicated to specific interaction modalities such as gesture recognition, speech recognition or the combined usage of speech and gesture. Two platforms, ICARE and OpenInterface are more general and are able to handle various interaction modalities. We describe in detail these two platforms and illustrate them by considering developed multimodal interfaces. [ABSTRACT FROM AUTHOR]