Search

Mixed reality systems seek to smoothly link the physical and data processing (digital) environments. Although mixed reality systems are becoming more prevalent, we still do not have a clear understanding of this interaction paradigm. Addressing this problem, this article introduces a new interaction model called Mixed Interaction model. It adopts a unified point of view on mixed reality systems by considering the interaction modalities and forms of multimodality that are involved for defining mixed environments. This article presents the model and its foundations. We then study its unifying and descriptive power by comparing it with existing classification schemes. We finally focus on the generative and evaluative power of the Mixed Interaction model by applying it to design and compare alternative interaction techniques in the context of RAZZLE, a mobile mixed reality game for which the goal of the mobile player is to collect digital jigsaw pieces localized in space.

L'Informatique Mobile definit un vaste domaine d'etudes pluridisciplinaires. Elle s'inscrit dans le mouvement de l'informatique pervasive, ubiquitaire et evanescente. La multiplication des ordinateurs de poche et des assistants personnels, la generalisation de l'utilisation des cartes a puces et la multiplication des systemes embarques dans des objets d'usage courant (automobile, television, etc.) sont des temoins de cet essor. L'explosion de la telephonie mobile constitue un autre exemple significatif de la formidable extension de ce domaine.

Mixed Reality seeks to smoothly link the physical and data processing (digital) environments. In recent years, mixed reality has been the subject of growing interest. Although mixed reality systems are becoming more prevalent, we still do not have a clear understanding of this interaction paradigm. This article introduces a new interaction model called Mixed Interaction model. It adopts a unifying point of view on mixed reality by considering the interaction modalities that are involved for defining mixed environments. This article presents the model after explaining its foundations, applies it to describe and compare existing systems and shows its generative power for designing new forms of mixed interaction.

In the physical world, most human activities involve the use of two hands. As part of ubiquitous computing that aims at combining the physical and the digital worlds, understanding how to design two-handed interaction is therefore important. This article presents a new design space for two-handed interaction, called 2M. The 2M design space takes its foundations in two complementary research domains: multimodal interaction and two-handed interaction. On the one hand we enrich the multimodal design space to take into account the roles of the two hands. On the other hand, we extend the two-handed interaction design space to cover the temporal as well as fusion aspects of multimodal interaction. This article presents the resulting 2M design space and demonstrates its descriptive power. We finally study the generative capacity of 2M for designing new forms of two-handed interaction in the context of the NavGraphe project that focuses on the interactive manipulation of very large biological structures (e.g., RNA structures).

Timelines are recurrently used to design visualization techniques of temporal data. In this paper, we present our perspective wall timeline to visualize temporal values. We illustrate how our timeline could be used by two examples of visualization techniques of temporal data: the star representation technique and the superposed histograms technique.In this paper, we propose to use a dedicated physical device for the perspective wall timeline, making the reusable corresponding software component a reusable physical device. Then, we present the results of an experiment we have conducted to compare the time device to the timeline (software component). Finally, we conclude and present our perspectives.

This article introduces a new kind of interface, the Phearcons, which are physical objects linked to audio sources that are spatialized according to the objects position and represent information sources. We aim at designing and developing a platfrom called Phearcons for specifying and running such interfaces. This article provides design rules as well as a process for the specification and implementation of such a platform.

Mobile and ubiquitous systems support multiple interaction techniques such as the synergistic use of active modalities (speech, gesture, etc.) and passive modalities (gaze, localization of a user, etc.). The flexibility they offer results in an increased complexity that current software development tools do not address appropriately. In this paper we describe a component-based approach, called ICARE, for specifying and developing interfaces combining active and passive modalities. Our approach relies on two types of components: (1) elementary components that describe pure modalities (active and passive) and (2) composition components (Complementarite, Redundancy and Equivalence) that enable the designer to specify combined usage of modalities. The designer graphically assembles the ICARE components and the code of the multimodal user interface is automatically generated. Although the ICARE platform is not fully developed, we illustrate the applicability of the approach with the implementation of three mobiles systems: two GeoNote systems and one prototype of cockpit commands of Rafale (French military plane).

Real needs of mobile users working in groups can be addressed by Mobile Collaborative Mixed Systems. But the design of such systems is a complex task due to the variety of design parameters. For example the characterization of the usage context is difficult because of the mobility of the users as well as the different types of collaboration amongst them. In this paper we present a characterization space and a notation as useful tools for the design of Mobile Collaborative Mixed Systems. Our two contributions are placed within a scenario based design process.

This paper describes a PAC-Amodeus software architectural solution for context-aware systems. The software architectural solution is based on an architecture dedicated to multimodal interaction. Indeed we argue in this paper that several software design issues are common between multimodal systems and context-aware systems. Our architectural solution is intentionally generic, intended to serve as the basis for a wide range of possible systems and compatible with the existing architectural approaches.

International audience; Dans cet article, nous présentons le modèle d'architecture logicielle Clover dédié aux collecticiels. Le modèle Clover résulte de la combinaison de l'approche en couches répliquées et partagées de Dewan avec la décomposition fonctionnelle du trèfle des collecticiels (Clover). Le trèfle des collecticiels définit trois classes de fonctionnalités, intitulées production, communication et coordination. Ces trois classes sont présentes dans chaque couche fonctionnelle de notre modèle. Nous illustrons la discussion au moyen du système CoVitesse, un système de navigation collaborative sur le WWW : l'architecture logicielle de CoVitesse découle de l'application de notre modèle Clover.

In this paper we present a unified framework for the design of audible interfaces. We describe the steps of the sonification process and their parameters. The process is modeled as a sequence of transformation functions from the data to present to the produced sounds.The usefulness of the framework for classifying existing audible interfaces and for designing new ones is then discussed.

Multimodal interactive systems support multiple interaction techniques such as the synergistic use of speech and direct manipulation. The flexibility they offer results in an increased complexity that current software tools do not address appropriately. One of the emerging technical problems in multimodal interaction is concerned with the fusion of information produced through distinct interaction techniques. In this article, we present a generic fusion engine that can be embedded in a multi-agent architecture modelling technique. We demonstrate the fruitful symbiosis of our fusion mechanism with PAC-Amodeus, our agentbased conceptual model, and illustrate the applicability of the approach with the implementation of an effective interactive system: MATIS, a Multimodal Airline Travel Information System.

Multimodal interaction enables the user to employ different modalities such as voice, gesture and typing for communicating with a computer. This paper presents an analysis of the integration of multiple communication modalities within an interactive system. To do so, a software engineering perspective is adopted. First, the notion of “multimodal system” is clarified. We aim at proving that two main features of a multimodal system are the concurrency of processing and the fusion of input/output data. On the basis of these two features, we then propose a design space and a method for classifying multimodal systems. In the last section, we present a software architecture model of multimodal systems which supports these two salient properties: concurrency of processing and data fusion. Two multimodal systems developed in our team, VoicePaint and NoteBook, are used to illustrate the discussion.