Search

The new ubiquitous interaction methods change people's life and facilitate their tasks in everyday life and in the workplace, enabling people to access their personal data as well as public resources at any time and in any place. We found two solutions to enable ubiquitous interaction and put a stop to the limits imposed by the desktop mode: namely nomadism and mobility. Based on these two solutions, we have proposed three interfaces (Zhou et al. in HCI international 2011: human-computer interaction. Interaction techniques and environments, Springer, Berlin, pp 500-509, ): in-environment interface (IEI), environment dependent interface (EDI), and environment independent interface (EII). In this paper, we first discuss an overview of IEI, EDI, and EII, before excluding IEI and focusing on EDI and EII, their background, and distinct characteristics. We also propose a continuum from physical paper-based interface to digital projected interface in relation with EDI and EII. Then, to validate EDI and EII concepts, we design and implement a MobilePaperAccess system, which is a wearable camera-glasses system with paper-based interface and original input techniques allowing mobile interaction. Furthermore, we discuss the evaluation of the MobilePaperAccess system; we compare two interfaces (EDI and EII) and three input techniques (finger input, mask input, and page input) to test the feasibility and usability of this system. Both the quantitative and qualitative results are reported and discussed. Finally, we provide the prospects and our future work for improving the current approaches. [ABSTRACT FROM AUTHOR]

A multiple display environment (MDE) networks personal and shared devices to form a virtual workspace, and designers are just beginning to grapple with the challenges of developing interfaces tailored for these environments. To develop effective interfaces for MDEs, designers must employ methods that allow them to rapidly generate and test alternative designs early in the design process. Paper prototyping offers one promising method, but needs to be adapted to effectively simulate the use of multiple displays and allow testing with groups of users. In this paper, we share experiences from two projects in which paper prototyping was utilized to explore interfaces for MDEs. We identify problems encountered when applying the traditional method, describe how these problems were overcome, and distill our experiences into recommendations that others can draw upon. By following our recommendations, designers need only make minor modifications to the existing method to better realize benefits of paper prototyping for MDEs. [ABSTRACT FROM AUTHOR]

Traditional desktop computers have a tendency to drag users towards the desk and away from the object of their work. Researchers working within fields covered by the term ‘tangible computing’ are trying to find ways to integrate computer technology in a more radical way and minimise the gravitation to a desk. In this paper, we argue for different interactional features that are connected to a design idea for monitoring treatment sessions of remote haemodialysis. We discuss interactional features with a non-traditional computer interface in a highly mobile work environment. [ABSTRACT FROM AUTHOR]

An introduction is presented in which the editor discusses various papers within the issue on topics including 3D map streaming system for mobile phones, use of an advanced multitouch-based interface, and the problem of volume rendering of medical data sets on mobile device.

Children are very different from adults—they have developing motor skills, limited reach, short attention spans, limited exposure to traditional user interfaces and social protocols. How do we evaluate user interfaces for children? In this paper, we present four evaluative techniques that we have successfully used with children. [ABSTRACT FROM AUTHOR]

This paper proposes the “Substitute Interface” to utilize the flat surfaces of objects around us as part of an ad hoc mobile device. The substitute interface is established by the combination of wearable devices such as a head-mounted display with camera and a ring-type microphone. The camera recognizes which object the user intends to employ. When the user picks up and taps the object, such as a notebook, a virtual display is overlaid on the object, and the user can operate the ad hoc mobile device as if the object were part of the device. Display size can be changed easily by selecting a larger object. The user’s pointing/selection action is recognized by the combination of the camera and the ring-type microphone. We first investigate the usage scene of tablet devices and create a prototype that can operate as a tablet device. Experiments on the prototype confirm that the proposal functions as intended. [ABSTRACT FROM AUTHOR]

Recent advances in EEG technology makes brain-computer-interface (BCI) an exciting field of research. BCI is primarily used to adopt with the paralyzed human body parts. However, BCI in envisioned speech recognition using electroencephalogram (EEG) signals has not been studied in details. Therefore, developing robust speech recognition system using EEG signals was proposed. In this paper, we propose a coarse-to-fine-level envisioned speech recognition framework with the help of EEG signals that can be thought of as a serious contribution in this field of research. Coarse-level classification is used to differentiate/categorize text and non-text classes using random forest (RF) classifier. Next, a finer-level imagined speech recognition of each class has been carried out. EEG data of 30 text and not-text classes including characters, digits, and object images have been imagined by 23 participants in this study. A recognition accuracy of 85.20 and 67.03% has been recorded at coarse- and fine-level classifications, respectively. The proposed framework outperforms the existing research work in terms of accuracy. We also show the robustness in envisioned speech recognition. [ABSTRACT FROM AUTHOR]

Ubiquitous computing brings new parts of the world into contact with each other through digital devices. Consequently, it is important to understand what meaning is attached across contexts to particular interface choices, especially for the display of identity-related information. This understanding is made all the more critical when there are politically sensitive differences in status between areas (such as those implied by labelling one a 'developing region'). This paper examines the politics of designing interfaces by looking at the situation of two producers in developing regions and relating this to the outcomes of studying a map metaphor with potential consumers from India and the UK. It does so in the context of exploring the representation of food production information as interactive user-generated content. Its findings challenge the popular notion of national cultural norms and suggest that an interface, which emphasises geography, can hide inter-continental similarities and intra-national variation and thus provide new support for stereotyping by region. [ABSTRACT FROM AUTHOR]

This paper presents a driver simulator, which takes into account the information about the user’s state of mind (level of attention, fatigue state, stress state). The user’s state of mind analysis is based on video data and biological signals. Facial movements such as eyes blinking, yawning, head rotations, etc., are detected on video data: they are used in order to evaluate the fatigue and the attention level of the driver. The user’s electrocardiogram and galvanic skin response are recorded and analyzed in order to evaluate the stress level of the driver. A driver simulator software is modified so that the system is able to appropriately react to these critical situations of fatigue and stress: some audio and visual messages are sent to the driver, wheel vibrations are generated and the driver is supposed to react to the alert messages. A multi-threaded system is proposed to support multi-messages sent by the different modalities. Strategies for data fusion and fission are also provided. Some of these components are integrated within the first prototype of OpenInterface: the multimodal similar platform. [ABSTRACT FROM AUTHOR]

The world of ubiquitous computing, which by definition includes mobile devices of every kind, leads us to an era of small computer devices, usable in everyday situations. Computers are becoming smaller and operate discreetly in the background. This paper deals with the disappearance of the screen that is described and specified according to Lev Manovich. In doing research on radio frequency identification, this paper shows one possible way to interact with ubiquitous computers—primarily exploring suitability and scenarios for audible interfaces. The paper describes a research project of the University of Arts Berlin and the University of St. Gallen and proposes future research questions. [ABSTRACT FROM AUTHOR]

More and more connected systems are entering the social and shared home environment. Interaction with these systems is often rather individual and based on personal preferences, leading to conflicts in multi-user situations. In this paper, we aim to develop a perspective on how to design for multi-user interaction with connected lighting systems, based on a better understanding of real-life interpersonal lighting conflicts. In order to understand everyday lighting conflicts, including their causes and resolution strategies, we present two studies. First, we observe real-life lighting conflicts between couples living in single-room apartments. Using probes for data gathering followed by dyadic interviews, we identify the role of agreements on use in conflicts and we identify different types of conflicts (preference, activity, and attitude conflicts). Next, we take a more disruptive approach based on technology probes, where we provoke lighting conflicts in family living rooms to observe resolution strategies. We find that people try to avoid conflicts at all costs. If there is a risk that others are negatively affected by an adjustment, people rather not interact with the system at all. Based on these insights, we defined a perspective on designing for multi-user interaction that provides the user with the confidence that interactions are socially accepted. This assurance can be given by presenting the user with information leading to awareness about the acceptance of a lighting change by the other users. We advise on what information can be visualized, based on the three conflict types we observed in the study. The combination of a deeper understanding of conflicts and a perspective on multi-user interface design can serve as a starting point to design better multi-user interfaces for domestic connected systems. [ABSTRACT FROM AUTHOR]

The computer is no longer the center of attention. Thus, what we design is no longer the interface to the computer. Rather, what we design is a thing or an environment in which a computer might be used to create certain desired effects. Indeed, interaction design in a sense becomes the practice of giving form to artifacts or environments rather like any of the other design disciplines that we have know for centuries. However, giving form to computational things is highly complex and somewhat different than most other form-giving practices due to its temporal form element-its ability to change between states. Thus, an interaction design practice needs to encompass this temporal form giving in combination with physical form giving and performances of the interaction gestalt. In this paper, I propose this trinity of forms as a framework to unfold the practice of interaction design. I further demonstrate how computational composites present a way to work with the temporal form and the physical form in a process not too different from any traditional form-giving practice. Lastly, I point to some tools and techniques to deal with the interdependencies of the three form elements and thereby also demonstrate that a form-giving practice of interaction design is already well under way. [ABSTRACT FROM AUTHOR]

This paper describes an issue-based method to evaluate the naturalness of an interface. The method consists of the execution of a series of tasks on that interface, which is subsequently systematically analyzed to identify breakdowns in the users' actions. The systematic analysis of breakdowns is allowed by the support of video-coding software (The Observer by Noldus). This method is described on its theoretical bases and then applied to the evaluation of a natural interface, a walk-in-place locomotion system for virtual spaces called Superfeet. The procedure is comparative, since Superfeet is compared to two locomotion devices, Superfeet enhanced with headtracker and a more traditional Joypad. The test involves 36 participants (mean age = 23.68, SD = 3.14). The outcomes of the breakdown analysis are illustrated at a progressively finer level of granularity from the amount and length of breakdowns, to the circumstances of the breakdowns, to the type of actions involved in the breakdowns. The potential of this procedure for usability studies is finally synthesized. [ABSTRACT FROM AUTHOR]

Gestures are an important modality for human–machine communication. Computer vision modules performing gesture recognition can be important components of intelligent homes, assistive environments, and human–computer interfaces. A key problem in recognizing gestures is that the appearance of a gesture can vary widely depending on variables such as the person performing the gesture, or the position and orientation of the camera. This paper presents a database-based approach for addressing this problem. The large variability in appearance among different examples of the same gesture is addressed by creating large gesture databases, that store enough exemplars from each gesture to capture the variability within that gesture. This database-based approach is applied to two gesture recognition problems: handshape categorization and motion-based recognition of American Sign Language signs. A key aspect of our approach is the use of database indexing methods, in order to address the challenge of searching large databases without violating the time constraints of an online interactive system, where system response times of over a few seconds are oftentimes considered unacceptable. Our experiments demonstrate the benefits of the proposed database-based framework, and the feasibility of integrating large gesture databases into online interacting systems. [ABSTRACT FROM AUTHOR]

In this paper, we describe an intelligent location-aware city guide system, which adapts to each user’s preferences, and uses an intuitive “metal detector” interface for navigation. Our system analyzes each user’s past location data history to estimate individual preferences, and allows users to find shops that match their tastes in the same way a metal detector would be used to detect metal objects. The procedure with which the system picks out shops that match each user’s preferences includes a newly developed place learning algorithm, which can efficiently find frequented places, complete with their proper names (e.g. “The Ueno Royal Museum”). We have conducted a series of evaluation tests at a popular shopping district inside Tokyo, and the results validate the effectiveness of our overall approach. [ABSTRACT FROM AUTHOR]

This paper presents a novel interactive framework for 3D content-based search and retrieval using as query model an object that is dynamically sketched by the user. In particular, two approaches are presented for generating the query model. The first approach uses 2D sketching and symbolic representation of the resulting gestures. The second utilizes non-linear least squares minimization to model the 3D point cloud that is generated by the 3D tracking of the user’s hands, using superquadrics. In the context of the proposed framework, three interfaces were integrated to the sketch-based 3D search system including (a) an unobtrusive interface that utilizes pointing gesture recognition to allow the user manipulate objects in 3D, (b) a haptic–VR interface composed by 3D data gloves and a force feedback device, and (c) a simple air–mouse. These interfaces were tested and comparative results were extracted according to usability and efficiency criteria. [ABSTRACT FROM AUTHOR]

This paper describes the Mobile Fair Diary (MFD), which is designed to allow a housing fair customer to make a personalized digital recording of his/her visit to a hectic fairground for later use. The MFD is a hybrid interface service comprising of an application for a camera equipped smart phone and a website accessed by a PC. The smart phone application is used for taking context-aware notes such as visual codes, photos, dictations and text. The notes are uploaded onto a website, where they can be viewed with a PC in a contextually ordered view for browsing, organizing and sharing. The MFD was empirically evaluated in a real-life environment of use with genuine end users by a large-scale field trial at a national housing fair. The results testify to the very successful design of the MFD with high usefulness. [ABSTRACT FROM AUTHOR]

Ubicomp technology faces many technical challenges, which makes it difficult to test in real world situations. However, understanding and building for everyday practices is crucial for ubicomp designers, in order to push the technological development in the directions needed. We have developed and tested a ubiquitous computing prototype supporting collaborative scheduling. It is based on Pin&Play, a surface-based networking technology with interactive pushpins. The team of a local film festival was engaged in the development process, which resulted in a partial implementation illustrating how their current work practice could be supported. Drawing on this particular design case, we report findings and discuss challenges for ubicomp technology in general. [ABSTRACT FROM AUTHOR]

According to the vision of Ambient Intelligence, technology will seamlessly merge into people’s everyday activities and environments. A challenge facing designers of such systems is to create interfaces that fit in people’s everyday contexts and incorporate the values of daily life. This paper focuses on tangible expressive interaction as one possible approach towards linking everyday experiences to intuitive forms of interaction and presents a number of principles for expressive interaction design in this field. A case study of a tangible expressive interface to control a living room atmosphere projection system (orchestrating living room lighting, audio and video-art) is presented to illustrate and reflect upon the design principles. Furthermore, the case study describes possible techniques towards integrating the design principles into a design method. [ABSTRACT FROM AUTHOR]

The wide distribution of mobile vehicles installed with various sensing devices and wireless communication interfaces has made vehicular mobile crowd sensing possible in practice. However, owing to the heterogeneity of vehicles in terms of sensing interfaces and mobilities, collecting comprehensive tempo-spatial sensing data with only one sensing vehicle is impossible. Moreover, the sensing data collected may expire in the future; as a result, sensing vehicles may have to continuously collect sensing data to ensure the relevance of such data. Although including more sensing vehicles can improve the quality of collected sensing data, this step also requires additional cost. Thus, how to continuously collect comprehensive tempo-spatial sensing data with a limited number of heterogeneous sensing vehicles is a critical issue in vehicular mobile crowd sensing systems. In this work, a heterogeneous sensing vehicle selection (HVS) method for the collection of comprehensive tempo-spatial sensing data is proposed. On the basis of the spatial distribution and sensing interfaces of sensing vehicles and the tempo-spatial coverage of collected sensing data, a utility function is designed in HVS to estimate the sensing capacity of sensing vehicles. Then, according to the utilities of sensing vehicles and the restriction on the number of recruited sensing vehicles, sensing vehicle selection is modeled as a knapsack problem. Finally, a greedy optimal sensing vehicle selection algorithm is designed. Real trace-driven simulations show that the HVS algorithm can collect sensing data with a higher coverage ratio in a more uniform and continuous manner than existing mobile crowd sensing methods. [ABSTRACT FROM AUTHOR]

Some people cannot use their hands to control a computer mouse due to conditions such as cerebral palsy or multiple sclerosis. For these individuals, there are various mouse-replacement solutions. One approach is to enable them to control the mouse pointer using head motions captured with a web camera. One such system, the Camera Mouse, uses an optical flow approach to track a manually-selected small patch of the subject's face, such as the nostril or the edge of the eyebrow. The optical flow tracker may lose the facial feature when the tracked image patch drifts away from the initially-selected feature or when a user makes a rapid head movement. To address the problem of feature loss, we developed and incorporated the Kernel-Subset-Tracker into the Camera Mouse. The Kernel-Subset-Tracker is an exemplar-based method that uses a training set of representative images to produce online templates for positional tracking. We designed the augmented Camera Mouse so that it can compute these templates in real time, employing kernel techniques traditionally used for classification. We propose three versions of the Kernel-Subset-Tracker, each using a different kernel, and compared their performance to the optical-flow tracker under five different experimental conditions. Our experiments with test subjects show that augmenting the Camera Mouse with the Kernel-Subset-Tracker improves communication bandwidth statistically significantly. Tracking of facial features was accurate, without feature drift, even during rapid head movements and extreme head orientations. We conclude by describing how the Camera Mouse augmented with the Kernel-Subset-Tracker enabled a stroke-victim with severe motion impairments to communicate via an on-screen keyboard. [ABSTRACT FROM AUTHOR]

Ubiquitous computing is aiming at providing users with intelligent human-centric context-aware services at anytime anywhere. However, mobility increases dynamism and uncertainty conditions. This study therefore explores the management and uses of various contexts for automatically providing appropriate services to individual users. This issue is explored from an open framework perspective referred to as ubiquitous gate (U-gate). In this framework, a distributed context management architecture and a communication model based on standard protocols are proposed. To fit user requirements and to achieve complete mobility management, a context-aware path planning mechanism (UbiPaPaGo) and a context-aware handoff mechanism (UbiHandoff) are proposed based on context stored in an open and distributed context management server U-gate. Based on the path planning results of UbiPaPaGo, UbiHandoff derives a minimum access point (AP) handoff plan that satisfies multiple QoS requirements for individual users and services. The UbiHandoff mechanism includes multiple-attribute decision making method (MADM)-based handoff planning, referred to as MADM-based UbiHandoff, and genetic algorithm (GA)-based handoff planning, referred to as GA-based UbiHandoff. In the proposed MADM-based UbiHandoff, analytic hierarchy process (AHP) and technique for order preference by similarity to ideal solution (TOPSIS) provide efficient and seamless AP handoff to gain higher QoS performance. In GA-based UbiHandoff, genetic algorithm is employed to minimize handoff by finding appropriate APs along the path under QoS constraints. Finally, the effectiveness of the proposed mechanisms is evaluated through simulations. Numerical results demonstrate that both mechanisms minimize handoffs and ensure compliance with QoS requirements. [ABSTRACT FROM AUTHOR]

Multi- field inputs are techniques driven by multiple short-range RFID-enabled artifacts like RFID-tags and RFID-tag readers. The technology is useful for designers so as to enable the construction of advanced interaction through the physical world. To take advantage of such opportunities, it is important to understand the technology in terms of what interactions it might offer designers. I address this issue by unwrapping and exposing elements that can be used to conceptualize multi-field interactions. This is done by way of a design driven inquiry in which design and research methods are used to investigate short-range RFID technology. My approach is informed by activity theory which I use to analyze RFID technology from a design perspective. The study presents multi- field relations as a conceptual framework that can be used to describe and generate multi- field inputs. Four types of multi- field relations are discussed: one-way, two-way, sequence and multiple relations. These are described and analyzed in context of a set of multi-field input examples. The multi-field relations expose elements that can be used to construct interactions. This is important for interaction designers, since new interactions presents designers with opportunities for making entirely new types of interfaces that can lead to interesting and surprising experiences. [ABSTRACT FROM AUTHOR]

Poultry are one of the most badly treated animals in the modern world. It has been shown that they have high levels of both cognition and feelings and as a result there has been a recent trend of promoting poultry welfare. There is also a tradition of keeping poultry as pets in some parts of the world. However, in modern cities and societies, it is often difficult to maintain contact with pets, particularly for office workers. We propose and describe a novel cybernetics system to use mobile and Internet technology to improve human–pet interaction. It can also be used for people who are allergic to touching animals and thus cannot stroke them directly. This interaction encompasses both visualization and tactile sensation of real objects. [ABSTRACT FROM AUTHOR]

Abstract The severe resource restrictions of computer augmented everyday artifacts imply substantial problems for the design of applications in smart environments. Some of these problems can be overcome by exploiting the resources, I/O interfaces, and computing capabilities of nearby mobile devices in an ad-hoc fashion. We identify the means by which smart objects can make use of handheld devices such as PDAs and mobile phones., and derive the following major roles of handhelds in smart environments: (1) mobile infrastructure access point; (2) user interface; (3) remote sensor; (4) mobile storage medium; (5) remote resource provider; and (6) weak user identifier. We present concrete applications that illustrate these roles, and describe how handhelds can serve as mobile mediators between computer-augmented everyday artifacts, their users, and background infrastructure services. The presented applications include a remote interaction scenario, a smart medicine cabinet, and an inventory monitoring application. [ABSTRACT FROM AUTHOR]

In many environments, the landscape of space and artefacts is evolving and changing with the tasks at hand. Integrating digital media and computation in these environments has to take into account the fact that people will configure space functions and tools according to the situation, organising use in unexpected ways. In this article, we present and discuss how the issue of configurability is dealt with, in a series of field trials with design students. The aim of these trials was to construct, for architecture and interaction design students, a mixed-media environment for inspirational learning. The results include physical infrastructure in space and in furniture as integral parts of the interaction technology and the creation of composite representations called “mixed objects‘’, which blend digital and non-digital media in one design artefact. Configurability has to be supported at different levels (infrastructures, artefacts, functions) and across the physical and digital realms. [ABSTRACT FROM AUTHOR]

This article provides a historical overview of educational computing research at MIT from the mid-1960s to the present day, focusing on physical interfaces. It discusses some of the results of this research: electronic toys that help children develop advanced modes of thinking through free-form play. In this historical context, the article then describes and discusses the author’s own research into tangible programming, culminating in the development of the Tangible Programming Bricks system—a platform for creating microworlds for children to explore computation and scientific thinking. [ABSTRACT FROM AUTHOR]

Information technology research has mainly focussed on supporting visitors with guides, whilst a lot of opportunities for the development of technology for collecting, manipulating and re-travelling through material from the visits remain unexplored. On the basis of observations from a specific setting, we have developed prototype technology to combine multimedia content with position information. In particular, we developed mobile support to record and organise multimedia using the walked path in order to preserve more of the experience of the visit. We created tools to edit and store multimedia paths. Additional components make it possible to configure a mixed-media environment to navigate a multimedia path, using such physical interfaces as gesturing, and link the recorded media to other artefacts. When discussing field observations of current practices and prototype trials, we investigate what kind of functionality is needed to support the whole activity of collecting, manipulating and playing multimedia content in combination with position information. [ABSTRACT FROM AUTHOR]