Search

Currently process engineers are using documents or authoring tools to bring the assembly instructions to the work floor. This is a time-consuming task, as instructions need to be created for each assembly operation. Furthermore, the engineer needs to be familiar with the assembly sequence. To assist the engineer, a tool is developed that i) uses a heuristic based on visibility, part similarity and proximity to semi-automatically determine the assembly sequence from a CAD model and ii) according to the computed sequence generates digital work instructions including visualizations and animations extracted from the CAD model. In essence, the assembly sequence generation works reversely: it determines the order in which components can be removed from the assembly, by evaluating whether the visibility of a component is obstructed by the remaining assembly. The reversed order is then returned as assembly sequence. During this process the engineer can modify the proposed sequence, add annotations and alter the vi-sualizations of the proposed instructions, i.e., images or 3D-animations. We illustrate that the developed tool effectively supports process engineers and speeds up the creation of digital work instructions by some industrial validation cases, e.g., the assembly of a weaving machine. This research is supported by VLAIO (Flanders Innovation & En- trepreneurship) Flanders Make, the strategic center for the man- ufacturing industry in Flanders, within the framework of the Op- eratorKnowledge ICON-project ( HBC.2017.0395 ). We like to thank Jens Brulmans, Niels Billen, Robbe Cools and Georges Verpoorten for their contributions to the implementation, and our partners Arkite, Augnition, Barco, CNHi, Atlas Copco, ASCO and Picanol for providing test data and context for this work.

Funding Information: This work was partly funded by the Department of Communications and Networking (Comnet, Aalto University), and Academy of Finland projects ‘Human Automata’ and ‘BAD’. This research was partially funded by Flanders Make, the strategic research centre for the manufacturing industry, through its projects ‘SmartHandler’ and ‘Ergo-EyeHand’. Publisher Copyright: © 2021 ACM. The promise of on-body interactions has led to widespread development of wearable displays. They manifest themselves in highly variable shapes and form, and are realized using technologies with fundamentally different properties. Through an extensive survey of the field of wearable displays, we characterize existing systems based on key qualities of displays and wearables, such as location on the body, intended viewers or audience, and the information density of rendered content. We present the results of this analysis in an open, web-based interactive design space that supports exploration and refinement along various parameters. The design space, which currently encapsulates 129 cases of wearable displays, aims to inform researchers and practitioners on existing solutions and designs, and enable the identification of gaps and opportunities for novel research and applications. Further, it seeks to provide them with a thinking tool to deliberate on how the displayed content should be adapted based on key design parameters. Through this work, we aim to facilitate progress in wearable displays, informed by existing solutions, by providing researchers with an interactive platform for discovery and reflection.

In domains where users are exposed to large variations in visuo-spatial features among designs, they often spend excess time searching for common elements (features) on an interface. This article contributes individualised predictive models of visual search, and a computational approach to restructure graphical layouts for an individual user such that features on a new, unvisited interface can be found quicker. It explores four technical principles inspired by the human visual system (HVS) to predict expected positions of features and create individualised layout templates: (I) the interface with highest frequency is chosen as the template; (II) the interface with highest predicted recall probability (serial position curve) is chosen as the template; (III) the most probable locations for features across interfaces are chosen (visual statistical learning) to generate the template; (IV) based on a generative cognitive model, the most likely visual search locations for features are chosen (visual sampling modelling) to generate the template. Given a history of previously seen interfaces, we restructure the spatial layout of a new (unseen) interface with the goal of making its features more easily findable. The four HVS principles are implemented in Familiariser, a web browser that automatically restructures webpage layouts based on the visual history of the user. Evaluation of Familiariser (using visual statistical learning) with users provides first evidence that our approach reduces visual search time by over 10%, and number of eye-gaze fixations by over 20%, during web browsing tasks. The project has partially received funding from the Academy of Finland project COMPUTED and the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement 637991).

Physical exercise training and medication compliance are primary components of cardiac rehabilitation. When rehabilitating independently at home, patients often fail to comply with their prescribed medication and find it challenging to interpret exercise targets or be aware of the expected efforts. Our work aims to assist cardiac patients in understanding their condition better, promoting medication adherence and motivating them to achieve their exercise targets in a tele-rehabilitation setting. We introduce a patient-centric intelligible visualization approach to present prescribed medication and exercise targets to patients. We assessed efficacy of intelligible visualizations on patients’ comprehension in two lab studies. We evaluated the impact on patient motivation and health outcomes in field studies. Patients were able to adhere to medication prescriptions, manage their physical exercises, monitor their progress and gained better self-awareness on how they achieved their rehabilitation targets. Patients confirmed that the intelligible visualizations motivated them to achieve their targets better. We observed an improvement in overall physical activity levels and health outcomes of patients. Research Highlights Presents challenges currently faced in cardiac tele-rehabilitation. Demonstrates how intelligibility was applied to two core aspects of cardiac rehabilitation- promoting medication adherence and physical exercise training. Lab., field and clinical studies to demonstrate efficacy of intelligible visualization, impact on patient motivation and resultant health outcomes. Reflection on how similar HCI approaches could be leveraged for technology-supported management of critical health conditions such as cardiac diseases.

Feedback and feedforward are two fundamental mechanisms that support users’ activities while interacting with computing devices. While feedback can be easily solved by providing information to the users following the triggering of an action, feedforward is much more complex as it must provide information before an action is performed. For interactive applications where making a mistake has more impact than just reduced user comfort, correct feedforward is an essential step toward correctly informed, and thus safe, usage. Our approach, Fortunettes, is a generic mechanism providing a systematic way of designing feedforward addressing both action and presentation problems. Including a feedforward mechanism significantly increases the complexity of the interactive application hardening developers’ tasks to detect and correct defects. We build upon an existing formal notation based on Petri Nets for describing the behavior of interactive applications and present an approach that allows for adding correct and consistent feedforward.

Makerspaces are creative and learning environments, home to activities such as fabrication processes and Do-It-Yourself (DIY) tasks. However, containing equipment that are not commonly seen or handled, these spaces can look rather challenging to novice users. This paper is based on the Smart Makerspace research from Autodesk, which uses a smart workbench for an immersive instructional space for DIY tasks. Having its functionalities in mind and trying to overcome some of its limitations, we approach the concept building an immersive instructional space as a web platform. The platform, introduced to users in a makerspace, had a feedback that reflects its potential between novice and intermediate users, for creating facilitators and encouraging these users.

This study focuses on innovative ways to digitally instrument the servicescape in bricks-and-mortar retailing. In the present digital era, technological developments allow for augmenting the shopping experience and capturing moments-of-truth along the shopper's path-to-purchase. This article provides an encompassing inventory of retail technologies resulting from a systematic screening of three secondary data sources, over 2008–2016: (1) the academic marketing literature, (2) retailing related scientific ICT publications, and (3) business practices (e.g., publications from retail labs and R&D departments). An affinity diagram approach allows for clustering the retail technologies from an HCI perspective. Additionally, a categorization of the technologies takes place in terms of the type of shopping value that they offer, and the stage in the path-to-purchase they prevail. This indepth analysis results in a comprehensive inventory of retail technologies that allows for verifying the suitability of these technologies for targeted in-store shopper marketing objectives (cf. the resulting online faceted-search repository at www.retail-tech.org). The findings indicate that the majority of the inventoried technologies provide cost savings, convenience and utilitarian value, whereas few offer hedonic or symbolic benefits. Moreover, at present the earlier stages of the path-to-purchase appear to be the most instrumented. The article concludes with a research agenda. This research has been funded by the Digitopia and Flanders Innovation & Entrepreneurship Baekeland grant number 150726 (‘In search of a sustainable competitive advantage: Digitally instrumenting bricks-and-mortar retailing in Flanders’; April 2016–March 2020). The authors furthermore acknowledge Randy Lauriers for her input in the data collection process for this study and the two anonymous reviewers whose suggestions helped in further strengthening a previous version of this manuscript.

Fig. 1. Based on self-sustaining predictions (e.g. the sun will set), FORTNIoT can deduce when trigger-condition-action rules (e.g. IF sun set AND anyone home THEN lower the rolling shutter) will trigger in the near future and what effects they will cause (e.g. the rolling shutter will lower). Ubiquitous environments, such as smart homes, are becoming more intelligent and autonomous. As a result, their behavior becomes harder to grasp and unintended behavior becomes more likely. Researchers have contributed tools to better understand and validate an environments' past behavior (e.g. logs, end-user debugging), and to prevent unintended behavior. There is, however, a lack of tools that help users understand the future behavior of such an environment. Information about the actions it will perform, and why it will perform them, remains concealed. In this paper, we contribute FORTNIoT, a well-defined approach that combines self-sustaining predictions (e.g. weather forecasts) and simulations of trigger-condition-action rules to deduce when these rules will trigger in the future and what state changes they will cause to connected smart home entities. We implemented a proof-of-concept of this approach, as well as a visual demonstrator that shows such predictions, including causes and effects, in an overview of a smart home's behavior. A between-subject evaluation with 42 participants indicates that FORTNIoT predictions lead to a more accurate understanding of the future behavior, more confidence in that understanding, and more appropriate trust in what the system will (not) do. We envision a wide variety of situations where predictions about the future are beneficial to inhabitants of smart homes, such as debugging unintended behavior and managing conflicts by exception, and hope to spark a new generation of intelligible tools for ubiquitous environments.

When using computer-aided translation systems in a typical, professional translation workflow, there are several stages at which there is room for improvement. The SCATE (Smart Computer-Aided Translation Environment) project investigated several of these aspects, both from a human-computer interaction point of view, as well as from a purely technological side. This paper describes the SCATE research with respect to improved fuzzy matching, parallel treebanks, the integration of translation memories with machine translation, quality estimation, terminology extraction from comparable texts, the use of speech recognition in the translation process, and human computer interaction and interface design for the professional translation environment. For each of these topics, we describe the experiments we performed and the conclusions drawn, providing an overview of the highlights of the entire SCATE project. The research in this project was funded by the Flemish Agency for Innovation and Technology IWT, project number 13007.

International audience; Feedback is commonly used to explain what happened in an interface. What if questions, on the other hand, remain mostly unanswered. In this paper, we present the concept of enhanced widgets capable of visualizing their future state, which helps users to understand what will happen without committing to an action. We describe two approaches to extend GUI toolkits to support widget-level feedforward, and illustrate the usefulness of widget-level feedforward in a standardized interface to control the weather radar in commercial aircraft. In our evaluation, we found that users require less clicks to achieve tasks and are more confident about their actions when feedforward information was available. These findings suggest that widget-level feedforward is highly suitable in applications the user is unfamiliar with, or when high confidence is desirable.

We present JigFab, an integrated end-to-end system that supports casual makers in designing and fabricating constructions with power tools. Starting from a digital version of the construction, JigFab achieves this by generating various types of constraints that configure and physically aid the movement of a power tool. Constraints are generated for every operation and are custom to the work piece. Constraints are laser cut and assembled together with predefined parts to reduce waste. JigFab's constraints are used according to an interactive step-by-step manual. JigFab internalizes all the required domain knowledge for designing and building intricate structures, consisting of various types of finger joints, tenon & mortise joints, grooves, and dowels. Building such structures is normally reserved for artisans or automated with advanced CNC machinery.

Part 4: Tools and Tool-Support; International audience; Collaborative robot-assisted production has great potential for high variety low volume production lines. These type of production lines are common in both personal fabrication settings as well as in several types of flexible production lines. Moreover, many assembly tasks are in fact hard to complete by a single user or a single robot, and benefit greatly from a fluent collaboration between both. However, programming such systems is cumbersome, given the wide variation of tasks and the complexity of instructing a robot how it should move and operate in collaboration with a human user.In this paper we explore the case of collaborative assembly for personal fabrication. Based on a CAD model of the envisioned product, our software analyzes how this can be composed from a set of standardized pieces and suggests a series of collaborative assembly steps to complete the product. The proposed tool removes the need for the end-user to perform additional programming of the robot. We use a low-cost robot setup that is accessible and usable for typical personal fabrication activities in Fab Labs and Makerspaces. Participants in a first experimental study testified that our approach leads to a fluent collaborative assembly process. Based on this preliminary evaluation, we present next steps and potential implications.

Translation environments offer various translation aids to support professional translators. However, translation aids typically provide only limited justification for the translation suggestions they propose. In this paper we present Intellingo, a translation environment that explores intelligibility for translation aids, to enable more sensible usage of translation suggestions. We performed a comparative study between an intelligible version and a non-intelligible version of Intellingo. The results show that although adding intelligibility does not necessarily result in significant changes to the user experience, translators can better assess translation suggestions without a negative impact on their performance. Intelligibility is preferred by translators when the additional information it conveys benefits the translation process and when this information is not part of the translator’s readily available knowledge. The SCATE (Smart Computer-Aided Translation Environment) project IWT 130041 is funded by the Flemish Institute for the Promotion of the Scientific-Technological Research in the Industry (IWT Vlaanderen).

The emergence of smart objects has the potential to radically change the way we interact with technology. Through embedded means for input and output, such objects allow for more natural and immediate interaction. The SmartObjects workshop will focus on how such embedded intelligence in objects situated in the user's physical environment can be used to provide more efficient and enjoyable interactions. We discuss the design from the technology and the user experience perspective.

Implementing multimodal interactions with event-driven languages results in a ‘callback soup', a source code littered with a multitude of flags that have to be maintained in a self-consistent manner and across different event handlers. Prototyping multimodal interactions adds to the complexity and error sensitivity, since the program code has to be refined iteratively as developers explore different possibilities and solutions. The authors present a declarative language for rapid prototyping multimodal interactions: Hasselt permits declaring composite events, sets of events that are logically related because of the interaction they support, that can be easily bound to dedicated event handlers for separate interactions. The authors' approach allows the description of multimodal interactions at a higher level of abstraction than event languages, which saves developers from dealing with the typical ‘callback soup' thereby resulting in a gain in programming efficiency and a reduction in errors when writing event handling code. They compared Hasselt with using a traditional programming language with strong support for events in a study with 12 participants each having a solid background in software development. When performing equivalent modifications to a multimodal interaction, the use of Hasselt leads to higher completion rates, lower completion times, and less code testing than when using a mainstream event-driven language.

Physical exercise is a primary component of cardiac rehabilitation. Interpreting exercise targets and being aware of the expected effort while rehabilitating independently at home is challenging for patients. Our work aims to assist cardiac patients in understanding their condition better and motivating them to achieve their exercise targets in a tele-rehabilitation setting. We introduce a patient-centric intelligible visualization approach to present prescribed rehabilitation targets to patients based on Metabolic Equivalent of Tasks (METs). We assessed efficacy of intelligible visualizations on patients’ comprehension in a lab study. We evaluated the impact on patient motivation and health outcomes in field studies. Patients were able to manage their prescribed activities, monitor their progress, and gained understanding on how their physical activities contribute to their rehabilitation targets. Patients confirmed that the intelligible visualizations motivated them to achieve their targets better. We observed an improvement in overall physical activity levels and health outcomes of patients.

In domains where users are exposed to large variations in visuo-spatial features among designs, they often spend excess time searching for common elements (features) in familiar locations. This paper contributes computational approaches to restructuring layouts such that features on a new, unvisited interface can be found quicker. We explore four concepts of familiarisation, inspired by the human visual system (HVS), to automatically generate a familiar design for each user. Given a history of previously visited interfaces, we restructure the spatial layout of the new (unseen) interface with the goal of making its elements more easily found. Familiariser is a browser-based implementation that automatically restructures webpage layouts based on the visual history of the user. Our evaluation with users provides first evidence favouring familiarisation. The project has partially received funding from the Academy of Finland project COMPUTED and the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement 637991).

Part 7: Design Rationale and Camera-Control; International audience; In the design process, designers make a wide variety of decisions that are essential to transform a design from a conceptual idea into a concrete solution. Recording and tracking design decisions, a first step to capturing the rationale of the design process, are tasks that until now are considered as cumbersome and too constraining. We used a holistic approach to design, deploy, and verify decision cards; a low threshold tool to capture, externalize, and contextualize design decisions during early stages of the design process. We evaluated the usefulness and validity of decision cards with both novice and expert designers. Our exploration results in valuable insights into how such decision cards are used, into the type of information that practitioners document as design decisions, and highlight the properties that make a recorded decision useful for supporting awareness and traceability on the design process.

Smart objects can be smart because of the information and communication technology that is added to human-made artifacts. It is not, however, the technology itself that makes them smart but rather the way in which the technology is integrated, and their smartness surfaces through how people are able to interact with these objects. Hence, the key challenge for making smart objects successful is to design usable and useful interactions with them. We list five features that can contribute to the smartness of an object, and we discuss how smart objects can help resolve the simplicity-featurism paradox. We conclude by introducing the three articles in this special issue, which dive into various aspects of smart object interaction: augmenting objects with projection, service-oriented interaction with smart objects via a mobile portal, and an analysis of input-output relations in interaction with tangible smart objects.

Part 7: Demonstrations; International audience; Patient coaching is integral to cardiac rehabilitation programs to help patients understand, cope better with their condition and become active participants in their care. The introduction of remote patient monitoring technologies and tele-monitoring solutions have proven to be effective and paved way for novel remote rehabilitation approaches. Nonetheless, these solutions focus largely on monitoring patients without a specific focus on coaching patients. Additionally, these systems lack personalization and a deeper understanding of individual patient needs. In our demonstration, we present a tool to personalize e-coaching based on individual patient risk factors, adherence rates and personal preferences of patients using a tele-rehabilitation solution. We developed the tool after conducting a workshop and multiple brainstorms with various caregivers involved in coaching cardiac patients to connect their perspectives with patient needs. It was integrated into a comprehensive tele-rehabilitation application.

Human-Robot Collaboration is increasingly considered in manufacturing to better combine the strengths of humans and robots. Establishing this human-robot collaboration may require multi-modal interaction; input to and output from the robot can both use multiple channels in sequence or in parallel. Designing effective interaction requires the expertise from different domains, possibly originating from people with different backgrounds. In our work we explore how composite events — hierarchical composition of events — can be used in a way that eases the communication within a multi-disciplinary team. In this paper, we present how the concept of composite events can be used to create different layers of abstraction that can be used to ease prototyping and discussion of human-robot collaboration with stakeholders through a supporting tool called Hasselt UIMS. At the lower level(s) of abstraction, the composite events can be mapped to the message-based communication as implemented in the Robotic Operating System (ROS), which is used to program collaborative robots, such as the Baxter robot from Rethink Robotics.

Localisation is a standard feature in many mobile applications today, and there are numerous techniques for determining a user’s location both indoors and outdoors. The provided location information is often organised in a format tailored to a particular localisation system’s needs and restrictions, making the use of several systems in one application cumbersome. The presented approach models the details of localisation systems and uses this model to create a unified view on localisation in which special attention is paid to uncertainty coming from different localisation conditions and to its presentation to the user. The work discusses technical considerations, challenges and issues of the approach, and reports on a user study on the acceptance of a mobile application’s behaviour reflecting the approach. The results of the study show the suitability of the approach and reveal users’ preference toward automatic and informed changes they experienced while using the application.

Professional video searchers typically have to search for particular video fragments in a vast video archive that contains many hours of video data. Without having the right video archive exploration tools, this is a difficult and time consuming task that induces hours of video skimming. We propose the video archive explorer, a video exploration tool that provides visual representations of automatically detected concepts to facilitate individual and collaborative video search tasks. This video archive explorer is developed by employing a user-centred methodology, which ensures that the tool is more likely to fit to the end user needs. A qualitative evaluation with professional video searchers shows that the combination of automatic video indexing, interactive visualisations and user-centred design can result in an increased usability, user satisfaction and productivity. Haesen M., Meskens J., Luyten K., Coninx K., Becker J.H., Tuytelaars T., Poulisse G.-J., Pham The P., Moens M.-F., ''Finding a needle in a haystack: an interactive video archive explorer for professional video searchers'', Multimedia tools and applications, vol. 63, no. 2, pp. 331-356, 2013. ispartof: Multimedia Tools and Applications vol:63 issue:2 pages:331-356 status: published

Design artefacts are vital to communicate design outcomes, both in remote and co-located settings. However, it is unclear how artefacts are used to mediate interactions between designers and stakeholders of the design process. The purpose of this paper is exploring how professional design teams use artefacts to guide and capture discussions involving multidisciplinary stakeholders while they work in a co-located setting. An earlier draft of this paper was paper published in the Proceedings of the European Conference on Cognitive Ergonomics (ECCE 2017). This work adds substantial clarification of the methodology followed, further details and photographs of the case studies, and an extended discussion about our findings and their relevance for designing interactive systems. We report the observations of six design meetings in three different projects, involving professional design teams that follow a user-centered design approach. Meetings with stakeholders are instrumental for design projects. However, design teams face the challenge of synthesizing large amounts of information, often in a limited time, and with minimal common ground between meeting attendees. We found that all the observed design meetings had a similar structure consisting of a series of particular phases, in which design activities were organized around artefacts. These artefacts were used as input to disseminate and gather feedback of previous design outcomes, or as output to collect and process a variety of perspectives. We discuss the challenges faced by design teams during design meetings, and propose three design directions for interactive systems to coordinate design meetings revolving around artefacts The research leading to these results has received funding from the European Research Council under the European Union's Seventh Framework Programme (FP7/2007-2013) / ERC grant agreement n° 610725 (COnCEPT project)

In mobile tele-rehabilitation applications for Coronary Artery Disease (CAD) patients, behavior change plays a central role in influencing better therapy adherence and prevention of disease recurrence. However, creating sustainable behavior change that holds a beneficial impact over a prolonged period of time remains an important challenge. In this paper we discuss various models and frameworks related to persuasion and behavior change, and investigate how to incorporate these with a multidisciplinary user-centered design approach for creating a mobile tele-rehabilitation application. By implementing different concepts that contribute to behavior change and applying a set of distinct persuasive design patterns, we were able to translate the high-level goals of behavior theory into a mobile application that explicitly incorporates behavior change techniques and also offers a good overall user experience. We evaluated our system, HeartHab, in a lab setting and show that our approach leads to a high user acceptance and willingness to use the system in daily activities.

International audience; We present Gestu-Wan, an intelligible gesture guidance system designed to support mid-air gesture-based interaction for walk-up-and-use displays. Although gesture-based interfaces have become more prevalent, there is currently very little uniformity with regard to gesture sets and the way gestures can be executed. This leads to confusion, bad user experiences and users who rather avoid than engage in interaction using mid-air gesturing. Our approach improves the visibility of gesture-based interfaces and facilitates execution of mid-air gestures without prior training. We compare Gestu-Wan with a static gesture guide, which shows that it can help users with both performing complex gestures as well as understanding how the gesture recognizer works.

International audience; Interactive large surfaces have recently become commonplace for interactions in public settings. The fact that people can engage with them and the spectrum of possible interactions, however, often remain invisible and can be confusing or ambiguous to passersby. In this paper, we explore the design of dynamic peripheral floor visualizations for revealing and mediating large surface interactions. Extending earlier work on interactive illuminated floors, we introduce a novel approach for leveraging floor displays in a secondary, assisting role to aid users in interacting with the primary display. We illustrate a series of visualizations with the illuminated floor of the Proxemic Flow system. In particular, we contribute a design space for peripheral floor visualizations that (a) provides peripheral information about tracking fidelity with personal halos, (b) makes interaction zones and borders explicit for easy opt-in and opt-out, and (c) gives cues inviting for spatial movement or possible next interaction steps through wave, trail, and footstep animations. We demonstrate our proposed techniques in the context of a large surface application and discuss important design considerations for assistive floor visualizations.

The increasing number of smart objects in our everyday life shapes how we interact beyond the desktop. In this workshop we discussed how the interaction with these smart objects should be designed from various perspectives. This year’s workshop put a special focus on affective computing with smart objects, as reflected by the keynote talk.

There is a growing demand for design support to create interactive systems that are deployed in ambient intelligent environments. Unlike traditional interactive systems, the wide diversity of situations these type of user interfaces need to work in require tool support that is close to the environment of the end-user on the one hand and provide a smooth integration with the application logic on the other hand. This paper shows how the model-based user interface development methodology can be applied for ambient intelligent environments; we propose a task-centered approach towards the design of interactive systems by means of appropriate visualizations and simulations of different models. Besides the use of these typical user interface models such as the task- and presentation-model to support interface design, we focus on user interfaces supporting situated task distributions and a visualization of context influences on deployed, possibly distributed, user interfaces. To enable this we introduce an environment model describing the device configuration at particular moment in time. To support the user interface designer while creating these complex interfaces for ambient intelligent environments, we discuss tool support using a visualization of the environment together with simulations of the user interface configurations. We also show how the concepts presented in the paper can be integrated within model-driven engineering, hereby narrowing the gap between HCI design and software engineering.

The usage of computing systems has evolved dramatically over the last years. Starting from a low level procedural usage, in which a process for executing one or several tasks is carried out, computers now tend to be used in a problem oriented way. Future computer usage will be more centered around particular services, and will not be focused on platforms or applications. These services should be independent of the technology used to interact with them. In this paper an approach will be presented which provides a uniform interface to such services, without any dependence on modality, platform or programming language. Through the usage of general user interface descriptions, presented in XML, and converted using XSLT, a uniform framework is presented for runtime migration of user interfaces. As a consequence, future services will become easily extensible for all kinds of devices and modalities. Special attention goes out to a component-based software development approach. Services represented by and grouped in components can offer a special interface for modal- and device-independent rendering. Components become responsible for describing their own possibilities and constraints for interacting. An implementation serving as a proof of concept, a runtime conversion of a joystick in a 3D virtual environment into a 2D dialog-based user interface, is developed.

We present PaperPulse, a design and fabrication approach that enables designers without a technical background to produce standalone interactive paper artifacts by augmenting them with electronics. With PaperPulse, designers overlay pre-designed visual elements with widgets available in our design tool. PaperPulse provides designers with three families of widgets designed for smooth integration with paper, for an overall of 20 different interactive components. We also contribute a logic demonstration and recording approach, Pulsation, that allows for specifying functional relationships between widgets. Using the final design and the recorded Pulsation logic, PaperPulse generates layered electronic circuit designs, and code that can be deployed on a microcontroller. By following automatically generated assembly instructions, designers can seamlessly integrate the microcontroller and widgets in the final paper artifact.

The present paper describes an empirical user study intended to compare the programming efficiency of our proposed domain-specific language versus a mainstream event language when it comes to modify multimodal interactions. By concerted use of observations, interviews, and standardized questionnaires, we managed to measure the completion rates, completion time, code testing effort, and perceived difficulty of the programming tasks along with the perceived usability and perceived learnability of the tool supporting our proposed language. Based on this experience, we propose some guidelines for designing comparative user studies of programming languages. The paper also discusses the considerations we took into account when designing a multimodal interaction description language that intends to be well regarded by its users.

Nonverbal and unconscious behaviour is an important component of daily human-human interaction. This is especially true in situations such as public speaking, job interviews or information sensitive conversations, where researchers have shown that an increased awareness of one's behaviour can improve the outcome of the interaction. With wearable technology, such as Google Glass, we now have the opportunity to augment social interactions and provide realtime feedback on one's behaviour in an unobtrusive way. In this paper we present Logue, a system that provides realtime feedback on the presenters' openness, body energy and speech rate during public speaking. The system analyses the user's nonverbal behaviour using social signal processing techniques and gives visual feedback on a head-mounted display. We conducted two user studies with a staged and a real presentation scenario which yielded that Logue's feedback was perceived helpful and had a positive impact on the speaker's performance.

Non-verbal and unconscious behavior plays an important role for efficient human-to-human communication but are often undervalued when training people to become better communicators. This is particularly true for public speakers who need not only behave according to a social etiquette but do so while generating enthusiasm and interest for dozens if not hundreds of other persons. In this paper we propose the concept of social augmentation using wearable computing with the goal of giving users the ability to continuously monitor their performance as a communicator. To this end we explore interaction modalities and feedback mechanisms which would lend themselves to this task.

International audience; In this short extended abstract, we present the doctoral consortium of the Engineering Interactive Computing Systems (EICS) 2014 Symposium. Our goal is to make the doctoral consortium a useful event with a maximum benefit for the participants by having a dedicated event the day before the conference as well as the opportunity to present their on-going doctoral work to a wider audience during the conference.

During remote video-mediated assistance, instructors often guide workers through problems and instruct them to perform unfamiliar or complex operations. However, the workers’ performance might deteriorate due to stress. We argue that informing biofeedback to the instructor, can improve communication and lead to lower stress. This paper presents a thorough investigation on mental workload and stress perceived by twenty participants, paired up in an instructor-worker scenario, performing remote video-mediated tasks. The interface conditions differ in task, facial and biofeedback communication. Two self-report measures are used to assess mental workload and stress. Results show that pairs reported lower mental workload and stress when instructors are using the biofeedback as compared to using interfaces with facial view. Significant correlations were found on task performance with reducing stress (i.e. increased task engagement and decreased worry) for instructors and declining mental workload (i.e. increased performance) for workers. Our findings provide insights to advance video-mediated interfaces for remote collaborative work.

Paddle is a highly deformable mobile device that leverages engineering principles from the design of the Rubik's Magic, a folding plate puzzle. The various transformations supported by Paddle bridges the gap between differently sized mobile devices available nowadays, such as phones, armbands, tablets and game controllers. Besides this, Paddle can be transformed to different physical controls in only a few steps, such as peeking options, a ring to scroll through lists and a book-like form factor to leaf through pages. These special-purpose physical controls have the advantage of providing clear physical affordances and exploiting people's innate abilities for manipulating objects in the real world. We investigated the benefits of these interaction techniques in detail in [1]. In contrast to traditional touch screens, physical controls are usually less flexible and therefore less suitable for mobile settings. Paddle, shows how mobile devices can be designed to bring physical controls to mobile devices and thus combine the flexibility of touch screens with the physical qualities that real world controls provide. Our current prototype is tracked with an optical tracking system and uses a projector to provide visual output. In the future, we envision devices similar to Paddle that are entirely self-contained, using tiny integrated displays.

There are currently toolkits that allow the specification of executable multimodal human-machine interaction models. Some provide domain-specific visual languages with which a broad range of interactions can be modeled but at the expense of bulky diagrams. Others instead, interpret concise specifications written in existing textual languages even though their non-specialized notations prevent the productivity improvement achievable through domain-specific ones. We propose a domain-specific textual language and its supporting toolkit; they both overcome the shortcomings of the existing approaches while retaining their strengths. The language provides notations and constructs specially tailored to compactly declare the event patterns raised during the execution of multimodal commands. The toolkit detects the occurrence of these patterns and invokes the functionality of a back-end system in response.

Smartphones leak personal information about their owner when they use it to connect to the Internet. Despite recent coverage of these issues in popular media, raising awareness remains problematic since it remains largely invisible to the users. We designed a system, SASQUATCH, consisting of a network scanner and a public display, to draw the visitor's attention and inform them about these issues. SASQUATCH first gathers private information about previous whereabouts, and then shows an anonymized version of this data on the public display to draw the visitor's attention. Next, SASQUATCH offers an interactive component that allows people to view the information their own smartphone is leaking in private, and then provides solutions (including a fully-automated smartphone application) for securing against future privacy leaks. A set of initial field trails has shown that SASQUATCH is highly effective in raising awareness.

Storyboarding is a technique that is often used for the conception of new interactive systems. A storyboard illustrates graphically how a system is used by its users and what a typical context of usage is. Although the informal notation of a storyboard stimulates creativity, and makes them easy to understand for everyone, it is more difficult to integrate in further steps in the engineering process. We present an approach, "Time In Storyboards" (Timisto), to extract valuable information on how various interactions with the system are positioned in time with respect to each other. Timisto does not interfere with the creative process of storyboarding, but maximizes the structured information about time that can be deduced from a storyboard. Design Methods;Analysis Methods

Feedback and affordances are two of the most well-known principles in interaction design. Unfortunately, the related and equally important notion of feedforward has not been given as much consideration. Nevertheless, feedforward is a powerful design principle for bridging Norman's Gulf of Execution. We reframe feedforward by disambiguating it from related design principles such as feedback and perceived affordances, and identify new classes of feedforward. In addition, we present a reference framework that provides a means for designers to explore and recognize different opportunities for feedforward. Copyright © 2013 ACM.

Modern knowledge work consists of both individual and highly collaborative activities that are typically composed of a number of configuration, coordination and articulation processes. The desktop interface today, however, provides very little support for these processes and rather forces knowledge workers to adapt to the technology. We introduce co-Activity Manager, an activity-centric desktop system that (i) provides tools for ad hoc dynamic configuration of a desktop working context, (ii) supports both explicit and implicit articulation of ongoing work through a built-in collaboration manager and (iii) provides the means to coordinate and share working context with other users and devices. In this paper, we discuss the activity theory informed design of co-Activity Manager and report on a 14 day field deployment in a multi-disciplinary software development team. The study showed that the activity-centric workspace supports different individual and collaborative work configuration practices and that activity-centric collaboration is a two-phase process consisting of an activity sharing and per-activity coordination phase.

Persons with neurological disorders or spinal cord injuries, such as Cerebrovascular Accident (CVA) or Paraplegia pa- tients, experience signi cantly reduced physical abilities dur- ing their activities of daily living. By frequent and intense physical therapy, these patients can sustain or even enhance their functional performance. However, physical therapy, whether or not it is supported by technology, can currently only be followed in a rehabilitation centre under supervi- sion of a therapist. To provide technology-supported physi- cal therapy for independent use by the patient in the home situation, our current research explores pervasive technolo- gies for rehabilitation systems. In this paper, we describe our pervasive prototype 'Liftacube' for training of upper ex- tremities. An initial evaluation with patients with a neuro- logical disorder or spinal cord injury (CVA and paraplegia patients) and their therapists reveals a great appreciation for this motivating pervasive gaming prototype. Re ections on the technical set-up (such as size, form factor, and mate- rials) and interaction preferences (such as feedback, games, and movements for interaction) for pervasive rehabilitation systems in a residential environment are elaborated upon.

Choosing an appropriate toolkit for creating a multimodal interface is a cumbersome task. Several specialized toolkits include fusion and fission engines that allow developers to combine and decompose modalities to capture multimodal input and provide multimodal output. Unfortunately, the extent to which these toolkits can facilitate the creation of a multimodal interface is hard or impossible to estimate, due to the absence of a scale where the toolkit's capabilities can be measured on. In this paper, we propose a measurement scale, which allows the assessment of specialized toolkits without need for time-consuming testing or source code analysis. This scale is used to measure and compare the capabilities of three toolkits: CoGenIVE, HephaisTK and ICon.

In todays videogames user feedback is often provided through raw sta-tistics and scoreboards. We envision that incorporating empathic feedback matching the player’s current mood will improve the overall gaming experience. In this paper we present Bro-cam, a novel system that provides empathic feedback to the player based on their body postures. Different body postures of the players are used as an indicator for their openness. From their level of openness, Bro-cam profiles the play-ers into different personality types ranging from introvert to extrovert. Empathic feedback is then automatically generated and matched to their preferences for certain humoristic feedback statements. We use a depth camera to track the player’s body postures and movements during the game and analyze these to provide customized feedback. We conducted a user study involving 32 players to investigate their subjec-tive assessment on the empathic game feedback. Semi-structured interviews reveal that participants were positive about the empathic feedback and Bro-cam significantly improves their game experience.

Intelligent User Interfaces can benefit from having knowledge on the user's emotion. However, current implementations to detect affective states, are often constraining the user's freedom of movement by instrumenting her with sensors. This prevents affective computing from being deployed in naturalistic and ubiquitous computing contexts. In this paper, we present a novel system called mASqUE, which uses a set of association rules to infer someone's affective state from their body postures. This is done without any user instrumentation and using off-the-shelf and non-expensive commodity hardware: a depth camera tracks the body posture of the users and their postures are also used as an indicator of their openness. By combining the posture information with physiological sensors measurements we were able to mine a set of association rules relating postures to affective states. We demonstrate the possibility of inferring affective states from body postures in ubiquitous computing environments and our study also provides insights how this opens up new possibilities for IUI to access the affective states of users from body postures in a nonintrusive way.

Informing caregivers by providing them with contextual medical information can significantly improve the quality of patient care activities. However, information flow in hospitals is still tied to traditional manual or digitised lengthy patient record files that are often not accessible while caregivers are attending to patients. Leveraging the proliferation of pervasive awareness technologies (sensors, actuators and mobile displays), recent studies have explored this information presentation aspect borrowing theories from context-aware computing, i.e., presenting subtle information contextually to support the activity at hand. However, the understanding of the information space (i.e., what information should be presented) is still fairly abstruse, which inhibits the deployment of such real-time activity support systems. To this end, this paper first presents situated glyphs, a graphical entity to encode situation specific information, and then presents our findings from an in-situ qualitative study addressing the information space tailored to such glyphs. Applying technology probes using situated glyphs and different glyph display form factors, the study aimed at uncovering the information space pertained to both primary and secondary medical care. Our analysis has resulted in a large set of information types as well as given us deeper insight on the principles for designing future situated glyphs. We report our findings in this paper that we expect would provide a solid foundation for designing future assistive systems to support patient care activities.