Search

The use of digital examples plays a critical role in mobile UI design. Yet, it remains unclear how UX/UI designers manage (i.e., collect, archive, and utilize) examples to facilitate their design processes at different stages, and what possible challenges are imposed on the design of proper tools to support these practices. In this paper, we conduct a qualitative interview study with mobile UI/UX designers (12 experts and 12 novices), deriving the commonality in practices and analyzing possible differences across four design phases (Discover, Define, Develop, and Deliver) and expertise. In brief, we find that there is more diverse and frequent use of examples in the Discover and Develop phases, and that experts take more diverse advantage of the information from examples compared to novices. We further identify the challenges faced by designers when using existing example management services, and propose potential design implications for the development of more supportive design tools in the future.

The perceived emotional intelligence of a conversational agent (CA) can significantly impact people’s interaction with the CA. Prior research applies text-based sentiment analysis and emotional response generation to improve CAs’ emotional intelligence. However, acoustic features in speech containing rich contexts are underexploited. In this work, we designed and implemented an emotionally aware CA, called HUE (Heard yoUr Emotion) that stylized responses with emotion regulation strategies and empathetic interjections. We conducted a user study with 75 participants to evaluate their perceived emotional intelligence (PEI) of HUE by having them observe conversations between people and HUE in different emotional scenarios. Our results show that participants’ PEI was significantly higher with the acoustic features than without.

Natural Braille reading, a demanding cognitive skill, poses a huge challenge for the brain network of the blind. Here, with behavioral measurement and functional MRI imaging data, we pinpointed the neural pathway and investigated the neural mechanisms of individual differences in Braille reading in late blindness. Using resting state fMRI, we identified a distinct neural link between the higher-tier ‘visual’ cortex—the lateral occipital cortex (LOC), and the inferior frontal cortex (IFC) in the late blind brain, which is significantly stronger than sighted controls. Individual Braille reading proficiency positively correlated with the left-lateralized LOC-IFC functional connectivity. In a natural Braille reading task, we found an enhanced bidirectional information flow with a stronger top-down modulation of the IFC-to-LOC effective connectivity. Greater top-down modulation contributed to higher Braille reading proficiency via a broader area of task-engaged LOC. Together, we established a model to predict Braille reading proficiency, considering both functional and effective connectivity of the LOC-IFC pathway. This ‘two-tale’ model suggests that developing the underpinning neural circuit and the top-down cognitive strategy contributes uniquely to superior Braille reading performance.SIGNIFICANCE STATEMENTFor late blind humans, one of the most challenging cognitive skills is natural Braille reading. However, little is known about the neural mechanisms of significant differences in individual Braille reading performance. Using functional imaging data, we identified a distinct neural link between the left lateral occipital cortex (LOC) and the left inferior frontal cortex (IFC) for natural Braille reading in the late blind brain. To better predict individual Braille reading proficiency, we proposed a linear model with two variables of the LOC-IFC link: the resting-state functional connectivity and the task-engaged top-down effective connectivity. These findings suggest that developing the underpinning neural pathway and the top-down cognitive strategy contributes uniquely to superior Braille reading performance.

Designing a tactile image for blind people is a significant challenge due to the difficulty of recognizing objects on a 2D line drawing image by touch compared to vision. In this paper, we proposed ”tactile accuracy”, a new criterion to evaluate the performance of recognizing 242 raised line images of common objects for 30 subjects (10 blindfolded sighted subjects, 10 congenitally blind subjects, and 10 late blind subjects), instead of the conventional ”naming accuracy” used in the visual image recognition tasks. We used multi-level evaluation criteria including ”tactile accuracy” to systematically analyze the design factors in tactile images. The results showed that using multi-level evaluate criteria could help unveil the tactile cognitive preferences of different types of subjects for personalized learning. Moreover, we reported important design factors that affect tactile image recognition, thus providing guidelines on the design of tactile images.

This paper presents WalkingBot, a modular robot system that allows non-expert users to build a multi-legged robot in various morphologies using a set of building blocks with sensors and actuators embedded. The kinematic model of the built robot is interpreted automatically and revealed in a customized GUI through an integrated hardware and software design, so that users can understand, control, and program the robot easily. A Model Predictive Control (MPC) scheme is introduced to generate a control policy for various motions (e.g. moving forward, turning left) corresponding to the sensed robot structure, affording rich robot motions right after assembling. Targeting different levels of programming skill, two programming methods, visual block programming and events programming, are also presented to enable users to create their own interactive legged robot.

Recently, researchers have become increasingly interested in finding new input methods for olfactory interfaces. Physical odor capture is a potential solution to this issue and, in order to make it more accessible for users, we designed a portable and fast smell capture prototype based on headspace technology and inspired by point-and-shoot cameras. We conducted a two-week diary study with 13 participants, in which they were allowed to freely use the prototype for odor capture activities. Through diary and interview feedback, we summarized factors such as the motivations of capturing, the collected odor types, and perceptual effects of odor replay. We found that the capture activities can positively affect user emotions, memory, or perception. User preferences on device parameters were also gathered to guide further design iterations. Physical odor capture has many potential applications in daily-life and other implications resulting from the study have been proposed for further research.

Driving from the novel technology of LIME (LIquid MEtal shape-changing interfaces), MetaLife presents a series of installations for educational, entertainment, or aesthetic purposes. In this paper, we discuss how we translate LIME's interaction paradigms into design vocabulary and how we use such vocabulary to design the installation. As industry and academia become disconnected from each other in the field of tangible user interfaces, MetaLife gives an example of how we can promote the bonds between them.

We present HeliCoach, an adaptive O&M (orientation and mobility) training system to help visually impaired people to gain audio-orientation ability efficiently. HeliCoach is a mastery-based adaptive learning system consists of three components: A drone-based physically simulated 3D audio space, a UWB (Ultra-Wide Band)-supported assessment system, an intelligent belt with haptic feedback as training scaffolds. We evaluated the acceptability and efficiency of HeliCoach on both blindfolded sighted participants and visually impaired participants in an audio-orientation task. The results showed that HeliCoach was efficient and interesting. Thus, We believe HeliCoach can be applied to many simulated 3D audio space related task to train people more efficiently.

In Dunhuang Mogao Grottoes, unique Buddhist murals of ancient China are preserved. Unfortunately, the exquisite murals are suffering from degradation. Experts have been trying to enhance public's awareness of mural protection, but there's no efficacious means to attract interest and popularize knowledge yet. In this paper, we propose RestoreVR, an interactive virtual reality (VR) system engaging users to experience Dunhuang mural restoration in a digital tour in the cave. Based on an online survey with the public and in-depth interviews with five Dunhuang experts, we derive a set of design requirements for generating embodied knowledge and situated experience in VR to bridge the gap between highly specialized experts and general audiences. Accordingly, we design RestoreVR and conduct a between-subjects user study to compare our system with traditional methods. The results suggest that RestoreVR significantly improves user experience and awareness of CH protection over existing methods.

Animations are widely used in mobile user interface design nowadays. By exploiting the noticeability of Motion Garaphics, interface designers can rapidly draw a user’s attention to salient objects. Such effects can be achieved by a variety of different visual stimuli including color, shape, size, motion, luminance, and flashing. However, it needs to be better understood and precisely assessed how the different animations affect user attention. In this paper, we analyzed and compared the noticeability of ten different animations. We investigated the extent to which they attract users’ attention and generate quantitative indications accordingly. We propose a quantitative method to measure the noticeability of animations in mobile interfaces and point out possible directions for future studies.

Touch plays a significant role in emotion communication, and affective haptics are becoming an emerging field in terms of designing and implementing multimodal experience in human-computer interactions. The present study explores the affective haptics and multimodal experiments. By the literature review of human emotional feedback on haptic signal and affective haptic devices, the multimodal experiments were designed and implemented. Three types of haptic signals (random, vibration and stroking stimuli) are designed and implemented. The results demonstrated that haptic affective signals affect similar emotion in both the haptic only and visual-haptic multimodal scenario, and visual stimuli largely dominate the emotional state compared with haptic stimuli.

Smudging is one of the most serious damages of Mogao Grottoes' murals. In this paper, confocal micro-Raman spectroscopy is adopted to investigate the smoked murals in Mogao Grottoes. The exutive fragments from the smoked murals in 71 Grotto have been analyzed. The white pigment, which is covered by a layer of black, is identified as calcite, and the black is amorphous carbon. No any other chromatic pigments were found in all exutive fragments. From which, we may speculate that the white pigment (CaCO3) is stable undergoing smoking and other chromatic pigments may have been destroyed by smoking. Based on the analysis and the records of the Mogao Grottoes' murals, a representative smoking process was designed to explore the degradation of the pigments in an imitative mural. The typical pigments, including white (aragonite), black (carbon) and red (vermilion), are analyzed from imitated smoked mural, and the results show that aragonite is stable and vermilion is damaged. It is judged that the vermilion has been destroyed by the smoking process, in which the high-temperature smoke makes the vermilion easily react with oxygen and produce the mercury vapor and sulfur dioxide. The simulative smoking experiment is reliable and feasible for investigating the degradation process of pigments and is proposed to investigate more pigments which can supply significant and important information for the restoration and conservation of smoked murals in Mogao Grottoes. Copyright © 2017 John Wiley & Sons, Ltd.

A color sketch creates a vivid depiction of a scene using sparse pencil strokes and casual colored brush strokes. The interactive drawing system ColorSketch can help novice users generate color sketches from photos. To preserve artistic freedom and expressiveness, the proposed system gives users full control over pencil strokes, while automatically augmenting pencil sketches using color mapping, brush stroke rendering, and blank area creation. Experimental and user study results demonstrate that users, especially novices, can create better color sketches with our system than when using traditional manual tools.

As a historical and cultural heritage, Chinese traditional mural art is not only the precious heritage of Chinese civilization, but also the shared wealth of world culture. Up to now, the original material resources of fresco art are very limited, some of them have entered the situation that the pictures and colors are diffuse and need to be protected urgently. In the "Outline of Chinese Frescoes' History", the first temple frescoes of the Ming Dynasty mentioned is frescoes of the Fahai Temple which painted by the palace painters. Especially, the painting style and color style of the frescoes of it, such as Blue-green Fine Brushwork and Leaching Powder and Pasting Gold" ("Li Fen Tie Jin"), have obvious artistic characteristics and color research value compared with Dunhuang frescoes and Yongle Palace's. Taking the fresco color of Fahai Temple as the object of practice, it is of positive significance to explore whether it expands the national chromatography range, protects the fresco itself digitally, or expresses and deduces scientifically and visually based on this color. On this basis, this paper takes mural color as the research object, through the cross-study and analysis of aesthetics, fine arts, information design, history, religions, computer graphics and other disciplines, tries to establish the digital map of mural color in Fahai Temple, and on this basis, tries to build a theoretical model and practice based on the application of digital dynamic deduction of cultural heritage color. Further more, it will provide interdisciplinary research paths and exploration experience for the digital display, dissemination and promotion for traditional color and its re-application.

Speech To Text (STT) plays a significant role in Voice User Interface (VUI). While preserving necessary semantic information in converted text, STT generally captures no or limited emotional information. In this paper, we present an emojilization tool to automatically attach related emojis to the STT-generated texts by analyzing both textual and acoustic features in speech signals. For a given voice message, the tool selects the most representative emoji from 64 most commonly used emojis. We conducted a pilot study with 34 participants. In our study, 159 utterances were labeled with emojis by our tool. The emotion restoration effect was evaluated. The results indicate that the proposed tool effectively compensates for the emotion loss.

We present IRelics, a tangible interaction platform for the popularization of field archeology. IRelics allows users to experience archaeological field work activities as a serious game by using a set of tangible tools. We developed an innovative LWIR (Long Wavelength Infrared Rays) sensing system, which implements the design of tangible tools that provide real manipulation experiences. By interacting with IRelics, a player may experience different archaeological activities such as excavation and cleaning. We conducted two observations to evaluate the usability and effectiveness at archeology popularizing. Findings suggest that the IRelics platform can enhance the engagement of the participants by providing a positive and interactive environment while teaching them unfamiliar knowledge.

Emotion is a salient property of the sense of touch. From a design perspective, the present study explores the affective haptics and human affective response, device analysis and interaction design. Firstly, the affective evaluation method - the Valence-Arousal Space - is analyzed in terms of haptic physiology, and the subjective and objective affective detection methods are illustrated, as well as the connection of human haptic valence and haptic stimulus by physiology research. Secondly, a series of affective haptic devices are analyzed according to human affective responses. Lastly, some affective haptics design guidelines and principles are introduced from a design perspective with the help of affective haptics research results.

Household appliances are becoming more varied. In daily life, people usually refer to printed documents while they learn to use different devices. However, augmented reality (AR) assistive systems providing visual and aural instructions have been proposed as an alternative solution. In this work, we evaluated users’ performance of instruction understanding in four different ways: (1) Baseline paper instructions, (2) Visual instructions based on head mounted displays (HMDs), (3) Visual instructions based on computer monitor, (4) Aural instructions. In a Wizard of Oz study, we found that, for the task of making espresso coffee, the helpfulness of visual and aural instructions depends on task complexity. Providing visual instructions is a better way of showing operation details, while aural instructions are suitable for presenting intention of operation. With the same visual instructions on displays, due to the limitation of hardware, the HMD-users complete the task in the longest duration and bear the heaviest perceived cognitive load.