Search

Musical pieces composed for the piano are registered on paper sheets. Since the space for sheets on the piano is limited, pianists have to turn pages while playing the piano to get access to the registered without having to stop playing. In this work, we present two methods to support page turning during piano play: 1) turning by touch and 2) turning by pedal in combination with two types of note layouts in (1) DIN A4 and (2) an adaptive display size. We compared these page turning methods to the baseline of using paper pages with 10 participants in a controlled lab experiment. Our results show that the pedal-adaptive turning method leads to the lowest interruption time while playing and pedal-based methods have a high usability and low cognitive load.

Touch screen interactions have been linked to repetitive strain injuries. Their use as an input channel to user interfaces can also have limitations due to screen size or mobility constraints. In this work, we investigate the feasibility of a neck-mounted wearable interface for software interaction, using only subtle head movements. Technologies and configurations for sensoring the neck region, including e-textiles and flex sensors, are considered. The type and placement of the sensors are evaluated. An end-to-end prototype system is developed, which takes the sensor readings from the bespoke neck hardware and wirelessly transmits them to a smart phone, for carrying-out the motion classification and interfacing with applications. The classification accuracy of common head movements using classical machine learning algorithms are evaluated. A classification accuracy of 91% is achieved with data collected from the prototype, for a library of five common head gestures and positions.

Foldables and 2-in-1 devices provide users with new ways to adapt their device to their mobile context, allowing users to switch device configuration, screen size and shape, and input mechanism in seconds. This has strong implications for the design of interactive media applications that need to adapt to the user's changing context and device parameters. In this study, we explored how users opportunistically make use of these new device affordances in their everyday lives through interviews with 15 diverse hybrid device owners, identifying use cases and pain points. Users reported regularly moving through multiple configurations or input modes while using a single product, or completing a single task. Based on our learnings, we provide recommendations for practitioners designing for hybrid mobile devices.

The use of Information and Communication Technologies (ICTs) can be an important ally in improving the quality of life of Visually Impaired People (VIP) because they provide greater independence in the performance of their daily tasks. During the development process of ICTs for VIP, the concepts of Accessibility, Usability, and User eXperience (UX) are significant because they can positively influence the system's quality. Concerning these concepts, some improvements can be considered already at the design stage, such as screen size, button format, color density, among others. However, not all designers consider Accessibility, Usability, and UX concepts when designing ICT for VIP. To identify and characterize which design technologies of Accessibility, Usability, and UX have been proposed in the literature, we performed a Systematic Mapping Study (SMS). We identified a total of 23 technologies. The analysis from these technologies showed that the majority are destined for mobile applications. Besides, some design technologies only address the concept of Accessibility, and only one technology considered the concepts of Accessibility, Usability, and UX together. To support professionals, students, and researchers in Human-Computer Interaction (HCI) and Software Engineering (SE), this SMS presents a set of Accessibility, Usability, and UX design technologies, which can be used in the initial phases of the ICT development process for VIP.

As Internet advertisements (also called "ads") revenue growth is being driven further than ever before, one challenge facing publishers, such as Google and Amazon, is to quickly select and place a group of ads in an ad space for each online user with the objective of maximizing the expected revenue. This is especially challenging in the context of mobile advertising due to the smaller screen size of mobile devices and longer user session. We notice that most existing models do not allow the publisher to place the same ad in multiple positions. However, it has been reported that people must see an advertisement at least several times before they will acquire enough interest to consider buying the product or service advertised. To capture this repetition effect we largely generalize the previous model by allowing the publisher to repeat the same ads multiple times. We also notice that many existing models assume that a user will leave the ad session permanently after clicking an ad. Our framework allows a more realistic but complicated user behavior by allowing a user to return to the previous ad session. Our model is able to capture many factors that may affect the click probability of an ad such as the intrinsic quality of the ad, the position of the ad, and all ads that have been previously displayed. We also extend our work to adaptive setting where publishers can dynamically adjust their ad display according to user's feedback. We develop effective algorithms with guarantees of finding either optimal or approximate solutions.

Online recommendation services recommend multiple commodities to users. Nowadays, a considerable proportion of users visit e-commerce platforms by mobile devices. Due to the limited screen size of mobile devices, positions of items have a significant influence on clicks: 1) Higher positions lead to more clicks for one commodity. 2) The 'pseudo-exposure' issue: Only a few recommended items are shown at first glance and users need to slide the screen to browse other items. Therefore, some recommended items ranked behind are not viewed by users and it is not proper to treat this kind of items as negative samples. While many works model the online recommendation as contextual bandit problems, they rarely take the influence of positions into consideration and thus the estimation of the reward function may be biased. In this paper, we aim at addressing these two issues to improve the performance of online mobile recommendation. Our contributions are four-fold. First, since we concern the reward of a set of recommended items, we model the online recommendation as a contextual combinatorial bandit problem and define the reward of a recommended set. Second, we propose a novel contextual combinatorial bandit method called UBM-LinUCB to address two issues related to positions by adopting the User Browsing Model (UBM), a click model for web search. Third, we provide a formal regret analysis and prove that our algorithm achieves sublinear regret independent of the number of items. Finally, we evaluate our algorithm on two real-world datasets by a novel unbiased estimator. An online experiment is also implemented in Taobao, one of the most popular e-commerce platforms in the world. Results on two CTR metrics show that our algorithm outperforms the other contextual bandit algorithms.

Smartwatches can be used independently from smartphones, but input tasks like messaging are cumbersome due to the small display size. Parts of the display are hidden during interaction, which can lead to incorrect input. For simplicity, instead of general text input a small set of answer options are often provided, but these are limited and impersonal. In contrast, free-form drawings can answer messages in a very personal way, but are difficult to produce on small displays. To enable precise drawing input on smartwatches we present a magnetic stylus that is tracked on the back of the hand. In an evaluation of several algorithms we show that 3D position estimation with a 7.5x20mm magnet reaches a worst-case 6% relative position error on the back of the hand. Furthermore, the results of a user study are presented, which show that in the case of drawing applications the presented technique is faster and more precise than direct finger input.

While smartwatches are widely adopted these days, their input and output space remains fairly limited by their screen size. We present StrapDisplays-interactive watchbands with embedded display and touch technologies-that enhance commodity watches and extend their input and output capabilities. After introducing the physical design space of these StrapDisplays, we explore how to combine a smartwatch and straps in a synergistic Watch+Strap system. Specifically, we propose multiple interface concepts that consider promising content distributions, interaction techniques, usage types, and display roles. For example, the straps can enrich watch apps, display visualizations, provide glanceable feedback, or help avoiding occlusion issues. Further, we provide a modular research platform incorporating three StrapDisplay prototypes and a flexible web-based software architecture, demonstrating the feasibility of our approach. Early brainstorming sessions with 15 participants informed our design process, while later interviews with six experts supported our concepts and provided valuable feedback for future developments.

© 2020 Association for Computing Machinery. This paper introduces TRANS-DOCK, a docking system for pin-based shape displays that enhances their interaction capabilities for both the output and input. By simply interchanging the transducer module, composed of passive mechanical structures, to be docked on a shape display, users can selectively switch between different configurations including display sizes, resolutions, and even motion modalities to allow pins moving in a linear motion to rotate, bend and inflate. We introduce a design space consisting of several mechanical elements and enabled interaction capabilities. We then explain the implementation of the docking system and transducer design components. Our implementation includes providing the limitations and characteristics of each motion transmission method as design guidelines. A number of transducer examples are then shown to demonstrate the range of interactivity and application space achieved with the approach of TRANS-DOCK. Potential use cases to take advantage of the interchangeability of our approach are discussed. Through this paper we intend to expand expressibility, adaptability and customizability of a single shape display for dynamic physical interaction. By converting arrays of linear motion to several types of dynamic motion in an adaptable and flexible manner, we advance shape displays to enable versatile embodied interactions.

Screen size has potential to affect the results of crowdsourced image classification and impact data quality, even for simple tasks like determining whether an animal is present in an image or not. We evaluated the rates at which volunteers judged images to be blank for six Snapshot Safari citizen science projects on the Zooniverse platform, and found that in all but one project, people using mobile devices were more likely to say an image was blank than those using a desktop computer. Qualitative evaluation further demonstrated that screen brightness can also play a role in reliability for volunteers using mobile devices. These findings can be taken into consideration in the design and testing of apps and platforms for crowdsourced image classification to support reliability and data quality.

New communication technologies afford individuals the ability to not only consume media, but also create and share content with others. The purpose of this study is to investigate various factors that influence perceptions of credibility and sharing behaviors. Unfortunately, information disseminated via the Internet does not always contain factual, unbiased information. This study randomly assigned 207 participants to one of six conditions where they were exposed to news articles containing factual or false information and one of three political frames (balanced, right-leaning, left-leaning) to identify the environmental (distraction levels and screen size) and individual factors (political interest and religiosity) that influence perceptions of credibility and sharing behavior. Results suggests that credibility positively influenced sharing behavior, regardless of condition. Additionally, political interest was found as a positive predictor of sharing behavior and religiosity was found as a positive predictor of credibility. Findings are discussed in terms of theoretical and practical implications.

The purpose of this paper is to introduce using scalable autostereoscopic displays that allow users to interact with floating images publicly, such as for digital signage. By tiling multiple display modules, it is possible to project larger autostereoscopic images in the air than by displaying it using a single display module. By projecting autostereoscopic images, it is possible for the user to understand the contents in better detail compared to a standard 2D public displays. Furthermore, by projecting floating images, it is possible to reduce the spatial divergence between the user's body and the autostereoscopic images, so that the user can interact intuitively with autostereoscopic images. Moreover, since this system has scalability, it is possible to expand the screen size just by adding more display modules. This expandability makes the system easy to move and arrange in public spaces. In this paper, we propose an autostereoscopic display module which can enlarge the screen size in the horizontal direction as the first step to realizing our concept.

Smartwatches are highly portable, ubiquitous devices, allowing rich interaction at a small scale. However, the display size can hinder user engagement, limit information display, and presentation style. Most research focuses on exploring ways in which the interaction area of smartwatches can be extended, although this mainly entails simple fold-out displays or additional screens. Conversely, added weight and size can hinder the wearable experience. In response, we took inspiration from origami and explored the design space for new types of lightweight, highly foldable smartwatch, by developing complex paper-prototypes which demonstrate novel ways of extending screen space. We collected data on potential input and output interaction with complex folded smartwatch displays during workshops with expert and non-expert users, discovering application ideas and additional input/output functionality. These insights were used to produce and evaluate a concept video for the FoldWatch prototype.

Available bandwidth is still a limiting factor for mobile communication applications. Multisensory communication has already been identified as an possibility to moderate this limitation. One of the strengths of mobile communication lies in its combination of visual and auditory modalities. However, one of the most salient features of mobile devices have are their small screen size. This paper explores how the potential for multimodal synergy relates to the small screen size. In an experiment with 54 participants, the intelligibility was tested using a standardized video-listening test. The videos had a signal-to-noise ratio of -9dB and were presented on three different screen sizes, whilst keeping the video and auditory signals equal. Intelligibility was found to be significantly higher when using a large screen in comparison to using either of both smaller screens. We conclude that multisensory synergy is key to mobile applications, yet that screen size is a substantial constraint to this synergy. We argue that knowledge about human sensory processing can alleviate this constraint and maximize the potential quality of service of mobile video technology.

The capacity of spatial multi-touch menus such as FastTap is limited by device screen size. We explore the idea of using multiple tabs to increase capacity - multiplexing the tablet's screen space so each location holds multiple items. Earlier work has shown potential of this idea for smartwatches, but no evaluations have considered larger devices. To assess issues with interference-based errors and spatial memory development, we built two FastTap systems with multiple tabs and conducted two studies. We first tested user learning of 16 targets with a training game, and found that participants easily adapted to the multi-tab model, were able to perform memory-based shortcuts, and made few interference-based errors. The second study used realistic drawing tasks and showed that people successfully used the multi-tab FastTap system, with 88% of selections made using shortcuts by the study's end. Our work demonstrates that spatial memory can successfully be multiplexed, and that tabs are a promising way to increase command set sizes for spatial interfaces.

As smartwatch functions expand, target selection among many items will probably become a common task. List search interfaces (LSIs) for a smartwatch use a prefix matching query to search for items, and need two modes because of the small screen size: query input mode and list navigation mode. Despite the modes, LSIs may be more efficient than list interfaces (LIs), which involve no text querying, for large pools. Actually, we could show that the LSI outperformed the LI for pool sizes over 60. However, we also found that the LSI users experience overhead when deciding whether to switch the modes or not. To reduce the overhead, we designed two auto-switching LSIs: input-length-based auto-switching LSI (IA-LSI) and list-size-based auto-switching LSI (LA-LSI). We could show both auto-switching LSIs outperformed the conventional LSI for pool sizes over 60. We also conducted experiments with the auto-switching LSIs for various pool sizes, and provided their results and guidelines for the optimal switching criteria for the LSIs.

Our goal is to present an initial concept of a novel haptic system to aid the learning of visual impairment in the context of computer science, with a focus on the representation and contextualization of digital images. Our system is divided in three stages, being: a software that receives the input images; an intermediate interface that creates an alternative representation for the visually impaired; and output devices for tactile and auditory representation. As proof of concept, an initial prototype of recognition of geometric shapes was developed, in which the haptic system was represented by a matrix of LEDs. Empirical evaluations were carried out of the prototype. The preliminary results show that the prototype achieves satisfactory representations, even though some points must be reviewed, such as a suitable display size. With this work, the proof of concept obtained indicates that the proposed system has potential to assist the visually impaired in computer science learning.

Gaze gesture-based interactions on a computer are promising, but the existing systems are limited by the number of supported gestures, recognition accuracy, need to remember the stroke order, lack of extensibility, and so on. We present a gaze gesture-based interaction framework where a user can design gestures and associate them to appropriate commands like minimize, maximize, scroll, and so on. This allows the user to interact with a wide range of applications using a common set of gestures. Furthermore, our gesture recognition algorithm is independent of the screen size, resolution, and the user can draw the gesture anywhere on the target application. Results from a user study involving seven participants showed that the system recognizes a set of nine gestures with an accuracy of 93% and a F-measure of 0.96. We envision, this framework can be leveraged in developing solutions for situational impairments, accessibility, and also for implementing rich a interaction paradigm.

The goal of our research was to understand the effects of display size on interaction zones as it applies to interactive systems. Interaction zone models for interactive displays are often static and do not consider the size of the display in their definition. As the interactive display ecosystem becomes more size diverse, current models for interaction are limited in their applicability. This paper describes the results of an exploratory study in which participants interacted with and discussed expectations with interactive displays ranging from personal to wall-sized. Our approach was open-ended rather than grounded in existing interaction zone models in order to explore potential differences in interaction zones and distances. We found that the existence of different interaction zones and the distance at which these zones are relevant are dependent on display size. In discussion of the results, we explore implications of our findings and offer guidelines for the design of interactive display systems.

In this work, we utilize visuo-haptic illusions to improve the perceived performance of encountered-type haptic devices, specifically shape displays, in virtual reality. Shape displays are matrices of actuated pins that travel vertically to render physical shapes; however, they have limitations such as low resolution, small display size, and low pin speed. To address these limitations, we employ illusions such as redirection, scaling, and retargeting that take advantage of the visual dominance effect, the idea that vision often dominates when senses conflict. Our evaluation of these techniques suggests that redirecting sloped lines with angles less than 40 degrees onto a horizontal line is an effective technique for increasing the perceived resolution of the display. Scaling up the virtual object onto the shape display by a factor less than 1.8x can also increase the perceived resolution. Finally, using vertical redirection a perceived 3x speed increase can be achieved.

With the proliferation of online news read on devices ranging from desktops to smart watches, the need for meaningful summaries of long texts is growing. Manual summaries are labour-intensive and cannot be offered for all display sizes, whereas today's abstracts of most news texts are teasers designed to attract the reader's interest more than to provide an overview of an article's content suited to the reader's information needs. We propose responsive news summarization as a technological approach for filling this gap. Responsive news summarization provides an automatically generated content summary that has the right length for the device requesting the article, plus access to the full text. We describe the system prototype available at multisizenews.com along with the initial user study results and give an outlook on future work.

Two experiments were conducted in order to investigate the effects of screen-size (3.5" vs. 7" vs. 10.1") on efficiency of mobile learning (m-learning) over time (immediate vs. delayed). Experiment 1 (N=141) revealed that students who took a mobile learning course on a large screen scored higher on a test administered immediately after the completion of the course. In Experiment 2 (N=111), another test was conducted two months after the first test. The Result indicated that the screen size did not make a significant difference on m-learning students over time. In detailed, test scores of students who took the course on a small screen were not relatively affected by the time of administering the test, whereas test scores of students with a large screen mobile device were decreased over time. Implications of notable findings are discussed. Also, limitations and future studies are examined.

The small screen size of a smartwatch limits user experience when watching or interacting with media. We propose a supplementary tactile feedback system to enhance the user experience with a method unique to the smartwatch form factor. Our system has a deformable surface on the back of the watch face, allowing the visual scene on screen to extend into 2.5D physical space. This allows the user to watch and feel virtual objects, such as experiencing a ball bouncing against the wrist. We devised two controlled experiments to analyze the influence of tactile display resolution on the illusion of virtual object presence. Our first study revealed that on average, a taxel can render virtual objects between 70% and 138% of its own size without shattering the illusion. From the second study, we found visual and haptic feedback can be separated by 4.5mm to 16.2mm for the tested taxels. Based on the results, we developed a prototype (called RetroShape) with 4×4 10mm taxels using micro servo motors, and demonstrated its unique capability through a set of tactile-enhanced games and videos. A preliminary user evaluation showed that participants welcome RetroShape as a useful addition to existing smartwatch output.

Location based services are a common application scenario in mobile and ubiquitous computing use. A typical issue with cartographic applications in this domain is the limited size of the displayed map, which makes interaction and visualization a difficult problem to solve. With the increasing popularity of head mounted displays for VR and AR systems, an opportunity is presented for map-based applications to overcome the limitation of the small display size, as the user's information visualization space can extend to his entire surroundings. In this paper we present a preliminary investigation into how interaction with such very large display maps can take place, using a virtual reality headset as the sole input and interaction method.

In this paper, we present a novel method of creating a waterproof wearable fingertip haptic display "AeroFinger" that is very light and small enough to fit on the fingertip and uses no electro-mechanical actuation. The display consists of 4 miniature airbags which are made out of 3D printed Rubber-Like material so that the display size, strength and shape can be customized by the user. A small sized full range speaker is mounted on a closed air chamber where the air is transferred back and forth through a tiny nozzle to the airbag. The Speaker movements creates a difference in air pressure and translated into airbag inflation and deflation. Therefore, AeroFinger can display the low frequency vibrations as force sensation and high frequency vibrations as tactile sensation. Unlike most ungrounded haptic devices which contains electrical components such as motors or vibration actuators, AeroFinger uses no electro-mechanical actuation and thus can be completely submerged on water or could be used in magnetic resonance environments.

Over the last ten years or so, mobile devices technology has changed significantly, with these devices and operating systems becoming more sophisticated. These developments have led to a huge variety of mobile applications designed for mobile operating systems. These mobile applications are typically harder to design and build because of several factors such as screen size and limited processing power and so forth. Therefore, it is important to clearly identify the characteristics of mobile application development and the issues and challenges related to it, as well as, the key features that characterize a great mobile application which make them valuable and useful. This paper has reviewed existing literature of the challenge and best practices of mobile application development. This study contributes towards a great understanding of the characteristics of mobile application development process, examines real challenges faced and explores the best practices that can be effectively applied to improve the development of mobile application.

Responsive Web Design (RWD) enables web applications to adapt to the characteristics of different devices such as screen size which is important for mobile browsing. Today, the only W3C standard to support this adaptability is CSS media queries. However, using media queries it is impossible to create applications in a modular way, because responsive elements then always depend on the global context. Hence, responsive elements can only be reused if the global context is exactly the same. This makes it extremely challenging to develop large responsive applications, because the lack of true modularity makes certain requirement changes either impossible or expensive to realize.In this paper we extend RWD to also include responsive modules, i.e., modules that adapt their design based on their local context, independently of the global context. We present the ELQ project that includes an approach to enabling modular responsivity, and a novel implementation of resize detection of DOM elements. ELQ provides an implementation of element queries which generalize CSS media queries. Importantly, our design conforms to existing web specifications, enabling adoption on a large scale. ELQ is designed to be heavily extensible using plugins. Experimental results show speed-ups of the core algorithms of up to 37x compared to previous approaches.

Supported by their large size and high resolution, display walls suit well for different collaboration types. However, in order to foster instead of impede collaboration processes, interaction techniques need to be carefully designed, taking into regard the possibilities and limitations of the display size, and their effects on human perception and performance. In this paper we investigate the impact of visual distractors (which, for instance, might be caused by other collaborators' input) in peripheral vision on short-term memory and attention. The distractors occur frequently when multiple users collaborate in large wall display systems and may draw attention away from the main task, as such potentially affecting performance and cognitive load. Yet, the effect of these distractors is hardly understood. Gaining a better understanding thus may provide valuable input for designing more effective user interfaces. In this article, we report on two interrelated studies that investigated the effect of distractors. Depending on when the distractor is inserted in the task performance sequence, as well as the location of the distractor, user performance can be disturbed: we will show that distractors may not affect short term memory, but do have an effect on attention. We will closely look into the effects, and identify future directions to design more effective interfaces.

In this paper, we explore the potential of using visual representations to support people in managing, organizing, and understanding relations between multiple visualization views. Multiple views can help people understand different facets of data and data processing, and are a crucial part of data analysis particularly when it is done collaboratively. Both the growing use of multiple views and the increasing display sizes have amplified the need to explore how to better help people to understand the relations between many views. To improve our understanding of how to visualize view relations, we invited visualization and interaction designers to critique and sketch representations of view relations. The participants provided design critiques, and sketched their own relation representations. Our findings expand the range and palette of ways of visually linking visualization views and suggest new directions for designing view relation representations to better support analysis with multiple views.

While smartwatches have become common for mobile interaction, one of their main limitation is the limited screen size. To facilitate reading activities despite these limitations, reading with Rapid Serial Visual Presentation (RSVP) has been shown to be feasible. However, when text is presented in rapid sequence, single words are easily missed due to blinking or briefly glancing up from the screen. This gets worse the more the reader is engaged in a secondary task, such as walking. To give implicit control over the reading flow we combined an RSVP reading application on a smartwatch with a head-worn eye tracker. When the reading flow is briefly interrupted, the text presentation automatically pauses or backtracks. In a user study with 15 participants we show that using eye tracking in combination with RSVP increases users' comprehension compared to a touch-based UI to control the text presentation. We argue that eye tracking will be a valuable extension for future smartwatch interaction.

Smartphone screen size becomes larger nowadays. However, it causes users hard to reach a target with the thumb when using the smartphone in one hand. We propose an interaction method MovingScreen to solve the hard-to-reach problem by making the smartphone screen view movable. Users move the screen view to make the target into their comfort zone, an area users comfortably performing touch input with the thumb. They then select a target easily. Using the proposed triggering gesture bezel-scroll, MovingScreen automatically calibrates the comfort zone. Bezel-scroll detects the comfort zone and provides different screen moving ratios for users in different poses to hold the smartphone. Users are even allowed to adjust screen moving speed by altering bezel-scroll length. We evaluate performance of MovingScreen and other methods in a user study. The results show that MovingScreen has similar selection time (1030.58ms) but lower error rate (4.57%) to the other methods.

Film and television content is moving out of the living room and onto mobile devices - viewers are now watching when and where it suits them, on devices of differing sizes. This freedom is convenient, but could lead to differing experiences across devices. Larger screens are often believed to be favourable, e.g. to watch films or sporting events. This is partially supported in the literature, which shows that larger screens lead to greater presence and more intense physiological responses. However, a more broadly-defined measure of experience, such as that of immersion from computer games research, has not been studied. In this study, 19 participants watched content on three different screens and reported their immersion level via questionnaire. Results showed that the 4.5-inch phone screen elicited lower immersion scores when compared to the 13-inch laptop and 30-inch monitor, but there was no difference when comparing the two larger screens. This suggests that very small screens lead to reduced immersion, but after a certain size the effect is less pronounced.

The screen size of a smartwatch provides limited space to enable expressive multi-touch input, resulting in a markedly difficult and limited experience. We present WatchMI: Watch Movement Input that enhances touch interaction on a smartwatch to support continuous pressure touch, twist, pan gestures and their combinations. Our novel approach relies on software that analyzes, in real-time, the data from a built-in Inertial Measurement Unit (IMU) in order to determine with great accuracy and different levels of granularity the actions performed by the user, without requiring additional hardware or modification of the watch. We report the results of an evaluation with the system, and demonstrate that the three proposed input interfaces are accurate, noise-resistant, easy to use and can be deployed on a variety of smartwatches. We then showcase the potential of this work with seven different applications including, map navigation, an alarm clock, a music player, pan gesture recognition, text entry, file explorer and controlling remote devices or a game character. Postprint

The Experience Sampling Method is widely used for collecting self-report responses from people in natural settings. While most traditional approaches rely on using a phone to trigger prompts and record information, wearable devices now offer new opportunities that may improve this method. This research quantitatively and qualitatively studies the experience sampling process on head-worn and wrist-worn wearable devices, and compares them to the traditional "smartphone in the pocket." To enable this work, we designed and implemented a custom application to provide similar prompts across the three types of devices and evaluated it with 15 individuals for five days (75 days total), in the context of real-life stress measurement. We found significant differences in response times across devices, and captured tradeoffs in interaction types, screen size, and device familiarity that can affect both users' experience and the reports made by users.

Wall displays almost universally assume a mostly flat and static shape. We ask two questions: Would people choose a flat display for a given interaction scenario and, if not, what are the display shapes they actually prefer? We conducted a design study around these two questions. Our results show that participants designed different screen shapes that varied based upon peoples' distance from the display and the content shown. Shapes ranged primarily between flat, separated, concave, L-shape and convex displays. Based on our findings, we designed a dynamic display that changes to these and other configurations. Shape-shifting is controlled either by explicit interaction (where the display responds to hand gestures) or implicitly (where the display infers a shape based both on its content and the sensed positions of the people around it). Overall, we contribute: a study that motivates research on shape-shifting wall displays, and a shape-shifting display system that responds to explicit and implicit controls to match particular activities.

Mobile devices are increasingly becoming an important part of technology. As the demand for handling information grows, so does the need for collaboration in integrating and managing multiple forms of information. With its small screen, it is relatively uncomfortable for a group of people to perform task collaboratively. This paper presents MobiTop - a multi-mobile system that integrates mobile devices that works seamlessly together -- promoting an extended screen size for the mobile devices to support collaboration. However, interactivity issues such as bezels and screen orientation are found to restrict users from performing collaborative tasks favourably. This paper discusses the interactivity issues pertaining multi-mobile systems and offers guidelines for designers.

International audience; This paper addresses the problem of mastering the complexity of interacting with a large set of applications on smartphones. In one hand, number of applications increases. In the other hand, screen size reduces. To tackle this paradoxical evolution, we investigate adaptive user interfaces. We assume that it is possible to predict the applications of interest for a user in a given situation. Based on this hypothesis, our challenge is to accelerate user interaction when prediction is correct, without penalizing it when prediction is wrong. The paper proposes the concept of Evanescent Adaptation. The principle is a two-layer based representation: the predicted items (first layer) are displayed above the full list of items (second layer). The first layer is said to be evanescent in the sense that it automatically disappears progressively. The paper claims for putting this disappearing process under the control of the end-user. Thereby the user can close the first-layer as soon as s/he perceives prediction as irrelevant.

The smartwatch is a representative of the wearables, mini-computers worn on the body, which often only require casual interaction. In this project the advantages of smartwatches should be used to add landmarks to a database as performantly as possible and provide them for further usage. The mobile terminal should therefore be operated with voice input in a clearly structured speech dialogue to make the interaction as simple and intuitive as possible. Additionally a smartphone paired with the smartwatch distributes the user's GPS-data which help to gather data for landmarks in URWalking, a research project for pedestrian navigation at the University of Regensburg. The special problem is on the one hand to find solutions for hardware-related limitations such as a very small display size. On the other hand a way for the translation of plain-language input to technically evaluable data has to be found. In a final evaluation the system should be tested under real-life circumstances.

Large displays are becoming prevalent, but little research has been conducted to quantify their effect on an individual user. We present an experiment in which users' reading comprehension of text displayed on physically large and small displays are compared through three types of tasks. By adjusting the viewing distance for both displays, we maintained a constant visual angle. This experiment demonstrated that differences in display size did not affect users' performance in brief reading comprehension tasks, that is, both search tasks involving small units such as characters or words and comprehension tasks involving larger units such as sentences or paragraphs. We found a large difference between the outcome of this experiment relating to text media and the results of previous research, which showed that for picture and video media, large displays bias users toward an egocentric perspective and small displays bias them toward an exocentric perspective.

This paper presents and discusses the internal operation of NCLSC (NCL Stereo Converter): a tool to convert a 2D interactive multimedia application annotated with depth information to a stereoscopic-multimedia application. Stereoscopic-multimedia applications are those that codify both the left-eye and right-eye views, as required by stereoscopic 3D displays. NCLSC takes as input an NCL (Nested Context Language) document and outputs an NCL stereoscopic application codified in side-by-side or top-bottom format (both common input formats for 3DTV sets). NCL is the declarative language adopted in most Latin America countries for terrestrial digital TV middleware systems and the ITU-T H.761 Recommendation for IPTV services. However, the proposed approach is not restricted to NCL and can be used by other languages. The depth annotation allows for positioning each 2D graphical component in a layered (2.5D or 2D+depth) user interface. It is used by NCLSC to compute the screen parallax (offset) between the graphical elements in the left and right views of the resulting stereoscopic application. When the resulting application is presented on stereoscopic 3D displays, such screen parallax induces retinal disparity, which creates the illusion of floating flat-2D graphical elements. NCLSC does not require any additional native middleware support to run in currently available 3D-enabled TV sets. Moreover, NCLSC can adapt, at run-time, the output application to different display sizes, viewer distances, and viewer preferences, which are usually required for a proper balance between artistic effects and user experience.

With a growing interest in wrist-worn devices, research has typically focused on expanding the available interaction area for smartwatches. In this paper, we instead investigate how different display sizes influence task performance, while maintaining a consistent input area. We conducted an experiment in which users completed a scrolling task using a small display, a large display, and a cylindrical display wrapped around the wrist. We found that the large and cylindrical displays resulted in faster task performances than the small display. We also found that the cylindrical display removed constraints on the participants' body pose, suggesting that cylindrical displays have unique benefits for mobile interactions.

In many disciplines, such as computer aided drug design, multiple simulation runs are performed with varying parameters, yielding ensembles of data sets. Comparative visualisation of these simulation results can help understanding the influence of different parameters. However, researchers might need to compare large numbers of variants. Single desktop monitors often do not have the resolution and screen size required for showing a whole ensemble at once with sufficient detail. Wall-sized high-resolution displays can be a solution for this problem. Although a number of studies has been conducted on how large high-resolution displays affect the speed and accuracy of certain tasks, only few of them are related to actual scientific visualisation tasks. We built a system for comparative visualisation of simulation results that can be used with conventional desktop monitors and with large high-resolution displays. We conducted a study using biochemical simulation data to evaluate the impact of screen size and 3D stereo display on a comparison task.

Providing the initiator of a conversation with awareness information about their target's availability can improve interruption timing. Providing this information, however, can be particularly challenging with mobile devices, which are carried by users in a range contexts and have limited audible and visual display possibilities due to possible background noise and small screen size. Haptic or tactile displays present a potentially useful alternative, as interaction with mobile devices is often by touch. This paper reports on a dual-task comparison of awareness information usage on a mobile variable-friction tactile display vs. a visual display under conditions that varied in task type and task workload. Participants using the tactile display performed marginally better on the primary task, but were less accurate and slower on the awareness task. However, there is evidence that some of the awareness task differences dissipated over time. Self-report data also suggests that people's experience with the tactile display was generally positive and improved over time.

Large high-resolution displays are becoming increasingly common in research settings, providing data scientists with visual interfaces for the analysis of large datasets. Numerous studies have demonstrated unique perceptual and cognitive benefits afforded by these displays in visual analytics and information visualization tasks. However, the effects of these displays on knowledge discovery in exploratory visual analysis are still poorly understood. We present the results of a small-scale study to better understand how display size and resolution affect insight. Analyzing participants' verbal statements, we find preliminary evidence that larger displays with more pixels can significantly increase the number of discoveries reported during visual exploration, while yielding broader, more integrative insights. Furthermore, we find important differences in how participants performed the same visual exploration task using displays of varying sizes. We tie these results to extant work and propose explanations by considering the cognitive and interaction costs associated with visual exploration.

This demo features smartwatch applications created by junior/senior CS students in a mobile development course. The course featured the Pebble smartwatch, with an e-ink display, vibrating motor, and accelerometer. Smartwatches enhance mobile device use by acting as a secondary display and providing immediate notifications, but designing for them has challenges in memory management, sensor programming, networking issues, and display size. Incorporating Pebble smartwatch development in a mobile app development class allowed students to exercise core computer science concepts to overcome Pebble hardware and software limitations and to develop applications that leverage Pebble affordances. Two sections of the class featured Pebble-focused assignments, including an open-ended homework, a focused homework, and a term design project coordinating smartphone-smartwatch use. The two-semester experience yielded apps in all Pebble-defined categories, including tools and utilities, notifications, remotes, health and fitness, and games. Examples of the students' work from each of the categories will be presented in this demo on topics such as music synthesizing, remote camera operation, course management alerts, and server management. The presented apps reflect student understanding of the sensor functionalities, hardware capabilities and limitations, and interactions of the phone-watch pair. This demo will serve as a valuable resource for those who plan to incorporate smartwatch development into their teaching practice. Based on the educational experience gleaned, future opportunities can explore the role of the smartwatch in mobile development and programming education for college and K-12 students.

Given the large number of installed apps and the limited screen size of mobile devices, it is often tedious for users to search for the app they want to use. Although some mobile OSs provide categorization schemes that enhance the visibility of useful apps among those installed, the emerging category of homescreen apps aims to take one step further by automatically organizing the installed apps in a more intelligent and personalized way. In this paper, we study how to improve homescreen apps' usage experience through a prediction mechanism that allows to show to users which app she is going to use in the immediate future. The prediction technique is based on a set of features representing the real-time spatiotemporal contexts sensed by the homescreen app. We model the prediction of the next app as a classification problem and propose an effective personalized method to solve it that takes full advantage of human-engineered features and automatically derived features. Furthermore, we study how to solve the two naturally associated cold-start problems: app cold-start and user cold-start. We conduct large-scale experiments on log data obtained from Yahoo Aviate, showing that our approach can accurately predict the next app that a person is going to use.

Traditional photo browsing systems developed for PCs are inefficient for browsing and searching of large photo albums on mobile devices due to the small screen size and limited mobile processing power. We propose a new concept in this paper, the multiscale timeline, where photos are grouped into clusters and displayed sequentially on a scaled timeline with user controllable time scales, enabling multiscale overview of the photo album for efficient browsing and searching. To address the slow speed of re-clustering when new photos are added, a new incremental spectral clustering algorithm is further developed, which is an order of magnitude faster than the traditional spectral clustering algorithm and its conventional incremental version. Our implementation of the system on mobile devices shows a better user experience and browsing efficiency based on the experiments over large real-world photo collections.

In a remote surveillance system, a high resolution surveillance camera streams its video to a user's handheld device. Such devices are unable to make use of the high resolution video due to their limited display size and bandwidth. In this paper, we propose a method to assist the mobile operator of the surveillance camera in focusing on sensitive regions of the video. Our system automatically identifies relevant regions. We introduce a pan and zoom strategy to ensure that the operator is able to see fine details in these areas while maintaining contextual knowledge. Regions of interest are identified using foreground detection as well as face and body detection. The efficacy of the proposed method is demonstrated through a user study. Our proposed method was reported to be more useful than two comparable approaches for getting an understanding of the activities in a surveillance scene while maintaining context.

We can see at these days a large number of users utilizing a smartphone or a tablet as their daily working tool. Many professionals are changing the use of their desktop computers by using their mobile devices due to trends in their business. Thus, we can find more enterprise mobile apps that allow users to manage their companies using a mobile device in a more efficient and effective way and everywhere. This growing demand for mobile applications has become a challenge for current developers. We can find more and more enterprise applications delivered in three versions: desktop, tablet and smartphone. However, it is difficult to determine how productive a user can be in each version. In this paper we present a comparative usability study of one management mobile application designed for smartphone and tablet platforms for measuring the efficiency of the tablet version versus the smartphone version. We conducted a study with 8 participants to measure the efficiency of one mobile app and also measured the perceived usability using SUS questionnaire. This way, we obtained a ratio of the efficiency of using a tablet compared to a smartphone that could be significant for the development of future apps in small screens.

Despite extensive usability research on virtual reality (VR) within academia and the increasing use of VR within industry, there is little research evaluating the usability and benefits of VR in applied settings. This is problematic for individuals desiring design principals or best practices for incorporating VR into their business. Furthermore, the literature that does exist often doesn't account for the characteristics of intended users. This shortage is problematic because individual differences have been shown to have a significant impact on performance in spatial tasks. The research presented here is an evaluation of a VR system in use at The Boeing Company, with 28 employees performing navigation and wayfinding tasks across two shading conditions (flat/smooth) and two display conditions (desktop/immersive). Performance was measured based on speed and accuracy. Individual difference factors were used as covariates. Results showed that women and those with high spatial orientation ability performed faster in smooth shading conditions, while flat shading benefited those with low spatial ability particularly for the navigation task. Unexpectedly, immersive presentation did not improve performance significantly. These results demonstrate the impact of individual differences on spatial performance and help determine appropriate tasks, display parameters, and suitable users for the VR system.