Search

In an era of rapidly evolving technology and increasing interconnection, full participation in society depends on the successful use of technology. Thus, to ensure equity and participation for people with disabilities, technology must be accessible - we must create and adapt interactive systems to improve access to technology and to the world at large. The University of Washington Center for Research and Education on Accessible Technology and Experiences (CREATE) is dedicated to propelling accessible technology research and education from incremental improvements to paradigm-shifting breakthroughs that enable greater inclusion and participation for people of all abilities.

Online data visualizations are widely used to communicate information from simple statistics to complex phenomena, supporting people in gaining important insights from data. However, due to the defining visual nature of data visualizations, extracting information from visualizations can be difficult or impossible for screen-reader users. To assess screen-reader users’ challenges with online data visualizations, we conducted two empirical studies: (1) A qualitative study with nine screen-reader users, and (2) a quantitative study with 36 screen-reader and 36 non-screen-reader users. Our results show that due to the inaccessibility of online data visualizations, screen-reader users extract information 61.48% less accurately and spend 210.96% more time interacting with online data visualizations compared to non-screen-reader users. Additionally, our findings show that online data visualizations are commonly indiscoverable to screen readers. In visualizations that are discoverable and comprehensible, screen-reader users suggested tabular and textual representation of data as techniques to improve the accessibility of online visualizations. Taken together, our results provide empirical evidence of the inequalities screen-readers users face in their interaction with online data visualizations.

Two-dimensional canvases are the core components of many digital productivity and creativity tools, with “artboards” containing objects rather than pixels. Unfortunately, the contents of artboards remain largely inaccessible to blind users relying on screen-readers, but the precise problems are not well understood. This study sought to understand how blind screen-reader users interact with artboards. Specifically, we conducted contextual interviews, observations, and task-based usability studies with 15 blind participants to understand their experiences of artboards found in Microsoft PowerPoint, Apple Keynote, and Google Slides. Participants expressed that the inaccessibility of these artboards contributes to significant educational and professional barriers. We found that the key problems faced were: (1) high cognitive loads from a lack of feedback about artboard contents and object state; (2) difficulty determining relationships among artboard objects; and (3) constant uncertainty about whether object manipulations were successful. We offer design remedies that improve feedback for object state, relationships, and manipulations.

Correcting errors in entered text is a common task but usually difficult to perform on mobile devices due to tedious cursor navigation steps. In this paper, we present JustCorrect, an intelligent post hoc text correction technique for smartphones. To make a correction, the user simply types the correct text at the end of their current input, and JustCorrect will automatically detect the error and apply the correction in the form of an insertion or a substitution. In this way, manual navigation steps are bypassed, and the correction can be committed with a single tap. We solved two critical problems to support JustCorrect: (1) Correction Algorithm: we propose an algorithm that infers the user's correction intention from the last typed word. (2) Input Modalities: our study revealed that both tap and gesture were suitable input modalities for performing JustCorrect. Based on our findings, we integrated JustCorrect into a soft keyboard. Our user studies show that using JustCorrect reduces the text correction time by 12.8% over the stock Android keyboard and by 9.7% over the "Type, then Correct" text correction technique by Zhang et al. (2019). Overall, JustCorrect complements existing post hoc text correction techniques, making error correction more automatic and intelligent.

Despite years of addressing disability in technology design and advocating user-centered design practices, popular mainstream technologies remain largely inaccessible for people with disabilities. We conducted a design course study investigating how student designers regard disability and explored how designing for multiple disabled and nondisabled users encouraged students to think about accessibility in the design process. Across two university course offerings one year apart, we examined how students focused on a design project while learning user-centered design concepts and techniques, working with people with and without disabilities throughout the project. In addition, we compared how students incorporated disability-focused design approaches within a classroom setting. We found that designing for multiple stakeholders with and without disabilities expanded student understanding of accessible design by demonstrating that people with the same disability could have diverse needs and by aligning such needs with those of nondisabled users. We also found that using approaches targeted toward designing for people with disabilities complemented interactions with users, particularly with regard to managing varying abilities across users, or incorporating social aspects. Our findings contribute to an understanding about how we might incur change in design practice by working with multiple stakeholders with and without disabilities whenever possible. We refined Design for Social Accessibility by incorporating these findings into three tenets emphasizing: (1) design for disability ought to incorporate users with and without disabilities, (2) design should address functional and social factors simultaneously, and (3) design should include tools to spur consideration of social factors in accessible design.

Current text correction processes on mobile touch devices are laborious: users either extensively use backspace, or navigate the cursor to the error position, make a correction, and navigate back, usually by employing multiple taps or drags over small targets. In this paper, we present three novel text correction techniques to improve the correction process: Drag-n-Drop, Drag-n-Throw, and Magic Key. All of the techniques skip error-deletion and cursor-positioning procedures, and instead allow the user to type the correction first, and then apply that correction to a previously committed error. Specifically, Drag-n-Drop allows a user to drag a correction and drop it on the error position. Drag-n-Throw lets a user drag a correction from the keyboard suggestion list and "throw" it to the approximate area of the error text, with a neural network determining the most likely error in that area. Magic Key allows a user to type a correction and tap a designated key to highlight possible error candidates, which are also determined by a neural network. The user can navigate among these candidates by directionally dragging from atop the key, and can apply the correction by simply tapping the key. We evaluated these techniques in both text correction and text composition tasks. Our results show that correction with the new techniques was faster than de facto cursor and backspace-based correction. Our techniques apply to any touch-based text entry method.

Method-independent text entry evaluation tools are often used to conduct text entry experiments and compute performance metrics, like words per minute and error rates. The input stream paradigm of Soukoreff & MacKenzie (2001, 2003) still remains prevalent, which presents a string for transcription and uses a strictly serial character representation for encoding the text entry process. Although an advance over prior paradigms, the input stream paradigm is unable to support many modern text entry features. To address these limitations, we present transcription sequences: for each new input, a snapshot of the entire transcribed string unto that point is captured. By comparing adjacent strings within a transcription sequence, we can compute all prior metrics, reduce artificial constraints on text entry evaluations, and introduce new metrics. We conducted a study with 18 participants who typed 1620 phrases using a laptop keyboard, on-screen keyboard, and smartphone keyboard using features such as auto-correction, word prediction, and copy/paste. We also evaluated non-keyboard methods Dasher, gesture typing, and T9. Our results show that modern text entry methods and features can be accommodated, prior metrics can be correctly computed, and new metrics can reveal insights. We validated our algorithms using ground truth based on cursor positioning, confirming 100% accuracy. We also provide a new tool, TextTest++, to facilitate web-based evaluations.

Online news sources have transformed civic discourse, and much has been made of their credibility. Although web page credibility has been investigated generally, most work has focused on the credibility of web page content. In this work, we study the isolated appearance of news-like web pages. Specifically, we report on a laboratory experiment involving 31 college students rating the perceived credibility of news-like web pages devoid of meaningful content. These pages contain only "lorem ipsum" text, indistinct videos and images, non-functional links, and various font settings. Our findings show that perceived credibility is indeed affected by some purely presentational factors. Specifically, video presence increased credibility, while large fonts and having no images reduced credibility. Having a few, but not too many, images increased credibility for short articles, especially in the presence of large fonts. We also conducted follow-up interviews, which revealed that participants noticed images, videos, and font sizes when making credibility judgments, corroborating our quantitative experimental results.

Situationally-induced impairments and disabilities (SIIDs) make it difficult for users of interactive computing systems to perform tasks due to context (e.g., listening to a phone call when in a noisy crowd) rather than a result of a congenital or acquired impairment (e.g., hearing damage). SIIDs are a great concern when considering the ubiquitousness of technology in a wide range of contexts. Considering our daily reliance on technology, and mobile technology in particular, it is increasingly important that we fully understand and model how SIIDs occur. Similarly, we must identify appropriate methods for sensing and adapting technology to reduce the effects of SIIDs. In this workshop, we will bring together researchers working on understanding, sensing, modelling, and adapting technologies to ameliorate the effects of SIIDs. This workshop will provide a venue to identify existing research gaps, new directions for future research, and opportunities for future collaboration.

Human-computer input performance inherently involves speed-accuracy tradeoffs---the faster users act, the more inaccurate those actions are. Therefore, comparing speeds and accuracies separately can result in ambiguous outcomes: Does a fast but inaccurate technique perform better or worse overall than a slow but accurate one? For pointing, speed and accuracy has been unified for over 60 years as throughput (bits/s) (Crossman 1957, Welford 1968), but to date, no similar metric has been established for text entry. In this paper, we introduce a text entry method-independent throughput metric based on Shannon information theory (1948). To explore the practical usability of the metric, we conducted an experiment in which 16 participants typed with a laptop keyboard using different cognitive sets, i.e., speed-accuracy biases. Our results show that as a performance metric, text entry throughput remains relatively stable under different speed-accuracy conditions. We also evaluated a smartphone keyboard with 12 participants, finding that throughput varied least compared to other text entry metrics. This work allows researchers to characterize text entry performance with a single unified measure of input efficiency.

End-user elicitation studies are a popular design method. Currently, such studies are usually confined to a lab, limiting the number and diversity of participants, and therefore the representativeness of their results. Furthermore, the quality of the results from such studies generally lacks any formal means of evaluation. In this paper, we address some of the limitations of elicitation studies through the creation of the Crowdlicit system along with the introduction of end-user identification studies, which are the reverse of elicitation studies. Crowdlicit is a new web-based system that enables researchers to conduct online and in-lab elicitation and identification studies. We used Crowdlicit to run a crowd-powered elicitation study based on Morris's "Web on the Wall" study (2012) with 78 participants, arriving at a set of symbols that included six new symbols different from Morris's. We evaluated the effectiveness of 49 symbols (43 from Morris and six from Crowdlicit) by conducting a crowd-powered identification study. We show that the Crowdlicit elicitation study resulted in a set of symbols that was significantly more identifiable than Morris's.

We compared three common usability testing methods—Concurrent Think-Aloud, Retrospective Think-Aloud and Co-discovery—with frail older adults. We found that Co-discovery is the most effective method for this group. Additionally, we interviewed Human-Computer Interaction experts who work with older adults. These experts discussed, for instance, the importance of leveraging usability tests to enhance participant motivation to engage with technology. We consolidated our findings from the usability studies with older adults and from interviews with experts to create a set of recommendations for performing usability tests with frail older adults. One of the core recommendations is to enhance participants’ sense of autonomy and self-confidence during usability tests.

End-user elicitation studies are a popular design method, but their data require substantial time and effort to analyze. In this paper, we present Crowdsensus, a crowd-powered tool that enables researchers to efficiently analyze the results of elicitation studies using subjective human judgment and automatic clustering algorithms. In addition to our own analysis, we asked six expert researchers with experience running and analyzing elicitation studies to analyze an end-user elicitation dataset of 10 functions for operating a web-browser, each with 43 voice commands elicited from end-users for a total of 430 voice commands. We used Crowdsensus to gather similarity judgments of these same 430 commands from 410 online crowd workers. The crowd outperformed the experts by arriving at the same results for seven of eight functions and resolving a function where the experts failed to agree. Also, using Crowdsensus was about four times faster than using experts.

Personal technologies are rarely designed to be accessible to disabled people, partly due to the perceived challenge of including disability in design. Through design workshops, we addressed this challenge by infusing user-centered design activities with Design for Social Accessibility-a perspective emphasizing social aspects of accessibility-to investigate how professional designers can leverage social factors to include accessibility in design. We focused on how professional designers incorporated Design for Social Accessibility's three tenets: (1) to work with users with and without visual impairments; (2) to consider social and functional factors; (3) to employ tools-a framework and method cards-to raise awareness and prompt reflection on social aspects toward accessible design. We then interviewed designers about their workshop experiences. We found DSA to be an effective set of tools and strategies incorporating social/functional and non/disabled perspectives that helped designers create accessible design.

Modern smartphones are built with capacitive-sensing touchscreens, which can detect anything that is conductive or has a dielectric differential with air. The human finger is an example of such a dielectric, and works wonderfully with such touchscreens. However, touch interactions are disrupted by raindrops, water smear, and wet fingers because capacitive touchscreens cannot distinguish finger touches from other conductive materials. When users' screens get wet, the screen's usability is significantly reduced. RainCheck addresses this hazard by filtering out potential touch points caused by water to differentiate fingertips from raindrops and water smear, adapting in real-time to restore successful interaction to the user. Specifically, RainCheck uses the low-level raw sensor data from touchscreen drivers and employs precise selection techniques to resolve water-fingertip ambiguity. Our study shows that RainCheck improves gesture accuracy by 75.7%, touch accuracy by 47.9%, and target selection time by 80.0%, making it a successful remedy to interference caused by rain and other water.

Breathalyzers, the standard quantitative method for assessing inebriation, are primarily owned by law enforcement and used only after a potentially inebriated individual is caught driving. However, not everyone has access to such specialized hardware. We present drunk user interfaces: smartphone user interfaces that measure how alcohol affects a person's motor coordination and cognition using performance metrics and sensor data. We examine five drunk user interfaces and combine them to form the "DUI app". DUI uses machine learning models trained on human performance metrics and sensor data to estimate a person's blood alcohol level (BAL). We evaluated DUI on 14 individuals in a week-long longitudinal study wherein each participant used DUI at various BALs. We found that with a global model that accounts for user-specific learning, DUI can estimate a person's BAL with an absolute mean error of 0.005% ± 0.007% and a Pearson's correlation coefficient of 0.96 with breathalyzer measurements.

With the recent influx of smartphones, tablets, and wearables such as watches and glasses, personal interactive device use is increasingly visible and commonplace in public and social spaces. Assistive Technologies (ATs) used by people with disabilities are observable to others and, as a result, can affect how AT users are perceived. This raises the possibility that what we call “social accessibility” may be as important as “functional accessibility” when considering ATs. But, to date, ATs have almost exclusively been regarded as functional aids. For example, ATs are defined by the Technical Assistance to the States Act as technologies that are “used to increase, maintain or improve functional capabilities of individuals with disabilities.” To investigate perceptions and self-perceptions of AT users, we conducted a diary study of two groups of participants: people with disabilities and people without disabilities. Our goal was to explore the types of interactions and perceptions that arise around AT use in social and public spaces. During our 4-week study, participants with sensory disabilities wrote about feeling either self-conscious or self-confident when using an assistive device in a social or public situation. Meanwhile, participants without disabilities were prompted to record their reactions and feelings whenever they saw ATs used in social or public situations. We found that AT form and function does influence social interactions by impacting self-efficacy and self-confidence. When the design of form or function is poor, or when inequality between technological accessibility exists, social inclusion is negatively affected, as are perceptions of ability. We contribute a definition for the “social accessibility” of ATs and subsequently offer Design for Social Accessibility (DSA) as a holistic design stance focused on balancing an AT user's sociotechnical identity with functional requirements.

Mobile sign language video conversations can become unintelligible if high video transmission rates cause network congestion and delayed video. In an effort to understand the perceived lower limits of intelligible sign language video intended for mobile communication, we evaluated sign language video transmitted at four low frame rates (1, 5, 10, and 15 frames per second [fps]) and four low fixed bit rates (15, 30, 60, and 120 kilobits per second [kbps]) at a constant spatial resolution of 320 × 240 pixels. We discovered an “intelligibility ceiling effect,” in which increasing the frame rate above 10fps did not improve perceived intelligibility, and increasing the bit rate above 60kbps produced diminishing returns. Given the study parameters, our findings suggest that relaxing the recommended frame rate and bit rate to 10fps at 60kbps will provide intelligible video conversations while reducing total bandwidth consumption to 25% of the ITU-T standard (at least 25fps and 100kbps). As part of this work, we developed the Human Signal Intelligibility Model , a new conceptual model useful for informing evaluations of video intelligibility and our methodology for creating linguistically accessible web surveys for deaf people. We also conducted a battery-savings experiment quantifying battery drain when sign language video is transmitted at the lower frame rates and bit rates. Results confirmed that increasing the transmission rates monotonically decreased the battery life.

The term "disability" connotes an absence of ability, but is like saying "dis-money" or "dis-height." All living people have some abilities [2]. Unfortunately, history is filled with examples of a focus on dis-ability, on what is missing, and on ensuing attempts to replace lost function to make people match a rigid world. Although often well intended, such a focus assumes humans must be adapted, and that interfaces, devices, and environments get to remain as they are. At the same time, our built things embody numerous "ability assumptions" imparted by their designers, and yet our built things remain unaware of their users' abilities. They also remain unaware of the situations their users are in, or how those situations affect their users' abilities [2,3]. An important shift in perspective comes by allowing people to "remain as they are," asking instead how interfaces, devices, and environments can bear the burden of becoming more suitable to their users' situated abilities. I call this perspective and the principles that accompany it "Ability-Based Design" [4,5], where the human abilities required to use a technology in a given context are questioned, and systems are made operable by or adaptable to alternative abilities. From this perspective, all people have varying degrees of ability, and different situations lead to different ability limitations, some long-term and some momentary. Some ability limitations come mostly from within the self, others from mostly outside the self. Ability-Based Design considers this whole "landscape of ability," respecting the human at its center and asking more of our technologies. In this talk, I will cover a decade's worth of projects related to Ability-Based Design, some directed at "people with disabilities" and others directed at "people in disabling situations." Rather than dive into any one project, I will convey a space of explored possibilities. I will also put forth a grand challenge: that anyone, anywhere, at any time can interact with technologies ideally suited to their specific situated abilities, and that our technologies do the work to achieve this fit.

Gesture-based system are becoming ubiquitous. Trablets, phones, large displays, and even laptop computers are now commonly equipped with multitouch-recognizing screens. Third-party accessories like the Wii Nunchuck and the Xbox Kinect can also detect.

We present two contributions toward improving the accessibility of touch screens for people with motor impairments. First, we provide an exploration of the touch behaviors of 10 people with motor impairments, e.g., we describe how touching with the back or sides of the hand, with multiple fingers, or with knuckles creates varied multi-point touches. Second, we introduce Smart Touch, a novel template-matching technique for touch input that maps any number of arbitrary contact-areas to a user's intended (x,y) target location. The result is that users with motor impairments can touch however their abilities allow, and Smart Touch will resolve their intended touch point. Smart Touch therefore allows users to touch targets in whichever ways are most comfortable and natural for them. In an experimental evaluation, we found that Smart Touch predicted (x,y) coordinates of the users' intended target locations over three times closer to the intended target than the native Land-on and Lift-off techniques reported by the built-in touch sensors found in the Microsoft PixelSense interactive tabletop. This result is an important step toward improving touch accuracy for people with motor impairments and others for whom touch screen operation was previously impossible.

Current approaches to accessible computing share a common goal of making technology accessible to users with disabilities. Perhaps because of this goal, they may also share a tendency to centralize disability rather than ability. We present a refinement to these approaches called ability-based design that consists of focusing on ability throughout the design process in an effort to create systems that leverage the full range of human potential. Just as user-centered design shifted the focus of interactive system design from systems to users, ability-based design attempts to shift the focus of accessible design from disability to ability. Although prior approaches to accessible computing may consider users’ abilities to some extent, ability-based design makes ability its central focus. We offer seven ability-based design principles and describe the projects that inspired their formulation. We also present a research agenda for ability-based design.

Prior research shows that people with motor impairments face considerable challenges when using conventional mice and trackballs. One challenge is positioning the mouse cursor within confined target areas; another is executing a precise click without slipping. These problems can make mouse pointing in graphical user interfaces very difficult for some people. This article explores goal crossing as an alternative strategy for more accessible target acquisition. In goal crossing, targets are boundaries that are simply crossed by the mouse cursor. Thus, goal crossing avoids the two aforementioned problems. To date, however, researchers have not examined the feasibility of goal crossing for people with motor difficulties. We therefore present a study comparing area pointing and goal crossing. Our performance results indicate that although Fitts' throughput for able-bodied users is higher for area pointing than for goal crossing (4.72 vs. 3.61 bits/s), the opposite is true for users with motor impairments (2.34 vs. 2.88 bits/s). However, error rates are higher for goal crossing than for area pointing under a strict definition of crossing errors (6.23% vs. 1.94%). We also present path analyses and an examination of submovement velocity, acceleration, and jerk (the change in acceleration over time). These results show marked differences between crossing and pointing and almost categorically favor crossing. An important finding is that crossing reduces jerk for both participant groups, indicating more fluid, stable motion. To help realize the potential of goal crossing for computer access, we offer design concepts for crossing widgets that address the occlusion problem, which occurs when one crossing goal obscures another in persistent mouse-cursor interfaces. This work provides the motivation and initial steps for further exploration of goal crossing on the desktop, and may help researchers and designers to radically reshape user interfaces to provide accessible goal crossing, thereby lowering barriers to access.

With the ubiquity of mobile touchscreen devices like smartphones, two widely used text entry methods have emerged: small touch-based keyboards and speech recognition. Although speech recognition has been available on desktop computers for years, it has continued to improve at a rapid pace, and it is currently unknown how today's modern speech recognizers compare to state-of-the-art mobile touch keyboards, which also have improved considerably since their inception. To discover both methods' "upper-bound performance," we evaluated them in English and Mandarin Chinese on an Apple iPhone 6 Plus in a laboratory setting. Our experiment was carried out using Baidu's Deep Speech 2, a deep learning-based speech recognition system, and the built-in Qwerty (English) or Pinyin (Mandarin) Apple iOS keyboards. We found that with speech recognition, the English input rate was 2.93 times faster (153 vs. 52 WPM), and the Mandarin Chinese input rate was 2.87 times faster (123 vs. 43 WPM) than the keyboard for short message transcription under laboratory conditions for both methods. Furthermore, although speech made fewer errors during entry (5.30% vs. 11.22% corrected error rate), it left slightly more errors in the final transcribed text (1.30% vs. 0.79% uncorrected error rate). Our results show that comparatively, under ideal conditions for both methods, upper-bound speech recognition performance has greatly improved compared to prior systems, and might see greater uptake in the future, although further study is required to quantify performance in non-laboratory settings for both methods., Comment: 23 pages

Work in human-computer interaction has generally assumed either a single user or a group of users working together in a shared virtual space. Recent crowd-powered systems use a different model in which a dynamic group of individuals (the crowd) collectively form a single actor that responds to real-time performance tasks, e.g., controlling an on-screen character, driving a robot, or operating an existing desktop interface. In this paper, we introduce the idea of the crowd actor as a way to model coordination strategies and resulting collective performance, and discuss how the crowd actor is influenced not only by the domain on which it is asked to operate but also by the personality endowed to it by algorithms used to combine the inputs of constituent participants. Nowhere is the focus on the individual performer more finely resolved than in the study of the human psychomotor system, a mainstay topic in psychology that, largely owing to Fitts' law, also has a legacy in HCI. Therefore, we explored our notion of a crowd actor by modeling the crowd as a individual motor system performing pointing tasks. We combined the input of 200 participants in a controlled offline experiment to demonstrate the inherent trade-offs between speed and errors based on personality, the number of constituent individuals, and the mechanism used to distribute work across the group. Finally, 10 workers participated in a synchronous experiment to explore how the crowd actor responds in a real online setting. This work contributes to the beginning of a predictive science for the general crowd actor model.

Smartphones and tablets are often used in dynamic environments that force users to break focus and attend to their surroundings, creating a form of "situational impairment." Current mobile devices have no ability to sense when users divert or restore their attention, let alone provide support for resuming tasks. We therefore introduce SwitchBack, a system that allows mobile device users to resume tasks more efficiently. SwitchBack is built upon Focus and Saccade Tracking (FAST), which uses the front-facing camera to determine when the user is looking and how their eyes are moving across the screen. In a controlled study, we found that FAST can identify how many lines the user has read in a body of text within a mean absolute percent error of just 3.9%. We then tested SwitchBack in a dual focus-of-attention task, finding that SwitchBack improved average reading speed by 7.7% in the presence of distractions.

As we increasingly strive for scientific rigor and generalizability in HCI research, should we entertain any hope that by doing good science, our discoveries will eventually be more transferrable to industry? We present an in-depth case study of how an HCI research innovation goes through the process of transitioning from a university project to a revenue-generating startup financed by venture capital. The innovation is a novel contextual help system for the Web, and we reflect on the different methods used to evaluate it and how research insights endure attempted dissemination as a commercial product. Although the extent to which any innovation succeeds commercially depends on a number of factors like market forces, we found that our HCI innovation with user-centered origins was in a unique position to gain traction with customers and garner buy-in from investors. However, since end users were not the buyers of our product, a strong user-centered focus obfuscated other critical needs of the startup and pushed out perspectives of non-user-centered stakeholders. To make the research-to-product transition, we had to focus on adoption-centered design, the process of understanding and designing for adopters and stakeholders of the product. Our case study raises questions about how we evaluate the novelty and research contributions of HCI innovations with respect to their potential for commercial impact.

Recent improvements in text entry error rate measurement have enabled the running of text entry experiments in which subjects are free to correct errors (or not) as they transcribe a presented string. In these “unconstrained” experiments, it is no longer necessary to force subjects to unnaturally maintain synchronicity with presented text for the sake of performing overall error rate calculations. However, the calculation of character-level error rates, which can be trivial in artificially constrained evaluations, is far more complicated in unconstrained text entry evaluations because it is difficult to infer a subject's intention at every character. For this reason, prior character-level error analyses for unconstrained experiments have only compared presented and transcribed strings, not input streams. But input streams are rich sources of character-level error information, since they contain all of the text entered (and erased) by a subject. The current work presents an algorithm for the automated analysis of character-level errors in input streams for unconstrained text entry evaluations. It also presents new character-level metrics that can aid method designers in refining text entry methods. To exercise these metrics, we perform two analyses on data from an actual text entry experiment. One analysis, available from the prior work, uses only presented and transcribed strings. The other analysis uses input streams, as described in the current work. The results confirm that input stream error analysis yields richer information for the same empirical data. To facilitate the use of these new analyses, we offer pseudocode and downloadable software for performing unconstrained text entry experiments and analyzing data.

Power wheelchair joysticks have be used to control a mouse cursor on desktop computers, but they offer no integrated text entry solution, confining users to point-and-click or point-and-dwell with on-screen keyboards. On-screen keyboards reduce useful screen real-estate, exacerbating the need for frequent window management, and impose a secondary focus of attention. By contrast, we present two integrated gestural text entry methods designed for use from power wheelchairs: one for use with joysticks and the other for use with touchpads. Both techniques are adaptations of EdgeWrite, originally a stylus-based unistroke method designed for people with tremor. In a preliminary text entry study of 7 power wheelchair users, we found that EdgeWrite with a touchpad was faster than the on-screen keyboard WiViK with a joystick, and EdgeWrite with a joystick was only slightly slower. These results warranted a multi-session comparison of text entry with EdgeWrite and WiViK using joysticks and touchpads, in which we fo...

Wireless technologies such as Bluetooth and IEEE 802.11 (Wi-Fi), handheld devices are communicating more frequently with conventional computers in offices, meeting rooms, classrooms, and homes. The smart homes of the future will have ubiquitous embedded computation, and an increasing number of appliances can already communicate wirelessly. The authors launched the Pebbles project to determine whether a handheld device, such as a personal digital assistant (PDA) or cell phone, could serve as a simpler, more effective remote control. Office-centered applications include using the PDA instead of a laser pointer, using a PDA to remotely control a PowerPoint presentation, and using a PDA with the nondominant hand to scroll windows on a PC. In the home, we are exploring how to use the PDA as a customizable, intelligent personal universal controller (PUC) for appliances, creating high-quality control panels on the handheld using a high-level specification of the appliance's capabilities. We are also interested in how PDAs can assist in providing both appliance and computer access for the disabled, including the development of new text entry methods for the motor-impaired user.

The computing literature often draws a sharp distinction between input and output; computer scientists are used to regarding a screen as a passive output device and a mouse as a pure input device. However, nearly all examples of human-computer interaction require both input and output to do anything useful. For example, what good would a mouse be without the corresponding feedback embodied by the cursor on the screen, as well as the sound and feel of the buttons when they are clicked? The distinction between output devices and input devices becomes even more blurred in the real world. A sheet of paper can be used to both record ideas (input) and display them (output). Clay reacts to the sculptor’s fingers yet also provides feedback through the curvature and texture of its surface. Indeed, the complete and seamless integration of input and output is becoming a common research theme in advanced computer interfaces such as ubiquitous computing (Weiser, 1991) and tangible interaction (Ishii & Ullmer, 1997). Input and output bridge the chasm between a computer’s inner world of bits, and the real world perceptible to the human senses. Input to computers consists of sensed information about the physical environment. Familiar examples include the mouse, which senses movement across a surface, and the keyboard, which detects a contact closure when the user presses a key. However, any sensed information about physical properties of people, places, or things can serve as input to computer systems. Output from computers can comprise any emission or modification to the physical environment, such as a display (including the cathode ray tube (CRT), flat-panel displays, or even light emitting diodes), speakers, or tactile and force feedback devices (sometimes referred to as haptic displays). An interaction technique is the fusion of input and output, consisting of all hardware and software elements, that provides a way for the user to accomplish a low-level task. For example, in the traditional graphical user interface, users can scroll through a document by clicking or dragging the mouse (input) within a scroll bar displayed on the screen (output). Input and output bridge the chasm between a computer’s inner world of bits, and the real world perceptible to the human senses. Input to computers consists of sensed information about the physical environment. Familiar examples include the mouse, which senses movement across a surface, and the keyboard, which detects a contact closure when the user presses a key. However, any sensed information about physical properties of people, places, or things can serve as input to computer systems. Output from computers can comprise any emission or modification to the physical environment, such as a display (including the cathode ray tube (CRT), flat-panel displays, or even light emitting diodes), speakers, or tactile and force feedback devices (sometimes referred to as haptic displays). An interaction technique is the fusion of input and output, consisting of all hardware and software elements, that provides a way for the user to accomplish a low-level task. For example, in the traditional graphical user interface, users can scroll through a document by clicking or dragging the mouse (input) within a scroll bar displayed on the screen (output).

We present the Bubble Lens, a new target acquisition technique that remedies the limitations of the Bubble Cursor to increase the speed and accuracy of acquiring small, dense targets--precisely those targets for which the Bubble Cursor degenerates to a point cursor. When targets are large and sparse, the Bubble Lens behaves like the Bubble Cursor. But when targets are small and dense, the Bubble Lens automatically magnifies nearby targets, making them larger in both visual- and motor-space. Importantly, magnification is not governed by an explicit user-invoked mode-switch. Rather, magnification is activated through kinematic triggering, a technique that continuously examines an unfolding velocity profile to automatically trigger mode changes based on observed features. In a first study, we found the Bubble Cursor performed poorly when targets had an effective size smaller than 10 pixels. Using this threshold for the Bubble Lens in a second study, we found that the Bubble Lens significantly outperformed the Bubble Cursor, decreasing movement time by 10.2% and error rates by 37.9%, making the Bubble Lens the fastest current pointing technique.

We present a new method of predicting the endpoints of mouse movements. While prior approaches to endpoint prediction have relied upon normative kinematic laws, regression, or control theory, our approach is straightforward but kinematically rich. Our key insight is to regard the unfolding velocity profile of a pointing movement as a 2-D stroke gesture and to use template matching to predict the endpoint based on prior observed movements. We call our technique kinematic template matching (KTM), which is simple to implement, user-adaptable, and kinematically expressive. In a study of 17 able-bodied participants evaluated over movement amplitudes ranging from 100-800 pixels, we found KTM to predict endpoints that were within 83 pixels of the true endpoint at 50% of the way through the movement, within 48 pixels at 75%, and within 39 pixels at 90%, using 1000 templates per participant. These accuracies make KTM as successful an approach to endpoint prediction as any prior technique, while being easier to implement and understand than most.

Pointing to targets in graphical user interfaces remains a frequent and fundamental necessity in modern computing systems. Yet for millions of people with motor impairments, children, and older users, pointing—whether with a mouse cursor, a stylus, or a finger on a touch screen—remains a major access barrier because of the fine-motor skills required. In a series of projects inspired by and contributing to ability-based design, we have reconsidered the nature and assumptions behind pointing, resulting in changes to how mouse cursors work, the types of targets used, the way interfaces are designed and laid out, and even how input devices are used. The results from these explorations show that people with motor difficulties can acquire targets in graphical user interfaces when interfaces are designed to better match the abilities of their users. Ability-based design, as both a design philosophy and a design approach, provides a route to realizing a future in which people can utilize whatever abilities they have to express themselves not only to machines, but to the world.

Adding tactile feedback to touch screens can improve their accessibility to blind users, but prior approaches to integrating tactile feedback with touch screens have either offered limited functionality or required extensive (and typically expensive) customization of the hardware. We introduce touchplates, carefully designed tactile guides that provide tactile feedback for touch screens in the form of physical guides that are overlaid on the screen and recognized by the underlying application. Unlike prior approaches to integrating tactile feedback with touch screens, touchplates are implemented with simple plastics and use standard touch screen software, making them versatile and inexpensive. Touchplates may be customized to suit individual users and applications, and may be produced on a laser cutter, 3D printer, or made by hand. We describe the design and implementation of touchplates, a "starter kit" of touchplates, and feedback from a formative evaluation with 9 people with visual impairments. Touchplates provide a low-cost, adaptable, and accessible method of adding tactile feedback to touch screen interfaces.

Despite the apparent popularity of touchscreens for older adults, little is known about the psychomotor performance of these devices. We compared performance between older adults and younger adults on four desktop and touchscreen tasks: pointing, dragging, crossing and steering. On the touchscreen, we also examined pinch-to-zoom. Our results show that while older adults were significantly slower than younger adults in general, the touchscreen reduced this performance gap relative to the desktop and mouse. Indeed, the touchscreen resulted in a significant movement time reduction of 35% over the mouse for older adults, compared to only 16% for younger adults. Error rates also decreased.

We present a multi-site field study to evaluate LemonAid, a crowdsourced contextual help approach that allows users to retrieve relevant questions and answers by making selections within the interface. We deployed LemonAid on 4 different web sites used by thousands of users and collected data over several weeks, gathering over 1,200 usage logs, 168 exit surveys, and 36 one-on-one interviews. Our results indicate that over 70% of users found LemonAid to be helpful, intuitive, and desirable for reuse. Software teams found LemonAid easy to integrate with their sites and found the analytics data aggregated by LemonAid a novel way of learning about users' popular questions. Our work provides the first holistic picture of the adoption and use of a crowdsourced contextual help system and offers several insights into the social and organizational dimensions of implementing such help systems for real-world applications.

Rapid prototyping of gesture interaction for emerging touch platforms requires that developers have access to fast, simple, and accurate gesture recognition approaches. The $-family of recognizers ($1, $N) addresses this need, but the current most advanced of these, $N-Protractor, has significant memory and execution costs due to its combinatoric gesture representation approach. We present $P, a new member of the $-family, that remedies this limitation by considering gestures as clouds of points. $P performs similarly to $1 on unistrokes and is superior to $N on multistrokes. Specifically, $P delivers >99% accuracy in user-dependent testing with 5+ training samples per gesture type and stays above 99% for user-independent tests when using data from 10 participants. We provide a pseudocode listing of $P to assist developers in porting it to their specific platform and a "cheat sheet" to aid developers in selecting the best member of the $-family for their specific application needs.

We introduce GripSense, a system that leverages mobile device touchscreens and their built-in inertial sensors and vibration motor to infer hand postures including one- or two-handed interaction, use of thumb or index finger, or use on a table. GripSense also senses the amount of pres-sure a user exerts on the touchscreen despite a lack of direct pressure sensors by inferring from gyroscope readings when the vibration motor is "pulsed." In a controlled study with 10 participants, GripSense accurately differentiated device usage on a table vs. in hand with 99.67% accuracy and when in hand, it inferred hand postures with 84.26% accuracy. In addition, GripSense distinguished three levels of pressure with 95.1% accuracy. A usability analysis of GripSense was conducted in three custom applications and showed that pressure input and hand-posture sensing can be useful in a number of scenarios.

The lack of tactile feedback on touch screens makes typing difficult, a challenge exacerbated when situational impairments like walking vibration and divided attention arise in mobile settings. We introduce WalkType, an adaptive text entry system that leverages the mobile device's built-in tri-axis accelerometer to compensate for extraneous movement while walking. WalkType's classification model uses the displacement and acceleration of the device, and inference about the user's footsteps. Additionally, WalkType models finger-touch location and finger distance traveled on the screen, features that increase overall accuracy regardless of movement. The final model was built on typing data collected from 16 participants. In a study comparing WalkType to a control condition, WalkType reduced uncorrected errors by 45.2% and increased typing speed by 12.9% for walking participants.

Web-based technical support such as discussion forums and social networking sites have been successful at ensuring that most technical support questions eventually receive helpful answers. Unfortunately, finding these answers is still quite difficult, since users' textual queries are often incomplete, imprecise, or use different vocabularies to describe the same problem. We present LemonAid, a new approach to help that allows users to find help by instead selecting a label, widget, link, image or other user interface (UI) element that they believe is relevant to their problem. LemonAid uses this selection to retrieve previously asked questions and their corresponding answers. The key insight that makes LemonAid work is that users tend to make similar selections in the interface for similar help needs and different selections for different help needs. Our initial evaluation shows that across a corpus of dozens of tasks and thousands of requests, LemonAid retrieved a result for 90% of help requests based on UI selections and, of those, over half had relevant matches in the top 2 results.

The HCI research community grows bigger each year, refining and expanding its boundaries in new ways. The ability to effectively review submissions is critical to the growth of CHI and related conferences. The review process is designed to produce a consistent supply of fair, high-quality reviews without overloading individual reviewers; yet, after each cycle, concerns are raised about limitations of the process. Every year, participants are left wondering why their papers were not accepted (or why they were). This SIG will explore reviewing through a critical and constructive lens, discussing current successes and future opportunities in the CHI review process. Goals will include actionable conclusions about ways to improve the system, potential alternative peer models, and the creation of materials to educate newcomer reviewers.

We explore using vibration on a smartphone to provide turn-by-turn walking instructions to people with visual impairments. We present two novel feedback methods called Wand and ScreenEdge and compare them to a third method called Pattern. We built a prototype and conducted a user study where 8 participants walked along a pre-programmed route using the 3 vibration feedback methods and no audio output. Participants interpreted the feedback with an average error rate of just 4 percent. Most preferred the Pattern method, where patterns of vibrations indicate different directions, or the ScreenEdge method, where areas of the screen correspond to directions and touching them may induce vibration.

Many touch screens remain inaccessible to blind users, and those approaches to providing access that do exist offer minimal support for interacting with large touch screens or spatial data. In this paper, we introduce a set of three software-based access overlays intended to improve the accessibility of large touch screen interfaces, specifically interactive tabletops. Our access overlays are called edge projection, neighborhood browsing, and touch-and-speak. In a user study, 14 blind users compared access overlays to an implementation of Apple's VoiceOver screen reader. Our results show that two of our techniques were faster than VoiceOver, that participants correctly answered more questions about the screen's layout using our techniques, and that participants overwhelmingly preferred our techniques. We developed several applications demonstrating the use of access overlays, including an accessible map kiosk and an accessible board game.

Blind and deaf-blind people often rely on public transit for everyday mobility, but using transit can be challenging for them. We conducted semi-structured interviews with 13 blind and deaf-blind people to understand how they use public transit and what human values were important to them in this domain. Two key values were identified: independence and safety. We developed GoBraille, two related Braille-based applications that provide information about buses and bus stops while supporting the key values. GoBraille is built on MoBraille, a novel framework that enables a Braille display to benefit from many features in a smartphone without knowledge of proprietary, device-specific protocols. Finally, we conducted user studies with blind people to demonstrate that GoBraille enables people to travel more independently and safely. We also conducted co-design with a deaf-blind person, finding that a minimalist interface, with short input and output messages, was most effective for this population.

Nonparametric data from multi-factor experiments arise often in human-computer interaction (HCI). Examples may include error counts, Likert responses, and preference tallies. But because multiple factors are involved, common nonparametric tests (e.g., Friedman) are inadequate, as they are unable to examine interaction effects. While some statistical techniques exist to handle such data, these techniques are not widely available and are complex. To address these concerns, we present the Aligned Rank Transform (ART) for nonparametric factorial data analysis in HCI. The ART relies on a preprocessing step that "aligns" data before applying averaged ranks, after which point common ANOVA procedures can be used, making the ART accessible to anyone familiar with the F-test. Unlike most articles on the ART, which only address two factors, we generalize the ART to N factors. We also provide ARTool and ARTweb, desktop and Web-based programs for aligning and ranking data. Our re-examination of some published HCI results exhibits advantages of the ART.

Despite the growing research on usability in the pre-development phase, we know little about post-deployment usability activities. To characterize these activities, we surveyed 333 full-time usability professionals and consultants working in large and small corporations from a wide range of industries. Our results show that, as a whole, usability professionals are currently not playing a substantial role in the post-deployment phase compared to other phases of user-centered design, but when they do, practitioners find their interactions quite valuable. We highlight opportunities in HCI research and practice to bridge this gap by working more closely with software support and maintenance teams. We also raise the need to understand what might be called 'usability maintenance,' that is, the process and procedures, by which usability is maintained after deployment.

In this paper we describe the Input Observer, a background application that will be capable of measuring a user's text entry and pointing abilities from everyday computer use "in the wild." The application runs quietly in the background of the user's computer and utilizes global Windows Hooks to observe the text entry input stream and use of the mouse, and will yield data equivalent to results from lab-based measures of text entry and target acquisition. A major challenge is the lack of a task model from which researchers can know the intent of the user at every moment. We describe our approach to handling this issue for both text entry and mouse pointing.

MobileASL is a video compression project for two-way, real-time video communication on cell phones, allowing Deaf people to communicate in the language most accessible to them, American Sign Language. Unfortunately, running MobileASL quickly depletes a full battery charge in a few hours. Previous work on MobileASL investigated a method called variable frame rate (VFR) to increase the battery duration. We expand on this previous work by creating two new power saving algorithms, variable spatial resolution (VSR), and the application of both VFR and VSR. These algorithms extend the battery life by altering the temporal and/or spatial resolutions of video transmitted on MobileASL. We found that implementing only VFR extended the battery life from 284 minutes to 307 minutes; implementing only VSR extended the battery life to 306 minutes, and implementing both VFR and VSR extended the battery life to 315 minutes. We evaluated all three algorithms by creating a linguistically accessible online survey to investigate Deaf people's perceptions of video quality when these algorithms were applied. In our survey results, we found that VFR produces perceived video choppiness and VSR produces perceived video blurriness; however, a surprising finding was that when both VFR and VSR are used together, they largely ameliorate the choppiness and blurriness perceived, i.e., they each improve the use of the other. This is a useful finding because using VFR and VSR together saves the most battery life.