Review of Quantitative Empirical Evaluations of Technology for People with Visual Impairments

This paper has been selected for a Honourable Mention award at CHI2020.The SIGCHI “Best of CHI” awards honour exceptional submissions to SIGCHI sponsored conferences.A total of 31 Papers received a Best Paper award (top 1% of submissions), 125 papers received Honourable Mention (remaining top 5% of submissions). Process: the Best Paper Committee, which includes the Awards Chairs and 19 committee members (one from each subcommittee), adjudicated the selection of the Best Papers and Honourable Mentions based on nomination scores and paragraphs that were provided by the CHI Associate Chairs.; International audience; Addressing the needs of visually impaired people is of continued interest in Human Computer Interaction (HCI) research. Yet, one of the major challenges facing researchers in this field continues to be how to design adequate quantitative empirical evaluation for these users in HCI. In this paper, we analyse a corpus of 178 papers on technologies designed for people with visual impairments, published since 1988, and including at least one quantitative empirical evaluation (243 evaluations in total). To inform future research in this area, we provide an overview, historic trends and a unified terminology to design and report quantitative empirical evaluations. We identify open issues and propose a set of guidelines to address them. Our analysis aims to facilitate and stimulate future research on this topic.