1. Generating Contextually-Relevant Navigation Instructions for Blind and Low Vision People
- Author
-
Merchant, Zain, Anwar, Abrar, Wang, Emily, Chattopadhyay, Souti, and Thomason, Jesse
- Subjects
Computer Science - Computation and Language ,Computer Science - Human-Computer Interaction - Abstract
Navigating unfamiliar environments presents significant challenges for blind and low-vision (BLV) individuals. In this work, we construct a dataset of images and goals across different scenarios such as searching through kitchens or navigating outdoors. We then investigate how grounded instruction generation methods can provide contextually-relevant navigational guidance to users in these instances. Through a sighted user study, we demonstrate that large pretrained language models can produce correct and useful instructions perceived as beneficial for BLV users. We also conduct a survey and interview with 4 BLV users and observe useful insights on preferences for different instructions based on the scenario., Comment: Accepted as RO-MAN 2024 Late Breaking Report
- Published
- 2024