Search

Goal: Distance information is highly requested in assistive smartphone Apps by people who are blind or low vision (PBLV). However, current techniques have not been evaluated systematically for accuracy and usability. Methods: We tested five smartphone-based distance-estimation approaches in the image center and periphery at 1–3 meters, including machine learning (CoreML), infrared grid distortion (IR_self), light detection and ranging (LiDAR_back), and augmented reality room-tracking on the front (ARKit_self) and back-facing cameras (ARKit_back). Results: For accuracy in the image center, all approaches had Conclusions: We provide key information that helps design reliable smartphone-based visual assistive technologies to enhance the functionality of PBLV.

People with blindness and low vision (pBLV) encounter substantial challenges when it comes to comprehensive scene recognition and precise object identification in unfamiliar environments. Additionally, due to the vision loss, pBLV have difficulty in accessing and identifying potential tripping hazards independently. Previous assistive technologies for the visually impaired often struggle in real-world scenarios due to the need for constant training and lack of robustness, which limits their effectiveness, especially in dynamic and unfamiliar environments, where accurate and efficient perception is crucial. Therefore, we frame our research question in this paper as: How can we assist pBLV in recognizing scenes, identifying objects, and detecting potential tripping hazards in unfamiliar environments, where existing assistive technologies often falter due to their lack of robustness? We hypothesize that by leveraging large pretrained foundation models and prompt engineering, we can create a system that effectively addresses the challenges faced by pBLV in unfamiliar environments. Motivated by the prevalence of large pretrained foundation models, particularly in assistive robotics applications, due to their accurate perception and robust contextual understanding in real-world scenarios induced by extensive pretraining, we present a pioneering approach that leverages foundation models to enhance visual perception for pBLV, offering detailed and comprehensive descriptions of the surrounding environment and providing warnings about potential risks. Specifically, our method begins by leveraging a large-image tagging model (i.e., Recognize Anything Model (RAM)) to identify all common objects present in the captured images. The recognition results and user query are then integrated into a prompt, tailored specifically for pBLV, using prompt engineering. By combining the prompt and input image, a vision-language foundation model (i.e., InstructBLIP) generates detailed and comprehensive descriptions of the environment and identifies potential risks in the environment by analyzing environmental objects and scenic landmarks, relevant to the prompt. We evaluate our approach through experiments conducted on both indoor and outdoor datasets. Our results demonstrate that our method can recognize objects accurately and provide insightful descriptions and analysis of the environment for pBLV.

Abstract Background Blind/low vision (BLV) severely limits information about our three-dimensional world, leading to poor spatial cognition and impaired navigation. BLV engenders mobility losses, debility, illness, and premature mortality. These mobility losses have been associated with unemployment and severe compromises in quality of life. VI not only eviscerates mobility and safety but also, creates barriers to inclusive higher education. Although true in almost every high-income country, these startling facts are even more severe in low- and middle-income countries, such as Thailand. We aim to use VIS4ION (Visually Impaired Smart Service System for Spatial Intelligence and Onboard Navigation), an advanced wearable technology, to enable real-time access to microservices, providing a potential solution to close this gap and deliver consistent and reliable access to critical spatial information needed for mobility and orientation during navigation. Methods We are leveraging 3D reconstruction and semantic segmentation techniques to create a digital twin of the campus that houses Mahidol University’s disability college. We will do cross-over randomization, and two groups of randomized VI students will deploy this augmented platform in two phases: a passive phase, during which the wearable will only record location, and an active phase, in which end users receive orientation cueing during location recording. A group will perform the active phase first, then the passive, and the other group will experiment reciprocally. We will assess for acceptability, appropriateness, and feasibility, focusing on experiences with VIS4ION. In addition, we will test another cohort of students for navigational, health, and well-being improvements, comparing weeks 1 to 4. We will also conduct a process evaluation according to the Saunders Framework. Finally, we will extend our computer vision and digital twinning technique to a 12-block spatial grid in Bangkok, providing aid in a more complex environment. Discussion Although electronic navigation aids seem like an attractive solution, there are several barriers to their use; chief among them is their dependence on either environmental (sensor-based) infrastructure or WiFi/cell “connectivity” infrastructure or both. These barriers limit their widespread adoption, particularly in low-and-middle-income countries. Here we propose a navigation solution that operates independently of both environmental and Wi-Fi/cell infrastructure. We predict the proposed platform supports spatial cognition in BLV populations, augmenting personal freedom and agency, and promoting health and well-being. Trial registration ClinicalTrials.gov under the identifier: NCT03174314, Registered 2017.06.02.

Objective: People with blindness and low vision face substantial challenges when navigating both indoor and outdoor environments. While various solutions are available to facilitate travel to and from public transit hubs, there is a notable absence of solutions for navigating within transit hubs, often referred to as the “middle mile”. Although research pilots have explored the middle mile journey, no solutions exist at scale, leaving a critical gap for commuters with disabilities. In this paper, we proposed a novel mobile application, Commute Booster, that offers full trip planning and real-time guidance inside the station. Methods and procedures: Our system consists of two key components: the general transit feed specification (GTFS) and optical character recognition (OCR). The GTFS dataset generates a comprehensive list of wayfinding signage within subway stations that users will encounter during their intended journey. The OCR functionality enables users to identify relevant navigation signs in their immediate surroundings. By seamlessly integrating these two components, Commute Booster provides real-time feedback to users regarding the presence or absence of relevant navigation signs within the field of view of their phone camera during their journey. Results: As part of our technical validation process, we conducted tests at three subway stations in New York City. The sign detection achieved an impressive overall accuracy rate of 0.97. Additionally, the system exhibited a maximum detection range of 11 meters and supported an oblique angle of approximately 110 degrees for field of view detection. Conclusion: The Commute Booster mobile application relies on computer vision technology and does not require additional sensors or infrastructure. It holds tremendous promise in assisting individuals with blindness and low vision during their daily commutes. Clinical and Translational Impact Statement: Commute Booster translates the combination of OCR and GTFS into an assistive tool, which holds great promise for assisting people with blindness and low vision in their daily commute.

Visual impairment represents a significant health and economic burden affecting 596 million globally. The incidence of visual impairment is expected to double by 2050 as our population ages. Independent navigation is challenging for persons with visual impairment, as they often rely on non-visual sensory signals to find the optimal route. In this context, electronic travel aids are promising solutions that can be used for obstacle detection and/or route guidance. However, electronic travel aids have limitations such as low uptake and limited training that restrict their widespread use. Here, we present a virtual reality platform for testing, refining, and training with electronic travel aids. We demonstrate the viability on an electronic travel aid developed in-house, consist of a wearable haptic feedback device. We designed an experiment in which participants donned the electronic travel aid and performed a virtual task while experiencing a simulation of three different visual impairments: age-related macular degeneration, diabetic retinopathy, and glaucoma. Our experiments indicate that our electronic travel aid significantly improves the completion time for all the three visual impairments and reduces the number of collisions for diabetic retinopathy and glaucoma. Overall, the combination of virtual reality and electronic travel aid may have a beneficial role on mobility rehabilitation of persons with visual impairment, by allowing early-phase testing of electronic travel aid prototypes in safe, realistic, and controllable settings.

IntroductionParkinson’s disease (PD) is the second most prevalent neurodegenerative disease. Complementary and alternative therapies are increasingly utilized to address its complex multisystem symptomatology. Art therapy involves motoric action and visuospatial processing while promoting broad biopsychosocial wellness. The process involves hedonic absorption, which provides an escape from otherwise persistent and cumulative PD symptoms, refreshing internal resources. It involves the expression in nonverbal form of multilayered psychological and somatic phenomena; once these are externalized in a symbolic arts medium, they can be explored, understood, integrated, and reorganized through verbal dialogue, effecting relief and positive change.Methods42 participants with mild to moderate PD were treated with 20 sessions of group art therapy. They were assessed before and after therapy with a novel arts-based instrument developed to match the treatment modality for maximum sensitivity. The House-Tree-Person PD Scale (HTP-PDS) assesses motoric and visuospatial processing–core PD symptoms–as well as cognition (thought and logic), affect/mood, motivation, self (including body-image, self-image, and self- efficacy), interpersonal functioning, creativity, and overall level of functioning. It was hypothesized that art therapy will ameliorate core PD symptoms and that this will correlate with improvements in all other variables.ResultsHTP-PDS scores across all symptoms and variables improved significantly, though causality among variables was indeterminate.DiscussionArt therapy is a clinically efficacious complementary treatment for PD. Further research is warranted to disentangle causal pathways among the aforementioned variables, and additionally, to isolate and examine the multiple, discrete healing mechanisms believed to operate simultaneously in art therapy.

Smart health applications have received significant attention in recent years. Novel applications hold significant promise to overcome many of the inconveniences faced by persons with disabilities throughout daily living. For people with blindness and low vision (BLV), environmental perception is compromised, creating myriad difficulties. Precise localization is still a gap in the field and is critical to safe navigation. Conventional GNSS positioning cannot provide satisfactory performance in urban canyons. 3D mapping-aided (3DMA) GNSS may serve as an urban GNSS solution, since the availability of 3D city models has widely increased. As a result, this study developed a real-time 3DMA GNSS-positioning system based on state-of-the-art 3DMA GNSS algorithms. Shadow matching was integrated with likelihood-based ranging 3DMA GNSS, generating positioning hypothesis candidates. To increase robustness, the 3DMA GNSS solution was then optimized with Doppler measurements using factor graph optimization (FGO) in a loosely-coupled fashion. This study also evaluated positioning performance using an advanced wearable system’s recorded data in New York City. The real-time forward-processed FGO can provide a root-mean-square error (RMSE) of about 21 m. The RMSE drops to 16 m when the data is post-processed with FGO in a combined direction. Overall results show that the proposed loosely-coupled 3DMA FGO algorithm can provide a better and more robust positioning performance for the multi-sensor integration approach used by this wearable for persons with BLV.

Number and picture rapid automatized naming (RAN) tests are useful sideline diagnostic tools. The main outcome measure of these RAN tests is the completion time, which is prolonged with a concussion, yet yields no information about eye movement behavior. We investigated eye movements during a digitized Mobile Universal Lexicon Evaluation System (MULES) test of rapid picture naming. A total of 23 participants with a history of concussion and 50 control participants performed MULES testing with simultaneous eye tracking. The test times were longer in participants with a concussion (32.4 s [95% CI 30.4, 35.8] vs. 26.9 s [95% CI 25.9, 28.0], t=6.1). The participants with a concussion made more saccades per picture than the controls (3.6 [95% CI 3.3, 4.1] vs. 2.7 [95% CI 2.5, 3.0]), and this increase was correlated with longer MULES times (r = 0.46, p = 0.026). The inter-saccadic intervals (ISI) did not differ between the groups, nor did they correlate with the test times. Following a concussion, eye movement behavior differs during number versus picture RAN performance. Prior studies have shown that ISI prolongation is the key finding for a number-based RAN test, whereas this study shows a primary finding of an increased saccade number per picture with a picture-based RAN test. Number-based and picture-based RAN tests may be complimentary in concussion detection, as they may detect different injury effects or compensatory strategies.

(1) Background: The King-Devick (KD) rapid number naming test is sensitive for concussion diagnosis, with increased test time from baseline as the outcome measure. Eye tracking during KD performance in concussed individuals shows an association between inter-saccadic interval (ISI) (the time between saccades) prolongation and prolonged testing time. This pilot study retrospectively assesses the relation between ISI prolongation during KD testing and cognitive performance in persistently-symptomatic individuals post-concussion. (2) Results: Fourteen participants (median age 34 years; 6 women) with prior neuropsychological assessment and KD testing with eye tracking were included. KD test times (72.6 ± 20.7 s) and median ISI (379.1 ± 199.1 msec) were prolonged compared to published normative values. Greater ISI prolongation was associated with lower scores for processing speed (WAIS-IV Coding, r = 0.72, p = 0.0017), attention/working memory (Trails Making A, r = −0.65, p = 0.006) (Digit Span Forward, r = 0.57, p = −0.017) (Digit Span Backward, r= −0.55, p = 0.021) (Digit Span Total, r = −0.74, p = 0.001), and executive function (Stroop Color Word Interference, r = −0.8, p = 0.0003). (3) Conclusions: This pilot study provides preliminary evidence suggesting that cognitive dysfunction may be associated with prolonged ISI and KD test times in concussion.

Failure of brainstem supranuclear centers for saccadic eye movements results in the clinical presence of a brainstem-mediated supranuclear saccadic gaze palsy (SGP), which is manifested as slowing of saccades with or without range of motion limitation of eye movements and as loss of quick phases of optokinetic nystagmus. Limitation in the range of motion of eye movements is typically worse with saccades than with smooth pursuit and is overcome with vestibular–ocular reflexive eye movements. The differential diagnosis of SGPs is broad, although acute-onset SGP is most often from brainstem infarction and chronic vertical SGP is most commonly caused by the neurodegenerative condition progressive supranuclear palsy. In this review, we discuss the brainstem anatomy and physiology of the brainstem saccade-generating network; we discuss the clinical features of SGPs, with an emphasis on insights from quantitative ocular motor recordings; and we consider the broad differential diagnosis of SGPs.

It is widely accepted that cerebral pathology can impair ocular motor and manual motor control. This is true in indolent and chronic processes, such as neurodegeneration and in acute processes such as stroke or those secondary to neurotrauma. More recently, it has been suggested that disruptions in these control systems are useful markers for prognostication and longitudinal monitoring. The utility of examining the relationship or the coupling between these systems has yet to be determined. We measured eye and hand-movement control in chronic, middle cerebral artery stroke, relative to healthy controls, in saccade-to-reach paradigms to assess eye–hand coordination. Primary saccades were initiated significantly earlier by stroke participants relative to control participants. However, despite these extremely early initial saccades to the target, reaches were nevertheless initiated at approximately the same time as those of control participants. Control participants minimized the time period between primary saccade onset and reach initiation, demonstrating temporal coupling between eye and hand. In about 90% of all trials, control participants produced no secondary, or corrective, saccades, instead maintaining fixation in the terminal position of the primary saccade until the end of the reach. In contrast, participants with stroke increased the time period between primary saccade onset and reach initiation. During this temporal decoupling, multiple saccades were produced in about 50% of the trials with stroke participants making between one and five additional saccades. Reaches made by participants with stroke were both longer in duration and less accurate. In addition to these increases in spatial reach errors, there were significant increases in saccade endpoint errors. Overall, the magnitude of the endpoint errors for reaches and saccades were correlated across participants. These findings suggest that in individuals with otherwise intact visual function, the spatial and temporal relationships between the eye and hand are disrupted poststroke, and may need to be specifically targeted during neurorehabilitation. Eye–hand coupling may be a useful biomarker in individuals with cerebral pathology in the setting of neurovascular, neurotraumatic, and neurodegenerative pathology.

Acute and chronic disease processes that lead to cerebral injury can often be clinically challenging diagnostically, prognostically, and therapeutically. Neurodegenerative processes are one such elusive diagnostic group, given their often diffuse and indolent nature, creating difficulties in pinpointing specific structural abnormalities that relate to functional limitations. A number of studies in recent years have focused on eye–hand coordination (EHC) in the setting of acquired brain injury (ABI), highlighting the important set of interconnected functions of the eye and hand and their relevance in neurological conditions. These experiments, which have concentrated on focal lesion-based models, have significantly improved our understanding of neurophysiology and underscored the sensitivity of biomarkers in acute and chronic neurological disease processes, especially when such biomarkers are combined synergistically. To better understand EHC and its connection with ABI, there is a need to clarify its definition and to delineate its neuroanatomical and computational underpinnings. Successful EHC relies on the complex feedback- and prediction-mediated relationship between the visual, ocular motor, and manual motor systems and takes advantage of finely orchestrated synergies between these systems in both the spatial and temporal domains. Interactions of this type are representative of functional sensorimotor control, and their disruption constitutes one of the most frequent deficits secondary to brain injury. The present review describes the visually mediated planning and control of eye movements, hand movements, and their coordination, with a particular focus on deficits that occur following neurovascular, neurotraumatic, and neurodegenerative conditions. Following this review, we also discuss potential future research directions, highlighting objective EHC as a sensitive biomarker complement within acute and chronic neurological disease processes.