Sensorial substitution system with encoding of visual objects into sounds

Visual and auditory prostheses involve surgeries that are complex, expensive, and invasive. They are limited to a small number of electrodes and can only be used when the impairment is peripheral. The Vibe and PSVA encode the entire image in one complex sound. The PSVA uses frequencies that are associated with each pixel and increase from left to right and from bottom to top of the image. The Vibe splits the image into several regions that are equivalent to receptive fields. The challenge in this project resides in the design of a suitable encoding of the visual scene into auditory stimuli such that the content of the sound carries the most important characteristics of the visual scene. These sounds should be shaped in a way that the subject can build mental representations of visual scenes even if the information carrier is the auditory pathway. The complex sound is the sum of all single sounds from each segment. One complex sound is generated for the right ear and another one for the left.