Search

Objective: Blindness deprives a person of a significant part of sensory information resulting in limited perceptual abilities and decreased quality of life. Although some aspects of visual information, like the shape of an object, can be conveyed by other senses, there is no easy way to perceive color without using sight. To address this issue, we developed the Colorophone sensory substitution device. Method: In a way analogous to pixels in visual displays, we introduced auxels (auditory pixels) that can be used as basic building blocks of a generic auditory display. The developed auxel-based system realizes real-time, spatial, color-to-sound conversion. We created a dedicated software suite that enables the independent introduction of various system features to ensure effective training. Four blind participants assessed the prototype's usability. The evaluation methods included: auditory color recognition, object identification, and virtual sound source localization tasks, as well as two self-descriptive methods: the System Usability Scale and the NASA Task Load Index. Results: The developed wearable system generates spatially calibrated colorful soundscapes. It enables auditory color recognition and object identification significantly above chance. However, analyzing complex natural scenes remains challenging. Users judged the system's usability from good to best imaginable. The identified usability issues are discussed together with the proposed solutions. Conclusion: The Colorophone device shows promise for the future development of a useful visual rehabilitation device; however, further work is needed to eliminate existing usability issues. Significance: The presented work contributes to developing a universal, affordable and user-friendly visual rehabilitation device.

The successful development of a system realizing color sonification would enable auditory representation of the visual environment. The primary beneficiary of such a system would be people that cannot directly access visual information—the visually impaired community. Despite the plethora of sensory substitution devices, developing systems that provide intuitive color sonification remains a challenge. This paper presents design considerations, development, and the usability audit of a sensory substitution device that converts spatial color information into soundscapes. The implemented wearable system uses a dedicated color space and continuously generates natural, spatialized sounds based on the information acquired from a camera. We developed two head-mounted prototype devices and two graphical user interface (GUI) versions. The first GUI is dedicated to researchers, and the second has been designed to be easily accessible for visually impaired persons. Finally, we ran fundamental usability tests to evaluate the new spatial color sonification algorithm and to compare the two prototypes. Furthermore, we propose recommendations for the development of the next iteration of the system.

The majority of interoceptive tasks (i.e. measuring the sensitivity to bodily signals) are based upon the heartbeat sensations perception. However, temporal perception of heartbeats varies among individuals and confidence and spatial perception of heartbeats in relation to temporal judgements have not yet been systematically investigated. This study examined the perception of heartbeat sensations in both time and spatial domains, using a multi-interval heartbeat discrimination task. The relationship between these domains was investigated, as well as the contribution of mental health conditions and cardiovascular parameters. Heartbeat sensations occurred on average ~250ms after the ECG R-wave and were more frequently sampled from the left part of the chest. Participants’ confidence in their experience of heartbeat sensations was maximal for the 0 ms interval. Interestingly, higher confidence was related to reduced dispersion of sampling locations but we found evidence toward the absence of relationship between temporal and spatial heartbeat sensations perception, using Bayesian statistics. Finally, we found evidence toward a relationship between spatial precision of heartbeat sensations and state anxiety score, which seems independent from the cardiovascular parameters. This systematic investigation of heartbeat sensations perception provides important fresh insights, informing the mechanistic understanding of the interoceptive signaling contribution to emotion, cognition and behaviour.

The rubber hand illusion (RHI) has been shown to alter the experience of pain, although studies have yielded inconsistent results. In this experiment we tested the influence of the RHI on the intensity of pain caused by electric stimuli. Electric stimuli were delivered to participants' experimental and control hands before RHI induction (control condition) and afterward (experimental condition), in a procedure that was double-blind with respect to location and strength of noxious stimulation. All hands were covered during the stimulation to avoid the analgesic effect of seeing one's own body part. The perceived location of the hand and of pain were measured after each trial in the experimental condition. The results showed that noxious stimuli were experienced as more painful on the hand under the illusion. In addition, in the experimental condition the perceived location of noxious stimulation applied to the experimental hand drifted toward the rubber hand. Our data suggest that the link between bodily illusions and pain could be modulated by uncertainty about location of pain and the affected body part. Future studies should aim to determine which aspects of altered body awareness lead to pain sensitization. Perspective We show that the RHI can change the perceived location of pain and increase pain ratings caused by electric stimuli. Our data suggest that the link between bodily illusions and pain could be modulated by uncertainty about location of pain and the affected body part.

The majority of scientific studies on consciousness have focused on vision, exploring the cognitive and neural mechanisms of conscious access to visual stimuli. In parallel, studies on bodily consciousness have revealed that bodily (i.e. tactile, proprioceptive, visceral, vestibular) signals are the basis for the sense of self. However, the role of bodily signals in the formation of visual consciousness is not well understood. Here we investigated how body-related visuo-tactile stimulation modulates conscious access to visual stimuli. We used a robotic platform to apply controlled tactile stimulation to the participants' backs while they viewed a dot moving either in synchrony or asynchrony with the touch on their back. Critically, the dot was rendered invisible through continuous flash suppression. Manipulating the visual context by presenting the dot moving on either a body form, or a non-bodily object we show that: (i) conflict induced by synchronous visuo-tactile stimulation in a body context is associated with a delayed conscious access compared to asynchronous visuo-tactile stimulation, (ii) this effect occurs only in the context of a visual body form, and (iii) is not due to detection or response biases. The results indicate that body-related visuo-tactile conflicts impact visual consciousness by facilitating access of non-conflicting visual information to awareness, and that these are sensitive to the visual context in which they are presented, highlighting the interplay between bodily signals and visual experience.

Interoception provides information about the saliency of external or internal sensory events and thus may inform perceptual decision-making. Error in performance is an example of a motivationally significant internal event that evokes autonomic nervous system response resembling the orienting response: heart rate deceleration, increased skin conductance response, and pupil dilation. Here, we investigate whether error-related cardiac activity may serve as a source of information when making metacognitive judgments in an orientation discrimination backward masking task. In the first experiment, we found that the heart accelerates less after an incorrect stimuli discrimination than after a correct one. Moreover, this difference becomes more pronounced with increasing subjective visibility of the stimuli. In the second experiment, this accuracy-dependent pattern of cardiac activity was found only when participants listened to their own heartbeats, but not someone else’s. We propose that decision accuracy coded in cardiac activity may be fed as a cue to subjective visibility judgments.

Information that is normally accessed through a sensory modality (substituted modality, e.g., vision) is provided by sensory substitution devices (SSDs) through an alternative modality such as hearing or touch (i.e., substituting modality). SSDs usually support disabled users by replacing sensory inputs that have been lost, but they also offer a unique opportunity to study adaptation and flexibility in human perception. Current debates in sensory substitution (SS) literature focus mostly on its neural correlates and behavioural consequences. In particular, studies have demonstrated the neural plasticity of the visual brain regions that are activated by the substituting modality. Participants also adapt to using the devices for a broad spectrum of cognitive tasks that usually require sight. However, little is known about the SS experience. Also, there is no agreement on how the phenomenology of SS should be studied. Here, we offer guidelines for the methodology of studies investigating behavioural adaptation to SS and the effects of this adaptation on the subjective SS experience. We also discuss factors that may influence the results of SS studies: (1) the type of SSD, (2) the effects of training, (3) the role of sensory deprivation, (4) the role of the experimental environment, (5) the role of the tasks participants follow, and (6) the characteristics of the participants. In addition, we propose combining qualitative and quantitative methods and discuss how this should be achieved when studying the neural, behavioural, and experiential consequences of SS.

International audience; Recent studies have highlighted the role of multisensory integration as a key mechanism of self-consciousness. In particular, integration of bodily signals within the peripersonal space (PPS) underlies the experience of the self in a body we own (self-identification) and that is experienced as occupying a specific location in space (self-location), two main components of bodily self-consciousness (BSC). Experiments investigating the effects of multisensory integration on BSC have typically employed supra-threshold sensory stimuli, neglecting the role of unconscious sensory signals in BSC, as tested in other consciousness research. Here, we used psychophysical techniques to test whether multisensory integration of bodily stimuli underlying BSC also occurs for multisensory inputs presented below the threshold of conscious perception. Our results indicate that visual stimuli rendered invisible through continuous flash suppression boost processing of tactile stimuli on the body (Exp. 1), and enhance the perception of near-threshold tactile stimuli (Exp. 2), only once they entered PPS. We then employed unconscious multisensory stimulation to manipulate BSC. Participants were presented with tactile stimulation on their body and with visual stimuli on a virtual body, seen at a distance, which were either visible or rendered invisible. We found that participants reported higher self-identification with the virtual body in the synchronous visuo-tactile stimulation (as compared to asynchronous stimulation; Exp. 3), and shifted their self-location toward the virtual body (Exp.4), even if stimuli were fully invisible. Our results indicate that multisensory inputs, even outside of awareness, are integrated and affect the phenomenological content of self-consciousness, grounding BSC firmly in the field of psychophysical consciousness studies.