Search

Introduction: Associating multimodal information is essential for human cognitive abilities including mathematical skills. Multimodal learning has also attracted attention in the field of machine learning, and it has been suggested that the acquisition of better latent representation plays an important role in enhancing task performance. This study aimed to explore the impact of multimodal learning on representation, and to understand the relationship between multimodal representation and the development of mathematical skills. Methods: We employed a multimodal deep neural network as the computational model for multimodal associations in the brain. We compared the representations of numerical information, that is, handwritten digits and images containing a variable number of geometric figures learned through single- and multimodal methods. Next, we evaluated whether these representations were beneficial for downstream arithmetic tasks. Results: Multimodal training produced better latent representation in terms of clustering quality, which is consistent with previous findings on multimodal learning in deep neural networks. Moreover, the representations learned using multimodal information exhibited superior performance in arithmetic tasks. Discussion: Our novel findings experimentally demonstrate that changes in acquired latent representations through multimodal association learning are directly related to cognitive functions, including mathematical skills. This supports the possibility that multimodal learning using deep neural network models may offer novel insights into higher cognitive functions. [ABSTRACT FROM AUTHOR]

The integration of stimuli from different sensory modalities forms the basis for human perception. While the relevant impact of visual stimuli on the perception of other sensory modalities is recognized, much less is known about the impact of auditory stimuli on general sensory processing. This study aims to investigate the effect of acoustic stimuli on the processing of somatosensory stimuli using real noise (i.e., unpleasant everyday noise, RN) and neutral white noise (WN). To this purpose, we studied 20 healthy human subjects between 20 and 29 years of age (mean: 24, SD: ±1.9 years sex ratio 1:1). Somatosensory perception was evaluated using mechanical detection threshold (MDT) of the skin on the back of the dominant hand. To investigate the underlying mechanisms in the brain, fMRI was performed while applying acoustic stimulation (RN and WN) and tactile stimulation of the dominant hand. Here we show that acoustic stimulation with noise alters the perception of touch on the skin. We found that the effect of RN and WN differed. RN leads to an improved tactile perception, whereas WN impaired tactile perception. These changes go along with significant differences in brain activity and connectivity. WN is associated with a significant increase in brain activity in multiple brain areas such as the auditory and somatosensory cortex, parietal association cortex, and the thalamus compared to RN. With tactile stimulation of the skin, the flow of information in these brain areas is altered. While with RN the information flow from the thalamus to the somatosensory cortex is prominent, the network activity pattern changes under WN revealing an increase in interaction between multiple networks. Unpleasant noise inhibits the multisensory integration and enables a more efficient unimodal perception in the somatosensory system, improving perception. Whether this is to be interpreted as a temporary increase in phasic alertness or by a stronger filter function of the thalamus with a preference for unimodal stimuli is still open for debate. [ABSTRACT FROM AUTHOR]

The purpose of this article is to examine evidence that broadens the clinical perspective on interoception as an imperative consideration for individuals with mental health and sensory processing challenges. The central supposition is that interoception is broader than just signals from the viscera. Rather, interoception refers to perceptions of bodily signals and bodily states that construct a subjective representation of the experience. These representations are then utilized for categorizing the sensory attributes and constructing meaning. Thus, this updated conceptualization presents interoception as a complex multidimensional system, with bidirectional features. The interplay between the brain and the body is necessary to maintain homeostasis as well as respond adaptively to the changes in one's internal and external environment. As a sensory capacity, interoceptive information must be processed and interpreted before it can be integrated into a personal experiential history. Interoception supports both body and mental functions and as such, interoceptive processes support health and wellness by establishing a felt sense of psychological and physiological safety that is foundational to meaningful participation in life. The information presented in this article is central to the pursuit of evidence-based best practices for any professional wishing to integrate consideration of interoception into their clinical practice. [ABSTRACT FROM AUTHOR]

The purpose of this article is to examine evidence that broadens the clinical perspective on interoception as an imperative consideration for individuals with mental health and sensory processing challenges. The central supposition is that interoception is broader than just signals from the viscera. Rather, interoception refers to perceptions of bodily signals and bodily states that construct a subjective representation of the experience. These representations are then utilized for categorizing the sensory attributes and constructing meaning. Thus, this updated conceptualization presents interoception as a complex multidimensional system, with bidirectional features. The interplay between the brain and the body is necessary to maintain homeostasis as well as respond adaptively to the changes in one’s internal and external environment. As a sensory capacity, interoceptive information must be processed and interpreted before it can be integrated into a personal experiential history. Interoception supports both body and mental functions and as such, interoceptive processes support health and wellness by establishing a felt sense of psychological and physiological safety that is foundational to meaningful participation in life. The information presented in this article is central to the pursuit of evidence-based best practices for any professional wishing to integrate consideration of interoception into their clinical practice.

The contribution of cervical proprioception, vision, and vestibular feedback to the dynamic head–trunk orientation error in the yaw direction was investigated to further the understanding over the mechanism of coordination among different sensory modalities for dynamic head–trunk orientation. To test the contribution of each sensory modality, individually and together, to dynamic head–trunk orientation, 10 healthy human subjects participated in the extended cervical joint position error test, measuring the ability of repositioning the head back to the reference orientation after 45° yaw rotation of head or trunk. The error between initial and returned angles was measured. The test was repeated under eight different conditions of sensory feedback, with or without each of three sensory modalities. Each subject completed 64 trials (8 per condition) in a random order for fair comparison. No change was found in bias when one of the three modalities was missing, while variance was largest at the lack of dynamic cervical proprioception. When two of the three modalities were missing (i.e., one of the three modalities was present), both bias and variance were minimum at the presence of cervical proprioception. Additionally, both visual and vestibular feedback was redundant (i.e., no further improvement in both bias and variance), if the other one (visual or vestibular feedback) was present with dynamic cervical proprioception. In sum, the experimental results suggest that dynamic cervical proprioception is the most significant sensory modality for reducing the dynamic head–trunk orientation error in the yaw direction.

The contribution of cervical proprioception, vision, and vestibular feedback to the dynamic head–trunk orientation error in the yaw direction was investigated to further the understanding over the mechanism of coordination among different sensory modalities for dynamic head–trunk orientation. To test the contribution of each sensory modality, individually and together, to dynamic head–trunk orientation, 10 healthy human subjects participated in the extended cervical joint position error test, measuring the ability of repositioning the head back to the reference orientation after 45° yaw rotation of head or trunk. The error between initial and returned angles was measured. The test was repeated under eight different conditions of sensory feedback, with or without each of three sensory modalities. Each subject completed 64 trials (8 per condition) in a random order for fair comparison. No change was found in bias when one of the three modalities was missing, while variance was largest at the lack of dynamic cervical proprioception. When two of the three modalities were missing (i.e., one of the three modalities was present), both bias and variance were minimum at the presence of cervical proprioception. Additionally, both visual and vestibular feedback was redundant (i.e., no further improvement in both bias and variance), if the other one (visual or vestibular feedback) was present with dynamic cervical proprioception. In sum, the experimental results suggest that dynamic cervical proprioception is the most significant sensory modality for reducing the dynamic head–trunk orientation error in the yaw direction. [ABSTRACT FROM AUTHOR]

Air turbulence ensures that in a natural environment insects tend to encounter odor stimuli in a pulsatile fashion. The frequency and duration of odor pulses varies with distance from the source, and hence successful mid-flight odor tracking requires resolution of spatiotemporal pulse dynamics. This requires both olfactory and mechanosensory input (from wind speed), a form of sensory integration observed within the antennal lobe (AL). In this work, we employ a model of the moth AL to study the effect of mechanosensory input on AL responses to pulsatile stimuli; in particular, we examine the ability of model neurons to: (1) encode the temporal length of a stimulus pulse; (2) resolve the temporal dynamics of a high frequency train of brief stimulus pulses. We find that AL glomeruli receiving olfactory input are adept at encoding the temporal length of a stimulus pulse but less effective at tracking the temporal dynamics of a pulse train, while glomeruli receiving mechanosensory input but little olfactory input can efficiently track the temporal dynamics of high frequency pulse delivery but poorly encode the duration of an individual pulse. Furthermore, we show that stronger intrinsic small-conductance calcium-dependent potassium (SK) currents tend to skew cells toward being better trackers of pulse frequency, while weaker SK currents tend to entail better encoding of the temporal length of individual pulses. We speculate a possible functional division of labor within the AL, wherein, for a particular odor, glomeruli receiving strong olfactory input exhibit prolonged spiking responses that facilitate detailed discrimination of odor features, while glomeruli receiving mechanosensory input (but little olfactory input) serve to resolve the temporal dynamics of brief, pulsatile odor encounters. Finally, we discuss how this hypothesis extends to explaining the functional significance of intraglomerular variability in observed phase II response patterns of AL neurons.

Air turbulence ensures that in a natural environment insects tend to encounter odor stimuli in a pulsatile fashion. The frequency and duration of odor pulses varies with distance from the source, and hence successful mid-flight odor tracking requires resolution of spatiotemporal pulse dynamics. This requires both olfactory and mechanosensory input (from wind speed), a form of sensory integration observed within the antennal lobe (AL). In this work, we employ a model of the moth AL to study the effect of mechanosensory input on AL responses to pulsatile stimuli; in particular, we examine the ability of model neurons to: (1) encode the temporal length of a stimulus pulse; (2) resolve the temporal dynamics of a high frequency train of brief stimulus pulses. We find that AL glomeruli receiving olfactory input are adept at encoding the temporal length of a stimulus pulse but less effective at tracking the temporal dynamics of a pulse train, while glomeruli receiving mechanosensory input but little olfactory input can efficiently track the temporal dynamics of high frequency pulse delivery but poorly encode the duration of an individual pulse. Furthermore, we show that stronger intrinsic small-conductance calcium-dependent potassium (SK) currents tend to skew cells toward being better trackers of pulse frequency, while weaker SK currents tend to entail better encoding of the temporal length of individual pulses. We speculate a possible functional division of labor within the AL, wherein, for a particular odor, glomeruli receiving strong olfactory input exhibit prolonged spiking responses that facilitate detailed discrimination of odor features, while glomeruli receiving mechanosensory input (but little olfactory input) serve to resolve the temporal dynamics of brief, pulsatile odor encounters. Finally, we discuss how this hypothesis extends to explaining the functional significance of intraglomerular variability in observed phase II response patterns of AL neurons. [ABSTRACT FROM AUTHOR]

This manuscript reviews historical and recent studies that focus on supplementary sensory feedback for use in upper limb prostheses. It shows that the inability of many studies to speak to the issue of meaningful performance improvements in real-life scenarios is caused by the complexity of the interactions of supplementary sensory feedback with other types of feedback along with other portions of the motor control process. To do this, the present manuscript frames the question of supplementary feedback from the perspective of computational motor control, providing a brief review of the main advances in that field over the last 20 years. It then separates the studies on the closed-loop prosthesis control into distinct categories, which are defined by relating the impact of feedback to the relevant components of the motor control framework, and reviews the work that has been done over the last 50+ years in each of those categories. It ends with a discussion of the studies, along with suggestions for experimental construction and connections with other areas of research, such as machine learning.

This manuscript reviews historical and recent studies that focus on supplementary sensory feedback for use in upper limb prostheses. It shows that the inability of many studies to speak to the issue of meaningful performance improvements in real-life scenarios is caused by the complexity of the interactions of supplementary sensory feedback with other types of feedback along with other portions of the motor control process. To do this, the present manuscript frames the question of supplementary feedback from the perspective of computational motor control, providing a brief review of the main advances in that field over the last 20 years. It then separates the studies on the closed-loop prosthesis control into distinct categories, which are defined by relating the impact of feedback to the relevant components of the motor control framework, and reviews the work that has been done over the last 50+ years in each of those categories. It ends with a discussion of the studies, along with suggestions for experimental construction and connections with other areas of research, such as machine learning. [ABSTRACT FROM AUTHOR]

Previous research showed that a simple target interception task reveals differences between younger adults (YA) and older adults (OA) on a large screen under laboratory conditions. Participants intercept downward moving objects while a horizontally moving background creates an illusion of the object moving in the opposite direction of the background. OA are more influenced by this illusory motion than YA. OA seem to be less able to ignore irrelevant sensory information than YA. Since sensory integration relates to the ability to perform Activities of Daily Living (ADL), this interception task can potentially signal ADL issues. Here we investigated whether the results of the target interception task could be replicated using a more portable setup, i.e., a tablet instead of a large touch screen. For YA from the same, homogeneous population, the main effects were replicated although the task was more difficult in the tablet set-up. After establishing the tablet’s validity, we analyzed the response patterns of OA that were less fit than the OA in previous research. We identified three different illusion patterns: a (large) illusion effect (indicating over integration), a reverse illusion effect, and no illusion effect. These different patterns are much more nuanced than previously reported for fit OA who only show over integration. We propose that the patterns are caused by differences in the samples of OA (OA in the current sample were older and had lower ADL scores), possibly modulated by increased task difficulty in the tablet setup. We discuss the effects of illusory background motion as a function of ADL scores using a transitional model. The first pattern commences when sensory integration capability starts to decrease, leading to a pattern of over-integration (illusion effect). The second pattern commences when compensatory mechanisms are not sufficient to counteract the effect of the background motion, leading to direction errors in the same direction as the background motion (reverse illusion). The third pattern commences when the task requirements are too high, leading OA to implement a probabilistic strategy by tapping toward the center of the screen.

Previous research showed that a simple target interception task reveals differences between younger adults (YA) and older adults (OA) on a large screen under laboratory conditions. Participants intercept downward moving objects while a horizontally moving background creates an illusion of the object moving in the opposite direction of the background. OA are more influenced by this illusory motion than YA. OA seem to be less able to ignore irrelevant sensory information than YA. Since sensory integration relates to the ability to perform Activities of Daily Living (ADL), this interception task can potentially signal ADL issues. Here we investigated whether the results of the target interception task could be replicated using a more portable setup, i.e., a tablet instead of a large touch screen. For YA from the same, homogeneous population, the main effects were replicated although the task was more difficult in the tablet set-up. After establishing the tablet's validity, we analyzed the response patterns of OA that were less fit than the OA in previous research. We identified three different illusion patterns: a (large) illusion effect (indicating over integration), a reverse illusion effect, and no illusion effect. These different patterns are much more nuanced than previously reported for fit OA who only show over integration. We propose that the patterns are caused by differences in the samples of OA (OA in the current sample were older and had lower ADL scores), possibly modulated by increased task difficulty in the tablet setup. We discuss the effects of illusory background motion as a function of ADL scores using a transitional model. The first pattern commences when sensory integration capability starts to decrease, leading to a pattern of over-integration (illusion effect). The second pattern commences when compensatory mechanisms are not sufficient to counteract the effect of the background motion, leading to direction errors in the same direction as the background motion (reverse illusion). The third pattern commences when the task requirements are too high, leading OA to implement a probabilistic strategy by tapping toward the center of the screen. [ABSTRACT FROM AUTHOR]