Search

We describe a new unified model to explain both binocular fusion and depth perception, over a broad range of depths. At each location, the model consists of an array of paired spatial frequency filters, with different relative horizontal shifts (position disparity) and interocular phase disparities of 0, 90, ±180, or -90°. The paired filters with different spatial profiles (non-zero phase disparity) compute interocular misalignment and provide phase-disparity energy (binocular fusion energy) to drive selection of the appropriate filters along the position disparity space until the misalignment is eliminated and sensory fusion is achieved locally. The paired filters with identical spatial profiles (0 phase disparity) compute the position-disparity energy. After sensory fusion, the combination of position and possible residual phase disparity energies is calculated for binocular depth perception. Binocular fusion occurs at multiple scales following a coarse-to-fine process. At a given location, the apparent depth is the weighted sum of fusion shifts combined with residual phase disparity in all spatial-frequency channels, and the weights depend on stimulus spatial frequency and stimulus contrast. To test the theory, we measured disparity minimum and maximum thresholds (Dmin and Dmax) at three spatial frequencies and with different intraocular contrast levels. The stimuli were Random-Gabor-Patch (RGP) stereograms consisting of Gabor patches with random positions and phases, but with a fixed spatial frequency. The two eyes viewed identical arrays of patches except that one eye's array could be shifted horizontally and could differ in contrast. Our experiments and modeling reveal two contrast normalization mechanisms: (1) Energy Normalization (EN): Binocular energy is normalized with monocular energy after the site of binocular combination. This predicts constant Dmin thresholds when varying stimulus contrast in the two eyes; (2) DSKL model Interocular interactions: Monocular contrasts are normalized before the binocular combination site through interocular contrast gain-control and gain-enhancement mechanisms. This predicts contrast dependent Dmax thresholds. We tested a range of models and found that a model consisting of a second-order pathway with DSKL interocular interactions and a first-order pathway with EN at each spatial-frequency band can account for both the Dmin and Dmax data very well. Simulations show that the model makes reasonable predictions of suprathreshold depth perception.

IntroductionEither non-invasive somatotopic or substitute sensory feedback is capable of conveying a single modality of sensory information from prosthetic hands to amputees. However, the neurocognitive ability of amputees to integrate multi-modality sensory information for functional discrimination is unclear. The purpose of this study was to assess the fusion of non-invasive somatotopic tactile and substitute aperture feedbacks for profile perception of multiple physical features during grasping objects.MethodsTwo left transradial amputees with somatotopic evoked tactile sensation (ETS) of five fingers participated in the study. The tactile information of prosthetic hand was provided to amputees by the ETS feedback elicited on the stump projected finger map. Hand aperture information was conveyed to amputees with substitute electrotactile stimulation on the forearm or upper arm. Two types of sensory feedback were integrated to a commercial prosthetic hand. The efficacy of somatotopic ETS feedback on object length identification task was assessed with or without substitute aperture stimulation. The object size identification task was utilized to assess how ETS stimulation at the stump may affect aperture perception with stimulation on the ipsilateral upper arm or forearm. Finally, the task of identifying combined length and size was conducted to evaluate the ability of amputees to integrate the dual modalities of sensory feedback for perceiving profile features.ResultsThe study revealed that amputee subjects can effectively integrate the ETS feedback with electrotactile substitutive feedback for object profile discrimination. Specifically, ETS was robust to provide object length information with electrotactile stimulation at either the forearm or upper arm. However, electrotactile stimulation at the upper arm for aperture perception was less susceptible to the interference of ETS stimulation than at the forearm.DiscussionAmputee subjects are able to combine somatotopic ETS and aperture feedbacks for identifying multi-dimensional features in object profiling. The two sensory streams of information can be fused effectively without mutual interference for functional discrimination.

The lack of intuitive and active human–robot interaction makes it difficult to use upper-limb-assistive devices. In this paper, we propose a novel learning-based controller that intuitively uses onset motion to predict the desired end-point position for an assistive robot. A multi-modal sensing system comprising inertial measurement units (IMUs), electromyographic (EMG) sensors, and mechanomyography (MMG) sensors was implemented. This system was used to acquire kinematic and physiological signals during reaching and placing tasks performed by five healthy subjects. The onset motion data of each motion trial were extracted to input into traditional regression models and deep learning models for training and testing. The models can predict the position of the hand in planar space, which is the reference position for low-level position controllers. The results show that using IMU sensor with the proposed prediction model is sufficient for motion intention detection, which can provide almost the same prediction performance compared with adding EMG or MMG. Additionally, recurrent neural network (RNN)-based models can predict target positions over a short onset time window for reaching motions and are suitable for predicting targets over a longer horizon for placing tasks. This study's detailed analysis can improve the usability of the assistive/rehabilitation robots. [ABSTRACT FROM AUTHOR]

Purpose: To analyze the efficacy of fusional vergence therapy (FVT) in management of consecutive esotropia with diplopia after intermittent exotropia (IXT) surgery. The current study is carried on how FVT affects the duration of treatment, sensory fusion, and exotropic drift. Methods: This was a retrospective study for the medical record of 11 patients with consecutive esotropia after IXT surgery of 543 patients over the period of 5 year, with mean surgery age of 9.5 (range: 4–33 y). FVT was planned after minimum 6 weeks of surgery and was considered for maximum 24 weeks. Patients underwent a combination of office-based and home-based FVT. Successful outcome of therapy was considered where diplopia resolves in free space and achieves sensory fusion, stereopsis with no manifest deviation. Results: Record of 543 patients who had horizontal muscle surgery for IXT were identified and reviewed. Records of 11 patients who showed consecutive esotropia of 10 prism diopter (PD) or more with normal retinal correspondence, with or without diplopia complaint, after 6 week of surgery and had undergone vision therapy management were reviewed. A successful outcome of binocular single vision with good sensory and motor fusion with no manifest deviation or prism requirement was achieved with in the mean duration of 4.8 month of therapy. With a mean duration of 4 weeks of therapy, the mean angle of deviation reduced by 53% for distance (17 PD to 8 PD) and 27% for near (11 PD to 8 PD) and mean stereopsis improvement by 80% with 94% patients demonstrating sensory fusion on Bagolini test and 94% of patients having no symptoms of diplopia or squint. Conclusion: With nonsurgical management involving refractive error correction, FVT, and prism, consecutive esotropia was resolved in 74% cases. Management of consecutive esotropia with FVT can result in satisfactory sensory fusion and successful motor alignment.

Purpose: To analyze the efficacy of fusional vergence therapy (FVT) in management of consecutive esotropia with diplopia after intermittent exotropia (IXT) surgery. The current study is carried on how FVT affects the duration of treatment, sensory fusion, and exotropic drift.Methods: This was a retrospective study for the medical record of 11 patients with consecutive esotropia after IXT surgery of 543 patients over the period of 5 year, with mean surgery age of 9.5 (range: 4-33 y). FVT was planned after minimum 6 weeks of surgery and was considered for maximum 24 weeks. Patients underwent a combination of office-based and home-based FVT. Successful outcome of therapy was considered where diplopia resolves in free space and achieves sensory fusion, stereopsis with no manifest deviation.Results: Record of 543 patients who had horizontal muscle surgery for IXT were identified and reviewed. Records of 11 patients who showed consecutive esotropia of 10 prism diopter (PD) or more with normal retinal correspondence, with or without diplopia complaint, after 6 week of surgery and had undergone vision therapy management were reviewed. A successful outcome of binocular single vision with good sensory and motor fusion with no manifest deviation or prism requirement was achieved with in the mean duration of 4.8 month of therapy. With a mean duration of 4 weeks of therapy, the mean angle of deviation reduced by 53% for distance (17 PD to 8 PD) and 27% for near (11 PD to 8 PD) and mean stereopsis improvement by 80% with 94% patients demonstrating sensory fusion on Bagolini test and 94% of patients having no symptoms of diplopia or squint.Conclusion: With nonsurgical management involving refractive error correction, FVT, and prism, consecutive esotropia was resolved in 74% cases. Management of consecutive esotropia with FVT can result in satisfactory sensory fusion and successful motor alignment. [ABSTRACT FROM AUTHOR]

Executing agile locomotion requires animals to integrate sensory feedback, often from multiple sources. For example, human gaze is mediated by multiple feedback loops that integrate visual and vestibular information. A central challenge in studying biological feedback loops is that they are nested and dynamically coupled. Here, we develop a framework based on control theory for unraveling nested feedback systems and apply it to study gaze stabilization in the fruit fly (Drosophila). By combining experimental and mathematical methods to manipulate control topologies, we uncovered the role of body-generated mechanosensory feedback nested within visual feedback in the control of head movements. We discovered that visual feedback changed the tuning of head movements across visual motion frequencies whereas mechanosensory feedback damped head movements. Head saccades had slower dynamics when the body was free to move, further pointing to the role of damping via mechanosensory feedback. By comparing head responses between self-generated and externally generated body motion, we revealed a nonlinear gating of mechanosensory feedback that is motor-context dependent. Altogether, our findings reveal the role of nested feedback loops in flies and uncover mechanisms that reconcile differences in head kinematics between body-free and body-fixed flies. Our framework is generalizable to biological and robotic systems relying on nested feedback control for guiding locomotion.

The lack of intuitive and active human–robot interaction makes it difficult to use upper-limb-assistive devices. In this paper, we propose a novel learning-based controller that intuitively uses onset motion to predict the desired end-point position for an assistive robot. A multi-modal sensing system comprising inertial measurement units (IMUs), electromyographic (EMG) sensors, and mechanomyography (MMG) sensors was implemented. This system was used to acquire kinematic and physiological signals during reaching and placing tasks performed by five healthy subjects. The onset motion data of each motion trial were extracted to input into traditional regression models and deep learning models for training and testing. The models can predict the position of the hand in planar space, which is the reference position for low-level position controllers. The results show that using IMU sensor with the proposed prediction model is sufficient for motion intention detection, which can provide almost the same prediction performance compared with adding EMG or MMG. Additionally, recurrent neural network (RNN)-based models can predict target positions over a short onset time window for reaching motions and are suitable for predicting targets over a longer horizon for placing tasks. This study’s detailed analysis can improve the usability of the assistive/rehabilitation robots.

In the modern world; technology is the asset of the development. There is demand of automation in every sector. Such type of autonomous applications requires inputs from multiple sensory resources and based on these inputs it processes and deliver intelligent decision i.e. output. The method used in proposed system focuses on collection of sensory information and few concluding parameters derivations from data which will help to decide taking best decisions for autonomous applications. Drawing conclusion based on only one aspect is not effective as compared to conclusions taken from many other heterogeneous sensors parameters. This system fulfills parameters such as accuracy, less request time and response time, performance, reliability etc. This paper explains safe and secure system which uses unsupervised algorithm that works on real time data and mixes the information from various resources. The main interest is by taking intelligent decision based on reduction of unreliability and disordered sensory data. Women being a major victim in today's world with their entry in almost each field their safety is a priority and with our concept and algorithm we have illustrated it as a system for sensory fusion with GPS enabled double security feature which will help a women when she feels insecure or harassed or in a difficult situation. System has smart feature to ensure complete safety of women even in dangerous situation where she cannot turn on the trigger. The system comprises of ATmega328P microcontroller, GPS, LCD display, fingerprint module, GPS, GSM etc. for proper and accurate working of the system to provide reliable, accurate and quickservice which efficiently work for safety of women. [ABSTRACT FROM AUTHOR]

We propose a novel controller for powered prosthetic arms, where fused EMG and gaze data predict the desired end-point for a full arm prosthesis, which could drive the forward motion of individual joints. We recorded EMG, gaze, and motion-tracking during pick-and-place trials with 7 able-bodied subjects. Subjects positioned an object above a random target on a virtual interface, each completing around 600 trials. On average across all trials and subjects gaze preceded EMG and followed a repeatable pattern that allowed for prediction. A computer vision algorithm was used to extract the initial and target fixations and estimate the target position in 2D space. Two SVRs were trained with EMG data to predict the x- and y- position of the hand; results showed that the y-estimate was significantly better than the x-estimate. The EMG and gaze predictions were fused using a Kalman Filter-based approach, and the positional error from using EMG-only was significantly higher than the fusion of EMG and gaze. The final target position Root Mean Squared Error (RMSE) decreased from 9.28 cm with an EMG-only prediction to 6.94 cm when using a gaze-EMG fusion. This error also increased significantly when removing some or all arm muscle signals. However, using fused EMG and gaze, there were no significant difference between predictors that included all muscles, or only a subset of muscles. [ABSTRACT FROM AUTHOR]

The work presents real-time data acquisition and monitoring for environmental sensors. This paper presents data fetching and processing implemented by modern approaches of Python. A functional diagram of a materialized system and algorithmic diagrams for Python 3 implemented programs are given. The functionality of the programmed system has been checked by testing the implemented program structures and performing experiments under different environmental conditions with actual results obtained. [ABSTRACT FROM AUTHOR]

Aims: Bagolini striated lenses are a useful test of binocular vision, but the variations in the striations (frequency and thickness) can impact on the perceived image. Also, responses can be difficult to interpret in young children. Therefore, the aims of this project were to evaluate the impact of striation frequency and the addition of coloured filters on subjective responses. Methods: Three sets of striated lenses were made (small, medium and large striations), each produced in two forms (both lenses clear, or with a red and blue lens). Also, Bagolini glasses (Optiker Ryser) were used, with and without the addition of red and blue filters. Subjects were asked to report what they perceived, with subsequent questions regarding the number and length of lines. Results: Forty-two adult subjects were tested, with uniocular VA ranging from –0.18 to 1.10 logMAR (mean 0.08 ± 0.25). The number of lines seen when varying the line thickness did not vary between coloured and clear lenses (post-hoc analysis following ANOVA, p > 0.1 in all comparisons). Adding red/blue filters to the original Bagolini glasses did not alter the rates of subjects perceiving a cross (Chi-square, p = 0.8). However, the laser-cut lenses produced a significantly shorter light streak than the original lenses (One-way ANOVA, p < 0.001), but the colour of the filters made no difference to the length of streak perceived (Tukey’s Test, p = 0.20). Conclusions: The addition of coloured filters did not impact on the responses given to the original or laser-cut lenses, suggesting this modification may aid responses in children. However, further evaluation is required with finer striations and thinner lenses to improve the visibility of the lines.

Introduction: Either non-invasive somatotopic or substitute sensory feedback is capable of conveying a single modality of sensory information from prosthetic hands to amputees. However, the neurocognitive ability of amputees to integrate multi-modality sensory information for functional discrimination is unclear. The purpose of this study was to assess the fusion of non-invasive somatotopic tactile and substitute aperture feedbacks for profile perception of multiple physical features during grasping objects., Methods: Two left transradial amputees with somatotopic evoked tactile sensation (ETS) of five fingers participated in the study. The tactile information of prosthetic hand was provided to amputees by the ETS feedback elicited on the stump projected finger map. Hand aperture information was conveyed to amputees with substitute electrotactile stimulation on the forearm or upper arm. Two types of sensory feedback were integrated to a commercial prosthetic hand. The efficacy of somatotopic ETS feedback on object length identification task was assessed with or without substitute aperture stimulation. The object size identification task was utilized to assess how ETS stimulation at the stump may affect aperture perception with stimulation on the ipsilateral upper arm or forearm. Finally, the task of identifying combined length and size was conducted to evaluate the ability of amputees to integrate the dual modalities of sensory feedback for perceiving profile features., Results: The study revealed that amputee subjects can effectively integrate the ETS feedback with electrotactile substitutive feedback for object profile discrimination. Specifically, ETS was robust to provide object length information with electrotactile stimulation at either the forearm or upper arm. However, electrotactile stimulation at the upper arm for aperture perception was less susceptible to the interference of ETS stimulation than at the forearm., Discussion: Amputee subjects are able to combine somatotopic ETS and aperture feedbacks for identifying multi-dimensional features in object profiling. The two sensory streams of information can be fused effectively without mutual interference for functional discrimination., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. The author(s) declared that they were an editorial board member of Frontiers, at the time of submission. This had no impact on the peer review process and the final decision., (Copyright © 2023 Zhang, Chou, Hao, Li, Yu and Lan.)

Restoring locomotion functionality of transfemoral amputees is essential for early rehabilitation treatment and for preserving mobility and independence in daily life. Research in wearable robotics fostered the development of innovative active mechatronic lower-limb prostheses designed with the goal to reduce the cognitive and physical effort of lower-limb amputees in rehabilitation and daily life activities. To ensure benefits to the users, active mechatronic prostheses are expected to be aware of the user intention and properly interact in a closed human-in-the-loop paradigm. In the state of the art various cognitive interfaces have been proposed to online decode the user's intention. Electromyography in combination with mechanical sensing such as inertial or pressure sensors is a widely adopted solution for driving active mechatronic prostheses. In this framework, researchers also explored targeted muscles re-innervation for an objective-oriented surgical amputation promoting wider usability of active prostheses. However, information kept by the neural component of the cognitive interface deteriorates in a prolonged use scenario due to electrodes-related issues, thereby undermining the correct functionality of the active prosthesis. The objective of this work is to present a novel controller for an active transfemoral prosthesis based on whole body awareness relying on a wireless distributed non-invasive sensory apparatus acting as cognitive interface. A finite-state machine controller based on signals monitored from the wearable interface performs subject-independent intention detection of functional tasks such as ground level walking, stair ascent, and sit-to-stand maneuvres and their main sub-phases. Experimental activities carried out with four transfemoral amputees (among them one dysvascular) demonstrated high reliability of the controller capable of providing 100% accuracy rate in treadmill walking even for weak subjects and low walking speeds. The minimum success rate was of 94.8% in performing sit-to-stand tasks. All the participants showed high confidence in using the transfemoral active prosthesis even without training period thanks to intuitiveness of the whole body awareness controller.

The direction of the Earth’s magnetic field is used as a reference vector to determine the heading in orientation estimation with wearable sensors. However, the magnetic field strength is weak and can be easily disturbed in the vicinity of ferromagnetic materials, which may result in inaccurate estimate of orientation. This paper presents a novel method for estimating and compensating for magnetic disturbances. The compensation algorithm is implemented within a kinematic-based extended Kalman filter and is based on an assessment of the magnetic disturbance and the change of orientation in each time step. The proposed algorithm was experimentally validated by measuring the orientation of a simple mechanical system with three degrees of freedom in an artificially disturbed magnetic field. The results of the experimental evaluation show that an Kalman filter algorithm that incorporates compensating for magnetic disturbances is capable of estimating the orientation with moderate error (the absolute median errors \Delta \widetilde \theta X,\Delta \widetilde \theta Z < {2}^{\circ } , \Delta \widetilde \theta Y < {9}^{\circ } ) when the Earth’s magnetic field is disturbed by magnetic disturbance with a magnitude equal to twice the magnitude of the Earth’s own magnetic field in different directions. [ABSTRACT FROM PUBLISHER]

Restoring locomotion functionality of transfemoral amputees is essential for early rehabilitation treatment and for preserving mobility and independence in daily life. Research in wearable robotics fostered the development of innovative activemechatronic lower-limb prostheses designed with the goal to reduce the cognitive and physical effort of lower-limb amputees in rehabilitation and daily life activities. To ensure benefits to the users, active mechatronic prostheses are expected to be aware of the user intention and properly interact in a closed human-in-the-loop paradigm. In the state of the art various cognitive interfaces have been proposed to online decode the user's intention. Electromyography in combination with mechanical sensing such as inertial or pressure sensors is a widely adopted solution for driving active mechatronic prostheses. In this framework, researchers also explored targeted muscles re-innervation for an objective-oriented surgical amputation promoting wider usability of active prostheses. However, information kept by the neural component of the cognitive interface deteriorates in a prolonged use scenario due to electrodes-related issues, thereby undermining the correct functionality of the active prosthesis. The objective of this work is to present a novel controller for an active transfemoral prosthesis based on whole body awareness relying on a wireless distributed non-invasive sensory apparatus acting as cognitive interface. A finite-state machine controller based on signals monitored from the wearable interface performs subject-independent intention detection of functional tasks such as ground level walking, stair ascent, and sit-to-stand maneuvres and their main sub-phases. Experimental activities carried out with four transfemoral amputees (among them one dysvascular) demonstrated high reliability of the controller capable of providing 100%accuracy rate in treadmill walking even for weak subjects and low walking speeds. The minimum success rate was of 94.8% in performing sit-to-stand tasks. All the participants showed high confidence in using the transfemoral active prosthesis even without training period thanks to intuitiveness of the whole body awareness controller. [ABSTRACT FROM AUTHOR]

This work proposes a new methodology for the detection of discontinuities in the weld bead applied in Shielded Metal ArcWelding (SMAW) processes. The detection system is based on two sensors--a microphone and piezoelectric--that acquire acoustic emissions generated during the welding. The feature vectors extracted from the sensor dataset are used to construct classifier models. The approaches based on Artificial Neural Network (ANN) and Support Vector Machine (SVM) classifiers are able to identify with a high accuracy the three proposed weld bead classes: desirable weld bead, shrinkage cavity and burn through discontinuities. Experimental results illustrate the system's high accuracy, greater than 90% for each class. A novel Hierarchical Support Vector Machine (HSVM) structure is proposed to make feasible the use of this system in industrial environments. This approach presented 96.6% overall accuracy. Given the simplicity of the equipment involved, this system can be applied in the metal transformation industries. [ABSTRACT FROM AUTHOR]

This work investigates an over-arching question: how can an immersive virtual environment be connected with its intelligent physical counterpart to allow for a more efficient man-machine collaboration. To this end, an immersive user interface for the purpose of robot tele-operation is designed. A large amount of sensory data is utilized to build models of the world and its inhabitants in a way that is intuitive to the operator and accurately represents the robot’s real-world state and environment. The game client is capable of handling multiple users, much like a traditional multiplayer game, while visualizing multiple robotic agents operating within the real world. The proposed Virtual Environment for Tele-Operation (VETO) architecture is a tele-operation system that provides a feature-rich framework to implement robotic agents into an immersive end-user game interface. Game levels are generated dynamically on a Graphic Processing Unit or GPU-accelerated server based on real-world sensor data from the robotic agents. A set of user studies are conducted to validate the performance of the proposed architecture compared to traditional tele-robotic applications. The experimental results show significant improvements in both task completion time and task completion rate over traditional tools.

Executing agile locomotion requires animals to integrate sensory feedback, often from multiple sources. For example, human gaze is mediated by multiple feedback loops that integrate visual and vestibular information. A central challenge in studying biological feedback loops is that they are nested and dynamically coupled. Here, we develop a framework based on control theory for unraveling nested feedback systems and apply it to study gaze stabilization in the fruit fly ( Drosophila ). By combining experimental and mathematical methods to manipulate control topologies, we uncovered the role of body-generated mechanosensory feedback nested within visual feedback in the control of head movements. We discovered that visual feedback changed the tuning of head movements across visual motion frequencies whereas mechanosensory feedback damped head movements. Head saccades had slower dynamics when the body was free to move, further pointing to the role of damping via mechanosensory feedback. By comparing head responses between self-generated and externally generated body motion, we revealed a nonlinear gating of mechanosensory feedback that is motor-context dependent. Altogether, our findings reveal the role of nested feedback loops in flies and uncover mechanisms that reconcile differences in head kinematics between body-free and body-fixed flies. Our framework is generalizable to biological and robotic systems relying on nested feedback control for guiding locomotion., Competing Interests: BC, JM No competing interests declared, (© 2022, Cellini and Mongeau.)

We describe a new unified model to explain both binocular fusion and depth perception, over a broad range of depths. At each location, the model consists of an array of paired spatial frequency filters, with different relative horizontal shifts (position disparity) and interocular phase disparities of 0, 90, ±180, or -90°. The paired filters with different spatial profiles (non-zero phase disparity) compute interocular misalignment and provide phase-disparity energy (binocular fusion energy) to drive selection of the appropriate filters along the position disparity space until the misalignment is eliminated and sensory fusion is achieved locally. The paired filters with identical spatial profiles (0 phase disparity) compute the position-disparity energy. After sensory fusion, the combination of position and possible residual phase disparity energies is calculated for binocular depth perception. Binocular fusion occurs at multiple scales following a coarse-to-fine process. At a given location, the apparent depth is the weighted sum of fusion shifts combined with residual phase disparity in all spatial-frequency channels, and the weights depend on stimulus spatial frequency and stimulus contrast. To test the theory, we measured disparity minimum and maximum thresholds (Dmin and Dmax) at three spatial frequencies and with different intraocular contrast levels. The stimuli were Random-Gabor-Patch (RGP) stereograms consisting of Gabor patches with random positions and phases, but with a fixed spatial frequency. The two eyes viewed identical arrays of patches except that one eye's array could be shifted horizontally and could differ in contrast. Our experiments and modeling reveal two contrast normalization mechanisms: (1) Energy Normalization (EN): Binocular energy is normalized with monocular energy after the site of binocular combination. This predicts constant Dmin thresholds when varying stimulus contrast in the two eyes; (2) DSKL model Interocular interactions: Monocular contrasts are normalized before the binocular combination site through interocular contrast gain-control and gain-enhancement mechanisms. This predicts contrast dependent Dmax thresholds. We tested a range of models and found that a model consisting of a second-order pathway with DSKL interocular interactions and a first-order pathway with EN at each spatial-frequency band can account for both the Dmin and Dmax data very well. Simulations show that the model makes reasonable predictions of suprathreshold depth perception., (Copyright © 2020 Elsevier Ltd. All rights reserved.)

When combining information across different senses, humans need to flexibly select cues of a common origin while avoiding distraction from irrelevant inputs. The brain could solve this challenge using a hierarchical principle by deriving rapidly a fused sensory estimate for computational expediency and, later and if required, filtering out irrelevant signals based on the inferred sensory cause(s). Analyzing time- and source-resolved human magnetoencephalographic data, we unveil a systematic spatiotemporal cascade of the relevant computations, starting with early segregated unisensory representations, continuing with sensory fusion in parietal-temporal regions, and culminating as causal inference in the frontal lobe. Our results reconcile previous computational accounts of multisensory perception by showing that prefrontal cortex guides flexible integrative behavior based on candidate representations established in sensory and association cortices, thereby framing multisensory integration in the generalized context of adaptive behavior. • Flexible use of multisensory information follows Bayesian inference • Sensory fusion emerges earlier than causal inference • Multisensory inference is represented in the frontal lobe • Inferior frontal regions guide perception in discrepant environments Humans combine multisensory information sharing a common cause while avoiding distraction from irrelevant sources. Cao et al. show how this flexible sensory inference is guided by prefrontal cortex and unfolds as sequential neural computations comprising functionally distinct multisensory representations. [ABSTRACT FROM AUTHOR]