Search

We describe a new unified model to explain both binocular fusion and depth perception, over a broad range of depths. At each location, the model consists of an array of paired spatial frequency filters, with different relative horizontal shifts (position disparity) and interocular phase disparities of 0, 90, ±180, or -90°. The paired filters with different spatial profiles (non-zero phase disparity) compute interocular misalignment and provide phase-disparity energy (binocular fusion energy) to drive selection of the appropriate filters along the position disparity space until the misalignment is eliminated and sensory fusion is achieved locally. The paired filters with identical spatial profiles (0 phase disparity) compute the position-disparity energy. After sensory fusion, the combination of position and possible residual phase disparity energies is calculated for binocular depth perception. Binocular fusion occurs at multiple scales following a coarse-to-fine process. At a given location, the apparent depth is the weighted sum of fusion shifts combined with residual phase disparity in all spatial-frequency channels, and the weights depend on stimulus spatial frequency and stimulus contrast. To test the theory, we measured disparity minimum and maximum thresholds (Dmin and Dmax) at three spatial frequencies and with different intraocular contrast levels. The stimuli were Random-Gabor-Patch (RGP) stereograms consisting of Gabor patches with random positions and phases, but with a fixed spatial frequency. The two eyes viewed identical arrays of patches except that one eye's array could be shifted horizontally and could differ in contrast. Our experiments and modeling reveal two contrast normalization mechanisms: (1) Energy Normalization (EN): Binocular energy is normalized with monocular energy after the site of binocular combination. This predicts constant Dmin thresholds when varying stimulus contrast in the two eyes; (2) DSKL model Interocular interactions: Monocular contrasts are normalized before the binocular combination site through interocular contrast gain-control and gain-enhancement mechanisms. This predicts contrast dependent Dmax thresholds. We tested a range of models and found that a model consisting of a second-order pathway with DSKL interocular interactions and a first-order pathway with EN at each spatial-frequency band can account for both the Dmin and Dmax data very well. Simulations show that the model makes reasonable predictions of suprathreshold depth perception.

IntroductionEither non-invasive somatotopic or substitute sensory feedback is capable of conveying a single modality of sensory information from prosthetic hands to amputees. However, the neurocognitive ability of amputees to integrate multi-modality sensory information for functional discrimination is unclear. The purpose of this study was to assess the fusion of non-invasive somatotopic tactile and substitute aperture feedbacks for profile perception of multiple physical features during grasping objects.MethodsTwo left transradial amputees with somatotopic evoked tactile sensation (ETS) of five fingers participated in the study. The tactile information of prosthetic hand was provided to amputees by the ETS feedback elicited on the stump projected finger map. Hand aperture information was conveyed to amputees with substitute electrotactile stimulation on the forearm or upper arm. Two types of sensory feedback were integrated to a commercial prosthetic hand. The efficacy of somatotopic ETS feedback on object length identification task was assessed with or without substitute aperture stimulation. The object size identification task was utilized to assess how ETS stimulation at the stump may affect aperture perception with stimulation on the ipsilateral upper arm or forearm. Finally, the task of identifying combined length and size was conducted to evaluate the ability of amputees to integrate the dual modalities of sensory feedback for perceiving profile features.ResultsThe study revealed that amputee subjects can effectively integrate the ETS feedback with electrotactile substitutive feedback for object profile discrimination. Specifically, ETS was robust to provide object length information with electrotactile stimulation at either the forearm or upper arm. However, electrotactile stimulation at the upper arm for aperture perception was less susceptible to the interference of ETS stimulation than at the forearm.DiscussionAmputee subjects are able to combine somatotopic ETS and aperture feedbacks for identifying multi-dimensional features in object profiling. The two sensory streams of information can be fused effectively without mutual interference for functional discrimination.

The lack of intuitive and active human–robot interaction makes it difficult to use upper-limb-assistive devices. In this paper, we propose a novel learning-based controller that intuitively uses onset motion to predict the desired end-point position for an assistive robot. A multi-modal sensing system comprising inertial measurement units (IMUs), electromyographic (EMG) sensors, and mechanomyography (MMG) sensors was implemented. This system was used to acquire kinematic and physiological signals during reaching and placing tasks performed by five healthy subjects. The onset motion data of each motion trial were extracted to input into traditional regression models and deep learning models for training and testing. The models can predict the position of the hand in planar space, which is the reference position for low-level position controllers. The results show that using IMU sensor with the proposed prediction model is sufficient for motion intention detection, which can provide almost the same prediction performance compared with adding EMG or MMG. Additionally, recurrent neural network (RNN)-based models can predict target positions over a short onset time window for reaching motions and are suitable for predicting targets over a longer horizon for placing tasks. This study's detailed analysis can improve the usability of the assistive/rehabilitation robots. [ABSTRACT FROM AUTHOR]

Purpose: To analyze the efficacy of fusional vergence therapy (FVT) in management of consecutive esotropia with diplopia after intermittent exotropia (IXT) surgery. The current study is carried on how FVT affects the duration of treatment, sensory fusion, and exotropic drift. Methods: This was a retrospective study for the medical record of 11 patients with consecutive esotropia after IXT surgery of 543 patients over the period of 5 year, with mean surgery age of 9.5 (range: 4–33 y). FVT was planned after minimum 6 weeks of surgery and was considered for maximum 24 weeks. Patients underwent a combination of office-based and home-based FVT. Successful outcome of therapy was considered where diplopia resolves in free space and achieves sensory fusion, stereopsis with no manifest deviation. Results: Record of 543 patients who had horizontal muscle surgery for IXT were identified and reviewed. Records of 11 patients who showed consecutive esotropia of 10 prism diopter (PD) or more with normal retinal correspondence, with or without diplopia complaint, after 6 week of surgery and had undergone vision therapy management were reviewed. A successful outcome of binocular single vision with good sensory and motor fusion with no manifest deviation or prism requirement was achieved with in the mean duration of 4.8 month of therapy. With a mean duration of 4 weeks of therapy, the mean angle of deviation reduced by 53% for distance (17 PD to 8 PD) and 27% for near (11 PD to 8 PD) and mean stereopsis improvement by 80% with 94% patients demonstrating sensory fusion on Bagolini test and 94% of patients having no symptoms of diplopia or squint. Conclusion: With nonsurgical management involving refractive error correction, FVT, and prism, consecutive esotropia was resolved in 74% cases. Management of consecutive esotropia with FVT can result in satisfactory sensory fusion and successful motor alignment.

Purpose: To analyze the efficacy of fusional vergence therapy (FVT) in management of consecutive esotropia with diplopia after intermittent exotropia (IXT) surgery. The current study is carried on how FVT affects the duration of treatment, sensory fusion, and exotropic drift.Methods: This was a retrospective study for the medical record of 11 patients with consecutive esotropia after IXT surgery of 543 patients over the period of 5 year, with mean surgery age of 9.5 (range: 4-33 y). FVT was planned after minimum 6 weeks of surgery and was considered for maximum 24 weeks. Patients underwent a combination of office-based and home-based FVT. Successful outcome of therapy was considered where diplopia resolves in free space and achieves sensory fusion, stereopsis with no manifest deviation.Results: Record of 543 patients who had horizontal muscle surgery for IXT were identified and reviewed. Records of 11 patients who showed consecutive esotropia of 10 prism diopter (PD) or more with normal retinal correspondence, with or without diplopia complaint, after 6 week of surgery and had undergone vision therapy management were reviewed. A successful outcome of binocular single vision with good sensory and motor fusion with no manifest deviation or prism requirement was achieved with in the mean duration of 4.8 month of therapy. With a mean duration of 4 weeks of therapy, the mean angle of deviation reduced by 53% for distance (17 PD to 8 PD) and 27% for near (11 PD to 8 PD) and mean stereopsis improvement by 80% with 94% patients demonstrating sensory fusion on Bagolini test and 94% of patients having no symptoms of diplopia or squint.Conclusion: With nonsurgical management involving refractive error correction, FVT, and prism, consecutive esotropia was resolved in 74% cases. Management of consecutive esotropia with FVT can result in satisfactory sensory fusion and successful motor alignment. [ABSTRACT FROM AUTHOR]

Executing agile locomotion requires animals to integrate sensory feedback, often from multiple sources. For example, human gaze is mediated by multiple feedback loops that integrate visual and vestibular information. A central challenge in studying biological feedback loops is that they are nested and dynamically coupled. Here, we develop a framework based on control theory for unraveling nested feedback systems and apply it to study gaze stabilization in the fruit fly (Drosophila). By combining experimental and mathematical methods to manipulate control topologies, we uncovered the role of body-generated mechanosensory feedback nested within visual feedback in the control of head movements. We discovered that visual feedback changed the tuning of head movements across visual motion frequencies whereas mechanosensory feedback damped head movements. Head saccades had slower dynamics when the body was free to move, further pointing to the role of damping via mechanosensory feedback. By comparing head responses between self-generated and externally generated body motion, we revealed a nonlinear gating of mechanosensory feedback that is motor-context dependent. Altogether, our findings reveal the role of nested feedback loops in flies and uncover mechanisms that reconcile differences in head kinematics between body-free and body-fixed flies. Our framework is generalizable to biological and robotic systems relying on nested feedback control for guiding locomotion.

The lack of intuitive and active human–robot interaction makes it difficult to use upper-limb-assistive devices. In this paper, we propose a novel learning-based controller that intuitively uses onset motion to predict the desired end-point position for an assistive robot. A multi-modal sensing system comprising inertial measurement units (IMUs), electromyographic (EMG) sensors, and mechanomyography (MMG) sensors was implemented. This system was used to acquire kinematic and physiological signals during reaching and placing tasks performed by five healthy subjects. The onset motion data of each motion trial were extracted to input into traditional regression models and deep learning models for training and testing. The models can predict the position of the hand in planar space, which is the reference position for low-level position controllers. The results show that using IMU sensor with the proposed prediction model is sufficient for motion intention detection, which can provide almost the same prediction performance compared with adding EMG or MMG. Additionally, recurrent neural network (RNN)-based models can predict target positions over a short onset time window for reaching motions and are suitable for predicting targets over a longer horizon for placing tasks. This study’s detailed analysis can improve the usability of the assistive/rehabilitation robots.

In the modern world; technology is the asset of the development. There is demand of automation in every sector. Such type of autonomous applications requires inputs from multiple sensory resources and based on these inputs it processes and deliver intelligent decision i.e. output. The method used in proposed system focuses on collection of sensory information and few concluding parameters derivations from data which will help to decide taking best decisions for autonomous applications. Drawing conclusion based on only one aspect is not effective as compared to conclusions taken from many other heterogeneous sensors parameters. This system fulfills parameters such as accuracy, less request time and response time, performance, reliability etc. This paper explains safe and secure system which uses unsupervised algorithm that works on real time data and mixes the information from various resources. The main interest is by taking intelligent decision based on reduction of unreliability and disordered sensory data. Women being a major victim in today's world with their entry in almost each field their safety is a priority and with our concept and algorithm we have illustrated it as a system for sensory fusion with GPS enabled double security feature which will help a women when she feels insecure or harassed or in a difficult situation. System has smart feature to ensure complete safety of women even in dangerous situation where she cannot turn on the trigger. The system comprises of ATmega328P microcontroller, GPS, LCD display, fingerprint module, GPS, GSM etc. for proper and accurate working of the system to provide reliable, accurate and quickservice which efficiently work for safety of women. [ABSTRACT FROM AUTHOR]

We propose a novel controller for powered prosthetic arms, where fused EMG and gaze data predict the desired end-point for a full arm prosthesis, which could drive the forward motion of individual joints. We recorded EMG, gaze, and motion-tracking during pick-and-place trials with 7 able-bodied subjects. Subjects positioned an object above a random target on a virtual interface, each completing around 600 trials. On average across all trials and subjects gaze preceded EMG and followed a repeatable pattern that allowed for prediction. A computer vision algorithm was used to extract the initial and target fixations and estimate the target position in 2D space. Two SVRs were trained with EMG data to predict the x- and y- position of the hand; results showed that the y-estimate was significantly better than the x-estimate. The EMG and gaze predictions were fused using a Kalman Filter-based approach, and the positional error from using EMG-only was significantly higher than the fusion of EMG and gaze. The final target position Root Mean Squared Error (RMSE) decreased from 9.28 cm with an EMG-only prediction to 6.94 cm when using a gaze-EMG fusion. This error also increased significantly when removing some or all arm muscle signals. However, using fused EMG and gaze, there were no significant difference between predictors that included all muscles, or only a subset of muscles. [ABSTRACT FROM AUTHOR]

The work presents real-time data acquisition and monitoring for environmental sensors. This paper presents data fetching and processing implemented by modern approaches of Python. A functional diagram of a materialized system and algorithmic diagrams for Python 3 implemented programs are given. The functionality of the programmed system has been checked by testing the implemented program structures and performing experiments under different environmental conditions with actual results obtained. [ABSTRACT FROM AUTHOR]

Aims: Bagolini striated lenses are a useful test of binocular vision, but the variations in the striations (frequency and thickness) can impact on the perceived image. Also, responses can be difficult to interpret in young children. Therefore, the aims of this project were to evaluate the impact of striation frequency and the addition of coloured filters on subjective responses. Methods: Three sets of striated lenses were made (small, medium and large striations), each produced in two forms (both lenses clear, or with a red and blue lens). Also, Bagolini glasses (Optiker Ryser) were used, with and without the addition of red and blue filters. Subjects were asked to report what they perceived, with subsequent questions regarding the number and length of lines. Results: Forty-two adult subjects were tested, with uniocular VA ranging from –0.18 to 1.10 logMAR (mean 0.08 ± 0.25). The number of lines seen when varying the line thickness did not vary between coloured and clear lenses (post-hoc analysis following ANOVA, p > 0.1 in all comparisons). Adding red/blue filters to the original Bagolini glasses did not alter the rates of subjects perceiving a cross (Chi-square, p = 0.8). However, the laser-cut lenses produced a significantly shorter light streak than the original lenses (One-way ANOVA, p < 0.001), but the colour of the filters made no difference to the length of streak perceived (Tukey’s Test, p = 0.20). Conclusions: The addition of coloured filters did not impact on the responses given to the original or laser-cut lenses, suggesting this modification may aid responses in children. However, further evaluation is required with finer striations and thinner lenses to improve the visibility of the lines.

Introduction: Either non-invasive somatotopic or substitute sensory feedback is capable of conveying a single modality of sensory information from prosthetic hands to amputees. However, the neurocognitive ability of amputees to integrate multi-modality sensory information for functional discrimination is unclear. The purpose of this study was to assess the fusion of non-invasive somatotopic tactile and substitute aperture feedbacks for profile perception of multiple physical features during grasping objects., Methods: Two left transradial amputees with somatotopic evoked tactile sensation (ETS) of five fingers participated in the study. The tactile information of prosthetic hand was provided to amputees by the ETS feedback elicited on the stump projected finger map. Hand aperture information was conveyed to amputees with substitute electrotactile stimulation on the forearm or upper arm. Two types of sensory feedback were integrated to a commercial prosthetic hand. The efficacy of somatotopic ETS feedback on object length identification task was assessed with or without substitute aperture stimulation. The object size identification task was utilized to assess how ETS stimulation at the stump may affect aperture perception with stimulation on the ipsilateral upper arm or forearm. Finally, the task of identifying combined length and size was conducted to evaluate the ability of amputees to integrate the dual modalities of sensory feedback for perceiving profile features., Results: The study revealed that amputee subjects can effectively integrate the ETS feedback with electrotactile substitutive feedback for object profile discrimination. Specifically, ETS was robust to provide object length information with electrotactile stimulation at either the forearm or upper arm. However, electrotactile stimulation at the upper arm for aperture perception was less susceptible to the interference of ETS stimulation than at the forearm., Discussion: Amputee subjects are able to combine somatotopic ETS and aperture feedbacks for identifying multi-dimensional features in object profiling. The two sensory streams of information can be fused effectively without mutual interference for functional discrimination., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. The author(s) declared that they were an editorial board member of Frontiers, at the time of submission. This had no impact on the peer review process and the final decision., (Copyright © 2023 Zhang, Chou, Hao, Li, Yu and Lan.)

Restoring locomotion functionality of transfemoral amputees is essential for early rehabilitation treatment and for preserving mobility and independence in daily life. Research in wearable robotics fostered the development of innovative active mechatronic lower-limb prostheses designed with the goal to reduce the cognitive and physical effort of lower-limb amputees in rehabilitation and daily life activities. To ensure benefits to the users, active mechatronic prostheses are expected to be aware of the user intention and properly interact in a closed human-in-the-loop paradigm. In the state of the art various cognitive interfaces have been proposed to online decode the user's intention. Electromyography in combination with mechanical sensing such as inertial or pressure sensors is a widely adopted solution for driving active mechatronic prostheses. In this framework, researchers also explored targeted muscles re-innervation for an objective-oriented surgical amputation promoting wider usability of active prostheses. However, information kept by the neural component of the cognitive interface deteriorates in a prolonged use scenario due to electrodes-related issues, thereby undermining the correct functionality of the active prosthesis. The objective of this work is to present a novel controller for an active transfemoral prosthesis based on whole body awareness relying on a wireless distributed non-invasive sensory apparatus acting as cognitive interface. A finite-state machine controller based on signals monitored from the wearable interface performs subject-independent intention detection of functional tasks such as ground level walking, stair ascent, and sit-to-stand maneuvres and their main sub-phases. Experimental activities carried out with four transfemoral amputees (among them one dysvascular) demonstrated high reliability of the controller capable of providing 100% accuracy rate in treadmill walking even for weak subjects and low walking speeds. The minimum success rate was of 94.8% in performing sit-to-stand tasks. All the participants showed high confidence in using the transfemoral active prosthesis even without training period thanks to intuitiveness of the whole body awareness controller.

The direction of the Earth’s magnetic field is used as a reference vector to determine the heading in orientation estimation with wearable sensors. However, the magnetic field strength is weak and can be easily disturbed in the vicinity of ferromagnetic materials, which may result in inaccurate estimate of orientation. This paper presents a novel method for estimating and compensating for magnetic disturbances. The compensation algorithm is implemented within a kinematic-based extended Kalman filter and is based on an assessment of the magnetic disturbance and the change of orientation in each time step. The proposed algorithm was experimentally validated by measuring the orientation of a simple mechanical system with three degrees of freedom in an artificially disturbed magnetic field. The results of the experimental evaluation show that an Kalman filter algorithm that incorporates compensating for magnetic disturbances is capable of estimating the orientation with moderate error (the absolute median errors \Delta \widetilde \theta X,\Delta \widetilde \theta Z < {2}^{\circ } , \Delta \widetilde \theta Y < {9}^{\circ } ) when the Earth’s magnetic field is disturbed by magnetic disturbance with a magnitude equal to twice the magnitude of the Earth’s own magnetic field in different directions. [ABSTRACT FROM PUBLISHER]

Restoring locomotion functionality of transfemoral amputees is essential for early rehabilitation treatment and for preserving mobility and independence in daily life. Research in wearable robotics fostered the development of innovative activemechatronic lower-limb prostheses designed with the goal to reduce the cognitive and physical effort of lower-limb amputees in rehabilitation and daily life activities. To ensure benefits to the users, active mechatronic prostheses are expected to be aware of the user intention and properly interact in a closed human-in-the-loop paradigm. In the state of the art various cognitive interfaces have been proposed to online decode the user's intention. Electromyography in combination with mechanical sensing such as inertial or pressure sensors is a widely adopted solution for driving active mechatronic prostheses. In this framework, researchers also explored targeted muscles re-innervation for an objective-oriented surgical amputation promoting wider usability of active prostheses. However, information kept by the neural component of the cognitive interface deteriorates in a prolonged use scenario due to electrodes-related issues, thereby undermining the correct functionality of the active prosthesis. The objective of this work is to present a novel controller for an active transfemoral prosthesis based on whole body awareness relying on a wireless distributed non-invasive sensory apparatus acting as cognitive interface. A finite-state machine controller based on signals monitored from the wearable interface performs subject-independent intention detection of functional tasks such as ground level walking, stair ascent, and sit-to-stand maneuvres and their main sub-phases. Experimental activities carried out with four transfemoral amputees (among them one dysvascular) demonstrated high reliability of the controller capable of providing 100%accuracy rate in treadmill walking even for weak subjects and low walking speeds. The minimum success rate was of 94.8% in performing sit-to-stand tasks. All the participants showed high confidence in using the transfemoral active prosthesis even without training period thanks to intuitiveness of the whole body awareness controller. [ABSTRACT FROM AUTHOR]

This work proposes a new methodology for the detection of discontinuities in the weld bead applied in Shielded Metal ArcWelding (SMAW) processes. The detection system is based on two sensors--a microphone and piezoelectric--that acquire acoustic emissions generated during the welding. The feature vectors extracted from the sensor dataset are used to construct classifier models. The approaches based on Artificial Neural Network (ANN) and Support Vector Machine (SVM) classifiers are able to identify with a high accuracy the three proposed weld bead classes: desirable weld bead, shrinkage cavity and burn through discontinuities. Experimental results illustrate the system's high accuracy, greater than 90% for each class. A novel Hierarchical Support Vector Machine (HSVM) structure is proposed to make feasible the use of this system in industrial environments. This approach presented 96.6% overall accuracy. Given the simplicity of the equipment involved, this system can be applied in the metal transformation industries. [ABSTRACT FROM AUTHOR]

This work investigates an over-arching question: how can an immersive virtual environment be connected with its intelligent physical counterpart to allow for a more efficient man-machine collaboration. To this end, an immersive user interface for the purpose of robot tele-operation is designed. A large amount of sensory data is utilized to build models of the world and its inhabitants in a way that is intuitive to the operator and accurately represents the robot’s real-world state and environment. The game client is capable of handling multiple users, much like a traditional multiplayer game, while visualizing multiple robotic agents operating within the real world. The proposed Virtual Environment for Tele-Operation (VETO) architecture is a tele-operation system that provides a feature-rich framework to implement robotic agents into an immersive end-user game interface. Game levels are generated dynamically on a Graphic Processing Unit or GPU-accelerated server based on real-world sensor data from the robotic agents. A set of user studies are conducted to validate the performance of the proposed architecture compared to traditional tele-robotic applications. The experimental results show significant improvements in both task completion time and task completion rate over traditional tools.

Aim: To determine whether the level of sensory fusion in normal participants is affected significantly by fixation light intensity. Methods: Seventeen normal adult participants had sensory fusion measurements taken using the Sbisa bar, combined with Bagolini glasses, as a control for binocular single vision, whilst fixating lights of three different intensities: 150 lux, 250 lux and 350 lux. Data were analysed using Friedman’s test and the Wilcoxon matched pairs signed ranks test. Results: Seventeen participants were tested (mean 35.3 years, range 23–60 years). Friedman’s test showed at least two of the light intensities had a statistically significant effect upon sensory fusion results ( p = 0.011). Wilcoxon matched pairs signed rank testing found no statistically significant difference in levels of sensory fusion when fixating the 150 lux and 250 lux fixation lights ( p = 0.194). There was a statistically significant difference in the level of sensory fusion when fixating the 150 lux and 350 lux lights ( p = 0.011) and the 250 lux and 350 lux lights ( p = 0.017). Conclusion: Changing the intensity of the fixation light from 150 lux to 350 lux and from 250 lux to 350 lux significantly affected the level of sensory fusion measured. The higher the intensity of the light, the greater the filter needed to break fusion. The clinical significance of these results is considered.

Deep Belief Networks (DBNs) have recently shown impressive performance on a broad range of classification problems. Their generative properties allow better understanding of the performance, and provide a simpler solution for sensor fusion tasks. However, because of their inherent need for feedback and parallel update of large numbers of units, DBNs are expensive to implement on serial computers. This paper proposes a method based on the Siegert approximation for Integrate-and-Fire neurons to map an offline-trained DBN onto an efficient event-driven spiking neural network suitable for hardware implementation. The method is demonstrated in simulation and by a real-time implementation of a 3-layer network with 2694 neurons used for visual classification of MNIST handwritten digits with input from a 128x128 Dynamic Vision Sensor (DVS) silicon retina, and sensory-fusion using additional input from a 64-channel AER-EAR silicon cochlea. The system is implemented through the open-source software in the jAER project and runs in real-time on a laptop computer. It is demonstrated that the system can recognize digits in the presence of distractions, noise, scaling, translation and rotation, and that the degradation of recognition performance by using an event-based approach is less than 1%. Recognition is achieved in an average of 5.8 ms after the onset of the presentation of a digit. By cue integration from both silicon retina and cochlea outputs we show that the system can be biased to select the correct digit from otherwise ambiguous input.

Abstract: Measuring the kinematic parameters in unconstrained human motion is becoming crucial for providing feedback information in wearable robotics and sports monitoring. This paper presents a novel sensory fusion algorithm for assessing the orientations of human body segments in long-term human walking based on signals from wearable sensors. The basic idea of the proposed algorithm is to constantly fuse the measured segment's angular velocity and linear acceleration via known kinematic relations between segments. The wearable sensory system incorporates seven inertial measurement units attached to the human body segments and two instrumented shoe insoles. The proposed system was experimentally validated in a long-term walking on a treadmill and on a polygon with stairs simulating different activities in everyday life. The outputs were compared to the reference parameters measured by a stationary optical system. Results show accurate joint angle measurements (error median below 5°) in all evaluated walking conditions with no expressed drift over time. [Copyright &y& Elsevier]

Abstract: Surface electrical potential and observational growth recordings were made of a protoplasmic tube of the slime mould Physarum polycephalum in response to a multitude of stimuli with regards to sensory fusion or multisensory integration. Each stimulus was tested alone and in combination in order to evaluate for the first time the effect that multiple stimuli have on the frequency of streaming oscillation. White light caused a decrease in frequency whilst increasing the temperature and applying a food source in the form of oat flakes both increased the frequency. Simultaneously stimulating P. polycephalum with light and oat flake produced no net change in frequency, while combined light and heat stimuli showed an increase in frequency smaller than that observed for heat alone. When the two positive stimuli, oat flakes and heat, were combined, there was a net increase in frequency similar to the cumulative increases caused by the individual stimuli. Boolean logic gates were derived from the measured frequency change. [Copyright &y& Elsevier]

Flies and other insects use vision to regulate their groundspeed in flight, enabling them to fly in varying wind conditions. Compared with mechanosensory modalities, however, vision requires a long processing delay (~100 ms) that might introduce instability if operated at high gain. Flies also sense air motion with their antennae, but how this is used in flight control is unknown. We manipulated the antennal function of fruit flies by ablating their aristae, forcing them to rely on vision alone to regulate groundspeed. Arista-ablated flies in flight exhibited significantly greater groundspeed variability than intact flies. We then subjected them to a series of controlled impulsive wind gusts delivered by an air piston and experimentally manipulated antennae and visual feedback. The results show that an antenna-mediated response alters wing motion to cause flies to accelerate in the same direction as the gust. This response opposes flying into a headwind, but flies regularly fly upwind. To resolve this discrepancy, we obtained a dynamic model of the fly's velocity regulator by fitting parameters of candidate models to our experimental data. The model suggests that the groundspeed variability of arista-ablated flies is the result of unstable feedback oscillations caused by the delay and high gain of visual feedback. The antenna response drives active damping with a shorter delay (~20 ms) to stabilize this regulator, in exchange for increasing the effect of rapid wind disturbances. This provides insight into flies' multimodal sensory feedback architecture and constitutes a previously unknown role for the antennae. [ABSTRACT FROM AUTHOR]

Deep Belief Networks (DBNs) have recently shown impressive performance on a broad range of classification problems. Their generative properties allow better understanding of the performance, and provide a simpler solution for sensor fusion tasks. However, because of their inherent need for feedback and parallel update of large numbers of units, DBNs are expensive to implement on serial computers. This paper proposes a method based on the Siegert approximation for Integrate-and-Fire neurons to map an offline-trained DBN onto an efficient event-driven spiking neural network suitable for hardware implementation. The method is demonstrated in simulation and by a real-time implementation of a 3-layer network with 2694 neurons used for visual classification of MNIST handwritten digits with input from a 128 × 128 Dynamic Vision Sensor (DVS) silicon retina, and sensoryfusion using additional input from a 64-channel AER-EAR silicon cochlea. The system is implemented through the open-source software in the jAER project and runs in real-time on a laptop computer. It is demonstrated that the system can recognize digits in the presence of distractions, noise, scaling, translation and rotation, and that the degradation of recognition performance by using an event-based approach is less than 1%. Recognition is achieved in an average of 5.8 ms after the onset of the presentation of a digit. By cue integration from both silicon retina and cochlea outputs we show that the system can be biased to select the correct digit from otherwise ambiguous input. [ABSTRACT FROM AUTHOR]

Executing agile locomotion requires animals to integrate sensory feedback, often from multiple sources. For example, human gaze is mediated by multiple feedback loops that integrate visual and vestibular information. A central challenge in studying biological feedback loops is that they are nested and dynamically coupled. Here, we develop a framework based on control theory for unraveling nested feedback systems and apply it to study gaze stabilization in the fruit fly ( Drosophila ). By combining experimental and mathematical methods to manipulate control topologies, we uncovered the role of body-generated mechanosensory feedback nested within visual feedback in the control of head movements. We discovered that visual feedback changed the tuning of head movements across visual motion frequencies whereas mechanosensory feedback damped head movements. Head saccades had slower dynamics when the body was free to move, further pointing to the role of damping via mechanosensory feedback. By comparing head responses between self-generated and externally generated body motion, we revealed a nonlinear gating of mechanosensory feedback that is motor-context dependent. Altogether, our findings reveal the role of nested feedback loops in flies and uncover mechanisms that reconcile differences in head kinematics between body-free and body-fixed flies. Our framework is generalizable to biological and robotic systems relying on nested feedback control for guiding locomotion., Competing Interests: BC, JM No competing interests declared, (© 2022, Cellini and Mongeau.)

Most conventional robots rely on controlling the location of the center of pressure to maintain balance, relying mainly on foot pressure sensors for information. By contrast, humans rely on sensory data from multiple sources, including proprioceptive, visual, and vestibular sources. Several models have been developed to explain how humans reconcile information from disparate sources to form a stable sense of balance. These models may be useful for developing robots that are able to maintain dynamic balance more readily using multiple sensory sources. Since these information sources may conflict, reliance by the nervous system on any one channel can lead to ambiguity in the system state. In humans, experiments that create conflicts between different sensory channels by moving the visual field or the support surface indicate that sensory information is adaptively reweighted. Unreliable information is rapidly down-weighted, then gradually up-weighted when it becomes valid again. Human balance can also be studied by building robots that model features of human bodies and testing them under similar experimental conditions. We implement a sensory reweighting model based on an adaptive Kalman filter in a bipedal robot, and subject it to sensory tests similar to those used on human subjects. Unlike other implementations of sensory reweighting in robots, our implementation includes vision, by using optic flow to calculate forward rotation using a camera (visual modality), as well as a three-axis gyro to represent the vestibular system (non-visual modality), and foot pressure sensors (proprioceptive modality). Our model estimates measurement noise in real time, which is then used to recompute the Kalman gain on each iteration, improving the ability of the robot to dynamically balance. We observe that we can duplicate many important features of postural sway in humans, including automatic sensory reweighting, effects, constant phase with respect to amplitude, and a temporal asymmetry in the reweighting gains. [ABSTRACT FROM AUTHOR]

Purpose: Anisometropia is reported to increase with age; however, the relationship between the development of anisometropia and binocular vision has not been determined. The purpose of this study was to determine the relationship between anisometropia and binocular vision after esotropia (ET) surgery. Methods: The medical records of 51 consecutive patients with ET who had undergone ET surgery were studied. Patients with amblyopia or anisometropia ≥2 diopters (D) before surgery were excluded. The average age of the patients at the time of surgery was 4.5 ± 3.7 years, and the average postoperative period was 5.0 ± 2.2 years. The refractive errors (spherical equivalent) in both eyes were measured preoperatively and postoperatively. Binocular visual function was evaluated with a modified Worth-4 dot test. Results: At the final visit, average anisometropia was 0.98 ± 1.30 D, which was significantly greater than the preoperative value of 0.36 ± 0.46 D ( P < 0.001). Anisometropia ≥2 D was observed in ten patients (23%) postoperatively. One patient was able to fuse a target of ≥2° (10%) in the anisometropia ≥2 D group, significantly fewer than the 13 patients (32%) in the anisometropia <2.0 D group ( P = 0.02). Conclusions: Anisometropia may develop in patients with poor sensory fusion after surgery for esotropia. [ABSTRACT FROM AUTHOR]