Search

Visually inferring material properties is crucial for many tasks, yet poses significant computational challenges for biological vision. Liquids and gels are particularly challenging due to their extreme variability and complex behaviour. We reasoned that measuring and modelling viscosity perception is a useful case study for identifying general principles of complex visual inferences. In recent years, artificial Deep Neural Networks (DNNs) have yielded breakthroughs in challenging real-world vision tasks. However, to model human vision, the emphasis lies not on best possible performance, but on mimicking the specific pattern of successes and errors humans make. We trained a DNN to estimate the viscosity of liquids using 100.000 simulations depicting liquids with sixteen different viscosities interacting in ten different scenes (stirring, pouring, splashing, etc). We find that a shallow feedforward network trained for only 30 epochs predicts mean observer performance better than most individual observers. This is the first successful image-computable model of human viscosity perception. Further training improved accuracy, but predicted human perception less well. We analysed the network's features using representational similarity analysis (RSA) and a range of image descriptors (e.g. optic flow, colour saturation, GIST). This revealed clusters of units sensitive to specific classes of feature. We also find a distinct population of units that are poorly explained by hand-engineered features, but which are particularly important both for physical viscosity estimation, and for the specific pattern of human responses. The final layers represent many distinct stimulus characteristics-not just viscosity, which the network was trained on. Retraining the fully-connected layer with a reduced number of units achieves practically identical performance, but results in representations focused on viscosity, suggesting that network capacity is a crucial parameter determining whether artificial or biological neural networks use distributed vs. localized representations., Competing Interests: The authors have declared that no competing interests exist.

Age-related changes in auditory and visual perception have an impact on the quality of life. It has been debated how perceptual organization is influenced by advancing age. From the neurochemical perspective, we investigated age effects on auditory and visual bistability. In perceptual bistability, a sequence of sensory inputs induces spontaneous switching between different perceptual objects. We used different modality tasks of auditory streaming and visual plaids. Young and middle-aged participants (20-60years) were instructed to indicate by a button press whenever their perception changed from one stable state to the other. The number of perceptual switches decreased with participants' ages. We employed magnetic resonance spectroscopy to measure non-invasively concentrations of the inhibitory neurotransmitter (γ-aminobutyric acid, GABA) in the brain regions of interest. When participants were asked to voluntarily modulate their perception, the amount of effective volitional control was positively correlated with the GABA concentration in the auditory and motion-sensitive areas corresponding to each sensory modality. However, no correlation was found in the prefrontal cortex and anterior cingulate cortex. In addition, effective volitional control was reduced with advancing age. Our results suggest that sequential scene analysis in auditory and visual domains is influenced by both age-related and neurochemical factors., (Copyright © 2017 The Author(s). Published by Elsevier Ltd.. All rights reserved.)

Perceptual organisation must select one interpretation from several alternatives to guide behaviour. Computational models suggest that this could be achieved through an interplay between inhibition and excitation across competing types of neural population coding for each interpretation. Here, to test for such models, we used magnetic resonance spectroscopy to measure non-invasively the concentrations of inhibitory γ-aminobutyric acid (GABA) and excitatory glutamate-glutamine (Glx) in several brain regions. Human participants first performed auditory and visual multistability tasks that produced spontaneous switching between percepts. Then, we observed that longer percept durations during behaviour were associated with higher GABA/Glx ratios in the sensory area coding for each modality. When participants were asked to voluntarily modulate their perception, a common factor across modalities emerged: the GABA/Glx ratio in the posterior parietal cortex tended to be positively correlated with the amount of effective volitional control. Our results provide direct evidence implicating that the balance between neural inhibition and excitation within sensory regions resolves perceptual competition. This powerful computational principle appears to be leveraged by both audition and vision, implemented independently across modalities, but modulated by an integrated control process.

Visual estimation of the material and shape of an object from a single image includes a hard ill-posed computational problem. However, in our daily life we feel we can estimate both reasonably well. The neural computation underlying this ability remains poorly understood. Here we propose that the human visual system uses different aspects of object images to separately estimate the contributions of the material and shape. Specifically, material perception relies mainly on the intensity gradient magnitude information, while shape perception relies mainly on the intensity gradient order information. A clue to this hypothesis was provided by the observation that luminance-histogram manipulation, which changes luminance gradient magnitudes but not the luminance-order map, effectively alters the material appearance but not the shape of an object. In agreement with this observation, we found that the simulated physical material changes do not significantly affect the intensity order information. A series of psychophysical experiments further indicate that human surface shape perception is robust against intensity manipulations provided they do not disturb the intensity order information. In addition, we show that the two types of gradient information can be utilized for the discrimination of albedo changes from highlights. These findings suggest that the visual system relies on these diagnostic image features to estimate physical properties in a distal world., Competing Interests: The authors have declared that no competing interests exist.

We perceive the world as stable and composed of discrete objects even though auditory and visual inputs are often ambiguous owing to spatial and temporal occluders and changes in the conditions of observation. This raises important questions regarding where and how 'scene analysis' is performed in the brain. Recent advances from both auditory and visual research suggest that the brain does not simply process the incoming scene properties. Rather, top-down processes such as attention, expectations and prior knowledge facilitate scene perception. Thus, scene analysis is linked not only with the extraction of stimulus features and formation and selection of perceptual objects, but also with selective attention, perceptual binding and awareness. This special issue covers novel advances in scene-analysis research obtained using a combination of psychophysics, computational modelling, neuroimaging and neurophysiology, and presents new empirical and theoretical approaches. For integrative understanding of scene analysis beyond and across sensory modalities, we provide a collection of 15 articles that enable comparison and integration of recent findings in auditory and visual scene analysis.This article is part of the themed issue 'Auditory and visual scene analysis'., (© 2017 The Author(s).)

Human vision has a remarkable ability to perceive two layers at the same retinal locations, a transparent layer in front of a background surface. Critical image cues to perceptual transparency, studied extensively in the past, are changes in luminance or color that could be caused by light absorptions and reflections by the front layer, but such image changes may not be clearly visible when the front layer consists of a pure transparent material such as water. Our daily experiences with transparent materials of this kind suggest that an alternative potential cue of visual transparency is image deformations of a background pattern caused by light refraction. Although previous studies have indicated that these image deformations, at least static ones, play little role in perceptual transparency, here we show that dynamic image deformations of the background pattern, which could be produced by light refraction on a moving liquid's surface, can produce a vivid impression of a transparent liquid layer without the aid of any other visual cues as to the presence of a transparent layer. Furthermore, a transparent liquid layer perceptually emerges even from a randomly generated dynamic image deformation as long as it is similar to real liquid deformations in its spatiotemporal frequency profile. Our findings indicate that the brain can perceptually infer the presence of "invisible" transparent liquids by analyzing the spatiotemporal structure of dynamic image deformation, for which it uses a relatively simple computation that does not require high-level knowledge about the detailed physics of liquid deformation.

Working memory is linked to the functions of the frontal areas, in which neural activity is mediated by dopaminergic and serotonergic tones. However, there is no consensus regarding how the dopaminergic and serotonergic systems influence working memory subprocesses. The present study used an imaging genetics approach to examine the interaction between neurochemical functions and working memory performance. We focused on functional polymorphisms of the catechol-O-methyltransferase (COMT) Val(158)Met and serotonin 2A receptor (HTR2A) -1438G/A genes, and devised a delayed recognition task to isolate the encoding, retention, and retrieval processes for visual information. The COMT genotypes affected recognition accuracy, whereas the HTR2A genotypes were associated with recognition response times. Activations specifically related to working memory were found in the right frontal and parietal areas, such as the middle frontal gyrus (MFG), inferior frontal gyrus (IFG), anterior cingulate cortex (ACC), and inferior parietal lobule (IPL). MFG and ACC/IPL activations were sensitive to differences between the COMT genotypes and between the HTR2A genotypes, respectively. Structural equation modeling demonstrated that stronger connectivity in the ACC-MFG and ACC-IFG networks is related to better task performance. The behavioral and fMRI results suggest that the dopaminergic and serotonergic systems play different roles in the working memory subprocesses and modulate closer cooperation between lateral and medial frontal activations.

In contrast to the classical findings of lightness constancy, recent psychophysical studies show the strong dependency of the perceived reflectance of a surface on the structure of the natural illumination. The present study examined this inconstancy for systematic variations in the light field and an image-based explanation for it. Observers matched the specular and diffuse reflectance of a three-dimensional object in a complex scene under a fixed light field to that in the scene under different light fields with variable mean, contrast, and gamma. For the both specular and diffuse components, the matched reflectance was relatively constant against changes in the mean illuminance but varied extensively with changes in the contrast and gamma of the light field. We found that the matching data were well predicted by the similarity of the subband histograms of the images. The results support the notion that early spatial filtering can provide a unified account of both the constancy in the perceived surface reflectance against mean illuminance and the inconstancy for higher-order illumination statistics., (Copyright © 2011 Elsevier Ltd. All rights reserved.)

Background: When simultaneous visual events appear to occur at different times, the discrepancy has generally been ascribed to time differences in neural transmission or cortical processing that lead to asynchronous awareness of the events., Results: We found, however, that an apparent delay of changes in motion direction relative to synchronous color changes occurs only for rapid alternations, and this delay is not accompanied by a difference in reaction time. We also found that perceptual asynchrony depends on the temporal structure of the stimuli (transitions [first-order temporal change] versus turning points [second-order temporal change]) rather than the attribute type (color versus motion)., Conclusions: We propose that the perception of the relative time of events is based on the relationship of representations of temporal pattern that we term time markers. We conclude that the perceptual asynchrony effects studied here do not reflect differential neural delays for different attributes; rather, they arise from a faulty correspondence match between color transitions and position transitions (motion), which in turn results from a difficulty in detecting turning points (direction reversals) and a preference for matching markers of the same type.

We analyzed how the visual response to orientation modulation in texture patterns varied as a function of the magnitude of orientation contrast. Using a contrast-discrimination technique, we measured threshold increments of orientation contrast (the orientation contrast required for discriminating between two textures) at various pedestal-orientation contrasts. The orientation-contrast-response function estimated for a step-orientation contrast, which produces a vivid percept of surface boundaries, saturated at approximately 30 degrees (experiment 1). The saturation was still evident even when the strength of the step-orientation contrast was reduced by orientation noise (experiment 2), but no strong saturation was found for textures that did not produce a vivid percept of surface boundaries (experiment 3). These results are consistent with the notion that orientation-based texture segregation involves the generation of a neural representation of the surface boundary whose strength is nearly independent of the magnitude of orientation contrast.

Recent studies have led to a better understanding of the mechanisms of object vision in the inferotemporal cortex. In particular, a columnar organization has been revealed, and a short-term change of responsiveness has been proposed as a neural correlate of working memory.