Search

The ability to read others' intentions is crucial when pooling knowledge to form a collective decision. Decision-making improves when communication is allowed through words or touch. Coucke et al. show that visual information communicated through actions can convey not only a decision but also decision confidence, improving collective decision-making., Competing Interests: Declaration of interests No interests are declared., (Copyright © 2024 Elsevier Ltd. All rights reserved.)

The neural and computational mechanisms underlying visual motion perception have been extensively investigated over several decades, but little attempt has been made to measure and analyze, how human observers perceive the map of motion vectors, or optical flow, in complex naturalistic scenes. Here, we developed a psychophysical method to assess human-perceived motion flows using local vector matching and a flash probe. The estimated perceived flow for naturalistic movies agreed with the physically correct flow (ground truth) at many points, but also showed consistent deviations from the ground truth (flow illusions) at other points. Comparisons with the predictions of various computational models, including cutting-edge computer vision algorithms and coordinate transformation models, indicated that some flow illusions are attributable to lower-level factors such as spatiotemporal pooling and signal loss, while others reflect higher-level computations, including vector decomposition. Our study demonstrates a promising data-driven psychophysical paradigm for an advanced understanding of visual motion perception., Competing Interests: T.F. and S.N. were employed by Nippon Telegraph and Telephone Corporation. The remaining author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (© 2023 The Author(s).)

Attention levels fluctuate during the course of daily activities. However, factors underlying sustained attention are still unknown. We investigated mechanisms of sustained attention using psychological, neuroimaging, and neurochemical approaches. Participants were scanned with functional magnetic resonance imaging (fMRI) while performing gradual-onset, continuous performance tasks (gradCPTs). In gradCPTs, narrations or visual scenes gradually changed from one to the next. Participants pressed a button for frequent Go trials as quickly as possible and withheld responses to infrequent No-go trials. Performance was better for the visual gradCPT than for the auditory gradCPT, but the 2 were correlated. The dorsal attention network was activated during intermittent responses, regardless of sensory modality. Reaction-time variability of gradCPTs was correlated with signal changes (SCs) in the left fronto-parietal regions. We also used magnetic resonance spectroscopy (MRS) to measure levels of glutamate-glutamine (Glx) and γ-aminobutyric acid (GABA) in the left prefrontal cortex (PFC). Glx levels were associated with performance under undemanding situations, whereas GABA levels were related to performance under demanding situations. Combined fMRI-MRS results demonstrated that SCs of the left PFC were positively correlated with neurometabolite levels. These findings suggest that a neural balance between excitation and inhibition is involved in attentional fluctuations and brain dynamics., (© The Author(s) 2023. Published by Oxford University Press.)

Understanding temporally attention fluctuations can benefit scientific knowledge and real-life applications. Temporal attention studies have typically used the reaction time (RT), which can be measured only after a target presentation, as an index of attention level. We have proposed the Micro-Pupillary Unrest Index (M-PUI) based on pupillary fluctuation amplitude to estimate RT before the target presentation. However, the kind of temporal attention effects that the M-PUI reflects remains unclear. We examined if the M-PUI shows two types of temporal attention effects initially reported for RTs in the variable foreperiod tasks: the variable foreperiod effect (FP effect) and the sequential effect (SE effect). The FP effect refers to a decrease in the RT due to an increase in the foreperiod of the current trial, whereas the SE effect refers to an increase in the RT in the early part of the foreperiod of the current trial due to an increase in the foreperiod of the previous trial. We used a simple reaction task with the medium-term variable foreperiods (Psychomotor Vigilance Task) and found that the M-PUI primarily reflects the FP effect. Inter-individual analyses showed that the FP effect on the M-PUI, unlike other eye movement indices, is correlated with the FP effect on RT. These results suggest that the M-PUI is a potentially powerful tool for investigating temporal attention fluctuations for a partly unpredictable target., Competing Interests: The authors have read the journal’s policy and have the following competing interests: The authors, JY, HT, MY, KM, and HO, are paid employees of NTT. NTT has signed a joint research agreement with TK and provided a research grant. The authors would like to declare the following patents/patent applications associated with this research: PCT/JP2020/043246 and PCT/JP2021/008837. NTT has a pending patent (in Japan, and possibly in the United States, Europe, and China) for estimating task performance using the amplitude of pupillary fluctuations. This does not alter our adherence to PLOS ONE policies on sharing data and materials.

Background: A brief measure of dispositional mindfulness is important for applied research on mindfulness. Although short forms of the Five Facet Mindfulness Questionnaire (FFMQ), which measures the five aspects of mindfulness (i.e., observing, describing, acting with awareness, non-judging, and non-reactivity), have been developed worldwide, the validity and reliability of the Japanese version has not been examined. This study aims to examine the validity and reliability of the 24-item and 15-item versions of the FFMQ in Japan, which are the most widely used versions worldwide., Methods: Online surveys were conducted for 889 adults in Japan through an online survey company using self-reported questionnaires including the FFMQ to confirm the factor structure and validity. To examine construct validity, we examined the relationship between the short form of FFMQ and mind wandering, interoceptive awareness, experiential avoidance, cognitive fusion, openness, neuroticism, self-compassion, depression, and anxiety, which have been theoretically or empirically shown to be related to mindfulness. In addition, 137 adults responded to the FFMQ again, after four weeks, for the test-retest reliability., Results: The correlated five-factor and four-factor (excluding observing) models and the higher-order factor hierarchical model did not show sufficient goodness of fit, while the 24-item version showed acceptable fit when uncorrelated method factors loaded on by the positive and negative (reverse-scored) items were added. However, the 15-item version did not show acceptable fits for any of the models. Regarding reliability, the 24-item version showed acceptable values. In terms of the relationship between the original and the shortened version of the FFMQ, the 24-item version shared approximately 80% of the variance with the original one. In addition, although the wording effects of positive and negative items seemed to affect the correlations between the FFMQ and the other scales, the associations with related concepts were as predicted generally, supporting the construct validity of the short form of the FFMQ., Conclusion: In Japan, the 24-item version of FFMQ showed acceptable validity and reliability similar to the original version, and we recommend that the 24-item version be used., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2022 Takahashi, Saito, Fujino, Sato and Kumano.)

The global virtual reality (VR) market is significantly expanding and being challenged with an increased demand owing to COVID-19. Unfortunately, VR is not useful for everyone due to large interindividual variability existing in VR suitability. To understand the neurobiological basis of this variability, we obtained neural structural and functional data from the participants using 3T magnetic resonance imaging. The participants completed one of two tasks (sports training or cognitive task) using VR, which differed in the time scale (months/minutes) and domain (motor learning/attention task). Behavioral results showed that some participants improved their motor skills in the real world after 1-month training in the virtual space or obtained high scores in the 3D attention task (high suitability for VR), whereas others did not (low suitability for VR). Brain structure analysis revealed that the structural properties of the superior and inferior parietal lobes contain information that can predict an individual's suitability for VR., (© 2021. The Author(s).)

Our daily activities require vigilance. Therefore, it is useful to externally monitor and predict our vigilance level using a straightforward method. It is known that the vigilance level is linked to pupillary fluctuations via Locus Coeruleus and Norepinephrine (LC-NE) system. However, previous methods of estimating long-term vigilance require monitoring pupillary fluctuations at rest over a long period. We developed a method of predicting the short-term vigilance level by monitoring pupillary fluctuation for a shorter period consisting of several seconds. The LC activity also fluctuates at a timescale of seconds. Therefore, we hypothesized that the short-term vigilance level could be estimated using pupillary fluctuations in a short period and quantified their amplitude as the Micro-Pupillary Unrest Index (M-PUI). We found an intra-individual trial-by-trial positive correlation between Reaction Time (RT) reflecting the short-term vigilance level and M-PUI in the period immediately before the target onset in a Psychomotor Vigilance Task (PVT). This relationship was most evident when the fluctuation was smoothed by a Hanning window of approximately 50 to 100 ms (including cases of down-sampled data at 100 and 50 Hz), and M-PUI was calculated in the period up to one or two seconds before the target onset. These results suggest that M-PUI can monitor and predict fluctuating levels of vigilance. M-PUI is also useful for examining pupillary fluctuations in a short period for elucidating the psychophysiological mechanisms of short-term vigilance., Competing Interests: The authors have read the journal’s policy and have the following competing interests: The authors, JY, HT, MY, KM, HK, HO, and HN, are paid employees of NTT. NTT has signed a joint research agreement with TK and provided a research grant. The authors would like to declare the following patents/patent applications associated with this research: PCT/JP2020/043246 and PCT/JP2021/008837. NTT has a pending patent (in Japan, and possibly in the United States, Europe, and China) for estimating task performance using the amplitude of pupillary fluctuations. This does not alter our adherence to PLOS ONE policies on sharing data and materials.

Frisson is characterised by tingling and tickling sensations with positive or negative feelings. However, it is still unknown what factors affect the intensity of frisson. We conducted experiments on the stimulus characteristics and individual's mood states and personality traits. Participants filled out self-reported questionnaires, including the Profile of Mood States, Beck Depression Inventory, and Big Five Inventory. They continuously indicated the subjective intensity of frisson throughout a 17-min experiment while listening to binaural brushing and tapping sounds through headphones. In the interviews after the experiments, participants reported that tingling and tickling sensations mainly originated on their ears, neck, shoulders, and back. Cross-correlation results showed that the intensity of frisson was closely linked to the acoustic features of auditory stimuli, including their amplitude, spectral centroid, and spectral bandwidth. This suggests that proximal sounds with dark and compact timbre trigger frisson. The peak of correlation between frisson and the acoustic feature was observed 2 s after the acoustic feature changed, suggesting that bottom-up auditory inputs modulate skin-related modalities. We also found that participants with anxiety were sensitive to frisson. Our results provide important clues to understanding the mechanisms of auditory-somatosensory interactions.

Humans can perceive a coherent visual scene despite a low spatial resolution in peripheral vision. How does the visual system determine whether an object exists in the periphery? We addressed this question by focusing on the extinction illusion in which a disk becomes subjectively invisible when presented at the intersection of grids. We hypothesized that the disk would go unnoticed when the stimuli with and without the disk produced the same strength of visual signals. The visual system would miss the disk by confounding the target signals with the intersection signals that should be discounted. Computational analysis revealed that the energy ratio between the stimuli with and without the disk decreased with stimulus eccentricity and such energy ratio could successfully explain the observer's d' to detect the disk. These results indicate that the discounting mechanism relying on stimulus energy determines the awareness toward a peripheral object., (Copyright © 2021 Elsevier Inc. All rights reserved.)

It is often assumed that the reaction time of a saccade toward visual and/or auditory stimuli reflects the sensitivities of our oculomotor-orienting system to stimulus saliency. Endogenous factors, as well as stimulus-related factors, would also affect the saccadic reaction time (SRT). However, it was not clear how these factors interact and to what extent visual and auditory-targeting saccades are accounted for by common mechanisms. The present study examined the effect of, and the interaction between, stimulus saliency and audiovisual spatial congruency on the SRT for visual- and for auditory-target conditions. We also analyzed pre-target pupil size to examine the relationship between saccade preparation and pupil size. Pupil size is considered to reflect arousal states coupling with locus-coeruleus (LC) activity during a cognitive task. The main findings were that (1) the pattern of the examined effects on the SRT varied between visual- and auditory-auditory target conditions, (2) the effect of stimulus saliency was significant for the visual-target condition, but not significant for the auditory-target condition, (3) Pupil velocity, not absolute pupil size, was sensitive to task set (i.e., visual-targeting saccade vs. auditory-targeting saccade), and (4) there was a significant correlation between the pre-saccade absolute pupil size and the SRTs for the visual-target condition but not for the auditory-target condition. The discrepancy between target modalities for the effect of pupil velocity and between the absolute pupil size and pupil velocity for the correlation with SRT may imply that the pupil effect for the visual-target condition was caused by a modality-specific link between pupil size modulation and the SC rather than by the LC-NE (locus coeruleus-norepinephrine) system. These results support the idea that different threshold mechanisms in the SC may be involved in the initiation of saccades toward visual and auditory targets., Competing Interests: Both authors are employees of NTT Communication Science Laboratories, which is a basic-science research section of Nippon Telegraph and Telecommunication corporation (NTT). This does not alter the authors’ adherence to policies of Frontiers., (Copyright © 2020 Yamagishi and Furukawa.)

Visually inferring material properties is crucial for many tasks, yet poses significant computational challenges for biological vision. Liquids and gels are particularly challenging due to their extreme variability and complex behaviour. We reasoned that measuring and modelling viscosity perception is a useful case study for identifying general principles of complex visual inferences. In recent years, artificial Deep Neural Networks (DNNs) have yielded breakthroughs in challenging real-world vision tasks. However, to model human vision, the emphasis lies not on best possible performance, but on mimicking the specific pattern of successes and errors humans make. We trained a DNN to estimate the viscosity of liquids using 100.000 simulations depicting liquids with sixteen different viscosities interacting in ten different scenes (stirring, pouring, splashing, etc). We find that a shallow feedforward network trained for only 30 epochs predicts mean observer performance better than most individual observers. This is the first successful image-computable model of human viscosity perception. Further training improved accuracy, but predicted human perception less well. We analysed the network's features using representational similarity analysis (RSA) and a range of image descriptors (e.g. optic flow, colour saturation, GIST). This revealed clusters of units sensitive to specific classes of feature. We also find a distinct population of units that are poorly explained by hand-engineered features, but which are particularly important both for physical viscosity estimation, and for the specific pattern of human responses. The final layers represent many distinct stimulus characteristics-not just viscosity, which the network was trained on. Retraining the fully-connected layer with a reduced number of units achieves practically identical performance, but results in representations focused on viscosity, suggesting that network capacity is a crucial parameter determining whether artificial or biological neural networks use distributed vs. localized representations., Competing Interests: The authors have declared that no competing interests exist.

Individuals with autism spectrum disorder and individuals with schizophrenia have impaired social and communication skills. They also have altered auditory perception. This study investigated autistic traits and schizotypy in a non-clinical population as well as the excitation-inhibition (EI) balance in different brain regions and their auditory multistable perception. Thirty-four healthy participants were assessed by the Autism-Spectrum Quotient (AQ) and Schizotypal Personality Questionnaire (SPQ). The EI balance was evaluated by measuring the resting-state concentrations of glutamate-glutamine (Glx) and ϒ-aminobutyric acid (GABA) in vivo by using magnetic resonance spectroscopy. To observe the correlation between their traits and perception, we conducted an auditory streaming task and a verbal transformation task, in which participants reported spontaneous perceptual switching while listening to a sound sequence. Their AQ and SPQ scores were positively correlated with the Glx/GABA ratio in the auditory cortex but not in the frontal areas. These scores were negatively correlated with the number of perceptual switches in the verbal transformation task but not in the auditory streaming task. Our results suggest that the EI balance in the auditory cortex and the perceptual formation of speech are involved in autistic traits and schizotypy.

Humans are quick to notice if an object is unstable. Does that assessment require attention or can instability serve as a preattentive feature that can guide the deployment of attention? This paper describes a series of visual search experiments, designed to address this question. Experiment 1 shows that less stable images among more stable images are found more efficiently than more stable among less stable; a search asymmetry that supports guidance by instability. Experiment 2 shows efficient search but no search asymmetry when the orientation of the objects is removed as a confound. Experiment 3 independently varies the orientation cues and perceived stability and finds a clear main effect of apparent stability. Experiment 4 shows converging evidence for a role of stability using different stimuli that lack an orientation cue. However, here both search for stable and unstable targets is inefficient. Experiment 5 is a control for Experiment 4, showing that the stability effect in Experiment 4 is not simple side-effects of the geometry of the stimuli. On balance, the data support a role for instability in the guidance of attention in visual search. (184 words).

Age-related changes in auditory and visual perception have an impact on the quality of life. It has been debated how perceptual organization is influenced by advancing age. From the neurochemical perspective, we investigated age effects on auditory and visual bistability. In perceptual bistability, a sequence of sensory inputs induces spontaneous switching between different perceptual objects. We used different modality tasks of auditory streaming and visual plaids. Young and middle-aged participants (20-60years) were instructed to indicate by a button press whenever their perception changed from one stable state to the other. The number of perceptual switches decreased with participants' ages. We employed magnetic resonance spectroscopy to measure non-invasively concentrations of the inhibitory neurotransmitter (γ-aminobutyric acid, GABA) in the brain regions of interest. When participants were asked to voluntarily modulate their perception, the amount of effective volitional control was positively correlated with the GABA concentration in the auditory and motion-sensitive areas corresponding to each sensory modality. However, no correlation was found in the prefrontal cortex and anterior cingulate cortex. In addition, effective volitional control was reduced with advancing age. Our results suggest that sequential scene analysis in auditory and visual domains is influenced by both age-related and neurochemical factors., (Copyright © 2017 The Author(s). Published by Elsevier Ltd.. All rights reserved.)

Perceptual organisation must select one interpretation from several alternatives to guide behaviour. Computational models suggest that this could be achieved through an interplay between inhibition and excitation across competing types of neural population coding for each interpretation. Here, to test for such models, we used magnetic resonance spectroscopy to measure non-invasively the concentrations of inhibitory γ-aminobutyric acid (GABA) and excitatory glutamate-glutamine (Glx) in several brain regions. Human participants first performed auditory and visual multistability tasks that produced spontaneous switching between percepts. Then, we observed that longer percept durations during behaviour were associated with higher GABA/Glx ratios in the sensory area coding for each modality. When participants were asked to voluntarily modulate their perception, a common factor across modalities emerged: the GABA/Glx ratio in the posterior parietal cortex tended to be positively correlated with the amount of effective volitional control. Our results provide direct evidence implicating that the balance between neural inhibition and excitation within sensory regions resolves perceptual competition. This powerful computational principle appears to be leveraged by both audition and vision, implemented independently across modalities, but modulated by an integrated control process.

When we drop an object from our hands, we use internal models of both our body height and object-motion to predict when it will hit the floor. What happens if the sensory feedback finally received from the impact conflicts with this prediction? The present study shows that such conflict results in changes in the internal estimates of our body height: When the object people dropped takes longer than expected to hit the floor, they report feeling taller and behave as if their legs were longer. This provides the first evidence of cross-modal recalibration of body-height representations as a function of changes in the distant environment. Crucially, the recalibration results from a mismatch between the predicted and actual outcome of an action, the ball's release and impact, which are causally-related but separated in space and time. These results suggest that implicit models of object-motion can interact with implicit and explicit models of one's body height., Competing Interests: ATJ was supported by a travel grant from the research project “DEEP BRAIN COMMUNICATION” funded by The Nippon Telegraph and Telephone Corporation of Japan. This does not alter our adherence to PLOS ONE policies on sharing data and materials.

The objectives of this study were to investigate middle finger movements and dynamics of ball movements around the instant of ball release during baseball pitching. Baseball pitching from an indoor mound among 14 semi-professional pitchers was captured using a motion capture system with 16 high-speed cameras (1,000 Hz). Kinematics of middle finger joints, ball rotation, and force applied to the ball were calculated. The proximal and distal interphalangeal joints continued to extend until the instant of ball release, then abruptly flexed. The abrupt flexion lasted for only several milliseconds, followed by a short extension phase. The finger made a quick double cycle of extension-flexion movement, suggesting that it attained high stiffness resulting from co-contraction. The ball began to roll up to the tip of the finger 8 ± 1 ms before ball release owing to the start of extension or the increased angular velocity of extension for the proximal interphalangeal joint. A mean force of 195 ± 27 N was applied in the proximal direction of the hand at the same time as the beginning of ball rolling, and a mean force of 109 ± 22 N was applied to the throwing direction just before ball release.

Visual estimation of the material and shape of an object from a single image includes a hard ill-posed computational problem. However, in our daily life we feel we can estimate both reasonably well. The neural computation underlying this ability remains poorly understood. Here we propose that the human visual system uses different aspects of object images to separately estimate the contributions of the material and shape. Specifically, material perception relies mainly on the intensity gradient magnitude information, while shape perception relies mainly on the intensity gradient order information. A clue to this hypothesis was provided by the observation that luminance-histogram manipulation, which changes luminance gradient magnitudes but not the luminance-order map, effectively alters the material appearance but not the shape of an object. In agreement with this observation, we found that the simulated physical material changes do not significantly affect the intensity order information. A series of psychophysical experiments further indicate that human surface shape perception is robust against intensity manipulations provided they do not disturb the intensity order information. In addition, we show that the two types of gradient information can be utilized for the discrimination of albedo changes from highlights. These findings suggest that the visual system relies on these diagnostic image features to estimate physical properties in a distal world., Competing Interests: The authors have declared that no competing interests exist.

Volitional control has been related to the excitatory/inhibitory (E/I) ratio of glutamate-glutamine to γ-aminobutyric acid concentration in the different parts of the frontal cortex. Yet, how the neurochemical balance in each of the brain areas modulates volitional control remains unclear. Here, participants performed an auditory Go/No-Go task with and without task-irrelevant face distractors. Neurochemical balance was measured with magnetic resonance spectroscopy at rest. Participants with higher E/I ratios in the dorsolateral prefrontal cortex (DLPFC) showed less control over No-Go cues under no distraction, whereas participants with higher E/I ratios in the anterior cingulate cortex (ACC) were more prompted to make speeded Go responses under distraction. Therefore, the neurochemical balance in the DLPFC and ACC may be involved in the control over task-relevant and -irrelevant cues respectively., (Copyright © 2018 Elsevier Inc. All rights reserved.)

With the growth in digital display technologies, dynamic text presentation is used widely in every day life, such as in electric advertisements and tickers on TV programs. Unlike static text reading, little is known about the basic characteristics underlying reading dynamically presented texts. Two experiments were performed to investigate this. Experiment 1 examined the optimum rate of dynamic text presentation in terms of a readability and favorability. This experiment demonstrated that, when the rate of text presentation was changed, there was an optimum presentation rate (around 6 letters/s in our condition) regardless of difficulty level. This indicates that the presentation rate of dynamic texts can affect the impression of reading. In Experiment 2, to elucidate the traits underlying dynamic text reading, we measured the reading speeds of silent and trace reading among the same participants and compared them with the optimum presentation rate obtained in Experiment 1. The results showed that the optimum rate was slower than with silent reading and faster than with trace reading, and, interestingly, the individual optimum rates of dynamic text presentation were correlated with the speeds of both silent and trace reading. In other words, the readers who preferred a fast rate in dynamic text presentation would also have a high reading speed for silent and trace reading.

Forces imparted by the fingers onto a baseball are the final, critical aspects for pitching, however these forces have not been quantified previously as no biomechanical technology was available. In this study, an instrumented baseball was developed for direct measurement of ball reaction force by individual fingers and used to provide fundamental information on the forces during a fastball pitch. A tri-axial force transducer with a cable having an easily-detachable connector were installed in an official baseball. Data were collected from 11 pitchers who placed the fingertip of their index, middle, ring, or thumb on the transducer, and threw four-seam fastballs to a target cage from a flat mound. For the index and middle fingers, resultant ball reaction force exhibited a bimodal pattern with initial and second peaks at 38-39ms and 6-7ms before ball release, and their amplitudes were around 97N each. The ring finger and thumb produced single-peak forces of approximately 50 and 83N, respectively. Shear forces for the index and middle fingers formed distinct peak at 4-5ms before release, and the peaks summed to 102N; a kinetic source for backspin on the ball. An additional experiment with submaximal pitching effort showed a linear relationship of peak forces with ball velocity. The peak ball reaction force for fastballs exceeded 80% of maximum finger strength measured, suggesting that strengthening of the distal muscles is important both for enhancing performance and for avoiding injuries., (Copyright © 2017 Elsevier B.V. All rights reserved.)

Characterization of the functional relationship between sensory inputs and neuronal or observers' perceptual responses is one of the fundamental goals of systems neuroscience and psychophysics. Conventional methods, such as reverse correlation and spike-triggered data analyses are limited in their ability to resolve complex and inherently nonlinear neuronal/perceptual processes because these methods require input stimuli to be Gaussian with a zero mean. Recent studies have shown that analyses based on a generalized linear model (GLM) do not require such specific input characteristics and have advantages over conventional methods. GLM, however, relies on iterative optimization algorithms and its calculation costs become very expensive when estimating the nonlinear parameters of a large-scale system using large volumes of data. In this paper, we introduce a new analytical method for identifying a nonlinear system without relying on iterative calculations and yet also not requiring any specific stimulus distribution. We demonstrate the results of numerical simulations, showing that our noniterative method is as accurate as GLM in estimating nonlinear parameters in many cases and outperforms conventional, spike-triggered data analyses. As an example of the application of our method to actual psychophysical data, we investigated how different spatiotemporal frequency channels interact in assessments of motion direction. The nonlinear interaction estimated by our method was consistent with findings from previous vision studies and supports the validity of our method for nonlinear system identification.

Though the computation of agency is thought to be based on prediction error, it is important for us to grasp our own reliability of that detected error. Here, the current study shows that we have a meta-monitoring ability over our own forward model, where the accuracy of motor prediction and therefore of the felt agency are implicitly evaluated. Healthy participants (N=105) conducted a simple motor control task and SELF or OTHER visual feedback was given. The relationship between the accuracy and confidence in a mismatch detection task and in a self-other attribution task was examined. The results suggest an accuracy-confidence correlation in both tasks, indicating our meta-monitoring ability over such decisions. Furthermore, a statistically identified group with low accuracy and low confidence was characterized as higher schizotypal people. Finally, what we can know about our own forward model and how we can know it is discussed., (Copyright © 2017 Elsevier Inc. All rights reserved.)

Multistability in perception is a powerful tool for investigating sensory-perceptual transformations, because it produces dissociations between sensory inputs and subjective experience. Spontaneous switching between different perceptual objects occurs during prolonged listening to a sound sequence of tone triplets or repeated words (termed auditory streaming and verbal transformations, respectively). We used these examples of auditory multistability to examine to what extent neurochemical and cognitive factors influence the observed idiosyncratic patterns of switching between perceptual objects. The concentrations of glutamate-glutamine (Glx) and γ-aminobutyric acid (GABA) in brain regions were measured by magnetic resonance spectroscopy, while personality traits and executive functions were assessed using questionnaires and response inhibition tasks. Idiosyncratic patterns of perceptual switching in the two multistable stimulus configurations were identified using a multidimensional scaling (MDS) analysis. Intriguingly, although switching patterns within each individual differed between auditory streaming and verbal transformations, similar MDS dimensions were extracted separately from the two datasets. Individual switching patterns were significantly correlated with Glx and GABA concentrations in auditory cortex and inferior frontal cortex but not with the personality traits and executive functions. Our results suggest that auditory perceptual organization depends on the balance between neural excitation and inhibition in different brain regions.This article is part of the themed issue 'Auditory and visual scene analysis'., (© 2017 The Author(s).)

We perceive the world as stable and composed of discrete objects even though auditory and visual inputs are often ambiguous owing to spatial and temporal occluders and changes in the conditions of observation. This raises important questions regarding where and how 'scene analysis' is performed in the brain. Recent advances from both auditory and visual research suggest that the brain does not simply process the incoming scene properties. Rather, top-down processes such as attention, expectations and prior knowledge facilitate scene perception. Thus, scene analysis is linked not only with the extraction of stimulus features and formation and selection of perceptual objects, but also with selective attention, perceptual binding and awareness. This special issue covers novel advances in scene-analysis research obtained using a combination of psychophysics, computational modelling, neuroimaging and neurophysiology, and presents new empirical and theoretical approaches. For integrative understanding of scene analysis beyond and across sensory modalities, we provide a collection of 15 articles that enable comparison and integration of recent findings in auditory and visual scene analysis.This article is part of the themed issue 'Auditory and visual scene analysis'., (© 2017 The Author(s).)

Recent studies have shown that interindividual variability can be a rich source of information regarding the mechanism of human visual perception. In this study, we examined the mechanisms underlying interindividual variability in the perception of visual motion, one of the fundamental components of visual scene analysis, by measuring neurotransmitter concentrations using magnetic resonance spectroscopy. First, by psychophysically examining two types of motion phenomena-motion assimilation and contrast-we found that, following the presentation of the same stimulus, some participants perceived motion assimilation, while others perceived motion contrast. Furthermore, we found that the concentration of the excitatory neurotransmitter glutamate-glutamine (Glx) in the dorsolateral prefrontal cortex (Brodmann area 46) was positively correlated with the participant's tendency to motion assimilation over motion contrast; however, this effect was not observed in the visual areas. The concentration of the inhibitory neurotransmitter γ-aminobutyric acid had only a weak effect compared with that of Glx. We conclude that excitatory process in the suprasensory area is important for an individual's tendency to determine antagonistically perceived visual motion phenomena.This article is part of the themed issue 'Auditory and visual scene analysis'., (© 2017 The Author(s).)

The paper presents a new compositional hierarchical model for robust music transcription. Its main features are unsupervised learning of a hierarchical representation of input data, transparency, which enables insights into the learned representation, as well as robustness and speed which make it suitable for real-world and real-time use. The model consists of multiple layers, each composed of a number of parts. The hierarchical nature of the model corresponds well to hierarchical structures in music. The parts in lower layers correspond to low-level concepts (e.g. tone partials), while the parts in higher layers combine lower-level representations into more complex concepts (tones, chords). The layers are learned in an unsupervised manner from music signals. Parts in each layer are compositions of parts from previous layers based on statistical co-occurrences as the driving force of the learning process. In the paper, we present the model's structure and compare it to other hierarchical approaches in the field of music information retrieval. We evaluate the model's performance for the multiple fundamental frequency estimation. Finally, we elaborate on extensions of the model towards other music information retrieval tasks., Competing Interests: The authors have declared that no competing interests exist.

Anomalous agency has been reported clinically and empirically for people with schizophrenia. This finding is expected to contribute to understanding positive symptomatology in schizophrenia in terms of a general neurocomputational model of motor control, because anomalous agency has also been reported in schizotypal traits in the general population. However, superficially opposite conclusions have been suggested: over-attributed or under-attributed agency in patients. In this work, healthy participants (N=104) were presented continuous morphed self-other visual feedback of their reaching movements and rated the agency they felt for it. The slope of the regression line in stimuli-response coordination as a function of self-other discriminability was estimated for each participant. The estimated slopes were significantly associated with positive schizotypal symptomatology. Higher schizotypal participants exhibited a lower slope, indicating poorer discriminability between their own and others' movements. Furthermore, the estimated regression lines in the high and low groups are predicted to cross at the approximately center point in the coordinates, which should produce both over-attribution and under-attribution errors for the high group compared with low group. The pattern of schizotypal attribution error depends on the S/N (signal-to-noise) ratio of the given stimuli within our sensorimotor system where the self-originated stimulus is the signal to be detected. The current study, for the first time, suggests both over- and under-attribution within participants scoring high on schizotypy., (Copyright © 2016 Elsevier Ireland Ltd. All rights reserved.)

In this series of behavioural experiments we investigated the effect of distraction on the maintenance of acoustic scene information in short-term memory. Stimuli are artificial acoustic 'scenes' composed of several (up to twelve) concurrent tone-pip streams ('sources'). A gap (1000 ms) is inserted partway through the 'scene'; Changes in the form of an appearance of a new source or disappearance of an existing source, occur after the gap in 50% of the trials. Listeners were instructed to monitor the unfolding 'soundscapes' for these events. Distraction was measured by presenting distractor stimuli during the gap. Experiments 1 and 2 used a dual task design where listeners were required to perform a task with varying attentional demands ('High Demand' vs. 'Low Demand') on brief auditory (Experiment 1a) or visual (Experiment 1b) signals presented during the gap. Experiments 2 and 3 required participants to ignore distractor sounds and focus on the change detection task. Our results demonstrate that the maintenance of scene information in short-term memory is influenced by the availability of attentional and/or processing resources during the gap, and that this dependence appears to be modality specific. We also show that these processes are susceptible to bottom up driven distraction even in situations when the distractors are not novel, but occur on each trial. Change detection performance is systematically linked with the, independently determined, perceptual salience of the distractor sound. The findings also demonstrate that the present task may be a useful objective means for determining relative perceptual salience., (Copyright © 2016 The Authors. Published by Elsevier B.V. All rights reserved.)

The scientific exploration of the self has progressed, with much attention focused on the Embodied Sense of Self (ESS). Empirical studies have suggested the mechanisms for self-representation. On the other hand, less attention has been paid to the subjectivity itself of the self. With reference to previous studies, the current study collected items that reflect the ESS and statistically extracted three factors for it: Ownership, Agency, and Narrative. The developed questionnaire [Embodied Sense of Self Scale (ESSS)] showed good enough validity and reliability for practical use. Furthermore, ESSS discriminated schizophrenia, a disorder of the ESS, from controls. We discuss the factorial structure of ESS and the relationship among factors on the basis of the current results.

The processing of time intervals is fundamental for sensorimotor and cognitive functions. Perceptual and motor timing are often performed concurrently (e.g., playing a musical instrument). Although previous studies have shown the influence of body movements on time perception, how we perceive self-produced time intervals has remained unclear. Furthermore, it has been suggested that the timing mechanisms are distinct for the sub- and suprasecond ranges. Here, we compared perceptual performances for self-produced and passively presented time intervals in random contexts (i.e., multiple target intervals presented in a session) across the sub- and suprasecond ranges (Experiment 1) and within the sub- (Experiment 2) and suprasecond (Experiment 3) ranges, and in a constant context (i.e., a single target interval presented in a session) in the sub- and suprasecond ranges (Experiment 4). We show that self-produced time intervals were perceived as shorter and more variable across the sub- and suprasecond ranges and within the suprasecond range but not within the subsecond range in a random context. In a constant context, the self-produced time intervals were perceived as more variable in the suprasecond range but not in the subsecond range. The impairing effects indicate that motor timing interferes with perceptual timing. The dependence of impairment on temporal contexts suggests multiple timing mechanisms for the subsecond and suprasecond ranges. In addition, violation of the scalar property (i.e., a constant variability to target interval ratio) was observed between the sub- and suprasecond ranges. The violation was clearer for motor timing than for perceptual timing. This suggests that the multiple timing mechanisms for the sub- and suprasecond ranges overlap more for perception than for motor. Moreover, the central tendency effect (i.e., where shorter base intervals are overestimated and longer base intervals are underestimated) disappeared with motor timing within the subsecond range, suggesting multiple subsecond timing system for perception and motor.

Multi-stability refers to the phenomenon of perception stochastically switching between possible interpretations of an unchanging stimulus. Despite considerable variability, individuals show stable idiosyncratic patterns of switching between alternative perceptions in the auditory streaming paradigm. We explored correlates of the individual switching patterns with executive functions, personality traits, and creativity. The main dimensions on which individual switching patterns differed from each other were identified using multidimensional scaling. Individuals with high scores on the dimension explaining the largest portion of the inter-individual variance switched more often between the alternative perceptions than those with low scores. They also perceived the most unusual interpretation more often, and experienced all perceptual alternatives with a shorter delay from stimulus onset. The ego-resiliency personality trait, which reflects a tendency for adaptive flexibility and experience seeking, was significantly positively related to this dimension. Taking these results together we suggest that this dimension may reflect the individual's tendency for exploring the auditory environment. Executive functions were significantly related to some of the variables describing global properties of the switching patterns, such as the average number of switches. Thus individual patterns of perceptual switching in the auditory streaming paradigm are related to some personality traits and executive functions.

Awareness of our own bodies (sense of body-ownership) and actions (sense of agency) is fundamental for self-consciousness. In the rubber hand illusion, watching a rubber hand being stroked synchronously as one's own unseen hand is also stroked causes the observer to attribute the rubber hand to their own body. The findings of the series of experiments reported here suggest that body-ownership, measured using proprioceptive drift, is elicited by the external acting proxy that drives the sense of agency. While participants clasped and unclasped their left hand for 60 s, they focused on video feedback on a monitor in front of them. Proprioceptive drift was observed only under the conditions, including synchronized conditions, where the sense of agency for the acting proxy occurred, suggesting an essential interaction between body-ownership and agency.

A pupillary dilation response is known to be evoked by salient deviant or contrast auditory stimuli, but so far a direct link between it and subjective salience has been lacking. In two experiments, participants listened to various environmental sounds while their pupillary responses were recorded. In separate sessions, participants performed subjective pairwise-comparison tasks on the sounds with respect to their salience, loudness, vigorousness, preference, beauty, annoyance, and hardness. The pairwise-comparison data were converted to ratings on the Thurstone scale. The results showed a close link between subjective judgments of salience and loudness. The pupil dilated in response to the sound presentations, regardless of sound type. Most importantly, this pupillary dilation response to an auditory stimulus positively correlated with the subjective salience, as well as the loudness, of the sounds (Exp. 1). When the loudnesses of the sounds were identical, the pupil responses to each sound were similar and were not correlated with the subjective judgments of salience or loudness (Exp. 2). This finding was further confirmed by analyses based on individual stimulus pairs and participants. In Experiment 3, when salience and loudness were manipulated by systematically changing the sound pressure level and acoustic characteristics, the pupillary dilation response reflected the changes in both manipulated factors. A regression analysis showed a nearly perfect linear correlation between the pupillary dilation response and loudness. The overall results suggest that the pupillary dilation response reflects the subjective salience of sounds, which is defined, or is heavily influenced, by loudness.

A unique sound that deviates from a repetitive background sound induces signature neural responses, such as mismatch negativity and novelty P3 response in electro-encephalography studies. Here we show that a deviant auditory stimulus induces a human pupillary dilation response (PDR) that is sensitive to the stimulus properties and irrespective whether attention is directed to the sounds or not. In an auditory oddball sequence, we used white noise and 2000-Hz tones as oddballs against repeated 1000-Hz tones. Participants' pupillary responses were recorded while they listened to the auditory oddball sequence. In Experiment 1, they were not involved in any task. Results show that pupils dilated to the noise oddballs for approximately 4 s, but no such PDR was found for the 2000-Hz tone oddballs. In Experiments 2, two types of visual oddballs were presented synchronously with the auditory oddballs. Participants discriminated the auditory or visual oddballs while trying to ignore stimuli from the other modality. The purpose of this manipulation was to direct attention to or away from the auditory sequence. In Experiment 3, the visual oddballs and the auditory oddballs were always presented asynchronously to prevent residuals of attention on to-be-ignored oddballs due to the concurrence with the attended oddballs. Results show that pupils dilated to both the noise and 2000-Hz tone oddballs in all conditions. Most importantly, PDRs to noise were larger than those to the 2000-Hz tone oddballs regardless of the attention condition in both experiments. The overall results suggest that the stimulus-dependent factor of the PDR appears to be independent of attention.

The performance of a lateralization task based on interaural time or level differences (ITDs or ILDs) often varies among listeners. This study examined the extent to which this inter-listener variation could be accounted for by the coding efficiency of the temporal-structure or level information below the stage of interaural interaction. Young listeners (20s to 30s) and early-elderly (60s) listeners with or without mild hearing loss were tested. The ITD, ILD, TIME, and LEVEL tasks were intended to measure sensitivities to ITDs, ILDs, the temporal structure of the stimulus encoded by the neural phase locking, and the stimulus level, respectively. The performances of the ITD and ILD tasks were not significantly different between the age groups, while the elderly listeners exhibited significantly poorer performance in the TIME task (and in the LEVEL with a high-frequency stimulus only) than the young listeners. Significant correlations were found between thresholds for the ILD and LEVEL tasks with low- and high-frequency stimuli and for the ITD and TIME tasks for the high-frequency stimulus, implying peripheral coding efficiency as a major factor determining lateralization performance. However, we failed to find a correlation between the ITD and TIME tasks for the low-frequency stimulus, despite a large range of threshold values in the TIME task. This implies that in a low frequency region, the peripheral coding efficiency of the stimulus temporal structure is a relatively minor factor in the ITD-based lateralization performance.

The sense of agency, the subjective experience of controlling one's own action, has an important function in motor control. When we move our own body or even external tools, we attribute that movement to ourselves and utilize that sensory information in order to correct "our own" movement in theory. The dynamic relationship between conscious self-other attribution and feedback control, however, is still unclear. Participants were required to make a sinusoidal reaching movement and received its visual feedback (i.e., cursor). When participants received a fake movement that was spatio-temporally close to their actual movement, illusory self-attribution of the fake movement was observed. In this situation, since participants tried to control the cursor but it was impossible to do so, the movement error was increased (Experiment 1). However, when the visual feedback was reduced to make self-other attribution difficult, there was no further increase in the movement error (Experiment 2). These results indicate that conscious self-other sensory attribution might coordinate sensory input and motor output., (Copyright © 2015 Elsevier Inc. All rights reserved.)

Sense of control refers to one's feelings to control environmental events through one's own action. A prevailing view is that the sense of control is strong (or is not diminished) when predicted sensory signals, which are generated in motor control mechanisms, are consistent with afferent sensory signals. Such intact sense of control often leads to the misjudgment of temporal relation between timings of one's action and its effect (so-called, intentional binding). The present study showed that the intentional binding could be enhanced by the delayed visual feedback of an agent's action. We asked participants to press a button to produce a tone as action outcome. In some conditions, they were given the delayed visual feedback of their button press. Participants judged whether the onset of the auditory outcome was delayed from the timing of their button press. Consequently, delay detection thresholds were significantly higher when the feedback was given 0.2 and 0.4 s delays than when no feedback was displayed to the participants. The results indicate that action agents misjudge the timing of their action (button press) in the presence of the delayed visual feedback of their action. Interestingly, delay detection thresholds were strongly correlated with the subjective magnitude of the sense of control. Thus, the sense of control is possibly determined by cross-modal processing for action-related and outcome-related sensory signals.

Individuals with autism spectrum disorder (ASD) show impaired social interaction and communication, which may be related to their difficulties in speech production. To investigate the mechanisms of atypical speech production in this population, we examined feedback control by delaying the auditory feedback of their own speech, which degraded speech fluency. We also examined feedforward control by adding loud pink noise to the auditory feedback, which led to increased vocal effort in producing speech. The results of Japanese speakers show that, compared with neurotypical (NT) individuals, high-functioning adults with ASD (including Asperger's disorder, autistic disorder, and pervasive developmental disorder not otherwise specified) were more affected by delayed auditory feedback but less affected by external noise. These findings indicate that, in contrast to NT individuals, those with ASD relied more on feedback control than on feedforward control in speech production, which is consistent with the hypothesis that this population exhibits attenuated Bayesian priors.