Search

Illusory contours (ICs) are borders that are perceived in the absence of contrast gradients. Until recently, IC processes were considered exclusively visual in nature and presumed to be unaffected by information from other senses. Electrophysiological data in humans indicates that sounds can enhance IC processes. Despite cross-modal enhancement being observed at the neurophysiological level, to date there is no evidence of direct amplification of behavioural performance in IC processing by sounds. We addressed this knowledge gap. Healthy adults () discriminated instances when inducers were arranged to form an IC from instances when no IC was formed (NC). Inducers were low-constrast and masked, and there was continuous background acoustic noise throughout a block of trials. On half of the trials, i.e., independently of IC vs NC, a 1000-Hz tone was presented synchronously with the inducer stimuli. Sound presence improved the accuracy of indicating when an IC was presented, but had no impact on performance with NC stimuli (significant IC presence/absence × Sound presence/absence interaction). There was no evidence that this was due to general alerting or to a speed–accuracy trade-off (no main effect of sound presence on accuracy rates and no comparable significant interaction on reaction times). Moreover, sound presence increased sensitivity and reduced bias on the IC vs NC discrimination task. These results demonstrate that multisensory processes augment mid-level visual functions, exemplified by IC processes. Aside from the impact on neurobiological and computational models of vision, our findings may prove clinically beneficial for low-vision or sight-restored patients.

The way fMRI data are analyzed varies greatly between researchers, and processing pipelines are often rewritten for each new dataset. This, the missing code transparency and the negligence of quality control is at the core of the current reproducibility crisis in neuroimaging.Here, we introduce fmriflows, a consortium of univariate and multivariate processing pipelines for task-based fMRI data. In addition to standard pre-processing steps, fmriflows also performs non-steady-state detection, intensity non-uniformity correction, allows for flexible temporal and spatial filtering, and computes many useful nuisance factors. Each pipeline creates an informative visual report, serving as a quality control for individual processing steps and ensuring desired processing outcomes. fmriflows can be run fully autonomously on any system via version-controlled Docker containers or in a more flexible and interactive mode via Jupyter Notebooks.Similar projects, such as FMRIPrep, have documented the need and interest for such automated analysis software pipelines. However and while FMRIPrep focuses only on preprocessing for univariate analysis, fmriflows also includes multivariate analysis as well as both 1st and 2nd-level statistical analyses.Establishing fully autonomous, community developed and open-source state-of-the-art analysis pipelines with quality control reports will make the analysis of fMRI data more objective, easier to share and therefore more reproducible.

Notter, M., Hanke, M., Murray, M.M., & Geiser, E. (2017). Temporal Gestalt perception: Where in the human brain do we encode rhythm?. Poster at Lemanic Neuroscience Annual Meeting (LNAM), September 1-2, 2017.

Multisensory interactions are observed in species from single-cell organisms to humans. Important early work was primarily carried out in the cat superior colliculus and a set of critical parameters for their occurrence were defined. Primary among these were temporal synchrony and spatial alignment of bisensory inputs. Here, we assessed whether spatial alignment was also a critical parameter for the temporally earliest multisensory interactions that are observed in lower-level sensory cortices of the human. While multisensory interactions in humans have been shown behaviorally for spatially disparate stimuli (e.g. the ventriloquist effect), it is not clear if such effects are due to early sensory level integration or later perceptual level processing. In the present study, we used psychophysical and electrophysiological indices to show that auditory-somatosensory interactions in humans occur via the same early sensory mechanism both when stimuli are in and out of spatial register. Subjects more rapidly detected multisensory than unisensory events. At just 50 ms post-stimulus, neural responses to the multisensory ‘whole' were greater than the summed responses from the constituent unisensory ‘parts'. For all spatial configurations, this effect followed from a modulation of the strength of brain responses, rather than the activation of regions specifically responsive to multisensory pairs. Using the local auto-regressive average source estimation, we localized the initial auditory-somatosensory interactions to auditory association areas contralateral to the side of somatosensory stimulation. Thus, multisensory interactions can occur across wide peripersonal spatial separations remarkably early in sensory processing and in cortical regions traditionally considered unisensory

This study analyzed high‐density event‐related potentials (ERPs) within an electrical neuroimaging framework to provide insights regarding the interaction between multisensory processes and stimulus probabilities. Specifically, we identified the spatiotemporal brain mechanisms by which the proportion of temporally congruent and task‐irrelevant auditory information influences stimulus processing during a visual duration discrimination task. The spatial position (top/bottom) of the visual stimulus was indicative of how frequently the visual and auditory stimuli would be congruent in their duration (i.e., context of congruence). Stronger influences of irrelevant sound were observed when contexts associated with a high proportion of auditory‐visual congruence repeated and also when contexts associated with a low proportion of congruence switched. Context of congruence and context transition resulted in weaker brain responses at 228 to 257 ms poststimulus to conditions giving rise to larger behavioral cross‐modal interactions. Importantly, a control oddball task revealed that both congruent and incongruent audiovisual stimuli triggered equivalent non‐linear multisensory interactions when congruence was not a relevant dimension. Collectively, these results are well explained by statistical learning, which links a particular context (here: a spatial location) with a certain level of top‐down attentional control that further modulates cross‐modal interactions based on whether a particular context repeated or changed. The current findings shed new light on the importance of context‐based control over multisensory processing, whose influences multiplex across finer and broader time scales. Hum Brain Mapp 37:273–288, 2016. © 2015 Wiley Periodicals, Inc.

Recent advances in signal analysis have engendered EEG with the status of a true brain mapping and brain imaging method capable of providing spatio-temporal information regarding brain (dys)function. Because of the increasing interest in the temporal dynamics of brain networks, and because of the straightforward compatibility of the EEG with other brain imaging techniques, EEG is increasingly used in the neuroimaging community. However, the full capability of EEG is highly underestimated. Many combined EEG-fMRI studies use the EEG only as a spike-counter or an oscilloscope. Many cognitive and clinical EEG studies use the EEG still in its traditional way and analyze grapho-elements at certain electrodes and latencies. We here show that this way of using the EEG is not only dangerous because it leads to misinterpretations, but it is also largely ignoring the spatial aspects of the signals. In fact, EEG primarily measures the electric potential field at the scalp surface in the same way as MEG measures the magnetic field. By properly sampling and correctly analyzing this electric field, EEG can provide reliable information about the neuronal activity in the brain and the temporal dynamics of this activity in the millisecond range. This review explains some of these analysis methods and illustrates their potential in clinical and experimental applications.

This paper describes methods to analyze the brain's electric fields recorded with multichannel Electroencephalogram (EEG) and demonstrates their implementation in the software CARTOOL. It focuses on the analysis of the spatial properties of these fields and on quantitative assessment of changes of field topographies across time, experimental conditions, or populations. Topographic analyses are advantageous because they are reference independents and thus render statistically unambiguous results. Neurophysiologically, differences in topography directly indicate changes in the configuration of the active neuronal sources in the brain. We describe global measures of field strength and field similarities, temporal segmentation based on topographic variations, topographic analysis in the frequency domain, topographic statistical analysis, and source imaging based on distributed inverse solutions. All analysis methods are implemented in a freely available academic software package called CARTOOL. Besides providing these analysis tools, CARTOOL is particularly designed to visualize the data and the analysis results using 3-dimensional display routines that allow rapid manipulation and animation of 3D images. CARTOOL therefore is a helpful tool for researchers as well as for clinicians to interpret multichannel EEG and evoked potentials in a global, comprehensive, and unambiguous way.

Time is a fundamental dimension of everyday experiences. We can unmistakably sense its passage and adjust our behavior accordingly. Despite its ubiquity, the neuronal mechanisms underlying the capacity to perceive time remains unclear. Here, in two experiments using ultrahigh-field 7-Tesla (7T) functional magnetic resonance imaging (fMRI), we show that in the medial premotor cortex (supplementary motor area [SMA]) of the human brain, neural units tuned to different durations are orderly mapped in contiguous portions of the cortical surface so as to form chronomaps. The response of each portion in a chronomap is enhanced by neighboring durations and suppressed by nonpreferred durations represented in distant portions of the map. These findings suggest duration-sensitive tuning as a possible neural mechanism underlying the recognition of time and demonstrate, for the first time, that the representation of an abstract feature such as time can be instantiated by a topographical arrangement of duration-sensitive neural populations.

This paper proposes and implements biophysical constraints to select a unique solution to the bioelectromagnetic inverse problem. It first shows that the brain's electric fields and potentials are predominantly due to ohmic currents. This serves to reformulate the inverse problem in terms of a restricted source model permitting noninvasive estimations of Local Field Potentials (LFPs) in depth from scalp-recorded data. Uniqueness in the solution is achieved by a physically derived regularization strategy that imposes a spatial structure on the solution based upon the physical laws that describe electromagnetic fields in biological media. The regularization strategy and the source model emulate the properties of brain activity's actual generators. This added information is independent of both the recorded data and head model and suffices for obtaining a unique solution compatible with and aimed at analyzing experimental data. The inverse solution's features are evaluated with event-related potentials (ERPs) from a healthy subject performing a visuo-motor task. Two aspects are addressed: the concordance between available neurophysiological evidence and inverse solution results, and the functional localization provided by fMRI data from the same subject under identical experimental conditions. The localization results are spatially and temporally concordant with experimental evidence, and the areas detected as functionally activated in both imaging modalities are similar, providing indices of localization accuracy. We conclude that biophysically driven inverse solutions offer a novel and reliable possibility for studying brain function with the temporal resolution required to advance our understanding of the brain's functional networks.

Grid cells in the entorhinal cortex, together with head direction, place, speed and border cells, are major contributors to the organization of spatial representations in the brain. In this work we introduce a novel theoretical and algorithmic framework able to explain the emergence of hexagonal grid-like response patterns from head direction cells' responses. We show that this pattern is a result of minimal variance encoding of neurons. The novelty lies into the formulation of the encoding problem through the modern Frame Theory language, specifically that of equiangular Frames, providing new insights about the optimality of hexagonal grid receptive fields. The model proposed overcomes some crucial limitations of the current attractor and oscillatory models. It is based on the well-accepted and tested hypothesis of Hebbian learning, providing a simplified cortical-based framework that does not require the presence of theta velocity-driven oscillations (oscillatory model) or translational symmetries in the synaptic connections (attractor model). We moreover demonstrate that the proposed encoding mechanism naturally explains axis alignment of neighbor grid cells and maps shifts, rotations and scaling of the stimuli onto the shape of grid cells' receptive fields, giving a straightforward explanation of the experimental evidence of grid cells remapping under transformations of environmental cues., Comment: 16 pages, 4 figures

Our understanding of how perception operates in real-world environments has been substantially advanced by studying both multisensory processes and “top-down” control processes influencing sensory processing via activity from higher-order brain areas, such as attention, memory, and expectations. As the two topics have been traditionally studied separately, the mechanisms orchestrating real-world multisensory processing remain unclear. Past work has revealed that the observer’s goals gate the influence of many multisensory processes on brain and behavioural responses, whereas some other multisensory processes might occur independently of these goals. Consequently, other forms of top-down control beyond goal dependence are necessary to explain the full range of multisensory effects currently reported at the brain and the cognitive level. These forms of control include sensitivity to stimulus context as well as the detection of matches (or lack thereof) between a multisensory stimulus and categorical attributes of naturalistic objects (e.g. tools, animals). In this review we discuss and integrate the existing findings that demonstrate the importance of such goal-, object- and context-based top-down control over multisensory processing. We then put forward a few principles emerging from this literature review with respect to the mechanisms underlying multisensory processing and discuss their possible broader implications. This research was supported by grants from the Ministerio de Economia y Competitividad (PSI2013-42626-P), AGAUR Generalitat de Catalunya (2014SGR856), and the European Research Council (StG-2010 263145) to S.S-F, and the Swiss National Science Foundation (Grant #320030-149982 as well as the National Centre of Competence in Research project “SYNAPSY, The Synaptic Bases of Mental Disease” [Project 51AU40-125759]) and the Swiss Brain League (2014 Research Prize) to MMM. StO receives support from the Dutch Organisation for Scientific Research (Grant 406-11-068).

It has been nearly 10 years since Ghazanfar and Schroeder (2006) proposed that the neocortex is essentially multisensory in nature. However, it is only recently that sufficient and hard evidence that supports this proposal has accrued. We review evidence that activity within the human primary visual cortex plays an active role in multisensory processes and directly impacts behavioural outcome. This evidence emerges from a full pallet of human brain imaging and brain mapping methods with which multisensory processes are quantitatively assessed by taking advantage of particular strengths of each technique as well as advances in signal analyses. Several general conclusions about multisensory processes in primary visual cortex of humans are supported relatively solidly. First, haemodynamic methods (fMRI/PET) show that there is both convergence and integration occurring within primary visual cortex. Second, primary visual cortex is involved in multisensory processes during early post-stimulus stages (as revealed by EEG/ERNERFs as well as TMS). Third, multisensory effects in primary visual cortex directly impact behaviour and perception, as revealed by correlational (EEG/ERPs/ERFs) as well as more causal measures (TMS/tACS). While the provocative claim of Ghazanfar and Schroeder (2006) that the whole of neocortex is multisensory in function has yet to be demonstrated, this can now be considered established in the case of the human primary visual cortex. (C) 2015 Elsevier Ltd. All rights reserved.

Approaching or looming sounds (L-sounds) have been shown to selectively increase visual cortex excitability [Romei, V., Murray, M. M., Cappe, C., & Thut, G. Preperceptual and stimulus-selective enhancement of low-level human visual cortex excitability by sounds. Current Biology, 19, 1799–1805, 2009]. These cross-modal effects start at an early, preperceptual stage of sound processing and persist with increasing sound duration. Here, we identified individual factors contributing to cross-modal effects on visual cortex excitability and studied the persistence of effects after sound offset. To this end, we probed the impact of different L-sound velocities on phosphene perception postsound as a function of individual auditory versus visual preference/dominance using single-pulse TMS over the occipital pole. We found that the boosting of phosphene perception by L-sounds continued for several tens of milliseconds after the end of the L-sound and was temporally sensitive to different L-sound profiles (velocities). In addition, we found that this depended on an individual's preferred sensory modality (auditory vs. visual) as determined through a divided attention task (attentional preference), but not on their simple threshold detection level per sensory modality. Whereas individuals with “visual preference” showed enhanced phosphene perception irrespective of L-sound velocity, those with “auditory preference” showed differential peaks in phosphene perception whose delays after sound-offset followed the different L-sound velocity profiles. These novel findings suggest that looming signals modulate visual cortex excitability beyond sound duration possibly to support prompt identification and reaction to potentially dangerous approaching objects. The observed interindividual differences favor the idea that unlike early effects this late L-sound impact on visual cortex excitability is influenced by cross-modal attentional mechanisms rather than low-level sensory processes.

Evidence of multisensory interactions within low-level cortices and at early post-stimulus latencies [1-6] has prompted a paradigm shift in conceptualizations of sensory organization [7-10]. However, the mechanisms of these interactions and their link to behavior remain largely unknown. One behaviorally salient stimulus is a rapidly approaching (looming) object, which can indicate potential threats [11-13]. Based on findings from humans [14] and nonhuman primates [15, 16] suggesting there to be selective multisensory (auditory-visual) integration of looming signals, we tested whether looming sounds would selectively modulate the excitability of visual cortex. We combined transcranial magnetic stimulation (TMS) over the occipital pole and psychophysics for "neurometric" and psychometric assays of changes in low-level visual cortex excitability (i.e., phosphene induction) and perception, respectively [17, 18]. Across three experiments we show that structured looming sounds considerably enhance visual cortex excitability relative to other sound categories and white-noise controls. The time course of this effect showed that modulation of visual cortex excitability started to differ between looming and stationary sounds for sound portions of very short duration (80 ms) that were significantly below (by 35 ms) perceptual discrimination threshold. Visual perceptions are thus rapidly and efficiently boosted by sounds through early, preperceptual and stimulus-selective modulation of neuronal excitability within low-level visual cortex. © 2009 Elsevier Ltd. All rights reserved.

Current models of brain organization include multisensory interactions at early processing stages and within low-level, including primary, cortices. Embracing this model with regard to auditory-visual (AV) interactions in humans remains problematic. Controversy surrounds the application of an additive model to the analysis of event-related potentials (ERPs), and conventional ERP analysis methods have yielded discordant latencies of effects and permitted limited neurophysiologic interpretability. While hemodynamic imaging and transcranial magnetic stimulation studies provide general support for the above model, the precise timing, superadditive/subadditive directionality, topographic stability, and sources remain unresolved. We recorded ERPs in humans to attended, but task-irrelevant stimuli that did not require an overt motor response, thereby circumventing paradigmatic caveats. We applied novel ERP signal analysis methods to provide details concerning the likely bases of AV interactions. First, nonlinear interactions occur at 60-95 ms after stimulus and are the consequence of topographic, rather than pure strength, modulations in the ERP. AV stimuli engage distinct configurations of intracranial generators, rather than simply modulating the amplitude of unisensory responses. Second, source estimations (and statistical analyses thereof) identified primary visual, primary auditory, and posterior superior temporal regions as mediating these effects. Finally, scalar values of current densities in all of these regions exhibited functionally coupled, subadditive nonlinear effects, a pattern increasingly consistent with the mounting evidence in nonhuman primates. In these ways, we demonstrate how neurophysiologic bases of multisensory interactions can be noninvasively identified in humans, allowing for a synthesis across imaging methods on the one hand and species on the other. Copyright © 2010 the authors.

An object's motion relative to an observer can confer ethologically meaningful information. Approaching or looming stimuli can signal threats/collisions to be avoided or prey to be confronted, whereas receding stimuli can signal successful escape or failed pursuit. Using movement detection and subjective ratings, we investigated the multisensory integration of looming and receding auditory and visual information by humans. While prior research has demonstrated a perceptual bias for unisensory and more recently multisensory looming stimuli, none has investigated whether there is integration of looming signals between modalities. Our findings reveal selective integration of multisensory looming stimuli. Performance was significantly enhanced for looming stimuli over all other multisensory conditions. Contrasts with static multisensory conditions indicate that only multisensory looming stimuli resulted in facilitation beyond that induced by the sheer presence of auditory-visual stimuli. Controlling for variation in physical energy replicated the advantage for multisensory looming stimuli. Finally, only looming stimuli exhibited a negative linear relationship between enhancement indices for detection speed and for subjective ratings. Maximal detection speed was attained when motion perception was already robust under unisensory conditions. The preferential integration of multisensory looming stimuli highlights that complex ethologically salient stimuli likely require synergistic cooperation between existing principles of multisensory integration. A new conceptualization of the neurophysiologic mechanisms mediating real-world multisensory perceptions and action is therefore supported. © 2008 Elsevier Ltd. All rights reserved.

This work addresses the problem how to extract components of the electric field of the brain providing a maximum of information on the intention of a subject operating a Brain-Computer Interface. A supervised learning algorithm is presented that computes spatial filters maximizing mutual information of extracted features and the user's intention. Furthermore, an unsupervised learning algorithm is developed that allows to optimally extract sources from specific areas of the brain. Both methods are validated on experimental data, and it is shown that in comparison to state-of-the-art algorithms an increase in classification accuracy and a decrease of required training time is achieved. Die hier vorliegende Dissertation behandelt die Fragestellung, wie aus dem elektrischen Feld des Gehirns jene Komponenten extrahiert werden können, die maximale Information über die Intention des Benutzers einer Gehirn-Computer Schnittstelle liefern. Zum einen wird ein überwachtes Lernverfahren entworfen, mit dessen Hilfe räumliche Filter berechnet werden können, welche die Transinformation von extrahierten Merkmalen und Intention des Benutzers maximieren. Weiterhin wird ein unüberwachtes Lernverfahren präsentiert, welches es ermöglicht Signale aus spezifischen Gehirnarealen optimal zu extrahieren. Anhand experimenteller Daten wird gezeigt, dass beide Methoden im Vergleich zu etablierten Verfahren eine höhere Klassifikationsgenauigkeit und kürzere Trainingszeit ermöglichen.

Multisensory interactions occur early in time and in low-level cortical areas, including primary cortices. To test current models of early auditory-visual (AV) convergence in unisensory visual brain areas, we studied the effect of transcranial magnetic stimulation (TMS) of visual cortex on behavioral responses to unisensory (auditory or visual) or multisensory (simultaneous auditory-visual) stimulus presentation. Single-pulse TMS was applied over the occipital pole at short delays (30-150 ms) after external stimulus onset. Relative to TMS over a control site, reactions times (RTs) to unisensory visual stimuli were prolonged by TMS at 60-75 ms poststimulus onset (visual suppression effect), confirming stimulation of functional visual cortex. Conversely, RTs to unisensory auditory stimuli were significantly shortened when visual cortex was stimulated by TMS at the same delays (beneficial interaction effect of auditory stimulation and occipital TMS). No TMS-effect on RTs was observed for AV stimulation. The beneficial interaction effect of combined unisensory auditory and TMS-induced visual cortex stimulation matched and was correlated with the RT-facilitation after external multisensory AV stimulation without TMS, suggestive of multisensory interactions between the stimulus-evoked auditory and TMS-induced visual cortex activities. A follow-up experiment showed that auditory input enhances excitability within visual cortex itself (using phosphene-induction via TMS as a measure) over a similarly early time-window (75-120 ms). The collective data support a mechanism of early auditory-visual interactions that is mediated by auditory-driven sensitivity changes in visual neurons that coincide in time with the initial volleys of visual input. Copyright © 2007 Society for Neuroscience.

Multisensory interactions are a fundamental feature of brain organization. Principles governing multisensory processing have been established by varying stimulus location, timing and efficacy independently. Determining whether and how such principles operate when stimuli vary dynamically in their perceived distance (as when looming/receding) provides an assay for synergy among the above principles and also means for linking multisensory interactions between rudimentary stimuli with higher-order signals used for communication and motor planning.Humanparticipants indicated movement of looming or receding versus static stimuli that were visual, auditory, or multisensory combinations while 160-channel EEG was recorded. Multivariate EEG analyses and distributed source estimations were performed. Nonlinear interactions between looming signals were observed at early poststimulus latencies (~75 ms) in analyses of voltage waveforms, global field power, and source estimations. These looming-specific interactions positively correlated with reaction time facilitation, providing direct links between neural and performance metrics of multisensory integration. Statistical analyses of source estimations identified looming-specific interactions within the right claustrum/insula extending inferiorly into the amygdala and also within the bilateral cuneus extending into the inferior and lateral occipital cortices. Multisensory effects common to all conditions, regardless of perceived distance and congruity, followed (~115 ms) and manifested as faster transition between temporally stable brain networks (vs summed responses to unisensory conditions). We demonstrate the early-latency, synergistic interplay between existing principles of multisensory interactions. Such findings change the manner in which to model multisensory interactions at neural and behavioral/perceptual levels. We also provide neurophysiologic backing for the notion that looming signals receive preferential treatment during perception. © 2012 the authors.