Search

Connectomic reconstruction of neural circuits relies on nanometer resolution microscopy which produces on the order of a petabyte of imagery for each cubic millimeter of brain tissue. The cost of storing such data is a significant barrier to broadening the use of connectomic approaches and scaling to even larger volumes. We present an image compression approach that uses machine learning-based denoising and standard image codecs to compress raw electron microscopy imagery of neuropil up to 17-fold with negligible loss of 3d reconstruction and synaptic detection accuracy.

We acquired a rapidly preserved human surgical sample from the temporal lobe of the cerebral cortex. We stained a 1 mm3 volume with heavy metals, embedded it in resin, cut more than 5000 slices at ∼30 nm and imaged these sections using a high-speed multibeam scanning electron microscope. We used computational methods to render the three-dimensional structure containing 57,216 cells, hundreds of millions of neurites and 133.7 million synaptic connections. The 1.4 petabyte electron microscopy volume, the segmented cells, cell parts, blood vessels, myelin, inhibitory and excitatory synapses, and 104 manually proofread cells are available to peruse online. Many interesting and unusual features were evident in this dataset. Glia outnumbered neurons 2:1 and oligodendrocytes were the most common cell type in the volume. Excitatory spiny neurons comprised 69% of the neuronal population, and excitatory synapses also were in the majority (76%). The synaptic drive onto spiny neurons was biased more strongly toward excitation (70%) than was the case for inhibitory interneurons (48%). Despite incompleteness of the automated segmentation caused by split and merge errors, we could automatically generate (and then validate) connections between most of the excitatory and inhibitory neuron types both within and between layers. In studying these neurons we found that deep layer excitatory cell types can be classified into new subsets, based on structural and connectivity differences, and that chandelier interneurons not only innervate excitatory neuron initial segments as previously described, but also each other’s initial segments. Furthermore, among the thousands of weak connections established on each neuron, there exist rarer highly powerful axonal inputs that establish multi-synaptic contacts (up to ∼20 synapses) with target neurons. Our analysis indicates that these strong inputs are specific, and allow small numbers of axons to have an outsized role in the activity of some of their postsynaptic partners.

Reconstruction of neural circuitry at single-synapse resolution is a key target for improving understanding of the nervous system in health and disease. Serial section transmission electron microscopy (ssTEM) is among the most prolific imaging methods employed in pursuit of such reconstructions. We demonstrate how Flood-Filling Networks (FFNs) can be used to computationally segment a forty-teravoxel whole-brain Drosophila ssTEM volume. To compensate for data irregularities and imperfect global alignment, FFNs were combined with procedures that locally re-align serial sections as well as dynamically adjust and synthesize image content. The proposed approach produced a largely merger-free segmentation of the entire ssTEM Drosophila brain, which we make freely available. As compared to manual tracing using an efficient skeletonization strategy, the segmentation enabled circuit reconstruction and analysis workflows that were an order of magnitude faster.

The neural circuits responsible for animal behavior remain largely unknown. We summarize new methods and present the circuitry of a large fraction of the brain of the fruit fly Drosophila melanogaster. Improved methods include new procedures to prepare, image, align, segment, find synapses in, and proofread such large data sets. We define cell types, refine computational compartments, and provide an exhaustive atlas of cell examples and types, many of them novel. We provide detailed circuits consisting of neurons and their chemical synapses for most of the central brain. We make the data public and simplify access, reducing the effort needed to answer circuit questions, and provide procedures linking the neurons defined by our analysis with genetic reagents. Biologically, we examine distributions of connection strengths, neural motifs on different scales, electrical consequences of compartmentalization, and evidence that maximizing packing density is an important criterion in the evolution of the fly’s brain., eLife digest Animal brains of all sizes, from the smallest to the largest, work in broadly similar ways. Studying the brain of any one animal in depth can thus reveal the general principles behind the workings of all brains. The fruit fly Drosophila is a popular choice for such research. With about 100,000 neurons – compared to some 86 billion in humans – the fly brain is small enough to study at the level of individual cells. But it nevertheless supports a range of complex behaviors, including navigation, courtship and learning. Thanks to decades of research, scientists now have a good understanding of which parts of the fruit fly brain support particular behaviors. But exactly how they do this is often unclear. This is because previous studies showing the connections between cells only covered small areas of the brain. This is like trying to understand a novel when all you can see is a few isolated paragraphs. To solve this problem, Scheffer, Xu, Januszewski, Lu, Takemura, Hayworth, Huang, Shinomiya et al. prepared the first complete map of the entire central region of the fruit fly brain. The central brain consists of approximately 25,000 neurons and around 20 million connections. To prepare the map – or connectome – the brain was cut into very thin 8nm slices and photographed with an electron microscope. A three-dimensional map of the neurons and connections in the brain was then reconstructed from these images using machine learning algorithms. Finally, Scheffer et al. used the new connectome to obtain further insights into the circuits that support specific fruit fly behaviors. The central brain connectome is freely available online for anyone to access. When used in combination with existing methods, the map will make it easier to understand how the fly brain works, and how and why it can fail to work correctly. Many of these findings will likely apply to larger brains, including our own. In the long run, studying the fly connectome may therefore lead to a better understanding of the human brain and its disorders. Performing a similar analysis on the brain of a small mammal, by scaling up the methods here, will be a likely next step along this path.

The neural circuits responsible for animal behavior remain largely unknown. We summarize new methods and present the circuitry of a large fraction of the brain of the fruit fly Drosophila melanogaster. Improved methods include new procedures to prepare, image, align, segment, find synapses in, and proofread such large data sets. We define cell types, refine computational compartments, and provide an exhaustive atlas of cell examples and types, many of them novel. We provide detailed circuits consisting of neurons and their chemical synapses for most of the central brain. We make the data public and simplify access, reducing the effort needed to answer circuit questions, and provide procedures linking the neurons defined by our analysis with genetic reagents. Biologically, we examine distributions of connection strengths, neural motifs on different scales, electrical consequences of compartmentalization, and evidence that maximizing packing density is an important criterion in the evolution of the fly’s brain.

The neural circuits responsible for behavior remain largely unknown. Previous efforts have reconstructed the complete circuits of small animals, with hundreds of neurons, and selected circuits for larger animals. Here we (the FlyEM project at Janelia and collaborators at Google) summarize new methods and present the complete circuitry of a large fraction of the brain of a much more complex animal, the fruit fly Drosophila melanogaster. Improved methods include new procedures to prepare, image, align, segment, find synapses, and proofread such large data sets; new methods that define cell types based on connectivity in addition to morphology; and new methods to simplify access to a large and evolving data set. From the resulting data we derive a better definition of computational compartments and their connections; an exhaustive atlas of cell examples and types, many of them novel; detailed circuits for most of the central brain; and exploration of the statistics and structure of different brain compartments, and the brain as a whole. We make the data public, with a web site and resources specifically designed to make it easy to explore, for all levels of expertise from the expert to the merely curious. The public availability of these data, and the simplified means to access it, dramatically reduces the effort needed to answer typical circuit questions, such as the identity of upstream and downstream neural partners, the circuitry of brain regions, and to link the neurons defined by our analysis with genetic reagents that can be used to study their functions.Note: In the next few weeks, we will release a series of papers with more involved discussions. One paper will detail the hemibrain reconstruction with more extensive analysis and interpretation made possible by this dense connectome. Another paper will explore the central complex, a brain region involved in navigation, motor control, and sleep. A final paper will present insights from the mushroom body, a center of multimodal associative learning in the fly brain.

Reconstruction of neural circuitry at single-synapse resolution is a key target for improving understanding of the nervous system in health and disease. Serial section transmission electron microscopy (ssTEM) is among the most prolific imaging methods employed in pursuit of such reconstructions. We demonstrate how Flood-Filling Networks (FFNs) can be used to computationally segment a forty-teravoxel whole-brain Drosophila ssTEM volume. To compensate for data irregularities and imperfect global alignment, FFNs were combined with procedures that locally re-align serial sections as well as dynamically adjust and synthesize image content. The proposed approach produced a largely merger-free segmentation of the entire ssTEM Drosophila brain, which we make freely available. As compared to manual tracing using an efficient skeletonization strategy, the segmentation enabled circuit reconstruction and analysis workflows that were an order of magnitude faster.

OBJECTIVEThe activation of the sensorimotor cortex as measured by electrocorticographic (ECoG) signals has been correlated with contralateral hand movements in humans, as precisely as the level of individual digits. However, the relationship between individual and multiple synergistic finger movements and the neural signal as detected by ECoG has not been fully explored. The authors used intraoperative high-resolution micro-ECoG (µECoG) on the sensorimotor cortex to link neural signals to finger movements across several context-specific motor tasks.METHODSThree neurosurgical patients with cortical lesions over eloquent regions participated. During awake craniotomy, a sensorimotor cortex area of hand movement was localized by high-frequency responses measured by an 8 × 8 µECoG grid of 3-mm interelectrode spacing. Patients performed a flexion movement of the thumb or index finger, or a pinch movement of both, based on a visual cue. High-gamma (HG; 70–230 Hz) filtered µECoG was used to identify dominant electrodes associated with thumb and index movement. Hand movements were recorded by a dataglove simultaneously with µECoG recording.RESULTSIn all 3 patients, the electrodes controlling thumb and index finger movements were identifiable approximately 3–6-mm apart by the HG-filtered µECoG signal. For HG power of cortical activation measured with µECoG, the thumb and index signals in the pinch movement were similar to those observed during thumb-only and index-only movement, respectively (all p > 0.05). Index finger movements, measured by the dataglove joint angles, were similar in both the index-only and pinch movements (p > 0.05). However, despite similar activation across the conditions, markedly decreased thumb movement was observed in pinch relative to independent thumb-only movement (all p < 0.05).CONCLUSIONSHG-filtered µECoG signals effectively identify dominant regions associated with thumb and index finger movement. For pinch, the µECoG signal comprises a combination of the signals from individual thumb and index movements. However, while the relationship between the index finger joint angle and HG-filtered signal remains consistent between conditions, there is not a fixed relationship for thumb movement. Although the HG-filtered µECoG signal is similar in both thumb-only and pinch conditions, the actual thumb movement is markedly smaller in the pinch condition than in the thumb-only condition. This implies a nonlinear relationship between the cortical signal and the motor output for some, but importantly not all, movement types. This analysis provides insight into the tuning of the motor cortex toward specific types of motor behaviors.

Reconstruction of neural circuits from volume electron microscopy data requires the tracing of cells in their entirety, including all their neurites. Automated approaches have been developed for tracing, but their error rates are too high to generate reliable circuit diagrams without extensive human proofreading. We present flood-filling networks, a method for automated segmentation that, similar to most previous efforts, uses convolutional neural networks, but contains in addition a recurrent pathway that allows the iterative optimization and extension of individual neuronal processes. We used flood-filling networks to trace neurons in a dataset obtained by serial block-face electron microscopy of a zebra finch brain. Using our method, we achieved a mean error-free neurite path length of 1.1 mm, and we observed only four mergers in a test set with a path length of 97 mm. The performance of flood-filling networks was an order of magnitude better than that of previous approaches applied to this dataset, although with substantially increased computational costs.

Electrocorticography (ECoG) signals can provide high spatio-temporal resolution and high signal to noise ratio recordings of local neural activity from the surface of the brain. Previous studies have shown that broad-band, spatially focal, high-frequency increases in ECoG signals are highly correlated with movement and other cognitive tasks and can be volitionally modulated. However, significant additional information may be present in inter-electrode interactions, but adding additional higher order inter-electrode interactions can be impractical from a computational aspect, if not impossible.In this paper we present a new method of calculating high frequency interactions between electrodes called Short-Time Windowed Covariance (STWC) that builds on mathematical techniques currently used in neural signal analysis, along with an implementation that accelerates the algorithm by orders of magnitude by leveraging commodity, off-the-shelf graphics processing unit (GPU) hardware.Using the hardware-accelerated implementation of STWC, we identify many types of event-related inter-electrode interactions from human ECoG recordings on global and local scales that have not been identified by previous methods. Unique temporal patterns are observed for digit flexion in both low- (10mm spacing) and high-resolution (3mm spacing) electrode arrays.Covariance is a commonly used metric for identifying correlated signals, but the standard covariance calculations do not allow for temporally varying covariance. In contrast STWC allows and identifies event-driven changes in covariance without identifying spurious noise correlations.STWC can be used to identify event-related neural interactions whose high computational load is well suited to GPU capabilities.

The majority of subjects who attempt to learn control of a brain–computer interface (BCI) can do so with adequate training. Much like when one learns to type or ride a bicycle, BCI users report transitioning from a deliberate, cognitively focused mindset to near automatic control as training progresses. What are the neural correlates of this process of BCI skill acquisition? Seven subjects were implanted with electrocorticography (ECoG) electrodes and had multiple opportunities to practice a 1D BCI task. As subjects became proficient, strong initial task-related activation was followed by lessening of activation in prefrontal cortex, premotor cortex, and posterior parietal cortex, areas that have previously been implicated in the cognitive phase of motor sequence learning and abstract task learning. These results demonstrate that, although the use of a BCI only requires modulation of a local population of neurons, a distributed network of cortical areas is involved in the acquisition of BCI proficiency.

Object The gold-standard method for determining cortical functional organization in the context of neurosurgical intervention is electrical cortical stimulation (ECS), which disrupts normal cortical function to evoke movement. This technique is imprecise, however, as motor responses are not limited to the precentral gyrus. Electrical cortical stimulation also can trigger seizures, is not always tolerated, and is often unsuccessful, especially in children. Alternatively, endogenous motor and sensory signals can be mapped by somatosensory evoked potentials (SSEPs), functional MRI (fMRI), and electrocorticography of high gamma (70–150 Hz) signal power, which reflect normal cortical function. The authors evaluated whether these 4 modalities of mapping sensorimotor function in children produce concurrent results. Methods The authors retrospectively examined the charts of all patients who underwent epilepsy surgery at Seattle Children's Hospital between July 20, 1999, and July 1, 2011, and they included all patients in whom the primary motor or somatosensory cortex was localized via 2 or more of the following tests: ECS, SSEP, fMRI, or high gamma electrocorticography (hgECoG). Results Inclusion criteria were met by 50 patients, whose mean age at operation was 10.6 years. The youngest patient who underwent hgECoG mapping was 2 years and 10 months old, which is younger than any patient reported on in the literature. The authors localized the putative sensorimotor cortex most often with hgECoG, followed by SSEP and fMRI; ECS was most likely to fail to localize the sensorimotor cortex. Conclusions Electrical cortical stimulation, SSEP, fMRI, and hgECoG generally produced concordant localization of motor and sensory function in children. When attempting to localize the sensorimotor cortex in children, hgECoG was more likely to produce results, was faster, safer, and did not require cooperation. The hgECoG maps in pediatric patients are similar to those in adult patients published in the literature. The sensorimotor cortex can be mapped by hgECoG and fMRI in children younger than 3 years old to localize cortical function.

Objective Human voluntary movements are a final product of complex interactions between multiple sensory, cognitive and motor areas of central nervous system. The objective was to investigate temporal sequence of activation of premotor (PM), primary motor (M1) and somatosensory (S1) areas during cued finger movements. Methods Electrocorticography (ECoG) was used to measure activation timing in human PM, S1, and M1 neurons in preparation for finger movements in 5 subjects with subdural grids for seizure localization. Cortical activation was determined by the onset of high gamma (HG) oscillation (70–150Hz). The three cortical regions were mapped anatomically using a common brain atlas and confirmed independently with direct electrical cortical stimulation, somatosensory evoked potentials and detection of HG response to tactile stimulation. Subjects were given visual cues to flex each finger or pinch the thumb and index finger. Movements were captured with a dataglove and time-locked with ECoG. A windowed covariance metric was used to identify the rising slope of HG power between two electrodes and compute time lag. Statistical constraints were applied to the time estimates to combat the noise. Rank sum testing was used to verify the sequential activation of cortical regions across 5 subjects. Results In all 5 subjects, HG activation in PM preceded S1 by an average of 53±13ms ( P =0.03), PM preceded M1 by 180±40ms ( P =0.001) and S1 activation preceded M1 by 136±40ms ( P =0.04). Conclusions Sequential HG activation of PM, S1 and M1 regions in preparation for movements is reported. Activity in S1 prior to any overt body movements supports the notion that these neurons may encode sensory information in anticipation of movements, i.e., an efference copy. Our analysis suggests that S1 modulation likely originates from PM. Significance First electrophysiological evidence of efference copy in humans.

Human subjects can learn to control a one-dimensional electrocorticographic (ECoG) brain-computer interface (BCI) using modulation of primary motor (M1) high-gamma activity (signal power in the 75–200 Hz range). However, the stability and dynamics of the signals over the course of new BCI skill acquisition have not been investigated. In this study, we report three characteristic periods in evolution of the high-gamma control signal during BCI training: initial, low task accuracy with corresponding low power modulation in the gamma spectrum, followed by a second period of improved task accuracy with increasing average power separation between activity and rest, and a final period of high task accuracy with stable (or decreasing) power separation and decreasing trial-to-trial variance. These findings may have implications in the design and implementation of BCI control algorithms.

Dorsolateral Prefrontal Cortex (DLPFC) has been associated with goal encoding in primates. Thus far, the majority of research involving DLPFC, including all electrophysiology studies, has been performed in non-human primates. In this paper, we explore the possibility of utilizing the cortical activity in DLPFC in humans for use in Brain-Computer Interfaces (BCIs). Electrocorticographic signals were recorded from seven patients with intractable epilepsy who had electrode coverage over DLPFC. These subjects performed a visuomotor target-based task to assess DLPFC's involvement in planning, execution, and accomplishment of the simple motor task. These findings demonstrate that there is a distinct high-frequency spectral component in DLPFC associated with accomplishment of the task. It is envisioned that these signals could potentially provide a novel verification of task accomplishment for a BCI.

Brain-computer interfaces (BCIs)—systems that can record neural activity and translate them into commands for computer systems—are sufficiently advanced to allow users to volitionally guide them through simple tasks. Contemporary BCI research focuses on squeezing additional functionality out of standardized paradigms, be it achieving more bits per second, increased degrees of freedom, or increasing accuracy. While these studies have shown marginal advancements in recent years, our lack of understanding concerning the underlying neurophysiology continues to be the limiting factor in BCI development. In this chapter, we propose turning the way research is done on BCI systems on its head; instead of using our understanding of neural signals to incrementally advance the state of brain-machine interfaces, we apply a BCI system as a form of experimental control to study changes in neural activity. By using current BCI systems as a tool for neuroscientific study, we can probe the underlying neuroanatomy in novel, behaviorally controlled ways.

The learning of a motor task is known to be improved by sleep, and sleep spindles are thought to facilitate this learning by enabling synaptic plasticity. In this study subjects implanted with electrocorticography (ECoG) arrays for long-term epilepsy monitoring were trained to control a cursor on a computer screen by modulating either the high-gamma or mu/beta power at a single electrode located over the motor or premotor area. In all trained subjects, spindle density in posttraining sleep was increased with respect to pretraining sleep in a remarkably spatially specific manner. The pattern of increased spindle activity reflects the functionally specific regions that were involved in learning of a highly novel and salient task during wakefulness, supporting the idea that sleep spindles are involved in learning to use a motor-based brain–computer interface device.

Invasive and non-invasive brain–computer interface (BCI) studies have long focused on the motor cortex for kinematic control of artificial devices. Most of these studies have used single-neuron recordings or electroencephalography (EEG). Electrocorticography (ECoG) is a relatively new recording modality in BCI research that has primarily been built on successes in EEG recordings. We built on prior experiments related to single-neuron recording and quantitatively compare the extent to which different brain regions reflect kinematic tuning parameters of hand speed, direction, and velocity in both a reaching and tracing task in humans. Hand and arm movement experiments using ECoG have shown positive results before, but the tasks were not designed to tease out which kinematics are encoded. In non-human primates, the relationships among these kinematics have been more carefully documented, and we sought to begin elucidating that relationship in humans using ECoG. The largest modulation in ECoG activity for direction, speed, and velocity representation was found in the primary motor cortex. We also found consistent cosine tuning across both tasks, to hand direction and velocity in the high gamma band (70–160 Hz). Thus, the results of this study clarify the neural substrates involved in encoding aspects of motor preparation and execution and confirm the important role of the motor cortex in BCI applications.

In this paper we present our low-level software framework that allows human neuromuscular properties and behaviors to be programmed modularly onto biologically inspired prosthetic devices. Specifically this software framework is used at the lowest level of the Anatomically Correct Testbed (ACT) Hand. It abstracts away from the robotic details and allows users including biologists to program individual muscle physiology, multiple muscle synergistic activities, neural commands, etc. independently. Our system has been designed to be fault-tolerant with the idea of safe and continued operation of prostheses if a new prototype behavior were to perform incorrectly or outright fail. Example coding of Hills model is shown.

We present results of cortical activity during phoneme pronunciation, recorded using miniaturized electrocorticography grids with high spatial resolution. A patient implanted with the miniature grid was instructed to audibly pronounce one of four phonemes. For each phoneme, we observed distinct spatial correlation patterns at the 3mm electrode spacing. We applied a support vector machine classification scheme and, for the first time, were able to distinguish discrete phonemes with high accuracy. In addition, we found that sub-regions of our miniature array were specific for distinct pairs of phonemes, showing that cortical phoneme processing occurs at a higher resolution than previously though.

Three human subjects participated in a closed-loop brain computer interface cursor control experiment mediated by implanted subdural electrocorticographic arrays. The paradigm consisted of several stages: baseline recording, hand and tongue motor tasks as the basis for feature selection, two closed-loop one-dimensional feedback experiments with each of these features, and a two-dimensional feedback experiment using both of the features simultaneously. The two selected features were simple channel and frequency band combinations associated with change during hand and tongue movement. Inter-feature correlation and cross-correlation between features during different epochs of each task were quantified for each stage of the experiment. Our anecdotal, three subject, result suggests that while high correlation between horizontal and vertical control signal can initially preclude successful two-dimensional cursor control, a feedback-based learning strategy can be successfully employed by the subject to overcome this limitation and progressively decorrelate these control signals.

All previous multiple-day brain-computer interface (BCI) experiments have dynamically adjusted the parameterization between the signals measured from the brain and the features used to control the interface. The authors present the results of a multiple-day electrocorticographic (ECoG) BCI experiment. A patient with a subdural electrode array implanted for seizure localization performed tongue motor tasks. After an initial screening and feature selection on the 1st day, 5 consecutive days of cursor-based feedback were performed with a fixed parameterization. Control of the interface was robust throughout all days, with performance increasing to a stable state in which high-frequency ECoG signal could immediately be translated into cursor control. These findings demonstrate that ECoG-based BCIs can be implemented for multiple-day control without the necessity for sophisticated retraining and adaptation.