Search

The neural circuits responsible for behavior remain largely unknown. Previous efforts have reconstructed the complete circuits of small animals, with hundreds of neurons, and selected circuits for larger animals. Here we (the FlyEM project at Janelia and collaborators at Google) summarize new methods and present the complete circuitry of a large fraction of the brain of a much more complex animal, the fruit fly Drosophila melanogaster. Improved methods include new procedures to prepare, image, align, segment, find synapses, and proofread such large data sets; new methods that define cell types based on connectivity in addition to morphology; and new methods to simplify access to a large and evolving data set. From the resulting data we derive a better definition of computational compartments and their connections; an exhaustive atlas of cell examples and types, many of them novel; detailed circuits for most of the central brain; and exploration of the statistics and structure of different brain compartments, and the brain as a whole. We make the data public, with a web site and resources specifically designed to make it easy to explore, for all levels of expertise from the expert to the merely curious. The public availability of these data, and the simplified means to access it, dramatically reduces the effort needed to answer typical circuit questions, such as the identity of upstream and downstream neural partners, the circuitry of brain regions, and to link the neurons defined by our analysis with genetic reagents that can be used to study their functions.Note: In the next few weeks, we will release a series of papers with more involved discussions. One paper will detail the hemibrain reconstruction with more extensive analysis and interpretation made possible by this dense connectome. Another paper will explore the central complex, a brain region involved in navigation, motor control, and sleep. A final paper will present insights from the mushroom body, a center of multimodal associative learning in the fly brain.

Recent advances in automatic image segmentation and synapse prediction in electron microscopy (EM) datasets of the brain enable more efficient reconstruction of neural connectivity. In these datasets, a single neuron can span thousands of images containing complex tree-like arbors with thousands of synapses. While image segmentation algorithms excel within narrow fields of views, the algorithms sometimes struggle to correctly segment large neurons, which require large context given their size and complexity. Conversely, humans are comparatively good at reasoning with large objects. In this paper, we introduce several semi-automated strategies that combine 3D visualization and machine guidance to accelerate connectome reconstruction. In particular, we introduce a strategy to quickly correct a segmentation through merging and cleaving, or splitting a segment along supervoxel boundaries, with both operations driven by affinity scores in the underlying segmentation. We deploy these algorithms as streamlined workflows in a tool called Neu3 and demonstrate superior performance compared to prior work, thus enabling efficient reconstruction of much larger datasets. The insights into proofreading from our work clarify the trade-offs to consider when tuning the parameters of image segmentation algorithms.

The aim in high-resolution connectomics is to reconstruct complete neuronal connectivity in a tissue. Currently, the only technology capable of resolving the smallest neuronal processes is electron microscopy (EM). Thus, a common approach to network reconstruction is to perform (error-prone) automatic segmentation of EM images, followed by manual proofreading by experts to fix errors. We have developed an algorithm and software library to not only improve the accuracy of the initial automatic segmentation, but also point out the image coordinates where it is likely to have made errors. Our software, called gala (graph-based active learning of agglomeration), improves the state of the art in agglomerative image segmentation. It is implemented in Python and makes extensive use of the scientific Python stack (numpy, scipy, networkx, scikit-learn, scikit-image, and others). We present here the software architecture of the gala library, and discuss several designs that we consider would be generally useful for other segmentation packages. We also discuss the current limitations of the gala library and how we intend to address them.

An artificial neural network can be trained to detect a structure of interest within medica1 images. The method entails the training of the neural net with hand-drawn masks using the backpropagation algorithm. Multiple neural nets, each a detector for a different structure, can be created in this fashion. In order to enhance the detection of a single target structure, a final neural net can be trained to incorporate the output of a primary target detector as well as related structure detectors. This paper describes a method for aorta detection using neural nets trained to detect the aorta, spinal cord, and inferior vena cava.

In this paper, algorithms for the segmentation of head structures within three-dimensional magnetic resonance images are described. The process can be described as passing the raw images through a series of image processing programs to form afinal image with voxel values corresponding to tissue type. This resulting image can be loaded into a commercial graphics package in order to produce near photorealistic pictures and spinning animations of head structures.

Three experience-based learning techniques are explored using the game Go-Moku (Connect-5). The first method, an exception tree, is used to prevent poor lines of play by recording critical moves in a move tree. The second method simply records all games in a large experience tree, backtracking the eventual outcomes in traditional minimax fashion. Both techniques allow a computer player to modify its behavior based on past experience, and, therefore, typically defeat static game programs. The last method explored is the use of a multilayer feedforward artificial neural network for strategy calculation. The network was trained on selected interior nodes of the experience tree using the backpropagation algorithm. The neural network strategy algorithm compares favorably with a fine-tuned hand-crafted strategy algorithm. >

With the advent of fast 3D magnetic resonance imaging (MRI) sequences, truly 3D volumes of data can be routinely acquired. While images produced by modalities such as positron emission tomography (PET;) and single photon emission computed tomography (SPECT) depict functional information, newer MRI techniques like magnetization-prepared rapid gradient-echo (MP-RAGE) capture a great deal of ana tomical detail [3,10]. Such 3D images are used by clinicians in two types of tasks, visualization and quLantifi cation. Visualization, in its most basic form, permits a user to see structures of interest within a volume of data. The "structures of interest" may not correspond to any physically visible phenomenon (e.g. the quan tity of blood flow to neural tissue in a functional image) but the process of visualization transforms the data into images which may then be displayed using computer graphics. On the other hand, quantification, while often depicted graphically, attempts to reduce the mass of data into numbers useful as clinical indicators. A major obstacle to both of these tasks is the prerequisite image segmentation. In order for the brain to be visualized beneath the overlying head, a segmentation step must determine which volume elements, or voxels, in the 3D head image correspond to the brain. Similarly, quantification requires the distinguishing of background voxels from voxels corresponding to the VOl. A growing number of clinical studies depend on volume measurements after segmentation. Examples include: tracking the progression/remission of disease processes (e.g. the size of intracranial tumors); evaluating the neuroanatomical abnormalities associated with schizophrenia (e.g. ventricular volumes); and determining atrophy associated with Alzheimer-type dementia and temporal lobe epilepsy (as in the hippocampal formation).

An automated procedure for segmentation of the brain from 3-D MR images of the head is described. This process combines some heuristics with a number of three-dimensional image processing and computer vision techniques including seed-based volume growing, DOG convolution and zero-crossing detection, convolution with the Zucker-Hummel operator, and watershed anaylsis. There are two broad steps: (1) rough estimation of brain voxels, and (2) refinement of the first step through a reverse-gravity watershed analysis. All operations are performed in three-dimensions in order to fully utilize the information present in the voxels generated by the 3-D MP-RAGE sequence.

This paper describes the rationale and importance of utilizing a priori, model based information to guide segmentation and classification in complex medical images represented by Magnetic Resonance Imaging (MRI). The incorporation of such a priori, model based information requires the development of a "top down" model driven system, rather than the more traditional "bottom up" data driven system. Two different examples which incorporate such model based a priori information are discussed: (1) The segmentation and classification of tissues involved in atherosclerosis; and (2) The segmentation and classification of brain tissue for neurosurgical applications.