From Images to Graphs: Machine Learning Methods for the Detection of Microtubules and Synapses in Large-Scale Electron Microscopy Data

Brain wiring diagrams showing every individual neuron and all the synaptic connections are becoming an important resource for neuroscientists. However, only a few such high-resolution wiring diagrams have been reconstructed so far. Emerging high-throughput electron microscopy (EM) technologies have started to fill this critical gap. EM images of neural tissue with sufficiently high resolution at large scale allow the extraction of the wiring diagram. The sheer size of acquired datasets precludes manual analysis and makes the development of computer-based automatic methods necessary. In this work, we propose and test new methods to address the problem of automatic identification of microtubules and synaptic partners in large-scale EM image datasets with the ultimate goal to aid circuit reconstruction. In the first part of the thesis, we introduce a method for the automatic reconstruction of microtubules. Microtubules follow the backbone of a neuron and can be an important source of constraints for the reconstruction of neurons. We formulate an energy-based model on short candidate segments of microtubules found by a local classifier. We enumerate and score possible links between candidates, in order to find a cost-minimal subset of candidates and links by solving an integer linear program. The model provides a way to incorporate biological priors including both hard constraints (e.g. microtubules are topologically chains of links) and soft constraints (e.g. high curvature is unlikely). We test our method on a challenging EM dataset of Drosophila neural tissue and show that our model reliably tracks microtubules spanning many image sections. In the second part of the thesis, we propose a method for the prediction of synaptic partners, which is, along with the segmentation of neurons, required for circuit reconstruction. For the prediction of synaptic partners, we propose a 3D U-Net architecture to directly identify pairs of voxels that are pre- and post-synaptic to each other. To that end, we formulate the problem of synaptic partner identification as a classification problem on long-range edges between voxels to encode both the presence of a synaptic pair and its direction. This formulation allows us to directly learn from synaptic point annotations instead of the more labor-intensive voxel-based synaptic cleft or vesicle annotations. We evaluate our method on the MICCAI 2016 CREMI challenge and improve over the current state of the art, producing 3% fewer errors than the next best method. The third and last part of the thesis is also dedicated to the reconstruction of synaptic partners. Compared to the previously introduced method, we propose to directly predict post-synaptic locations and the direction to their pre-synaptic partner. We use our method to extract 244 million putative synaptic partners in the fifty-teravoxel full adult fly brain (FAFB) EM dataset and evaluated its accuracy on 146,643 synapses of 702 neurons with a total cable length of 312 mm in four different brain regions. We find that the predicted synaptic connections can be used together with a neuron segmentation to infer a connectivity graph with high accuracy. Our synaptic partner predictions for the FAFB dataset are publicly available, together with a query library allowing automatic retrieval of up- and downstream neurons. The three methods described in this work have produced state-of-the-art results on two very challenging, highly anisotropic EM datasets. The 244 million putative synaptic partners in the FAFB dataset will be a valuable resource for neuroscientists. While the fully automatic extraction of large wiring diagrams is still impeded by high accuracy requirements, this work is an important contribution for the acceleration of reconstruction efforts.