1. A Multicore Path to Connectomics-on-Demand
- Author
-
Aleksandar Zlateski, Hayk Saribekyan, Tim Kaler, David Budden, Alexander Matveev, Yaron Meirovitch, Gergely Ódor, Wiktor Jakubiuk, Nir Shavit, Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology. Department of Mathematics, Matveev, Alexander, Meirovitch, Yaron, Saribekyan, Hayk, Jakubiuk, Wiktor B., Kaler, Timothy, Odor, Gergely, Budden, David, Zlateski, Aleksandar, and Shavit, Nir N.
- Subjects
0301 basic medicine ,Multi-core processor ,Connectomics ,Computer science ,Image processing ,Parallel computing ,Terabyte ,Computer Graphics and Computer-Aided Design ,Mass storage ,03 medical and health sciences ,030104 developmental biology ,0302 clinical medicine ,Path (graph theory) ,Enhanced Data Rates for GSM Evolution ,030217 neurology & neurosurgery ,Software - Abstract
The current design trend in large scale machine learning is to use distributed clusters of CPUs and GPUs with MapReduce-style programming. Some have been led to believe that this type of horizontal scaling can reduce or even eliminate the need for traditional algorithm development, careful parallelization, and performance engineering. This paper is a case study showing the contrary: that the benefits of algorithms, parallelization, and performance engineering, can sometimes be so vast that it is possible to solve "cluster-scale" problems on a single commodity multicore machine. Connectomics is an emerging area of neurobiology that uses cutting edge machine learning and image processing to extract brain connectivity graphs from electron microscopy images. It has long been assumed that the processing of connectomics data will require mass storage, farms of CPU/GPUs, and will take months (if not years) of processing time. We present a high-throughput connectomics-on-demand system that runs on a multicore machine with less than 100 cores and extracts connectomes at the terabyte per hour pace of modern electron microscopes., National Science Foundation (U.S.) (grant IIS-1447786), National Science Foundation (U.S.) (grant CCF1563880), United States. Intelligence Advanced Research Projects Activity (grant 138076-5093555)
- Published
- 2017