1. Dynamic allocation of computational resources for deep learning-enabled cellular image analysis with Kubernetes
- Author
-
Bannon, Dylan, Moen, Erick, Schwartz, Morgan, Borba, Enrico, Cui, Sunny, Huang, Kevin, Camplisson, Isabella, Koe, Nora, Kyme, Daniel, Kudo, Takamasa, Chang, Brian, Pao, Edward, Osterman, Erik, Graf, William, and Van Valen, David
- Abstract
Deep learning is transforming the ability of life scientists to extract information from images. These techniques have better accuracy than conventional approaches and enable previously impossible analyses. As the capability of deep learning methods expands, they are increasingly being applied to large imaging datasets. The computational demands of deep learning present a significant barrier to large-scale image analysis. To meet this challenge, we have developed DeepCell 2.0, a platform for deploying deep learning models on large imaging datasets (>10 5 -megapixel images) in the cloud. This software enables the turnkey deployment of a Kubernetes cluster on all commonly used operating systems. By using a microservice architecture, our platform matches computational operations with their hardware requirements to reduce operating costs. Further, it scales computational resources to meet demand, drastically reducing the time necessary for analysis of large datasets. A thorough analysis of costs demonstrates that cloud computing is economically competitive for this application. By treating hardware infrastructure as software, this work foreshadows a new generation of software packages for biology in which computational resources are a dynamically allocated resource.
- Published
- 2018
- Full Text
- View/download PDF