Bannon, Dylan and Moen, Erick and Schwartz, Morgan and Borba, Enrico and Kudo, Takamasa and Greenwald, Noah and Vijayakumar, Vibha and Chang, Brian and Pao, Edward and Osterman, Erik and Graf, William and Van Valen, David (2021) DeepCell Kiosk: scaling deep learning–enabled cellular image analysis with Kubernetes. Nature Methods, 18 (1). pp. 43-45. ISSN 1548-7091. https://resolver.caltech.edu/CaltechAUTHORS:20190916-101510993
![[img]](https://authors.library.caltech.edu/style/images/fileicons/application_pdf.png) |
PDF (Sept. 16, 2020)
- Submitted Version
Creative Commons Attribution Non-commercial. 2MB |
|
PDF (Supplementary Notes 1–8)
- Supplemental Material
See Usage Policy. 2MB |
|
PDF (Reporting Summary)
- Supplemental Material
See Usage Policy. 2MB |
Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20190916-101510993
Abstract
Deep learning is transforming the analysis of biological images, but applying these models to large datasets remains challenging. Here we describe the DeepCell Kiosk, cloud-native software that dynamically scales deep learning workflows to accommodate large imaging datasets. To demonstrate the scalability and affordability of this software, we identified cell nuclei in 10⁶ 1-megapixel images in ~5.5 h for ~US$250, with a cost below US$100 achievable depending on cluster configuration. The DeepCell Kiosk can be downloaded at https://github.com/vanvalenlab/kiosk-console; a persistent deployment is available at https://deepcell.org/.
| Item Type: | Article | ||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Related URLs: |
| ||||||||||||||||||||||||||||||
| ORCID: |
| ||||||||||||||||||||||||||||||
| Alternate Title: | Dynamic allocation of computational resources for deep learning-enabled cellular image analysis with Kubernetes | ||||||||||||||||||||||||||||||
| Additional Information: | © The Author(s), under exclusive licence to Springer Nature America, Inc. 2021. Received 15 October 2019; Accepted 23 November 2020; Published 04 January 2021. We thank numerous colleagues including A. Anandkumar, M. Angelo, J. Bois, I. Brown, A. Butkovic, L. Cai, I. Camplisson, M. Covert, M. Elowitz, J. Freeman, C. Frick, L. Geontoro, A. Ho, K. Huang, K. C. Huang, G. Johnson, L. Keren, D. Litovitz, D. Macklin, U. Manor, S. Patel, A. Raj, N. Pelaez Restrepo, C. Pavelchek, S. Shah and M. Thomson for helpful discussions and contributing data. We gratefully acknowledge support from the Shurl and Kay Curci Foundation, the Rita Allen Foundation, the Paul Allen Family Foundation through the Allen Discovery Center at Stanford University, the Rosen Center for Bioengineering at Caltech, Google Research Cloud, Figure 8’s AI For Everyone award, and a subaward from NIH U24-CA224309-01. Data availability: All data that were used to generate the figures in this paper are available at https://deepcell.org/data and at https://github.com/vanvalenlab/deepcell-tf under the deepcell.datasets module. Code availability: We used Kubernetes and TensorFlow, along with the scientific computing stack for Python. A persistent deployment of the software described can be accessed at https://deepcell.org/. All source code, including version requirements and explicit usage, is under a modified Apache license and is available at https://github.com/vanvalenlab. Detailed instructions are available at https://deepcell-kiosk.readthedocs.io. Author Contributions: D.B., W.G. and D.V.V. conceived the project; D.B., W.G., E.O. and D.V.V. designed the software architecture; D.B., E.O. and W.G. wrote the core components of the software; D.B., E.M., M.S., E.B., V.V., B.C., E.O., W.G. and D.V.V. contributed to the code base; T.K. and E.P. collected data for annotation; E.M., M.S., N.G., D.B., W.G. and D.V.V. wrote documentation; D.B., E.M., W.G. and D.V.V. wrote the paper; D.V.V. supervised the project. Competing interests: The authors have filed a provisional patent for the described work; the software described here is available under a modified Apache license and is free for non-commercial uses. Peer review information: Nature Methods thanks Ola Spjuth and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Rita Strack was the primary editor on this article and managed its editorial process and peer review in collaboration with the rest of the editorial team. | ||||||||||||||||||||||||||||||
| Errata: | Bannon, D., Moen, E., Schwartz, M. et al. Publisher Correction: DeepCell Kiosk: scaling deep learning–enabled cellular image analysis with Kubernetes. Nat Methods (2021). https://doi.org/10.1038/s41592-021-01059-w | ||||||||||||||||||||||||||||||
| Group: | Rosen Bioengineering Center | ||||||||||||||||||||||||||||||
| Funders: |
| ||||||||||||||||||||||||||||||
| Issue or Number: | 1 | ||||||||||||||||||||||||||||||
| Record Number: | CaltechAUTHORS:20190916-101510993 | ||||||||||||||||||||||||||||||
| Persistent URL: | https://resolver.caltech.edu/CaltechAUTHORS:20190916-101510993 | ||||||||||||||||||||||||||||||
| Official Citation: | Bannon, D., Moen, E., Schwartz, M. et al. DeepCell Kiosk: scaling deep learning–enabled cellular image analysis with Kubernetes. Nat Methods 18, 43–45 (2021). https://doi.org/10.1038/s41592-020-01023-0 | ||||||||||||||||||||||||||||||
| Usage Policy: | No commercial reproduction, distribution, display or performance rights in this work are provided. | ||||||||||||||||||||||||||||||
| ID Code: | 98655 | ||||||||||||||||||||||||||||||
| Collection: | CaltechAUTHORS | ||||||||||||||||||||||||||||||
| Deposited By: | Tony Diaz | ||||||||||||||||||||||||||||||
| Deposited On: | 16 Sep 2019 17:42 | ||||||||||||||||||||||||||||||
| Last Modified: | 18 Aug 2021 01:28 |
Repository Staff Only: item control page
