CaltechAUTHORS
  A Caltech Library Service

DeepCell Kiosk: scaling deep learning–enabled cellular image analysis with Kubernetes

Bannon, Dylan and Moen, Erick and Schwartz, Morgan and Borba, Enrico and Kudo, Takamasa and Greenwald, Noah and Vijayakumar, Vibha and Chang, Brian and Pao, Edward and Osterman, Erik and Graf, William and Van Valen, David (2021) DeepCell Kiosk: scaling deep learning–enabled cellular image analysis with Kubernetes. Nature Methods, 18 (1). pp. 43-45. ISSN 1548-7091. https://resolver.caltech.edu/CaltechAUTHORS:20190916-101510993

[img] PDF (Sept. 16, 2020) - Submitted Version
Creative Commons Attribution Non-commercial.

2657Kb
[img] PDF (Supplementary Notes 1–8) - Supplemental Material
See Usage Policy.

2316Kb
[img] PDF (Reporting Summary) - Supplemental Material
See Usage Policy.

2612Kb

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20190916-101510993

Abstract

Deep learning is transforming the analysis of biological images, but applying these models to large datasets remains challenging. Here we describe the DeepCell Kiosk, cloud-native software that dynamically scales deep learning workflows to accommodate large imaging datasets. To demonstrate the scalability and affordability of this software, we identified cell nuclei in 10⁶ 1-megapixel images in ~5.5 h for ~US$250, with a cost below US$100 achievable depending on cluster configuration. The DeepCell Kiosk can be downloaded at https://github.com/vanvalenlab/kiosk-console; a persistent deployment is available at https://deepcell.org/.


Item Type:Article
Related URLs:
URLURL TypeDescription
https://doi.org/10.1038/s41592-020-01023-0DOIArticle
https://rdcu.be/cdr01PublisherFree ReadCube access
https://doi.org/10.1038/s41592-021-01059-wDOIPublisher Correction
https://rdcu.be/cdRlLPublisherFree ReadCube access - Publisher Correction
https://doi.org/10.1101/505032DOIDiscussion Paper
https://github.com/vanvalenlab/kiosk-consoleRelated ItemDeepCell Kiosk
https://deepcell.org/dataRelated ItemData
https://github.com/vanvalenlab/deepcell-tfRelated ItemCode
https://deepcell-kiosk.readthedocs.ioRelated ItemDetailed instructions
ORCID:
AuthorORCID
Moen, Erick0000-0002-5947-7044
Schwartz, Morgan0000-0002-1449-0026
Kudo, Takamasa0000-0002-9709-5549
Van Valen, David0000-0001-7534-7621
Alternate Title:Dynamic allocation of computational resources for deep learning-enabled cellular image analysis with Kubernetes
Additional Information:© The Author(s), under exclusive licence to Springer Nature America, Inc. 2021. Received 15 October 2019; Accepted 23 November 2020; Published 04 January 2021. We thank numerous colleagues including A. Anandkumar, M. Angelo, J. Bois, I. Brown, A. Butkovic, L. Cai, I. Camplisson, M. Covert, M. Elowitz, J. Freeman, C. Frick, L. Geontoro, A. Ho, K. Huang, K. C. Huang, G. Johnson, L. Keren, D. Litovitz, D. Macklin, U. Manor, S. Patel, A. Raj, N. Pelaez Restrepo, C. Pavelchek, S. Shah and M. Thomson for helpful discussions and contributing data. We gratefully acknowledge support from the Shurl and Kay Curci Foundation, the Rita Allen Foundation, the Paul Allen Family Foundation through the Allen Discovery Center at Stanford University, the Rosen Center for Bioengineering at Caltech, Google Research Cloud, Figure 8’s AI For Everyone award, and a subaward from NIH U24-CA224309-01. Data availability: All data that were used to generate the figures in this paper are available at https://deepcell.org/data and at https://github.com/vanvalenlab/deepcell-tf under the deepcell.datasets module. Code availability: We used Kubernetes and TensorFlow, along with the scientific computing stack for Python. A persistent deployment of the software described can be accessed at https://deepcell.org/. All source code, including version requirements and explicit usage, is under a modified Apache license and is available at https://github.com/vanvalenlab. Detailed instructions are available at https://deepcell-kiosk.readthedocs.io. Author Contributions: D.B., W.G. and D.V.V. conceived the project; D.B., W.G., E.O. and D.V.V. designed the software architecture; D.B., E.O. and W.G. wrote the core components of the software; D.B., E.M., M.S., E.B., V.V., B.C., E.O., W.G. and D.V.V. contributed to the code base; T.K. and E.P. collected data for annotation; E.M., M.S., N.G., D.B., W.G. and D.V.V. wrote documentation; D.B., E.M., W.G. and D.V.V. wrote the paper; D.V.V. supervised the project. Competing interests: The authors have filed a provisional patent for the described work; the software described here is available under a modified Apache license and is free for non-commercial uses. Peer review information: Nature Methods thanks Ola Spjuth and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Rita Strack was the primary editor on this article and managed its editorial process and peer review in collaboration with the rest of the editorial team.
Errata:Bannon, D., Moen, E., Schwartz, M. et al. Publisher Correction: DeepCell Kiosk: scaling deep learning–enabled cellular image analysis with Kubernetes. Nat Methods (2021). https://doi.org/10.1038/s41592-021-01059-w
Group:Rosen Bioengineering Center
Funders:
Funding AgencyGrant Number
Shurl and Kay Curci FoundationUNSPECIFIED
Rita Allen FoundationUNSPECIFIED
Paul Allen Family FoundationUNSPECIFIED
Donna and Benjamin M. Rosen Bioengineering CenterUNSPECIFIED
Google Research CloudUNSPECIFIED
Figure 8’s AI for EveryoneUNSPECIFIED
NIHU24-CA224309-01
Issue or Number:1
Record Number:CaltechAUTHORS:20190916-101510993
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20190916-101510993
Official Citation:Bannon, D., Moen, E., Schwartz, M. et al. DeepCell Kiosk: scaling deep learning–enabled cellular image analysis with Kubernetes. Nat Methods 18, 43–45 (2021). https://doi.org/10.1038/s41592-020-01023-0
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:98655
Collection:CaltechAUTHORS
Deposited By: Tony Diaz
Deposited On:16 Sep 2019 17:42
Last Modified:19 Jan 2021 21:40

Repository Staff Only: item control page