CaltechAUTHORS
  A Caltech Library Service

Expert surgeons and deep learning models can predict the outcome of surgical hemorrhage from 1 min of video

Pangal, Dhiraj J. and Kugener, Guillaume and Zhu, Yichao and Sinha, Aditya and Unadkat, Vyom and Cote, David J. and Strickland, Ben and Rutkowski, Martin and Hung, Andrew and Anandkumar, Animashree and Han, X. Y. and Papyan, Vardan and Wrobel, Bozena and Zada, Gabriel and Donoho, Daniel A. (2022) Expert surgeons and deep learning models can predict the outcome of surgical hemorrhage from 1 min of video. Scientific Reports, 12 . Art. No. 8137. ISSN 2045-2322. PMCID PMC9114003. doi:10.1038/s41598-022-11549-2. https://resolver.caltech.edu/CaltechAUTHORS:20220124-214564000

[img] PDF - Published Version
Creative Commons Attribution.

1MB
[img] PDF - Submitted Version
Creative Commons Attribution Non-commercial No Derivatives.

1MB
[img] MS Word (Supplementary Legends) - Supplemental Material
Creative Commons Attribution.

29kB
[img] PDF (Supplementary Information) - Supplemental Material
Creative Commons Attribution.

967kB

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20220124-214564000

Abstract

Major vascular injury resulting in uncontrolled bleeding is a catastrophic and often fatal complication of minimally invasive surgery. At the outset of these events, surgeons do not know how much blood will be lost or whether they will successfully control the hemorrhage (achieve hemostasis). We evaluate the ability of a deep learning neural network (DNN) to predict hemostasis control ability using the first minute of surgical video and compare model performance with human experts viewing the same video. The publicly available SOCAL dataset contains 147 videos of attending and resident surgeons managing hemorrhage in a validated, high-fidelity cadaveric simulator. Videos are labeled with outcome and blood loss (mL). The first minute of 20 videos was shown to four, blinded, fellowship trained skull-base neurosurgery instructors, and to SOCALNet (a DNN trained on SOCAL videos). SOCALNet architecture included a convolutional network (ResNet) identifying spatial features and a recurrent network identifying temporal features (LSTM). Experts independently assessed surgeon skill, predicted outcome and blood loss (mL). Outcome and blood loss predictions were compared with SOCALNet. Expert inter-rater reliability was 0.95. Experts correctly predicted 14/20 trials (Sensitivity: 82%, Specificity: 55%, Positive Predictive Value (PPV): 69%, Negative Predictive Value (NPV): 71%). SOCALNet correctly predicted 17/20 trials (Sensitivity 100%, Specificity 66%, PPV 79%, NPV 100%) and correctly identified all successful attempts. Expert predictions of the highest and lowest skill surgeons and expert predictions reported with maximum confidence were more accurate. Experts systematically underestimated blood loss (mean error − 131 mL, RMSE 350 mL, R2 0.70) and fewer than half of expert predictions identified blood loss > 500 mL (47.5%, 19/40). SOCALNet had superior performance (mean error − 57 mL, RMSE 295 mL, R² 0.74) and detected most episodes of blood loss > 500 mL (80%, 8/10). In validation experiments, SOCALNet evaluation of a critical on-screen surgical maneuver and high/low-skill composite videos were concordant with expert evaluation. Using only the first minute of video, experts and SOCALNet can predict outcome and blood loss during surgical hemorrhage. Experts systematically underestimated blood loss, and SOCALNet had no false negatives. DNNs can provide accurate, meaningful assessments of surgical video. We call for the creation of datasets of surgical adverse events for quality improvement research.


Item Type:Article
Related URLs:
URLURL TypeDescription
https://doi.org/10.1038/s41598-022-11549-2DOIArticle
http://www.ncbi.nlm.nih.gov/pmc/articles/pmc9114003/PubMed CentralArticle
https://doi.org/10.6084/m9.figshare.15132468.v1DOIDatasets
https://doi.org/10.1101/2022.01.22.22269640DOIDiscussion Paper
ORCID:
AuthorORCID
Pangal, Dhiraj J.0000-0001-7391-9825
Kugener, Guillaume0000-0002-4697-2847
Strickland, Ben0000-0002-4620-9542
Rutkowski, Martin0000-0002-5188-3419
Hung, Andrew0000-0002-7201-6736
Anandkumar, Animashree0000-0002-6974-6797
Papyan, Vardan0000-0002-5028-2144
Zada, Gabriel0000-0001-5821-902X
Donoho, Daniel A.0000-0002-0531-1436
Alternate Title:Expert Surgeons and Deep Learning Models Can Predict the Outcome of Surgical Hemorrhage from One Minute of Video
Additional Information:© The Author(s) 2022. This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. Received 22 December 2021; Accepted 18 April 2022; Published 17 May 2022. Data availability: The datasets generated during and/or analyzed during the current study are available in the figshare repository, link: https://doi.org/10.6084/m9.figshare.15132468.v1. Contributions: Study design: D.J.P., G.K., A.S., G.Z., D.A.D. Data acquisition: D.J.P., G.K., B.S., M.R., G.Z., D.A.D. Model development: D.J.P., G.K., A.S., V.U., X.H., V.P., D.A.D. Statistical analysis: D.J.P., G.K., D.A.D. Writing—original draft: D.J.P., G.K., D.A.D. Writing—revisions: All authors. Final approval: All authors. Study supervision: G.Z., D.A.D. The authors declare no competing interests.
Subject Keywords:Machine learning; Outcomes research
PubMed Central ID:PMC9114003
DOI:10.1038/s41598-022-11549-2
Record Number:CaltechAUTHORS:20220124-214564000
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20220124-214564000
Official Citation:Pangal, D.J., Kugener, G., Zhu, Y. et al. Expert surgeons and deep learning models can predict the outcome of surgical hemorrhage from 1 min of video. Sci Rep 12, 8137 (2022). https://doi.org/10.1038/s41598-022-11549-2
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:113084
Collection:CaltechAUTHORS
Deposited By: George Porter
Deposited On:25 Jan 2022 15:08
Last Modified:23 May 2022 21:05

Repository Staff Only: item control page