A Caltech Library Service

Computer Vision in the Operating Room: Opportunities and Caveats

Kennedy-Metz, Lauren R. and Mascagni, Pietro and Torralba, Antonio and Dias, Roger D. and Perona, Pietro and Shah, Julie A. and Padoy, Nicolas and Zenati, Marco A. (2021) Computer Vision in the Operating Room: Opportunities and Caveats. IEEE Transactions on Medical Robotics and Bionics, 3 (1). pp. 2-10. ISSN 2576-3202. PMCID PMC7908934. doi:10.1109/tmrb.2020.3040002.

[img] PDF - Accepted Version
See Usage Policy.


Use this Persistent URL to link to this item:


Effectiveness of computer vision techniques has been demonstrated through a number of applications, both within and outside healthcare. The operating room environment specifically is a setting with rich data sources compatible with computational approaches and high potential for direct patient benefit. The aim of this review is to summarize major topics in computer vision for surgical domains. The major capabilities of computer vision are described as an aid to surgical teams to improve performance and contribute to enhanced patient safety. Literature was identified through leading experts in the fields of surgery, computational analysis and modeling in medicine, and computer vision in healthcare. The literature supports the application of computer vision principles to surgery. Potential applications within surgery include operating room vigilance, endoscopic vigilance, and individual and team-wide behavioral analysis. To advance the field, we recommend collecting and publishing carefully annotated datasets. Doing so will enable the surgery community to collectively define well-specified common objectives for automated systems, spur academic research, mobilize industry, and provide benchmarks with which we can track progress. Leveraging computer vision approaches through interdisciplinary collaboration and advanced approaches to data acquisition, modeling, interpretation, and integration promises a powerful impact on patient safety, public health, and financial costs.

Item Type:Article
Related URLs:
URLURL TypeDescription CentralArticle
Kennedy-Metz, Lauren R.0000-0002-2696-3943
Mascagni, Pietro0000-0001-7288-3023
Dias, Roger D.0000-0003-4959-5052
Perona, Pietro0000-0002-7583-5809
Shah, Julie A.0000-0003-1338-8107
Padoy, Nicolas0000-0002-5010-4137
Zenati, Marco A.0000-0001-7139-0323
Additional Information:© 2020 IEEE. Manuscript received September 25, 2020; accepted November 10, 2020. Date of publication November 24, 2020; date of current version February 22, 2021. This article was recommended for publication by Associate Editor P. Poignet and Editor P. Dario upon evaluation of the reviewers’ comments. This work was supported by the National Heart, Lung, and Blood Institute of NIH (PI Zenati) under Grant R01HL126896. The work of Nicolas Padoy was supported in part by the French State Funds managed by the Agence Nationale de la Recherche through the Investissements d’Avenir Program under Grant ANR-16-CE33-0009 (DeepSurg), Grant ANR-11-LABX-0004 (Labex CAMI), Grant ANR-10-IDEX-0002-02 (Idex Unistra), and Grant ANR-10-IAHU-02 (IHU Strasbourg), and in part by BPI France through Project CONDOR.
Funding AgencyGrant Number
Agence Nationale pour la Recherche (ANR)ANR-16-CE33-0009
Agence Nationale pour la Recherche (ANR)ANR-11- LABX-0004
Agence Nationale pour la Recherche (ANR)ANR-10-IDEX-0002-02
Agence Nationale pour la Recherche (ANR)ANR-10-IAHU-02
Issue or Number:1
PubMed Central ID:PMC7908934
Record Number:CaltechAUTHORS:20210301-131304383
Persistent URL:
Official Citation:L. R. Kennedy-Metz et al., "Computer Vision in the Operating Room: Opportunities and Caveats," in IEEE Transactions on Medical Robotics and Bionics, vol. 3, no. 1, pp. 2-10, Feb. 2021, doi: 10.1109/TMRB.2020.3040002
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:108252
Deposited By: Tony Diaz
Deposited On:01 Mar 2021 21:29
Last Modified:10 Feb 2022 18:13

Repository Staff Only: item control page