CaltechAUTHORS
  A Caltech Library Service

A Guide to Annotation of Neurosurgical Intraoperative Video for Machine Learning Analysis and Computer Vision

Pangal, Dhiraj J. and Kugener, Guillaume and Shahrestani, Shane and Attenello, Frank and Zada, Gabriel and Donoho, Daniel A. (2021) A Guide to Annotation of Neurosurgical Intraoperative Video for Machine Learning Analysis and Computer Vision. World Neurosurgery, 150 . pp. 26-30. ISSN 1878-8750. doi:10.1016/j.wneu.2021.03.022. https://resolver.caltech.edu/CaltechAUTHORS:20210322-135133478

[img] PDF - Accepted Version
See Usage Policy.

3800Kb
[img] Video (MPEG) - Supplemental Material
See Usage Policy.

3754Kb

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20210322-135133478

Abstract

Objective: Computer vision (CV) is a subset of artificial intelligence that performs computations on image or video data, permitting the quantitative analysis of visual information. Common CV tasks that may be relevant to surgeons include image classification, object detection and tracking, and extraction of higher order features. Despite the potential applications of CV to intraoperative video, however, few surgeons describe the use of CV. A primary roadblock in implementing CV is the lack of a clear workflow to create an intraoperative video dataset to which CV can be applied. We report general principles for creating usable surgical video datasets and the result of their applications. Methods: Video annotations from cadaveric endoscopic endonasal skull base simulations (n = 20 trials of 1–5 minutes, size = 8 GB) were reviewed by 2 researcher-annotators. An internal, retrospective analysis of workflow for development of the intraoperative video annotations was performed to identify guiding practices. Results: Approximately 34,000 frames of surgical video were annotated. Key considerations in developing annotation workflows include 1) overcoming software and personnel constraints; 2) ensuring adequate storage and access infrastructure; 3) optimization and standardization of annotation protocol; and 4) operationalizing annotated data. Potential tools for use include CVAT (Computer Vision Annotation Tool) and Vott: open-sourced annotation software allowing for local video storage, easy setup, and the use of interpolation. Conclusions: CV techniques can be applied to surgical video, but challenges for novice users may limit adoption. We outline principles in annotation workflow that can mitigate initial challenges groups may have when converting raw video into useable, annotated datasets.


Item Type:Article
Related URLs:
URLURL TypeDescription
https://doi.org/10.1016/j.wneu.2021.03.022DOIArticle
ORCID:
AuthorORCID
Kugener, Guillaume0000-0002-4697-2847
Shahrestani, Shane0000-0001-7561-4590
Zada, Gabriel0000-0001-5821-902X
Alternate Title:Annotating Surgical Video for Machine Learning
Additional Information:© 2021 Elsevier Inc. Received 11 January 2021, Revised 2 March 2021, Accepted 3 March 2021, Available online 17 March 2021. CRediT authorship contribution statement: Dhiraj J. Pangal: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Writing - original draft, Writing - review & editing. Guillaume Kugener: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Writing - original draft, Writing - review & editing. Shane Shahrestani: Writing - review & editing. Frank Attenello: Writing - review & editing. Gabriel Zada: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Supervision, Writing - original draft, Writing - review & editing. Daniel A. Donoho: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Supervision, Writing - original draft, Writing - review & editing. The authors declare that the article content was composed in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Subject Keywords:artificial intelligence; machine learning; computer vision; intraoperative video
DOI:10.1016/j.wneu.2021.03.022
Record Number:CaltechAUTHORS:20210322-135133478
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20210322-135133478
Official Citation:Dhiraj J. Pangal, Guillaume Kugener, Shane Shahrestani, Frank Attenello, Gabriel Zada, Daniel A. Donoho, A Guide to Annotation of Neurosurgical Intraoperative Video for Machine Learning Analysis and Computer Vision, World Neurosurgery, Volume 150, 2021, Pages 26-30, ISSN 1878-8750, https://doi.org/10.1016/j.wneu.2021.03.022. (https://www.sciencedirect.com/science/article/pii/S1878875021003909)
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:108513
Collection:CaltechAUTHORS
Deposited By: Tony Diaz
Deposited On:24 Mar 2021 20:23
Last Modified:12 Apr 2021 14:53

Repository Staff Only: item control page