A Caltech Library Service

A systematic review of virtual reality for the assessment of technical skills in neurosurgery

Chan, Justin and Pangal, Dhiraj J. and Cardinal, Tyler and Kugener, Guillaume and Zhu, Yichao and Roshannai, Arman and Markarian, Nicholas and Sinha, Aditya and Anandkumar, Anima and Hung, Andrew and Zada, Gabriel and Donoho, Daniel A. (2021) A systematic review of virtual reality for the assessment of technical skills in neurosurgery. Neurosurgical Focus, 51 (2). Art. No. E15. ISSN 1092-0684. doi:10.3171/2021.5.focus21210.

[img] PDF (Supplemental Tables 1-6) - Supplemental Material
See Usage Policy.


Use this Persistent URL to link to this item:


Objective: Virtual reality (VR) and augmented reality (AR) systems are increasingly available to neurosurgeons. These systems may provide opportunities for technical rehearsal and assessments of surgeon performance. The assessment of neurosurgeon skill in VR and AR environments and the validity of VR and AR feedback has not been systematically reviewed. Methods: A systematic review following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines was conducted through MEDLINE and PubMed. Studies published in English between January 1990 and February 2021 describing the use of VR or AR to quantify surgical technical performance of neurosurgeons without the use of human raters were included. The types and categories of automated performance metrics (APMs) from each of these studies were recorded. Results: Thirty-three VR studies were included in the review; no AR studies met inclusion criteria. VR APMs were categorized as either distance to target, force, kinematics, time, blood loss, or volume of resection. Distance and time were the most well-studied APM domains, although all domains were effective at differentiating surgeon experience levels. Distance was successfully used to track improvements with practice. Examining volume of resection demonstrated that attending surgeons removed less simulated tumor but preserved more normal tissue than trainees. More recently, APMs have been used in machine learning algorithms to predict level of training with a high degree of accuracy. Key limitations to enhanced-reality systems include limited AR usage for automated surgical assessment and lack of external and longitudinal validation of VR systems. Conclusions: VR has been used to assess surgeon performance across a wide spectrum of domains. The VR environment can be used to quantify surgeon performance, assess surgeon proficiency, and track training progression. AR systems have not yet been used to provide metrics for surgeon performance assessment despite potential for intraoperative integration. VR-based APMs may be especially useful for metrics that are difficult to assess intraoperatively, including blood loss and extent of resection.

Item Type:Article
Related URLs:
URLURL TypeDescription
Additional Information:© AANS 2021. Submitted March 31, 2021. Accepted May 19, 2021. Author Contributions: Conception and design: Chan, Pangal. Acquisition of data: Chan, Pangal. Analysis and interpretation of data: Chan, Pangal. Drafting the article: Chan, Pangal, Cardinal. Critically revising the article: all authors. Reviewed submitted version of manuscript: all authors. Approved the final version of the manuscript on behalf of all authors: Zada. Administrative/technical/material support: Zada, Chan, Pangal, Cardinal, Donoho. Study supervision: Zada, Donoho. Disclosures: Dr. Hung is a consultant for Johnson & Johnson, Quantgene, and Mimic Technologies.
Subject Keywords:virtual reality; augmented reality; technical assessment
Issue or Number:2
Record Number:CaltechAUTHORS:20210910-182725636
Persistent URL:
Official Citation:Chan, J., Pangal, D. J., Cardinal, T., Kugener, G., Zhu, Y., Roshannai, A., Markarian, N., Sinha, A., Anandkumar, A., Hung, A., Zada, G., & Donoho, D. A. (2021). A systematic review of virtual reality for the assessment of technical skills in neurosurgery, Neurosurgical Focus, 51(2), E15; DOI: 10.3171/2021.5.focus21210
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:110808
Deposited By: Tony Diaz
Deposited On:10 Sep 2021 20:11
Last Modified:10 Sep 2021 20:11

Repository Staff Only: item control page