A Caltech Library Service

Capturing fine-grained details for video-based automation of suturing skills assessment

Hung, Andrew J. and Bao, Richard and Sunmola, Idris O. and Huang, De-An and Nguyen, Jessica H. and Anandkumar, Anima (2023) Capturing fine-grained details for video-based automation of suturing skills assessment. International Journal of Computer Assisted Radiology and Surgery, 18 (3). pp. 545-552. ISSN 1861-6429. PMCID PMC9975072. doi:10.1007/s11548-022-02778-x.

[img] PDF - Accepted Version
See Usage Policy.


Use this Persistent URL to link to this item:


Objectives: Manually-collected suturing technical skill scores are strong predictors of continence recovery after robotic radical prostatectomy. Herein, we automate suturing technical skill scoring through computer vision (CV) methods as a scalable method to provide feedback. Methods: Twenty-two surgeons completed a suturing exercise three times on the Mimic™ Flex VR simulator. Instrument kinematic data (XYZ coordinates of each instrument and pose) were captured at 30 Hz. After standardized training, three human raters manually video segmented suturing task into four sub-stitch phases (Needle handling, Needle targeting, Needle driving, Needle withdrawal) and labeled the corresponding technical skill domains (Needle positioning, Needle entry, Needle driving, and Needle withdrawal). The CV framework extracted RGB features and optical flow frames using a pre-trained AlexNet. Additional CV strategies including auxiliary supervision (using kinematic data during training only) and attention mechanisms were implemented to improve performance. Results: This study included data from 15 expert surgeons (median caseload 300 [IQR 165–750]) and 7 training surgeons (0 [IQR 0–8]). In all, 226 virtual sutures were captured. Automated assessments for Needle positioning performed best with the simplest approach (1 s video; AUC 0.749). Remaining skill domains exhibited improvements with the implementation of auxiliary supervision and attention mechanisms when deployed separately (AUC 0.604–0.794). All techniques combined produced the best performance, particularly for Needle driving and Needle withdrawal (AUC 0.959 and 0.879, respectively). Conclusions: This study demonstrated the best performance of automated suturing technical skills assessment to date using advanced CV techniques. Future work will determine if a “human in the loop” is necessary to verify surgeon evaluations.

Item Type:Article
Related URLs:
URLURL TypeDescription CentralArticle
Hung, Andrew J.0000-0002-7201-6736
Huang, De-An0000-0002-6945-7768
Nguyen, Jessica H.0000-0003-0454-8463
Anandkumar, Anima0000-0002-6974-6797
Additional Information:© 2023 Springer Nature. We thank Daniel Sanford, Balint Der, Ryan Hakim, Runzhuo Ma, and Taseen Haque for data collection and grading of technical skill scores through video review. Mimic Technologies, Inc. provided access to the raw kinematic instrument data for each exercise. Center for Robotic Simulation & Education, Catherine & Joseph Aresty Department of Urology, USC Institute of Urology, University of Southern California, Los Angeles, California. Research reported in this publication was supported in part by the National Cancer Institute under Award No. R01CA251579-01A1. Ethics approval: All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. Our study complied with protocols put forth by the University of Southern California’s IRB. Informed consent was obtained from all individual participants included in the study. Statement and Declarations: Andrew J. Hung has financial disclosures with Intuitive Surgical, Inc.
Funding AgencyGrant Number
Issue or Number:3
PubMed Central ID:PMC9975072
Record Number:CaltechAUTHORS:20221110-430801400.16
Persistent URL:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:117830
Deposited By: Research Services Depository
Deposited On:27 Nov 2022 20:55
Last Modified:23 May 2023 16:45

Repository Staff Only: item control page