A Caltech Library Service

Capturing fine-grained details for video-based automation of suturing skills assessment

Hung, Andrew J. and Bao, Richard and Sunmola, Idris O. and Huang, De-An and Nguyen, Jessica H. and Anandkumar, Anima (2022) Capturing fine-grained details for video-based automation of suturing skills assessment. International Journal of Computer Assisted Radiology and Surgery . ISSN 1861-6429. doi:10.1007/s11548-022-02778-x. (In Press)

Full text is not posted in this repository. Consult Related URLs below.

Use this Persistent URL to link to this item:


Objectives: Manually-collected suturing technical skill scores are strong predictors of continence recovery after robotic radical prostatectomy. Herein, we automate suturing technical skill scoring through computer vision (CV) methods as a scalable method to provide feedback. Methods: Twenty-two surgeons completed a suturing exercise three times on the Mimic™ Flex VR simulator. Instrument kinematic data (XYZ coordinates of each instrument and pose) were captured at 30 Hz. After standardized training, three human raters manually video segmented suturing task into four sub-stitch phases (Needle handling, Needle targeting, Needle driving, Needle withdrawal) and labeled the corresponding technical skill domains (Needle positioning, Needle entry, Needle driving, and Needle withdrawal). The CV framework extracted RGB features and optical flow frames using a pre-trained AlexNet. Additional CV strategies including auxiliary supervision (using kinematic data during training only) and attention mechanisms were implemented to improve performance. Results: This study included data from 15 expert surgeons (median caseload 300 [IQR 165–750]) and 7 training surgeons (0 [IQR 0–8]). In all, 226 virtual sutures were captured. Automated assessments for Needle positioning performed best with the simplest approach (1 s video; AUC 0.749). Remaining skill domains exhibited improvements with the implementation of auxiliary supervision and attention mechanisms when deployed separately (AUC 0.604–0.794). All techniques combined produced the best performance, particularly for Needle driving and Needle withdrawal (AUC 0.959 and 0.879, respectively). Conclusions: This study demonstrated the best performance of automated suturing technical skills assessment to date using advanced CV techniques. Future work will determine if a “human in the loop” is necessary to verify surgeon evaluations.

Item Type:Article
Related URLs:
URLURL TypeDescription
Hung, Andrew J.0000-0002-7201-6736
Huang, De-An0000-0002-6945-7768
Nguyen, Jessica H.0000-0003-0454-8463
Anandkumar, Anima0000-0002-6974-6797
Additional Information:We thank Daniel Sanford, Balint Der, Ryan Hakim, Runzhuo Ma, and Taseen Haque for data collection and grading of technical skill scores through video review. Mimic Technologies, Inc. provided access to the raw kinematic instrument data for each exercise. Center for Robotic Simulation & Education, Catherine & Joseph Aresty Department of Urology, USC Institute of Urology, University of Southern California, Los Angeles, California. Research reported in this publication was supported in part by the National Cancer Institute under Award No. R01CA251579-01A1.
Funding AgencyGrant Number
Record Number:CaltechAUTHORS:20221110-430801400.16
Persistent URL:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:117830
Deposited By: Research Services Depository
Deposited On:27 Nov 2022 20:55
Last Modified:28 Nov 2022 18:25

Repository Staff Only: item control page