CaltechAUTHORS
  A Caltech Library Service

Temporal Segmentation of Surgical Sub-tasks through Deep Learning with Multiple Data Sources

Qin, Yidan and Aghajani Pedram, Sahba and Feyzabadi, Seyedshams and Allan, Max and McLeod, A. Jonathan and Burdick, Joel W. and Azizian, Mahdi (2020) Temporal Segmentation of Surgical Sub-tasks through Deep Learning with Multiple Data Sources. In: 2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE , Piscataway, NJ, pp. 371-377. ISBN 978-1-7281-7395-5. https://resolver.caltech.edu/CaltechAUTHORS:20200527-081301991

[img] PDF - Submitted Version
See Usage Policy.

2168Kb

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20200527-081301991

Abstract

Many tasks in robot-assisted surgeries (RAS) can be represented by finite-state machines (FSMs), where each state represents either an action (such as picking up a needle) or an observation (such as bleeding). A crucial step towards the automation of such surgical tasks is the temporal perception of the current surgical scene, which requires a real-time estimation of the states in the FSMs. The objective of this work is to estimate the current state of the surgical task based on the actions performed or events occurred as the task progresses. We propose Fusion-KVE, a unified surgical state estimation model that incorporates multiple data sources including the Kinematics, Vision, and system Events. Additionally, we examine the strengths and weaknesses of different state estimation models in segmenting states with different representative features or levels of granularity. We evaluate our model on the JHU-ISI Gesture and Skill Assessment Working Set (JIGSAWS), as well as a more complex dataset involving robotic intra-operative ultrasound (RIOUS) imaging, created using the da Vinci® Xi surgical system. Our model achieves a superior frame-wise state estimation accuracy up to 89.4%, which improves the state-of-the-art surgical state estimation models in both JIGSAWS suturing dataset and our RIOUS dataset.


Item Type:Book Section
Related URLs:
URLURL TypeDescription
https://doi.org/10.1109/ICRA40945.2020.9196560DOIArticle
https://arxiv.org/abs/2002.02921arXivDiscussion Paper
ORCID:
AuthorORCID
Aghajani Pedram, Sahba0000-0001-7439-8013
Additional Information:© 2020 IEEE. This work was funded by Intuitive Surgical, Inc. We would like to thank Dr. Azad Shademan and Dr. Pourya Shirazian for their support of this research.
Funders:
Funding AgencyGrant Number
Intuitive Surgical, Inc.UNSPECIFIED
Record Number:CaltechAUTHORS:20200527-081301991
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20200527-081301991
Official Citation:Y. Qin et al., "Temporal Segmentation of Surgical Sub-tasks through Deep Learning with Multiple Data Sources," 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France, 2020, pp. 371-377, doi: 10.1109/ICRA40945.2020.9196560
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:103486
Collection:CaltechAUTHORS
Deposited By: Tony Diaz
Deposited On:27 May 2020 15:51
Last Modified:22 Sep 2020 18:31

Repository Staff Only: item control page