CaltechAUTHORS
  A Caltech Library Service

daVinciNet: Joint Prediction of Motion and Surgical State in Robot-Assisted Surgery

Qin, Yidan and Feyzabadi, Seyedshams and Allan, Max and Burdick, Joel W. and Azizian, Mahdi (2020) daVinciNet: Joint Prediction of Motion and Surgical State in Robot-Assisted Surgery. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE , Piscataway, NJ, pp. 2921-2928. https://resolver.caltech.edu/CaltechAUTHORS:20210119-161653290

[img] PDF - Accepted Version
See Usage Policy.

7MB

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20210119-161653290

Abstract

This paper presents a technique to concurrently and jointly predict the future trajectories of surgical instruments and the future state(s) of surgical subtasks in robot-assisted surgeries (RAS) using multiple input sources. Such predictions are a necessary first step towards shared control and supervised autonomy of surgical subtasks. Minute-long surgical subtasks, such as suturing or ultrasound scanning, often have distinguishable tool kinematics and visual features, and can be described as a series of fine-grained states with transition schematics. We propose daVinciNet - an end-to-end dual-task model for robot motion and surgical state predictions. daVinciNet performs concurrent end-effector trajectory and surgical state predictions using features extracted from multiple data streams, including robot kinematics, endoscopic vision, and system events. We evaluate our proposed model on an extended Robotic Intra-Operative Ultrasound (RIOUS+) imaging dataset collected on a da Vinci® Xi surgical system and the JHU-ISI Gesture and Skill Assessment Working Set (JIGSAWS). Our model achieves up to 93.85% short-term (0.5s) and 82.11% long-term (2s) state prediction accuracy, as well as 1.07mm short-term and 5.62mm long-term trajectory prediction error.


Item Type:Book Section
Related URLs:
URLURL TypeDescription
https://doi.org/10.1109/IROS45743.2020.9340723DOIArticle
https://ieeexplore.ieee.org/document/9340723PublisherArticle
https://arxiv.org/abs/2009.11937arXivDiscussion Paper
ORCID:
AuthorORCID
Feyzabadi, Seyedshams0000-0002-2357-2955
Allan, Max0000-0002-6495-065X
Additional Information:© 2020 IEEE. This work was funded by Intuitive Surgical, Inc. We would like to thank Dr. Azad Shademan and Dr. A. Jonathan McLeod for their support of this research.
Funders:
Funding AgencyGrant Number
Intuitive Surgical, Inc.UNSPECIFIED
Record Number:CaltechAUTHORS:20210119-161653290
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20210119-161653290
Official Citation:Y. Qin, S. Feyzabadi, M. Allan, J. W. Burdick and M. Azizian, "daVinciNet: Joint Prediction of Motion and Surgical State in Robot-Assisted Surgery," 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA, 2020, pp. 2921-2928, doi: 10.1109/IROS45743.2020.9340723
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:107578
Collection:CaltechAUTHORS
Deposited By: George Porter
Deposited On:20 Jan 2021 15:06
Last Modified:12 Feb 2021 18:37

Repository Staff Only: item control page