Studier-Fischer, Alexander and Seidlitz, Silvia and Sellner, Jan and Özdemir, Berkin and Wiesenfarth, Manuel and Ayala, Leonardo and Odenthal, Jan and Knödler, Samuel and Kowalewski, Karl-Friedrich and Haney, Caelán-Max and Camplisson, Isabella and Dietrich, Maximilian and Schmidt, Karsten and Salg, Gabriel-Alexander and Kenngott, Hannes-Goetz and Adler, Tim-Julian and Schreck, Nicholas and Kopp-Schneider, Annette and Maier-Hein, Klaus and Maier-Hein, Lena and Müller-Stich, Beat-Peter and Nickel, Felix (2022) Spectral organ fingerprints for machine learning-based intraoperative tissue classification with hyperspectral imaging in a porcine model. Scientific Reports, 12 . Art. No. 11028. ISSN 2045-2322. PMCID PMC9247052. doi:10.1038/s41598-022-15040-w. https://resolver.caltech.edu/CaltechAUTHORS:20211130-152255247
![[img]](https://authors.library.caltech.edu/style/images/fileicons/application_pdf.png) |
PDF
- Published Version
Creative Commons Attribution. 5MB |
![[img]](https://authors.library.caltech.edu/style/images/fileicons/application_msword.png) |
MS Word
- Supplemental Material
Creative Commons Attribution. 3MB |
|
PDF
- Submitted Version
Creative Commons Attribution Non-commercial No Derivatives. 8MB |
Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20211130-152255247
Abstract
Visual discrimination of tissue during surgery can be challenging since different tissues appear similar to the human eye. Hyperspectral imaging (HSI) removes this limitation by associating each pixel with high-dimensional spectral information. While previous work has shown its general potential to discriminate tissue, clinical translation has been limited due to the method’s current lack of robustness and generalizability. Specifically, the scientific community is lacking a comprehensive spectral tissue atlas, and it is unknown whether variability in spectral reflectance is primarily explained by tissue type rather than the recorded individual or specific acquisition conditions. The contribution of this work is threefold: (1) Based on an annotated medical HSI data set (9059 images from 46 pigs), we present a tissue atlas featuring spectral fingerprints of 20 different porcine organs and tissue types. (2) Using the principle of mixed model analysis, we show that the greatest source of variability related to HSI images is the organ under observation. (3) We show that HSI-based fully-automatic tissue differentiation of 20 organ classes with deep neural networks is possible with high accuracy (> 95%). We conclude from our study that automatic tissue discrimination based on HSI data is feasible and could thus aid in intraoperative decisionmaking and pave the way for context-aware computer-assisted surgery systems and autonomous robotics.
| Item Type: | Article | ||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Related URLs: |
| ||||||||||||||||||||||||||||||||||||||||||||||
| ORCID: |
| ||||||||||||||||||||||||||||||||||||||||||||||
| Additional Information: | © The Author(s) 2022. This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. Received 04 February 2022; Accepted 16 June 2022; Published 30 June 2022. The authors thank Minu Tizabi (IMSY, DKFZ) for proofreading the manuscript and gratefully acknowledge the data storage service SDS@hd supported by the Ministry of Science, Research and the Arts Baden-Württemberg (MWK) and the German Research Foundation (DFG) through Grant INST 35/1314-1 FUGG and INST 35/1503-1 FUGG. The present contribution is supported by the Helmholtz Association under the joint research school HIDSS4Health (Helmholtz Information and Data Science School for Health). This project received funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation program (NEURAL SPICING, Grant Agreement No. [101002198]) as well as from the Willi Robert Pitzer foundation, from the Heidelberg Foundation for Surgery and from the RISE program by the German Academic Exchange Service (DAAD). Depicted images in the figures were drawn by the authors. The image of the TIVITA® Tissue system has been made available with kind approval from Diaspective Vision. Data availability: Data and code that support the findings of this study are available from the corresponding authors upon reasonable request. Open Access funding enabled and organized by Projekt DEAL. Contributions: A.S.F., F.N. and B.P.M.S. had the original idea for the project. ASF and FN initiated the project. A.S.F., F.N. and C.M.H. performed the initial review of existing literature and the planning. A.S.F., C.M.H., K.F.K., B.Ö., M.D., B.P.M.S. and F.N. performed the surgeries. A.S.F., I.C., B.Ö. and J.O. developed the Python codes for data organization and annotation. A.S.F., G.A.S. and S.K. annotated data. A.S.F., G.A.S., B.Ö., J.O., S.K., K.F.K., S.S., L.A., J.S., L.M.H., T.A. and M.W. analyzed and interpreted data. L.M.H, T.A., L.A., S.S., J.S., M.W., N.S., A.K.S. and A.S.F. developed and implemented the statistical and machine learning-based data analysis strategy. L.A. provided the t-SNE analysis and the relevant manuscript passages. M.W., N.S. and A.K.S. provided the mixed model analysis, the structured model and the relevant manuscript passages. S.S. and J.S. provided the machine learning-based classification and the relevant manuscript passages. F.N., L.M.H, H.G.K., K.M.H., K.S. and B.P.M.S. provided expert knowledge throughout the project. A.S.F., B.Ö. and F.N. wrote the manuscript. S.S., J.S., L.A., M.W., B.P.M.S., H.G.K., K.M.H. and L.M.H. revised the manuscript. All authors have read and approved the final manuscript. The authors declare no competing interests. | ||||||||||||||||||||||||||||||||||||||||||||||
| Funders: |
| ||||||||||||||||||||||||||||||||||||||||||||||
| PubMed Central ID: | PMC9247052 | ||||||||||||||||||||||||||||||||||||||||||||||
| DOI: | 10.1038/s41598-022-15040-w | ||||||||||||||||||||||||||||||||||||||||||||||
| Record Number: | CaltechAUTHORS:20211130-152255247 | ||||||||||||||||||||||||||||||||||||||||||||||
| Persistent URL: | https://resolver.caltech.edu/CaltechAUTHORS:20211130-152255247 | ||||||||||||||||||||||||||||||||||||||||||||||
| Official Citation: | Studier-Fischer, A., Seidlitz, S., Sellner, J. et al. Spectral organ fingerprints for machine learning-based intraoperative tissue classification with hyperspectral imaging in a porcine model. Sci Rep 12, 11028 (2022). https://doi.org/10.1038/s41598-022-15040-w | ||||||||||||||||||||||||||||||||||||||||||||||
| Usage Policy: | No commercial reproduction, distribution, display or performance rights in this work are provided. | ||||||||||||||||||||||||||||||||||||||||||||||
| ID Code: | 112084 | ||||||||||||||||||||||||||||||||||||||||||||||
| Collection: | CaltechAUTHORS | ||||||||||||||||||||||||||||||||||||||||||||||
| Deposited By: | Tony Diaz | ||||||||||||||||||||||||||||||||||||||||||||||
| Deposited On: | 30 Nov 2021 16:35 | ||||||||||||||||||||||||||||||||||||||||||||||
| Last Modified: | 11 Jul 2022 18:07 |
Repository Staff Only: item control page
