Search

Abstract Accurate intraoperative assessment of organ perfusion is a pivotal determinant in preserving organ function e.g. during kidney surgery including partial nephrectomy or kidney transplantation. Hyperspectral imaging (HSI) has great potential to objectively describe and quantify this perfusion as opposed to conventional surrogate techniques such as ultrasound flowmeter, indocyanine green or the subjective eye of the surgeon. An established live porcine model under general anesthesia received median laparotomy and renal mobilization. Different scenarios that were measured using HSI were (1) complete, (2) gradual and (3) partial malperfusion. The differences in spectral reflectance as well as HSI oxygenation (StO2) between different perfusion states were compelling and as high as 56.9% with 70.3% (± 11.0%) for “physiological” vs. 13.4% (± 3.1%) for “venous congestion”. A machine learning (ML) algorithm was able to distinguish between these perfusion states with a balanced prediction accuracy of 97.8%. Data from this porcine study including 1300 recordings across 57 individuals was compared to a human dataset of 104 recordings across 17 individuals suggesting clinical transferability. Therefore, HSI is a highly promising tool for intraoperative microvascular evaluation of perfusion states with great advantages over existing surrogate techniques. Clinical trials are required to prove patient benefit.

Accurate intraoperative assessment of organ perfusion is a pivotal determinant in preserving organ function e.g. during kidney surgery including partial nephrectomy or kidney transplantation. Hyperspectral imaging (HSI) has great potential to objectively describe and quantify this perfusion as opposed to conventional surrogate techniques such as ultrasound flowmeter, indocyanine green or the subjective eye of the surgeon. An established live porcine model under general anesthesia received median laparotomy and renal mobilization. Different scenarios that were measured using HSI were (1) complete, (2) gradual and (3) partial malperfusion. The differences in spectral reflectance as well as HSI oxygenation (StO 2 ) between different perfusion states were compelling and as high as 56.9% with 70.3% (± 11.0%) for "physiological" vs. 13.4% (± 3.1%) for "venous congestion". A machine learning (ML) algorithm was able to distinguish between these perfusion states with a balanced prediction accuracy of 97.8%. Data from this porcine study including 1300 recordings across 57 individuals was compared to a human dataset of 104 recordings across 17 individuals suggesting clinical transferability. Therefore, HSI is a highly promising tool for intraoperative microvascular evaluation of perfusion states with great advantages over existing surrogate techniques. Clinical trials are required to prove patient benefit., (© 2024. The Author(s).)

Introduction: Oncologic esophagectomy is a two-cavity procedure with considerable morbidity and mortality. Complex anatomy and the proximity to major vessels constitute a risk for massive intraoperative hemorrhage. Currently, there is no conclusive consensus on the ideal anesthesiologic countermeasure in case of such immense blood loss. The objective of this work was to identify the most promising anesthesiologic management in case of intraoperative hemorrhage with regards to tissue perfusion of the gastric conduit during esophagectomy using hyperspectral imaging (HSI)., Material and Methods: An established live porcine model (n=32) for esophagectomy was used with gastric conduit formation and simulation of a linear stapled side-to-side esophagogastrostomy. After a standardized procedure of controlled blood loss of about 1 L per pig, the four experimental groups (n=8 each) differed in anesthesiologic intervention i.e. (I) permissive hypotension, (II) catecholamine therapy using noradrenaline, (III) crystalloid volume supplementation and (IV) combined crystalloid volume supplementation with noradrenaline therapy. HSI tissue oxygenation (StO2) of the gastric conduit was evaluated and correlated with systemic perfusion parameters. Measurements were conducted before (T0) and after (T1) laparotomy, after hemorrhage (T2) and 60 minutes (T3) and 120 minutes (T4) after anesthesiologic intervention., Results: StO2 values of the gastric conduit showed significantly different results between the four experimental groups with 63.3% (±7.6%) after permissive hypotension (I), 45.9% (±6.4%) after catecholamine therapy (II), 70.5% (±6.1%) after crystalloid volume supplementation (III) and 69.0% (±3.7%) after combined therapy (IV). StO2 values correlated strongly with systemic lactate values (r=-0.67; CI -0.77 to -0.54), which is an established prognostic factor., Conclusion: Crystalloid volume supplementation (III) yields the highest StO2 values and lowest systemic lactate values and therefore appears to be the superior primary treatment strategy after hemorrhage during esophagectomy with regards to microcirculatory tissue oxygenation of the gastric conduit., (Copyright © 2024 The Author(s). Published by Wolters Kluwer Health, Inc.)

Purpose: Surgical scene segmentation is crucial for providing context-aware surgical assistance. Recent studies highlight the significant advantages of hyperspectral imaging (HSI) over traditional RGB data in enhancing segmentation performance. Nevertheless, the current hyperspectral imaging (HSI) datasets remain limited and do not capture the full range of tissue variations encountered clinically., Methods: Based on a total of 615 hyperspectral images from a total of 16 pigs, featuring porcine organs in different perfusion states, we carry out an exploration of distribution shifts in spectral imaging caused by perfusion alterations. We further introduce a novel strategy to mitigate such distribution shifts, utilizing synthetic data for test-time augmentation., Results: The effect of perfusion changes on state-of-the-art (SOA) segmentation networks depended on the organ and the specific perfusion alteration induced. In the case of the kidney, we observed a performance decline of up to 93% when applying a state-of-the-art (SOA) network under ischemic conditions. Our method improved on the state-of-the-art (SOA) by up to 4.6 times., Conclusion: Given its potential wide-ranging relevance to diverse pathologies, our approach may serve as a pivotal tool to enhance neural network generalization within the realm of spectral imaging., (© 2024. The Author(s).)

Spectral imaging has the potential to become a key technique in interventional medicine as it unveils much richer optical information compared to conventional RBG (red, green, and blue)-based imaging. Thus allowing for high-resolution functional tissue analysis in real time. Its higher information density particularly shows promise for the development of powerful perfusion monitoring methods for clinical use. However, even though in vivo validation of such methods is crucial for their clinical translation, the biomedical field suffers from a lack of publicly available datasets for this purpose. Closing this gap, we generated the SPECTRAL Perfusion Arm Clamping dAtaset (SPECTRALPACA). It comprises ten spectral videos (∼20 Hz, approx. 20,000 frames each) systematically recorded of the hands of ten healthy human participants in different functional states. We paired each spectral video with concisely tracked regions of interest, and corresponding diffuse reflectance measurements recorded with a spectrometer. Providing the first openly accessible in human spectral video dataset for perfusion monitoring, our work facilitates the development and validation of new functional imaging methods., (© 2024. The Author(s).)

In recent discussions in the European Parliament, the need for regulations for so-called high-risk artificial intelligence (AI) systems was identified, which are currently codified in the upcoming EU Artificial Intelligence Act (AIA) and approved by the European Parliament. The AIA is the first document to be turned into European Law. This initiative focuses on turning AI systems in decision support systems (human-in-the-loop and human-in-command), where the human operator remains in control of the system. While this supposedly solves accountability issues, it includes, on one hand, the necessary human-computer interaction as a potential new source of errors; on the other hand, it is potentially a very effective approach for decision interpretation and verification. This paper discusses the necessary requirements for high-risk AI systems once the AIA comes into force. Particular attention is paid to the opportunities and limitations that result from the decision support system and increasing the explainability of the system. This is illustrated using the example of the media forensic task of DeepFake detection.

Background: Small bowel malperfusion (SBM) can cause high morbidity and severe surgical consequences. However, there is no standardized objective measuring tool for the quantification of SBM. Indocyanine green (ICG) imaging can be used for visualization, but lacks standardization and objectivity. Hyperspectral imaging (HSI) as a newly emerging technology in medicine might present advantages over conventional ICG fluorescence or in combination with it., Methods: HSI baseline data from physiological small bowel, avascular small bowel and small bowel after intravenous application of ICG was recorded in a total number of 54 in-vivo pig models. Visualizations of avascular small bowel after mesotomy were compared between HSI only (1), ICG-augmented HSI (IA-HSI) (2), clinical evaluation through the eyes of the surgeon (3) and conventional ICG imaging (4). The primary research focus was the localization of resection borders as suggested by each of the four methods. Distances between these borders were measured and histological samples were obtained from the regions in between in order to quantify necrotic changes 6 h after mesotomy for every region., Results: StO2 images (1) were capable of visualizing areas of physiological perfusion and areas of clearly impaired perfusion. However, exact borders where physiological perfusion started to decrease could not be clearly identified. Instead, IA-HSI (2) suggested a sharp-resection line where StO2 values started to decrease. Clinical evaluation (3) suggested a resection line 23 mm (±7 mm) and conventional ICG imaging (4) even suggested a resection line 53 mm (±13 mm) closer towards the malperfused region. Histopathological evaluation of the region that was sufficiently perfused only according to conventional ICG (R3) already revealed a significant increase in pre-necrotic changes in 27% (±9%) of surface area. Therefore, conventional ICG seems less sensitive than IA-HSI with regards to detection of insufficient tissue perfusion., Conclusions: In this experimental animal study, IA-HSI (2) was superior for the visualization of segmental SBM compared to conventional HSI imaging (1), clinical evaluation (3) or conventional ICG imaging (4) regarding histopathological safety. ICG application caused visual artifacts in the StO2 values of the HSI camera as values significantly increase. This is caused by optical properties of systemic ICG and does not resemble a true increase in oxygenation levels. However, this empirical finding can be used to visualize segmental SBM utilizing ICG as contrast agent in an approach for IA-HSI. Clinical applicability and relevance will have to be explored in clinical trials., Level of Evidence: Not applicable. Translational animal science. Original article., (Copyright © 2023 The Author(s). Published by Wolters Kluwer Health, Inc.)

Hyperspectral Imaging (HSI) is a relatively new medical imaging modality that exploits an area of diagnostic potential formerly untouched. Although exploratory translational and clinical studies exist, no surgical HSI datasets are openly accessible to the general scientific community. To address this bottleneck, this publication releases HeiPorSPECTRAL ( https://www.heiporspectral.org ; https://doi.org/10.5281/zenodo.7737674 ), the first annotated high-quality standardized surgical HSI dataset. It comprises 5,758 spectral images acquired with the TIVITA ® Tissue and annotated with 20 physiological porcine organs from 8 pigs per organ distributed over a total number of 11 pigs. Each HSI image features a resolution of 480 × 640 pixels acquired over the 500-1000 nm wavelength range. The acquisition protocol has been designed such that the variability of organ spectra as a function of several parameters including the camera angle and the individual can be assessed. A comprehensive technical validation confirmed both the quality of the raw data and the annotations. We envision potential reuse within this dataset, but also its reuse as baseline data for future research questions outside this dataset. Measurement(s) Spectral Reflectance Technology Type(s) Hyperspectral Imaging Sample Characteristic - Organism Sus scrofa., (© 2023. The Author(s).)

Introduction: Esophagectomy is the mainstay of esophageal cancer treatment, but anastomotic insufficiency related morbidity and mortality remain challenging for patient outcome. Therefore, the objective of this work was to optimize anastomotic technique and gastric conduit perfusion with hyperspectral imaging (HSI) for total minimally invasive esophagectomy (MIE) with linear stapled anastomosis., Material and Methods: A live porcine model (n = 58) for MIE was used with gastric conduit formation and simulation of linear stapled side-to-side esophagogastrostomy. Four main experimental groups differed in stapling length (3 vs. 6 cm) and simulation of anastomotic position on the conduit (cranial vs. caudal). Tissue oxygenation around the anastomotic simulation site was evaluated using HSI and was validated with histopathology., Results: The tissue oxygenation (ΔStO 2 ) after the anastomotic simulation remained constant only for the short stapler in caudal position (-0.4 ± 4.4%, n.s.) while it was impaired markedly in the other groups (short-cranial: -15.6 ± 11.5%, p = 0.0002; long-cranial: -20.4 ± 7.6%, p = 0.0126; long-caudal: -16.1 ± 9.4%, p < 0.0001). Tissue samples from avascular stomach as measured by HSI showed correspondent eosinophilic pre-necrotic changes in 35.7 ± 9.7% of the surface area., Conclusion: Tissue oxygenation at the site of anastomotic simulation of the gastric conduit during MIE is influenced by stapling technique. Optimal oxygenation was achieved with a short stapler (3 cm) and sufficient distance of the simulated anastomosis to the cranial end of the gastric conduit. HSI tissue deoxygenation corresponded to histopathologic necrotic tissue changes. The experimental model with HSI and ML allow for systematic optimization of gastric conduit perfusion and anastomotic technique while clinical translation will have to be proven., Competing Interests: Declaration of competing interest The authors declare that there is no conflict of interest for the submission of the manuscript “Optimization of anastomotic technique and gastric conduit perfusion with hyperspectral imaging and machine learning in an experimental model for minimally invasive esophagectomy”., (Copyright © 2023 The Authors. Published by Elsevier Ltd.. All rights reserved.)

Laparoscopic surgery has evolved as a key technique for cancer diagnosis and therapy. While characterization of the tissue perfusion is crucial in various procedures, such as partial nephrectomy, doing so by means of visual inspection remains highly challenging. We developed a laparoscopic real-time multispectral imaging system featuring a compact and lightweight multispectral camera and the possibility to complement the conventional surgical view of the patient with functional information at a video rate of 25 Hz. To enable contrast agent-free ischemia monitoring during laparoscopic partial nephrectomy, we phrase the problem of ischemia detection as an out-of-distribution detection problem that does not rely on data from any other patient and uses an ensemble of invertible neural networks at its core. An in-human trial demonstrates the feasibility of our approach and highlights the potential of spectral imaging combined with advanced deep learning-based analysis tools for fast, efficient, reliable, and safe functional laparoscopic imaging.

Background: Intraoperative imaging assists surgeons during minimally invasive procedures. Hyperspectral imaging (HSI) is a noninvasive and noncontact optical technique with great diagnostic potential in medicine. The combination with artificial intelligence (AI) approaches to analyze HSI data is called intelligent HSI in this article., Objective: What are the medical applications and advantages of intelligent HSI for minimally invasive visceral surgery?, Material and Methods: Within various clinical studies HSI data from multiple in vivo tissue types and oncological resections were acquired using an HSI camera system. Different AI algorithms were evaluated for detection and discrimination of organs, risk structures and tumors., Results: In an experimental animal study 20 different organs could be differentiated with high precision (> 95%) using AI. In vivo, the parathyroid glands could be discriminated from surrounding tissue with an F1 score of 47% and sensitivity of 75%, and the bile duct with an F1 score of 79% and sensitivity of 90%. Furthermore, ex vivo tumor tissue could be successfully detected with an area under the receiver operating characteristic (ROC) curve (AUC) larger than 0.91., Discussion: This study demonstrates that intelligent HSI can automatically and accurately detect different tissue types. Despite great progress in the last decade intelligent HSI still has limitations. Thus, accurate AI algorithms that are easier to understand for the user and an extensive standardized and continuously growing database are needed. Further clinical studies should support the various medical applications and lead to the adoption of intelligent HSI in the clinical routine practice., (© 2022. The Author(s), under exclusive licence to Springer Medizin Verlag GmbH, ein Teil von Springer Nature.)

Semantic image segmentation is an important prerequisite for context-awareness and autonomous robotics in surgery. The state of the art has focused on conventional RGB video data acquired during minimally invasive surgery, but full-scene semantic segmentation based on spectral imaging data and obtained during open surgery has received almost no attention to date. To address this gap in the literature, we are investigating the following research questions based on hyperspectral imaging (HSI) data of pigs acquired in an open surgery setting: (1) What is an adequate representation of HSI data for neural network-based fully automated organ segmentation, especially with respect to the spatial granularity of the data (pixels vs. superpixels vs. patches vs. full images)? (2) Is there a benefit of using HSI data compared to other modalities, namely RGB data and processed HSI data (e.g. tissue parameters like oxygenation), when performing semantic organ segmentation? According to a comprehensive validation study based on 506 HSI images from 20 pigs, annotated with a total of 19 classes, deep learning-based segmentation performance increases - consistently across modalities - with the spatial context of the input data. Unprocessed HSI data offers an advantage over RGB data or processed data from the camera provider, with the advantage increasing with decreasing size of the input to the neural network. Maximum performance (HSI applied to whole images) yielded a mean DSC of 0.90 ((standard deviation (SD)) 0.04), which is in the range of the inter-rater variability (DSC of 0.89 ((standard deviation (SD)) 0.07)). We conclude that HSI could become a powerful image modality for fully-automatic surgical scene understanding with many advantages over traditional imaging, including the ability to recover additional functional tissue information. Our code and pre-trained models are available at https://github.com/IMSY-DKFZ/htc., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. The Authors declare that there is no conflict of interest., (Copyright © 2022 The Authors. Published by Elsevier B.V. All rights reserved.)

Visual discrimination of tissue during surgery can be challenging since different tissues appear similar to the human eye. Hyperspectral imaging (HSI) removes this limitation by associating each pixel with high-dimensional spectral information. While previous work has shown its general potential to discriminate tissue, clinical translation has been limited due to the method's current lack of robustness and generalizability. Specifically, the scientific community is lacking a comprehensive spectral tissue atlas, and it is unknown whether variability in spectral reflectance is primarily explained by tissue type rather than the recorded individual or specific acquisition conditions. The contribution of this work is threefold: (1) Based on an annotated medical HSI data set (9059 images from 46 pigs), we present a tissue atlas featuring spectral fingerprints of 20 different porcine organs and tissue types. (2) Using the principle of mixed model analysis, we show that the greatest source of variability related to HSI images is the organ under observation. (3) We show that HSI-based fully-automatic tissue differentiation of 20 organ classes with deep neural networks is possible with high accuracy (> 95%). We conclude from our study that automatic tissue discrimination based on HSI data is feasible and could thus aid in intraoperative decisionmaking and pave the way for context-aware computer-assisted surgery systems and autonomous robotics., (© 2022. The Author(s).)

Academic research in media forensics mainly focuses on methods for the detection of the traces or artefacts left by media manipulations in media objects. While the resulting detectors often achieve quite impressive detection performances, when tested under lab conditions, hardly any of those have yet come close to the ultimate benchmark for any forensic method, which would be courtroom readiness. This paper tries first to facilitate the different stakeholder perspectives in this field and then to partly address the apparent gap between the academic research community and the requirements imposed onto forensic practitioners. The intention is to facilitate the mutual understanding of these two classes of stakeholders and assist with first steps intended at closing this gap. To do so, first a concept for modelling media forensic investigation pipelines is derived from established guidelines. Then, the applicability of such modelling is illustrated on the example of a fusion-based media forensic investigation pipeline aimed at the detection of DeepFake videos using five exemplary detectors (hand-crafted, in one case neural network supported) and testing two different fusion operators. At the end of the paper, the benefits of such a planned realisation of AI-based investigation methods are discussed and generalising effects are mapped out.

DeepFake detection is a novel task for media forensics and is currently receiving a lot of research attention due to the threat these targeted video manipulations propose to the trust placed in video footage. The current trend in DeepFake detection is the application of neural networks to learn feature spaces that allow them to be distinguished from unmanipulated videos. In this paper, we discuss, with features hand-crafted by domain experts, an alternative to this trend. The main advantage that hand-crafted features have over learned features is their interpretability and the consequences this might have for plausibility validation for decisions made. Here, we discuss three sets of hand-crafted features and three different fusion strategies to implement DeepFake detection. Our tests on three pre-existing reference databases show detection performances that are under comparable test conditions (peak AUC > 0.95) to those of state-of-the-art methods using learned features. Furthermore, our approach shows a similar, if not better, generalization behavior than neural network-based methods in tests performed with different training and test sets. In addition to these pattern recognition considerations, first steps of a projection onto a data-centric examination approach for forensics process modeling are taken to increase the maturity of the present investigation.

Image-based tracking of medical instruments is an integral part of surgical data science applications. Previous research has addressed the tasks of detecting, segmenting and tracking medical instruments based on laparoscopic video data. However, the proposed methods still tend to fail when applied to challenging images and do not generalize well to data they have not been trained on. This paper introduces the Heidelberg Colorectal (HeiCo) data set - the first publicly available data set enabling comprehensive benchmarking of medical instrument detection and segmentation algorithms with a specific emphasis on method robustness and generalization capabilities. Our data set comprises 30 laparoscopic videos and corresponding sensor data from medical devices in the operating room for three different types of laparoscopic surgery. Annotations include surgical phase labels for all video frames as well as information on instrument presence and corresponding instance-wise segmentation masks for surgical instruments (if any) in more than 10,000 individual frames. The data has successfully been used to organize international competitions within the Endoscopic Vision Challenges 2017 and 2019.

Purpose: Live intra-operative functional imaging has multiple potential clinical applications, such as localization of ischemia, assessment of organ transplantation success and perfusion monitoring. Recent research has shown that live monitoring of functional tissue properties, such as tissue oxygenation and blood volume fraction, is possible using multispectral imaging in laparoscopic surgery. While the illuminant spectrum is typically kept constant in laparoscopic surgery and can thus be estimated from preoperative calibration images, a key challenge in open surgery originates from the dynamic changes of lighting conditions., Methods: The present paper addresses this challenge with a novel approach to light source calibration based on specular highlight analysis. It involves the acquisition of low-exposure time images serving as a basis for recovering the illuminant spectrum from pixels that contain a dominant specular reflectance component., Results: Comprehensive in silico and in vivo experiments with a range of different light sources demonstrate that our approach enables an accurate and robust recovery of the illuminant spectrum in the field of view of the camera, which results in reduced errors with respect to the estimation of functional tissue properties. Our approach further outperforms state-of-the-art methods proposed in the field of computer vision., Conclusion: Our results suggest that low-exposure multispectral images are well suited for light source calibration via specular highlight analysis. This work thus provides an important first step toward live functional imaging in open surgery.

Glioblastoma multiforme is characterized by high accumulation of microglia/macrophages. The function of these tumor-infiltrating myeloid cells is not sufficiently elucidated. Therefore, a better understanding of the precise immune cell composition and function in brain tumors is required. In rodent glioma models, two different myeloid cell populations exist, determined by the expression level of CD45, namely CD11b + CD45 low and CD11b + CD45 high . Previous analyses of cytokine and marker expression profiles were almost exclusively performed on the entire myeloid cell fraction. Consequently, described pro- and anti-tumoral characteristics were not assigned to the evident subpopulations. In the present study, we used a syngeneic glioblastoma mouse model and subsequent flow cytometric analyses to demonstrate the distinct properties of CD11b + CD45 high and the CD11b + CD45 low cells. First, the majority of CD11b + CD45 high cells expressed high level of GR1 and around 6% of IL10 representing in part features of myeloid-derived suppressor cells, while the CD11b + CD45 low fraction displayed no upregulation of these molecules. Second, we detected that specifically the CD11b + CD45 high population showed antigen-presenting, co-stimulatory, and inflammatory features. Here, we identified up to 80% of MHCII and approximately 50% of CD86 and TNFα-expressing cells. Investigation of MHCI and CD80 revealed a moderate upregulation. By contrast, in the CD11b + CD45 low cell fraction, merely MHCII and TNFα were marginally overexpressed. In summary, these data emphasize the specific phenotype of CD11b + CD45 high cells in glioma with suppressive as well as pro-inflammatory characteristics whereas the CD11b + CD45 low cells were almost unaffected. Hence, primarily, the subpopulation consisting of CD45 high -expressing cells is activated by the tumor and should be considered as therapeutic target.