Search

Semantic segmentation is an essential component of medical image analysis research, with recent deep learning algorithms offering out-of-the-box applicability across diverse datasets. Despite these advancements, segmentation failures remain a significant concern for real-world clinical applications, necessitating reliable detection mechanisms. This paper introduces a comprehensive benchmarking framework aimed at evaluating failure detection methodologies within medical image segmentation. Through our analysis, we identify the strengths and limitations of current failure detection metrics, advocating for the risk-coverage analysis as a holistic evaluation approach. Utilizing a collective dataset comprising five public 3D medical image collections, we assess the efficacy of various failure detection strategies under realistic test-time distribution shifts. Our findings highlight the importance of pixel confidence aggregation and we observe superior performance of the pairwise Dice score (Roy et al., 2019) between ensemble predictions, positioning it as a simple and robust baseline for failure detection in medical image segmentation. To promote ongoing research, we make the benchmarking framework available to the community., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2024 The Authors. Published by Elsevier B.V. All rights reserved.)

Objectives: To evaluate the performance and potential biases of deep-learning models in detecting chronic obstructive pulmonary disease (COPD) on chest CT scans across different ethnic groups, specifically non-Hispanic White (NHW) and African American (AA) populations., Materials and Methods: Inspiratory chest CT and clinical data from 7549 Genetic epidemiology of COPD individuals (mean age 62 years old, 56-69 interquartile range), including 5240 NHW and 2309 AA individuals, were retrospectively analyzed. Several factors influencing COPD binary classification performance on different ethnic populations were examined: (1) effects of training population: NHW-only, AA-only, balanced set (half NHW, half AA) and the entire set (NHW + AA all); (2) learning strategy: three supervised learning (SL) vs. three self-supervised learning (SSL) methods. Distribution shifts across ethnicity were further assessed for the top-performing methods., Results: The learning strategy significantly influenced model performance, with SSL methods achieving higher performances compared to SL methods (p < 0.001), across all training configurations. Training on balanced datasets containing NHW and AA individuals resulted in improved model performance compared to population-specific datasets. Distribution shifts were found between ethnicities for the same health status, particularly when models were trained on nearest-neighbor contrastive SSL. Training on a balanced dataset resulted in fewer distribution shifts across ethnicity and health status, highlighting its efficacy in reducing biases., Conclusion: Our findings demonstrate that utilizing SSL methods and training on large and balanced datasets can enhance COPD detection model performance and reduce biases across diverse ethnic populations. These findings emphasize the importance of equitable AI-driven healthcare solutions for COPD diagnosis., Critical Relevance Statement: Self-supervised learning coupled with balanced datasets significantly improves COPD detection model performance, addressing biases across diverse ethnic populations and emphasizing the crucial role of equitable AI-driven healthcare solutions., Key Points: Self-supervised learning methods outperform supervised learning methods, showing higher AUC values (p < 0.001). Balanced datasets with non-Hispanic White and African American individuals improve model performance. Training on diverse datasets enhances COPD detection accuracy. Ethnically diverse datasets reduce bias in COPD detection models. SimCLR models mitigate biases in COPD detection across ethnicities., (© 2024. The Author(s).)

Objectives: To quantify regional manifestations related to COPD as anomalies from a modeled distribution of normal-appearing lung on chest CT using a deep learning (DL) approach, and to assess its potential to predict disease severity., Materials and Methods: Paired inspiratory/expiratory CT and clinical data from COPDGene and COSYCONET cohort studies were included. COPDGene data served as training/validation/test data sets (N = 3144/786/1310) and COSYCONET as external test set (N = 446). To differentiate low-risk (healthy/minimal disease, [GOLD 0]) from COPD patients (GOLD 1-4), the self-supervised DL model learned semantic information from 50 × 50 × 50 voxel samples from segmented intact lungs. An anomaly detection approach was trained to quantify lung abnormalities related to COPD, as regional deviations. Four supervised DL models were run for comparison. The clinical and radiological predictive power of the proposed anomaly score was assessed using linear mixed effects models (LMM)., Results: The proposed approach achieved an area under the curve of 84.3 ± 0.3 (p < 0.001) for COPDGene and 76.3 ± 0.6 (p < 0.001) for COSYCONET, outperforming supervised models even when including only inspiratory CT. Anomaly scores significantly improved fitting of LMM for predicting lung function, health status, and quantitative CT features (emphysema/air trapping; p < 0.001). Higher anomaly scores were significantly associated with exacerbations for both cohorts (p < 0.001) and greater dyspnea scores for COPDGene (p < 0.001)., Conclusion: Quantifying heterogeneous COPD manifestations as anomaly offers advantages over supervised methods and was found to be predictive for lung function impairment and morphology deterioration., Clinical Relevance Statement: Using deep learning, lung manifestations of COPD can be identified as deviations from normal-appearing chest CT and attributed an anomaly score which is consistent with decreased pulmonary function, emphysema, and air trapping., Key Points: • A self-supervised DL anomaly detection method discriminated low-risk individuals and COPD subjects, outperforming classic DL methods on two datasets (COPDGene AUC = 84.3%, COSYCONET AUC = 76.3%). • Our contrastive task exhibits robust performance even without the inclusion of expiratory images, while voxel-based methods demonstrate significant performance enhancement when incorporating expiratory images, in the COPDGene dataset. • Anomaly scores improved the fitting of linear mixed effects models in predicting clinical parameters and imaging alterations (p < 0.001) and were directly associated with clinical outcomes (p < 0.001)., (© 2023. The Author(s).)

Purpose: To assess the feasibility and diagnostic accuracy of MRI-derived 3D volumetry of lower lumbar vertebrae and dural sac segments using shape-based machine learning for the detection of Marfan syndrome (MFS) compared with dural sac diameter ratios (the current clinical standard)., Materials and Methods: The final study sample was 144 patients being evaluated for MFS from 01/2012 to 12/2016, of whom 81 were non-MFS patients (46 [67%] female, 36 ± 16 years) and 63 were MFS patients (36 [57%] female, 35 ± 11 years) according to the 2010 Revised Ghent Nosology. All patients underwent 1.5T MRI with isotropic 1 × 1 × 1 mm 3 3D T2-weighted acquisition of the lumbosacral spine. Segmentation and quantification of vertebral bodies L3-L5 and dural sac segments L3-S1 were performed using a shape-based machine learning algorithm. For comparison with the current clinical standard, anteroposterior diameters of vertebral bodies and dural sac were measured. Ratios between dural sac volume/diameter at the respective level and vertebral body volume/diameter were calculated., Results: Three-dimensional volumetry revealed larger dural sac volumes ( p < 0.001) and volume ratios ( p < 0.001) at L3-S1 levels in MFS patients compared with non-MFS patients. For the detection of MFS, 3D volumetry achieved higher AUCs at L3-S1 levels (0.743, 0.752, 0.808, and 0.824) compared with dural sac diameter ratios (0.673, 0.707, 0.791, and 0.848); a significant difference was observed only for L3 ( p < 0.001)., Conclusion: MRI-derived 3D volumetry of the lumbosacral dural sac and vertebral bodies is a feasible method for quantifying dural ectasia using shape-based machine learning. Non-inferior diagnostic accuracy was observed compared with dural sac diameter ratio (the current clinical standard for MFS detection).

Objectives: Achieving a consensus on a definition for different aspects of radiomics workflows to support their translation into clinical usage. Furthermore, to assess the perspective of experts on important challenges for a successful clinical workflow implementation., Materials and Methods: The consensus was achieved by a multi-stage process. Stage 1 comprised a definition screening, a retrospective analysis with semantic mapping of terms found in 22 workflow definitions, and the compilation of an initial baseline definition. Stages 2 and 3 consisted of a Delphi process with over 45 experts hailing from sites participating in the German Research Foundation (DFG) Priority Program 2177. Stage 2 aimed to achieve a broad consensus for a definition proposal, while stage 3 identified the importance of translational challenges., Results: Workflow definitions from 22 publications (published 2012-2020) were analyzed. Sixty-nine definition terms were extracted, mapped, and semantic ambiguities (e.g., homonymous and synonymous terms) were identified and resolved. The consensus definition was developed via a Delphi process. The final definition comprising seven phases and 37 aspects reached a high overall consensus (> 89% of experts "agree" or "strongly agree"). Two aspects reached no strong consensus. In addition, the Delphi process identified and characterized from the participating experts' perspective the ten most important challenges in radiomics workflows., Conclusion: To overcome semantic inconsistencies between existing definitions and offer a well-defined, broad, referenceable terminology, a consensus workflow definition for radiomics-based setups and a terms mapping to existing literature was compiled. Moreover, the most relevant challenges towards clinical application were characterized., Critical Relevance Statement: Lack of standardization represents one major obstacle to successful clinical translation of radiomics. Here, we report a consensus workflow definition on different aspects of radiomics studies and highlight important challenges to advance the clinical adoption of radiomics., Key Points: Published radiomics workflow terminologies are inconsistent, hindering standardization and translation. A consensus radiomics workflow definition proposal with high agreement was developed. Publicly available result resources for further exploitation by the scientific community., (© 2024. The Author(s).)

Background: Chronic obstructive pulmonary disease (COPD) poses a substantial global health burden, demanding advanced diagnostic tools for early detection and accurate phenotyping. In this line, this study seeks to enhance COPD characterization on chest computed tomography (CT) by comparing the spatial and quantitative relationships between traditional parametric response mapping (PRM) and a novel self-supervised anomaly detection approach, and to unveil potential additional insights into the dynamic transitional stages of COPD., Methods: Non-contrast inspiratory and expiratory CT of 1,310 never-smoker and GOLD 0 individuals and COPD patients (GOLD 1-4) from the COPDGene dataset were retrospectively evaluated. A novel self-supervised anomaly detection approach was applied to quantify lung abnormalities associated with COPD, as regional deviations. These regional anomaly scores were qualitatively and quantitatively compared, per GOLD class, to PRM volumes (emphysema: PRM Emph , functional small-airway disease: PRM fSAD ) and to a Principal Component Analysis (PCA) and Clustering, applied on the self-supervised latent space. Its relationships to pulmonary function tests (PFTs) were also evaluated., Results: Initial t-Distributed Stochastic Neighbor Embedding (t-SNE) visualization of the self-supervised latent space highlighted distinct spatial patterns, revealing clear separations between regions with and without emphysema and air trapping. Four stable clusters were identified among this latent space by the PCA and Cluster Analysis. As the GOLD stage increased, PRM Emph , PRM fSAD , anomaly score, and Cluster 3 volumes exhibited escalating trends, contrasting with a decline in Cluster 2. The patient-wise anomaly scores significantly differed across GOLD stages ( p  < 0.01), except for never-smokers and GOLD 0 patients. In contrast, PRM Emph , PRM fSAD , and cluster classes showed fewer significant differences. Pearson correlation coefficients revealed moderate anomaly score correlations to PFTs (0.41-0.68), except for the functional residual capacity and smoking duration. The anomaly score was correlated with PRM Emph ( r  = 0.66, p  < 0.01) and PRM fSAD ( r  = 0.61, p  < 0.01). Anomaly scores significantly improved fitting of PRM-adjusted multivariate models for predicting clinical parameters ( p  < 0.001). Bland-Altman plots revealed that volume agreement between PRM-derived volumes and clusters was not constant across the range of measurements., Conclusion: Our study highlights the synergistic utility of the anomaly detection approach and traditional PRM in capturing the nuanced heterogeneity of COPD. The observed disparities in spatial patterns, cluster dynamics, and correlations with PFTs underscore the distinct - yet complementary - strengths of these methods. Integrating anomaly detection and PRM offers a promising avenue for understanding of COPD pathophysiology, potentially informing more tailored diagnostic and intervention approaches to improve patient outcomes., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2024 Almeida, Norajitra, Lüth, Wald, Weru, Nolden, Jäger, von Stackelberg, Heußel, Weinheimer, Biederer, Kauczor and Maier-Hein.)

Background: Image-guidance promises to make complex situations in liver interventions safer. Clinical success is limited by intraoperative organ motion due to ventilation and surgical manipulation. The aim was to assess influence of different ventilatory and operative states on liver motion in an experimental model., Methods: Liver motion due to ventilation (expiration, middle, and full inspiration) and operative state (native, laparotomy, and pneumoperitoneum) was assessed in a live porcine model (n = 10). Computed tomography (CT)-scans were taken for each pig for each possible combination of factors. Liver motion was measured by the vectors between predefined landmarks along the hepatic vein tree between CT scans after image segmentation., Results: Liver position changed significantly with ventilation. Peripheral regions of the liver showed significantly higher motion (maximal Euclidean motion 17.9 ± 2.7 mm) than central regions (maximal Euclidean motion 12.6 ± 2.1 mm, p < 0.001) across all operative states. The total average motion measured 11.6 ± 0.7 mm (p < 0.001). Between the operative states, the position of the liver changed the most from native state to pneumoperitoneum (14.6 ± 0.9 mm, p < 0.001). From native state to laparotomy comparatively, the displacement averaged 9.8 ± 1.2 mm (p < 0.001). With pneumoperitoneum, the breath-dependent liver motion was significantly reduced when compared to other modalities. Liver motion due to ventilation was 7.7 ± 0.6 mm during pneumoperitoneum, 13.9 ± 1.1 mm with laparotomy, and 13.5 ± 1.4 mm in the native state (p < 0.001 in all cases)., Conclusions: Ventilation and application of pneumoperitoneum caused significant changes in liver position. Liver motion was reduced but clearly measurable during pneumoperitoneum. Intraoperative guidance/navigation systems should therefore account for ventilation and intraoperative changes of liver position and peripheral deformation., (© 2023. The Author(s).)

This research addresses the assessment of adipose tissue (AT) and spatial distribution of visceral (VAT) and subcutaneous fat (SAT) in the trunk from standardized magnetic resonance imaging at 3 T, thereby demonstrating the feasibility of deep learning (DL)-based image segmentation in a large population-based cohort in Germany (five sites). Volume and distribution of AT play an essential role in the pathogenesis of insulin resistance, a risk factor of developing metabolic/cardiovascular diseases. Cross-validated training of the DL-segmentation model led to a mean Dice similarity coefficient of >0.94, corresponding to a mean absolute volume deviation of about 22 ml. SAT is significantly increased in women compared to men, whereas VAT is increased in males. Spatial distribution shows age- and body mass index-related displacements. DL-based image segmentation provides robust and fast quantification of AT (≈15 s per dataset versus 3 to 4 hours for manual processing) and assessment of its spatial distribution from magnetic resonance images in large cohort studies.

Automated image analysis plays an increasing role in radiology in detecting and quantifying image features outside of the perception of human eyes. Common AI-based approaches address a single medical problem, although patients often present with multiple interacting, frequently subclinical medical conditions. A holistic imaging diagnostics tool based on artificial intelligence (AI) has the potential of providing an overview of multi-system comorbidities within a single workflow. An interdisciplinary, multicentric team of medical experts and computer scientists designed a pipeline, comprising AI-based tools for the automated detection, quantification and characterization of the most common pulmonary, metabolic, cardiovascular and musculoskeletal comorbidities in chest computed tomography (CT). To provide a comprehensive evaluation of each patient, a multidimensional workflow was established with algorithms operating synchronously on a decentralized Joined Imaging Platform (JIP). The results of each patient are transferred to a dedicated database and summarized as a structured report with reference to available reference values and annotated sample images of detected pathologies. Hence, this tool allows for the comprehensive, large-scale analysis of imaging-biomarkers of comorbidities in chest CT, first in science and then in clinical routine. Moreover, this tool accommodates the quantitative analysis and classification of each pathology, providing integral diagnostic and prognostic value, and subsequently leading to improved preventive patient care and further possibilities for future studies.

Fluoroscopy-guided trauma and orthopedic surgeries involve the repeated acquisition of correct anatomy-specific standard projections for guidance, monitoring, and evaluating the surgical result. C-arm positioning is usually performed by hand, involving repeated or even continuous fluoroscopy at a cost of radiation exposure and time. We propose to automate this procedure and estimate the pose update for C-arm repositioning directly from a first X-ray without the need for a patient-specific computed tomography scan (CT) or additional technical equipment. Our method is trained on digitally reconstructed radiographs (DRRs) which uniquely provide ground truth labels for an arbitrary number of training examples. The simulated images are complemented with automatically generated segmentations, landmarks, and with simulated k-wires and screws. To successfully achieve a transfer from simulated to real X-rays, and also to increase the interpretability of results, the pipeline was designed to closely reflect the actual clinical decision-making process followed by spinal neurosurgeons. It explicitly incorporates steps such as region-of-interest (ROI) localization, detection of relevant and view-independent landmarks, and subsequent pose regression. The method was validated on a large human cadaver study simulating a real clinical scenario, including k-wires and screws. The proposed procedure obtained superior C-arm positioning accuracy of dθ=8.8°±4.2° average improvement (p t-test ≪0.01), robustness, and generalization capabilities compared to the state-of-the-art direct pose regression framework., Competing Interests: Declaration of Competing Interest The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Lisa Kausch reports financial support was provided by Siemens Healthineers. Sarina Thomas, Jan El Babari, Eric Mandelka, Maxim Privalov, Sven Vetter, Klaus Maier-Hein reports financial support was provided by Siemens Healthineers., (Copyright © 2022 The Author(s). Published by Elsevier B.V. All rights reserved.)

Background Context: Navigation and robotic systems have been increasingly applied to spinal instrumentation but dedicated screw planning is a time-consuming prerequisite to tap the full potential of these techniques., Purpose: To develop and validate an automated planning tool for lumbosacral pedicle screw placement using a convolutional neural network (CNN) to facilitate the planning process., Study Design/setting: Retrospective analysis and processing of CT and screw planning data randomly selected from a consecutive registry of CT-navigated instrumentations from a single academic institution., Patient Sample: Data from 179 cases was processed for CNN training and validation (155 for training, 24 for validation) leveraging a total of 1182 screws (1052 for training, 130 for validation)., Outcome Measures: Quantitative and qualitative (Gertzbein-Robbins classification [GR]) validation via comparison of automatically and manually planned reference screws, inter-rater and intra-rater variability., Methods: Annotated data from CT-navigated instrumentation was used to train a CNN operating in a vertebra instance-based approach employing a state-of-the-art U-Net framework. Internal five-fold cross-validation and external validation on an independent cohort not previously involved in training was performed. Quantitative validation of automatically planned screws was performed in comparison to corresponding manually planned screws by calculating the minimal absolute difference (MAD) of screw head and tip points, length and diameter, screw direction and Dice coefficient. Results were evaluated in relation to inter-rater and intra-rater variability of manual screw planning., Results: Automated screw planning was successful in all targeted 130 screws. Compared with manually planned screws as a reference, mean MAD of automatically planned screws was 4.61±2.27 mm for screw head, 3.96±2.19 mm for tip points and 5.51±3.64° for screw direction. These differences were either statistically comparable or significantly smaller when compared with interrater variability of manual screw planning (p>.99 for head point and direction, p=.004 for tip point, respectively). Mean Dice coefficient of 0.61±0.16 indicated significantly greater agreement of automatic screws with the manual reference compared with interrater agreement (Dice 0.56±0.18, p<.001). Automatically planned screws were marginally shorter (MAD 3.4±3.2 mm) and thinner (MAD mean 0.3±0.6 mm) compared with the manual reference, but with statistical significance (p<.0001, respectively). Automatically planned screws were GR grade A in 96.2% in qualitative validation. Planning time was significantly shorter with the automatic approach (0:41 min vs. 6:41 min, p<.0001)., Conclusions: We derived and validated a fully automated planning tool for lumbosacral pedicle screws using a CNN. Our validation showed noninferiority to manual screw planning and provided sufficient accuracy to facilitate and expedite the screw planning process. These results offer a high potential to improve workflows in spine surgery when integrated into navigation or robotic assistance systems., Competing Interests: Declarations of Competing Interests All authors declare that they have no known competing financial interests or personal relationships to disclose that could have appeared to influence the work reported in this paper., (Copyright © 2022 Elsevier Inc. All rights reserved.)

Background: Hepatectomy, living donor liver transplantations and other major hepatic interventions rely on precise calculation of the total, remnant and graft liver volume. However, liver volume might differ between the pre- and intraoperative situation. To model liver volume changes and develop and validate such pre- and intraoperative assistance systems, exact information about the influence of lung ventilation and intraoperative surgical state on liver volume is essential., Methods: This study assessed the effects of respiratory phase, pneumoperitoneum for laparoscopy, and laparotomy on liver volume in a live porcine model. Nine CT scans were conducted per pig (N = 10), each for all possible combinations of the three operative (native, pneumoperitoneum and laparotomy) and respiratory states (expiration, middle inspiration and deep inspiration). Manual segmentations of the liver were generated and converted to a mesh model, and the corresponding liver volumes were calculated., Results: With pneumoperitoneum the liver volume decreased on average by 13.2% (112.7 ml ± 63.8 ml, p < 0.0001) and after laparotomy by 7.3% (62.0 ml ± 65.7 ml, p = 0.0001) compared to native state. From expiration to middle inspiration the liver volume increased on average by 4.1% (31.1 ml ± 55.8 ml, p = 0.166) and from expiration to deep inspiration by 7.2% (54.7 ml ± 51.8 ml, p = 0.007)., Conclusions: Considerable changes in liver volume change were caused by pneumoperitoneum, laparotomy and respiration. These findings provide knowledge for the refinement of available preoperative simulation and operation planning and help to adjust preoperative imaging parameters to best suit the intraoperative situation., (© 2020. The Author(s).)

Objective: Evaluation of software tools for segmentation, quantification, and characterization of fibrotic pulmonary parenchyma changes will strengthen the role of CT as biomarkers of disease extent, evolution, and response to therapy in idiopathic pulmonary fibrosis (IPF) patients., Methods: 418 nonenhanced thin-section MDCTs of 127 IPF patients and 78 MDCTs of 78 healthy individuals were analyzed through 3 fully automated, completely different software tools: YACTA, LUFIT, and IMBIO. The agreement between YACTA and LUFIT on segmented lung volume and 80th (reflecting fibrosis) and 40th (reflecting ground-glass opacity) percentile of the lung density histogram was analyzed using Bland-Altman plots. The fibrosis and ground-glass opacity segmented by IMBIO (lung texture analysis software tool) were included in specific regression analyses., Results: In the IPF-group, LUFIT outperformed YACTA by segmenting more lung volume (mean difference 242 mL, 95% limits of agreement -54 to 539 mL), as well as quantifying higher 80th (76 HU, -6 to 158 HU) and 40th percentiles (9 HU, -73 to 90 HU). No relevant differences were revealed in the control group. The 80th/40th percentile as quantified by LUFIT correlated positively with the percentage of fibrosis/ground-glass opacity calculated by IMBIO (r = 0.78/r = 0.92)., Conclusions: In terms of segmentation of pulmonary fibrosis, LUFIT as a shape model-based segmentation software tool is superior to the threshold-based YACTA, tool, since the density of (severe) fibrosis is similar to that of the surrounding soft tissues. Therefore, shape modeling as used in LUFIT may serve as a valid tool in the quantification of IPF, since this mainly affects the subpleural space., (© 2021 S. Karger AG, Basel.)

3D Statistical Shape Models (3D-SSM) are widely used for medical image segmentation. However, during segmentation, they typically perform a very limited unidirectional search for suitable landmark positions in the image, relying on weak learners or use-case specific appearance models that solely take local image information into account. As a consequence, segmentation errors arise, and results in general depend on the accuracy of a previous model initialization. Furthermore, these methods become subject to a tedious and use-case dependent parameter tuning in order to obtain optimized results. To overcome these limitations, we propose an extension of 3D-SSM by landmark-wise random regression forests that perform an enhanced omni-directional search for landmark positions, thereby taking rich non-local image information into account. In addition, we provide a long distance model fitting based on a multi-scale approach, that allows an accurate and reproducible segmentation even from distant image positions, thus enabling an application without model initialization. Finally, translation of the proposed method to different organs is straightforward and requires no adaptation of the training process. In segmentation experiments on 45 clinical CT volumes, the proposed omni-directional search significantly increased accuracy and displayed great precision regardless of model initialization. Furthermore, for liver, spleen and kidney segmentation in a competitive multi-organ labeling challenge on publicly available data, the proposed method achieved similar or better results than the state of the art. Finally, liver segmentation results were obtained that successfully compete with specialized state-of-the-art methods from the well-known liver segmentation challenge SLIVER.