Search

Purpose: Recent development in positron emission tomography (PET) dramatically increased the effective sensitivity by increasing the geometric coverage leading to total-body PET imaging. This encouraging breakthrough brings the hope of ultra-low dose PET imaging equivalent to transatlantic flight with the assistance of deep learning (DL)-based methods. However, conventional DL approaches face limitations in addressing the heterogeneous domain of PET imaging. This study aims to develop a wavelet-based DL method capable of restoring high-quality imaging from ultra-low-dose PET scans., Materials and Methods: In contrast to conventional DL techniques that denoise images in the spatial domain, we introduce WaveNet, a novel approach that inputs wavelet-decomposed frequency components of PET imaging to perform denoising in the frequency domain. A dataset comprising total-body 18 F -FDG PET images of 1447, acquired using total-body PET scanners including Biograph Vision Quadra (Siemens Healthineers) and uEXPLORER (United Imaging) in Bern and Shanghai, was utilized for developing and testing the proposed method. The quality of enhanced images was assessed using a customized scoring system, which incorporated weighted global physical metrics and local indices., Results: Our proposed WaveNet consistently outperforms the baseline UNet model across all levels of dose reduction factors (DRF), with greater improvements observed as image quality decreases. Statistical analysis (p < 0.05) and visual inspection validated the superiority of WaveNet. Moreover, WaveNet demonstrated superior generalizability when applied to two cross-scanner datasets (p < 0.05)., Conclusion: WaveNet developed with total-body PET scanners may offer a computational-friendly and robust approach to recover image quality from ultra-low-dose PET imaging. Its adoption may enhance the reliability and clinical acceptance of DL-based dose reduction techniques., Competing Interests: Declarations. Competing interests: H.S. is a full-time employee of Siemens Healthineers AG in Switzerland. K.S. and A.R. have received research grants from Siemens Healthineers AG and Novartis AG. Authors A.R., B.L. and K.S. are editors in Eur J Nucl Med Mol Imaging. The remaining authors declare no competing interests. Ethics approval: This retrospective study complies with all relevant ethical regulations of the respective local ethics committees in Switzerland (Waiver from Cantonal Ethics Committee of Bern, Switzerland) and China (Approval from Ruijin Hospital Ethics Committee Shanghai Jiao Tong University School of Medicine). Consent to participate: Informed consent was obtained from all patients included in this study. Consent to publish: All authors have reviewed the final version of the manuscript and approved it for submission to your journal., (© 2024. The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature.)

Purpose: To identify optical coherence tomography (OCT) biomarkers for macula-off rhegmatogenous retinal detachment (RRD) with artificial intelligence (AI) and to correlate these biomarkers with functional outcomes., Methods: Patients with macula-off RRD treated with single vitrectomy and gas tamponade were included. OCT volumes, taken at 4 to 6 weeks and 1 year postoperative, were uploaded on an AI-derived platform (Discovery OCT Biomarker Detector; RetinAI AG, Bern, Switzerland), measuring different retinal layer thicknesses, including outer nuclear layer (ONL), photoreceptor and retinal pigmented epithelium (PR + RPE), intraretinal fluid (IRF), subretinal fluid, and biomarker probability detection, including hyperreflective foci (HF). A random forest model assessed the predictive factors for final best-corrected visual acuity (BCVA)., Results: Fifty-nine patients (42 male, 17 female) were enrolled. Baseline BCVA was 0.5 logarithmic minimum angle of resolution (logMAR) ± 0.1, significantly improving to 0.3 ± 0.1 logMAR at the final visit (P < 0.001). Average thickness analysis indicated a significant increase after the last follow-up visit for ONL (from 95.16 ± 5.47 µm to 100.8 ± 5.27 µm, P = 0.0007) and PR + RPE thicknesses (60.9 ± 2.6 µm to 66.2 ± 1.8 µm, P = 0.0001). Average occurrence rate of HF was 0.12 ± 0.06 at initial visit and 0.08 ± 0.05 at last follow-up visit (P = 0.0093). Random forest model revealed baseline BCVA as the most critical predictor for final BCVA, followed by ONL thickness, HF, and IRF presence at the initial visit., Conclusions: Increased ONL and PR-RPE thickness associate with better outcomes, while HF presence indicates poorer results, with initial BCVA remaining a primary visual predictor., Translational Relevance: The study underscores the role of novel biomarkers like HF in understanding visual function in macula-off RRD.

In Ultrasound Localization Microscopy (ULM), achieving high-resolution images relies on the precise localization of contrast agent particles across a series of beamformed frames. However, our study uncovers an enormous potential: The process of delay-and-sum beamforming leads to an irreversible reduction of Radio-Frequency (RF) channel data, while its implications for localization remain largely unexplored. The rich contextual information embedded within RF wavefronts, including their hyperbolic shape and phase, offers great promise for guiding Deep Neural Networks (DNNs) in challenging localization scenarios. To fully exploit this data, we propose to directly localize scatterers in RF channel data. Our approach involves a custom super-resolution DNN using learned feature channel shuffling, non-maximum suppression, and a semi-global convolutional block for reliable and accurate wavefront localization. Additionally, we introduce a geometric point transformation that facilitates seamless mapping to the B-mode coordinate space. To understand the impact of beamforming on ULM, we validate the effectiveness of our method by conducting an extensive comparison with State-Of-The-Art (SOTA) techniques. We present the inaugural in vivo results from a wavefront-localizing DNN, highlighting its real-world practicality. Our findings show that RF-ULM bridges the domain shift between synthetic and real datasets, offering a considerable advantage in terms of precision and complexity. To enable the broader research community to benefit from our findings, our code and the associated SOTA methods are made available at https://github.com/hahnec/rf-ulm.

The release of Ca 2+ ions from intracellular stores plays a crucial role in many cellular processes, acting as a secondary messenger in various cell types, including cardiomyocytes, smooth muscle cells, hepatocytes, and many others. Detecting and classifying associated local Ca 2+ release events is particularly important, as these events provide insight into the mechanisms, interplay, and interdependencies of local Ca 2+ release events underlying global intracellular Ca 2+ signaling. However, time-consuming and labor-intensive procedures often complicate analysis, especially with low signal-to-noise ratio imaging data. Here, we present an innovative deep learning-based approach for automatically detecting and classifying local Ca 2+ release events. This approach is exemplified with rapid full-frame confocal imaging data recorded in isolated cardiomyocytes. To demonstrate the robustness and accuracy of our method, we first use conventional evaluation methods by comparing the intersection between manual annotations and the segmentation of Ca 2+ release events provided by the deep learning method, as well as the annotated and recognized instances of individual events. In addition to these methods, we compare the performance of the proposed model with the annotation of six experts in the field. Our model can recognize more than 75 % of the annotated Ca 2+ release events and correctly classify more than 75 %. A key result was that there were no significant differences between the annotations produced by human experts and the result of the proposed deep learning model. We conclude that the proposed approach is a robust and time-saving alternative to conventional full-frame confocal imaging analysis of local intracellular Ca 2+ events., Competing Interests: Declaration of competing interest The authors declare no competing interests., (Copyright © 2024 The Author(s). Published by Elsevier Ltd.. All rights reserved.)

Purpose: To explore the structural-functional loss relationship from optic-nerve-head- and macula-centred spectral-domain (SD) Optical Coherence Tomography (OCT) images in the full spectrum of glaucoma patients using deep-learning methods., Methods: A cohort comprising 5238 unique eyes classified as suspects or diagnosed with glaucoma was considered. All patients underwent ophthalmologic examination consisting of standard automated perimetry (SAP), macular OCT, and peri-papillary OCT on the same day. Deep learning models were trained to estimate G-pattern visual field (VF) mean deviation (MD) and cluster MD using retinal thickness maps from seven layers: retinal nerve fiber layer (RNFL), ganglion cell layer and inner plexiform layer (GCL + IPL), inner nuclear layer and outer plexiform layer (INL + OPL), outer nuclear layer (ONL), photoreceptors and retinal pigmented epithelium (PR + RPE), choriocapillaris and choroidal stroma (CC + CS), total retinal thickness (RT)., Results: The best performance on MD prediction is achieved by RNFL, GCL + IPL and RT layers, with R2 scores of 0.37, 0.33, and 0.31, respectively. Combining macular and peri-papillary scans outperforms single modality prediction, achieving an R2 value of 0.48. Cluster MD predictions show promising results, notably in central clusters, reaching an R2 of 0.56., Conclusions: The combination of multiple modalities, such as optic-nerve-head circular B-scans and retinal thickness maps from macular SD-OCT images, improves the performance of MD and cluster MD prediction. Our proposed model demonstrates the highest level of accuracy in predicting MD in the early-to-mid stages of glaucoma., Translational Relevance: Objective measures recorded with SD-OCT can optimize the number of visual field tests and improve individualized glaucoma care by adjusting VF testing frequency based on deep-learning estimates of functional damage.

Aim: To adopt a novel artificial intelligence (AI) optical coherence tomography (OCT)-based program to identify the presence of biomarkers associated with central serous chorioretinopathy (CSC) and whether these can differentiate between acute and chronic central serous chorioretinopathy (aCSC and cCSC)., Methods: Multicenter, observational study with a retrospective design enrolling treatment-naïve patients with aCSC and cCSC. The diagnosis of aCSC and cCSC was established with multimodal imaging and for the current study subsequent follow-up visits were also considered. Baseline OCTs were analyzed by an AI-based platform (Discovery® OCT Fluid and Biomarker Detector, RetinAI AG, Switzerland). This software allows to detect several different biomarkers in each single OCT scan, including subretinal fluid (SRF), intraretinal fluid (IRF), hyperreflective foci (HF) and flat irregular pigment epithelium detachment (FIPED). The presence of SRF was considered as a necessary inclusion criterion for performing biomarker analysis and OCT slabs without SRF presence were excluded from the analysis., Results: Overall, 160 eyes of 144 patients with CSC were enrolled, out of which 100 (62.5%) eyes were diagnosed with cCSC and 60 eyes (34.5%) with aCSC. In the OCT slabs showing presence of SRF the presence of biomarkers was found to be clinically relevant (> 50%) for HF and FIPED in aCSC and cCSC. HF had an average percentage of 81% (± 20) in the cCSC group and 81% (± 15) in the aCSC group (p = 0.4295) and FIPED had a mean percentage of 88% (± 18) in cCSC vs. 89% (± 15) in the aCSC (p = 0.3197)., Conclusion: We demonstrate that HF and FIPED are OCT biomarkers positively associated with CSC when present at baseline. While both HF and FIPED biomarkers could aid in CSC diagnosis, they could not distinguish between aCSC and cCSC at the first visit. AI-assisted biomarker detection shows promise for reducing invasive imaging needs, but further validation through longitudinal studies is needed., (© 2024. The Author(s).)

Purpose: Semantic segmentation plays a pivotal role in many applications related to medical image and video analysis. However, designing a neural network architecture for medical image and surgical video segmentation is challenging due to the diverse features of relevant classes, including heterogeneity, deformability, transparency, blunt boundaries, and various distortions. We propose a network architecture, DeepPyramid+, which addresses diverse challenges encountered in medical image and surgical video segmentation., Methods: The proposed DeepPyramid+ incorporates two major modules, namely "Pyramid View Fusion" (PVF) and "Deformable Pyramid Reception" (DPR), to address the outlined challenges. PVF replicates a deduction process within the neural network, aligning with the human visual system, thereby enhancing the representation of relative information at each pixel position. Complementarily, DPR introduces shape- and scale-adaptive feature extraction techniques using dilated deformable convolutions, enhancing accuracy and robustness in handling heterogeneous classes and deformable shapes., Results: Extensive experiments conducted on diverse datasets, including endometriosis videos, MRI images, OCT scans, and cataract and laparoscopy videos, demonstrate the effectiveness of DeepPyramid+ in handling various challenges such as shape and scale variation, reflection, and blur degradation. DeepPyramid+ demonstrates significant improvements in segmentation performance, achieving up to a 3.65% increase in Dice coefficient for intra-domain segmentation and up to a 17% increase in Dice coefficient for cross-domain segmentation., Conclusions: DeepPyramid+ consistently outperforms state-of-the-art networks across diverse modalities considering different backbone networks, showcasing its versatility. Accordingly, DeepPyramid+ emerges as a robust and effective solution, successfully overcoming the intricate challenges associated with relevant content segmentation in medical images and surgical videos. Its consistent performance and adaptability indicate its potential to enhance precision in computerized medical image and surgical video analysis applications., (© 2024. The Author(s).)

Malaria, a significant global health challenge, is caused by Plasmodium parasites. The Plasmodium liver stage plays a pivotal role in the establishment of the infection. This study focuses on the liver stage development of the model organism Plasmodium berghei, employing fluorescent microscopy imaging and convolutional neural networks (CNNs) for analysis. Convolutional neural networks have been recently proposed as a viable option for tasks such as malaria detection, prediction of host-pathogen interactions, or drug discovery. Our research aimed to predict the transition of Plasmodium-infected liver cells to the merozoite stage, a key development phase, 15 hours in advance. We collected and analyzed hourly imaging data over a span of at least 38 hours from 400 sequences, encompassing 502 parasites. Our method was compared to human annotations to validate its efficacy. Performance metrics, including the area under the receiver operating characteristic curve (AUC), sensitivity, and specificity, were evaluated on an independent test dataset. The outcomes revealed an AUC of 0.873, a sensitivity of 84.6%, and a specificity of 83.3%, underscoring the potential of our CNN-based framework to predict liver stage development of P. berghei . These findings not only demonstrate the feasibility of our methodology but also could potentially contribute to the broader understanding of parasite biology., Competing Interests: The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (© 2024 The Author(s).)

In recent years, the landscape of computer-assisted interventions and post-operative surgical video analysis has been dramatically reshaped by deep-learning techniques, resulting in significant advancements in surgeons' skills, operation room management, and overall surgical outcomes. However, the progression of deep-learning-powered surgical technologies is profoundly reliant on large-scale datasets and annotations. In particular, surgical scene understanding and phase recognition stand as pivotal pillars within the realm of computer-assisted surgery and post-operative assessment of cataract surgery videos. In this context, we present the largest cataract surgery video dataset that addresses diverse requisites for constructing computerized surgical workflow analysis and detecting post-operative irregularities in cataract surgery. We validate the quality of annotations by benchmarking the performance of several state-of-the-art neural network architectures for phase recognition and surgical scene segmentation. Besides, we initiate the research on domain adaptation for instrument segmentation in cataract surgery by evaluating cross-domain instrument segmentation performance in cataract surgery videos. The dataset and annotations are publicly available in Synapse., (© 2024. The Author(s).)

Purpose: The purpose of this study was to develop a deep learning algorithm, to detect retinal breaks and retinal detachments on ultra-widefield fundus (UWF) optos images using artificial intelligence (AI)., Methods: Optomap UWF images of the database were annotated to four groups by two retina specialists: (1) retinal breaks without detachment, (2) retinal breaks with retinal detachment, (3) retinal detachment without visible retinal breaks, and (4) a combination of groups 1 to 3. The fundus image data set was split into a training set and an independent test set following an 80% to 20% ratio. Image preprocessing methods were applied. An EfficientNet classification model was trained with the training set and evaluated with the test set., Results: A total of 2489 UWF images were included into the dataset, resulting in a training set size of 2008 UWF images and a test set size of 481 images. The classification models achieved an area under the receiver operating characteristic curve (AUC) on the testing set of 0.975 regarding lesion detection, an AUC of 0.972 for retinal detachment and an AUC of 0.913 for retinal breaks., Conclusions: A deep learning system to detect retinal breaks and retinal detachment using UWF images is feasible and has a good specificity. This is relevant for clinical routine as there can be a high rate of missed breaks in clinics. Future clinical studies will be necessary to evaluate the cost-effectiveness of applying such an algorithm as an automated auxiliary tool in a large practices or tertiary referral centers., Translational Relevance: This study demonstrates the relevance of applying AI in diagnosing peripheral retinal breaks in clinical routine in UWF fundus images.

Purpose: To identify optical coherence tomography (OCT) features to predict the course of central serous chorioretinopathy (CSC) with an artificial intelligence-based program., Methods: Multicenter, observational study with a retrospective design. Treatment-naïve patients with acute CSC and chronic CSC were enrolled. Baseline OCTs were examined by an artificial intelligence-developed platform (Discovery OCT Fluid and Biomarker Detector, RetinAI AG, Switzerland). Through this platform, automated retinal layer thicknesses and volumes, including intaretinal and subretinal fluid, and pigment epithelium detachment were measured. Baseline OCT features were compared between acute CSC and chronic CSC patients., Results: One hundred and sixty eyes of 144 patients with CSC were enrolled, of which 100 had chronic CSC and 60 acute CSC. Retinal layer analysis of baseline OCT scans showed that the inner nuclear layer, the outer nuclear layer, and the photoreceptor-retinal pigmented epithelium complex were significantly thicker at baseline in eyes with acute CSC in comparison with those with chronic CSC ( P < 0.001). Similarly, choriocapillaris and choroidal stroma and retinal thickness (RT) were thicker in acute CSC than chronic CSC eyes ( P = 0.001). Volume analysis revealed average greater subretinal fluid volumes in the acute CSC group in comparison with chronic CSC ( P = 0.041)., Conclusion: Optical coherence tomography features may be helpful to predict the clinical course of CSC. The baseline presence of an increased thickness in the outer retinal layers, choriocapillaris and choroidal stroma, and subretinal fluid volume seems to be associated with acute course of the disease.

Recent developments in deep learning have shown success in accurately predicting the location of biological markers in Optical Coherence Tomography (OCT) volumes of patients with Age-Related Macular Degeneration (AMD) and Diabetic Retinopathy (DR). We propose a method that automatically locates biological markers to the Early Treatment Diabetic Retinopathy Study (ETDRS) rings, only requiring B-scan-level presence annotations. We trained a neural network using 22,723 OCT B-Scans of 460 eyes (433 patients) with AMD and DR, annotated with slice-level labels for Intraretinal Fluid (IRF) and Subretinal Fluid (SRF). The neural network outputs were mapped into the corresponding ETDRS rings. We incorporated the class annotations and domain knowledge into a loss function to constrain the output with biologically plausible solutions. The method was tested on a set of OCT volumes with 322 eyes (189 patients) with Diabetic Macular Edema, with slice-level SRF and IRF presence annotations for the ETDRS rings. Our method accurately predicted the presence of IRF and SRF in each ETDRS ring, outperforming previous baselines even in the most challenging scenarios. Our model was also successfully applied to en-face marker segmentation and showed consistency within C-scans, despite not incorporating volume information in the training process. We achieved a correlation coefficient of 0.946 for the prediction of the IRF area., (© 2023. The Author(s).)

Purpose: To evaluate the effectiveness of automated liver segmental volume quantification and calculation of the liver segmental volume ratio (LSVR) on a non-contrast T1-vibe Dixon liver MRI sequence using a deep learning segmentation pipeline., Method: A dataset of 200 liver MRI with a non-contrast 3 mm T1-vibe Dixon sequence was manually labeledslice-by-sliceby an expert for Couinaud liver segments, while portal and hepatic veins were labeled separately. A convolutional neural networkwas trainedusing 170 liver MRI for training and 30 for evaluation. Liver segmental volumes without liver vessels were retrieved and LSVR was calculated as the liver segmental volumes I-III divided by the liver segmental volumes IV-VIII. LSVR was compared with the expert manual LSVR calculation and the LSVR calculated on CT scans in 30 patients with CT and MRI within 6 months., Results: Theconvolutional neural networkclassified the Couinaud segments I-VIII with an average Dice score of 0.770 ± 0.03, ranging between 0.726 ± 0.13 (segment IVb) and 0.810 ± 0.09 (segment V). The calculated mean LSVR with liver MRI unseen by the model was 0.32 ± 0.14, as compared with manually quantified LSVR of 0.33 ± 0.15, resulting in a mean absolute error (MAE) of 0.02. A comparable LSVR of 0.35 ± 0.14 with a MAE of 0.04 resulted with the LSRV retrieved from the CT scans. The automated LSVR showed significant correlation with the manual MRI LSVR (Spearman r = 0.97, p < 0.001) and CT LSVR (Spearman r = 0.95, p < 0.001)., Conclusions: A convolutional neural network allowed for accurate automated liver segmental volume quantification and calculation of LSVR based on a non-contrast T1-vibe Dixon sequence., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2023 The Authors. Published by Elsevier B.V. All rights reserved.)

Polarimetry is an optical characterization technique capable of analyzing the polarization state of light reflected by materials and biological samples. In this study, we investigate the potential of Müller matrix polarimetry (MMP) to analyze fresh pancreatic tissue samples. Due to its highly heterogeneous appearance, pancreatic tissue type differentiation is a complex task. Furthermore, its challenging location in the body makes creating direct imaging difficult. However, accurate and reliable methods for diagnosing pancreatic diseases are critical for improving patient outcomes. To this end, we measured the Müller matrices of ex-vivo unfixed human pancreatic tissue and leverage the feature-learning capabilities of a machine-learning model to derive an optimized data representation that minimizes normal-abnormal classification error. We show experimentally that our approach accurately differentiates between normal and abnormal pancreatic tissue. This is, to our knowledge, the first study to use ex-vivo unfixed human pancreatic tissue combined with feature-learning from raw Müller matrix readings for this purpose., (© 2023. Springer Nature Limited.)

Purpose: Surgical scene understanding plays a critical role in the technology stack of tomorrow's intervention-assisting systems in endoscopic surgeries. For this, tracking the endoscope pose is a key component, but remains challenging due to illumination conditions, deforming tissues and the breathing motion of organs., Method: We propose a solution for stereo endoscopes that estimates depth and optical flow to minimize two geometric losses for camera pose estimation. Most importantly, we introduce two learned adaptive per-pixel weight mappings that balance contributions according to the input image content. To do so, we train a Deep Declarative Network to take advantage of the expressiveness of deep learning and the robustness of a novel geometric-based optimization approach. We validate our approach on the publicly available SCARED dataset and introduce a new in vivo dataset, StereoMIS, which includes a wider spectrum of typically observed surgical settings., Results: Our method outperforms state-of-the-art methods on average and more importantly, in difficult scenarios where tissue deformations and breathing motion are visible. We observed that our proposed weight mappings attenuate the contribution of pixels on ambiguous regions of the images, such as deforming tissues., Conclusion: We demonstrate the effectiveness of our solution to robustly estimate the camera pose in challenging endoscopic surgical scenes. Our contributions can be used to improve related tasks like simultaneous localization and mapping (SLAM) or 3D reconstruction, therefore advancing surgical scene understanding in minimally invasive surgery., (© 2023. The Author(s).)

Recent advances in machine learning models have greatly increased the performance of automated methods in medical image analysis. However, the internal functioning of such models is largely hidden, which hinders their integration in clinical practice. Explainability and trust are viewed as important aspects of modern methods, for the latter's widespread use in clinical communities. As such, validation of machine learning models represents an important aspect and yet, most methods are only validated in a limited way. In this work, we focus on providing a richer and more appropriate validation approach for highly powerful Visual Question Answering (VQA) algorithms. To better understand the performance of these methods, which answer arbitrary questions related to images, this work focuses on an automatic visual Turing test (VTT). That is, we propose an automatic adaptive questioning method, that aims to expose the reasoning behavior of a VQA algorithm. Specifically, we introduce a reinforcement learning (RL) agent that observes the history of previously asked questions, and uses it to select the next question to pose. We demonstrate our approach in the context of evaluating algorithms that automatically answer questions related to diabetic macular edema (DME) grading. The experiments show that such an agent has similar behavior to a clinician, whereby asking questions that are relevant to key clinical concepts., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2023. Published by Elsevier B.V.)

Purpose: A fundamental problem in designing safe machine learning systems is identifying when samples presented to a deployed model differ from those observed at training time. Detecting so-called out-of-distribution (OoD) samples is crucial in safety-critical applications such as robotically guided retinal microsurgery, where distances between the instrument and the retina are derived from sequences of 1D images that are acquired by an instrument-integrated optical coherence tomography (iiOCT) probe., Methods: This work investigates the feasibility of using an OoD detector to identify when images from the iiOCT probe are inappropriate for subsequent machine learning-based distance estimation. We show how a simple OoD detector based on the Mahalanobis distance can successfully reject corrupted samples coming from real-world ex vivo porcine eyes., Results: Our results demonstrate that the proposed approach can successfully detect OoD samples and help maintain the performance of the downstream task within reasonable levels. MahaAD outperformed a supervised approach trained on the same kind of corruptions and achieved the best performance in detecting OoD cases from a collection of iiOCT samples with real-world corruptions., Conclusion: The results indicate that detecting corrupted iiOCT data through OoD detection is feasible and does not need prior knowledge of possible corruptions. Consequently, MahaAD could aid in ensuring patient safety during robotically guided microsurgery by preventing deployed prediction models from estimating distances that put the patient at risk., (© 2023. The Author(s).)

Medical imaging has been intensively employed in screening, diagnosis and monitoring during the COVID-19 pandemic. With the improvement of RT-PCR and rapid inspection technologies, the diagnostic references have shifted. Current recommendations tend to limit the application of medical imaging in the acute setting. Nevertheless, efficient and complementary values of medical imaging have been recognized at the beginning of the pandemic when facing unknown infectious diseases and a lack of sufficient diagnostic tools. Optimizing medical imaging for pandemics may still have encouraging implications for future public health, especially for long-lasting post-COVID-19 syndrome theranostics. A critical concern for the application of medical imaging is the increased radiation burden, particularly when medical imaging is used for screening and rapid containment purposes. Emerging artificial intelligence (AI) technology provides the opportunity to reduce the radiation burden while maintaining diagnostic quality. This review summarizes the current AI research on dose reduction for medical imaging, and the retrospective identification of their potential in COVID-19 may still have positive implications for future public health., Competing Interests: Conflict of interestThe authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (© The Author(s) 2023.)

We evaluated the effectiveness of automated segmentation of the liver and its vessels with a convolutional neural network on non-contrast T1 vibe Dixon acquisitions. A dataset of non-contrast T1 vibe Dixon liver magnetic resonance images was labelled slice-by-slice for the outer liver border, portal, and hepatic veins by an expert. A 3D U-Net convolutional neural network was trained with different combinations of Dixon in-phase, opposed-phase, water, and fat reconstructions. The neural network trained with the single-modal in-phase reconstructions achieved a high performance for liver parenchyma (Dice 0.936 ± 0.02), portal veins (0.634 ± 0.09), and hepatic veins (0.532 ± 0.12) segmentation. No benefit of using multi-modal input was observed (p = 1.0 for all experiments), combining in-phase, opposed-phase, fat, and water reconstruction. Accuracy for differentiation between portal and hepatic veins was 99% for portal veins and 97% for hepatic veins in the central region and slightly lower in the peripheral region (91% for portal veins, 80% for hepatic veins). In conclusion, deep learning-based automated segmentation of the liver and its vessels on non-contrast T1 vibe Dixon was highly effective. The single-modal in-phase input achieved the best performance in segmentation and differentiation between portal and hepatic veins., (© 2022. The Author(s).)

Background: Diagnosing heparin-induced thrombocytopenia (HIT) at the bedside remains challenging, exposing a significant number of patients at risk of delayed diagnosis or overtreatment. We hypothesized that machine-learning algorithms could be utilized to develop a more accurate and user-friendly diagnostic tool that integrates diverse clinical and laboratory information and accounts for complex interactions., Methods: We conducted a prospective cohort study including 1393 patients with suspected HIT between 2018 and 2021 from 10 study centers. Detailed clinical information and laboratory data were collected, and various immunoassays were conducted. The washed platelet heparin-induced platelet activation assay (HIPA) served as the reference standard., Findings: HIPA diagnosed HIT in 119 patients (prevalence 8.5%). The feature selection process in the training dataset (75% of patients) yielded the following predictor variables: (1) immunoassay test result, (2) platelet nadir, (3) unfractionated heparin use, (4) CRP, (5) timing of thrombocytopenia, and (6) other causes of thrombocytopenia. The best performing models were a support vector machine in case of the chemiluminescent immunoassay (CLIA) and the ELISA, as well as a gradient boosting machine in particle-gel immunoassay (PaGIA). In the validation dataset (25% of patients), the AUROC of all models was 0.99 (95% CI: 0.97, 1.00). Compared to the currently recommended diagnostic algorithm (4Ts score, immunoassay), the numbers of false-negative patients were reduced from 12 to 6 (-50.0%; ELISA), 9 to 3 (-66.7%, PaGIA) and 14 to 5 (-64.3%; CLIA). The numbers of false-positive individuals were reduced from 87 to 61 (-29.8%; ELISA), 200 to 63 (-68.5%; PaGIA) and increased from 50 to 63 (+29.0%) for the CLIA., Interpretation: Our user-friendly machine-learning algorithm for the diagnosis of HIT (https://toradi-hit.org) was substantially more accurate than the currently recommended diagnostic algorithm. It has the potential to reduce delayed diagnosis and overtreatment in clinical practice. Future studies shall validate this model in wider settings., Funding: Swiss National Science Foundation (SNSF), and International Society on Thrombosis and Haemostasis (ISTH)., Competing Interests: AC has served as a consultant for Synergy; has received authorship royalties for UpToDate; and his institution has received research support on his behalf from Alexion, Bayer, Novartis, Novo Nordisk, Pfizer, Sanofi, Spark, and Takeda. The institution of BG received grant support and CME support from Pfizer, Thermo Fisher Scientific, Axonlab, Sanofi, Alnylam, Bayer, BMS, Daiichi-Sankyo, Octapharma, Takeda, SOBI, Janssen, Novo Nordisk, Mitsubishi Taneba, outside of the current work. The institution of JKH received grant support, consultancy fees, or honoraria from SNSF, Baxter/Takeda, Bayer, CSL-Behring, NovoNordisk, Octapharma, Roche, SOBI, Roche, Sanofi, FOPH, and Swiss Hemophilia Society, outside of the current work. MN received research grants from Bayer Healthcare, Roche diagnostics, Siemens healthineers, Pentapharm, and Bühlmann laboratories, outside of the current work. Dr. Greinacher reports personal fees from Aspen, grants from Ergomed, grants from Boehringer Ingelheim, personal fees from Bayer Vital, grants from Rovi, grants from Sagent, personal fees from Chromatec, personal fees from Instrumentation Laboratory, grants and personal fees from Macopharma, grants from Portola, grants from Biokit, personal fees from Sanofi-Aventis, grants from Fa. Blau Farmaceutics, grants from Prosensa/Biomarin, grants and other from DRK-BSD NSTOB, grants from DRK-BSD Baden-Würtemberg/Hessen, personal fees from Roche, personal fees from GTH e.V., grants from Deutsche Forschungsgemeinschaft, grants from Deutsche Forschungsgemeinschaft, grants from Deutsche Forschungsgemeinschaft, grants from Robert-Koch-Institut, non-financial support from Veralox, grants from Dilaflor, non-financial support from Vakzine Projekt Management GmbH, grants from GIZ Else-Körner-Stiftung, grants from GIZ Else-Körner-Stiftung, non-financial support from AstraZeneca, non-financial support from Janssen Vaccines & Prevention B.V., personal fees from Takeda Pharma, personal fees from Falk Foundation e.V., grants from European Medicines Agency, outside the submitted work; In addition, AG has a patent Screening Methods for transfusion related acute lung injury (TRALI) with royalties paid to EP2321644, 18.05.2011. TM reports grant support, consultancy fees, honoraria, or support for attending meetings from DFG, Stiftung Transfusionsmedizin und Immunhämatologie e.V, DRK Blutspendedienst, Deutsche Herzstiftung, Ministerium für Wissenschaft, Forschung und Kunst Baden Würtemberg, Gesellschaft für Thrombose-und Hämostaseforschung, Berufsverband Deutscher Internisten, CoaChrom Diagnostica GmbH, Robert Bosch GmbH, Ergomed, Bayer, Bristol-Myers Squibb, Doctrina Med AG, Leo Pharma GmbH, Schöchl medical education GmbH, Mitsubishi Tanabe GmbH, Novo Nordisk GmbH, Swedish Orphan Biovitrium GmbH. All other authors declare that no conflict of interest exists., (© 2022 The Author(s).)

Despite the potential of deep learning (DL)-based methods in substituting CT-based PET attenuation and scatter correction for CT-free PET imaging, a critical bottleneck is their limited capability in handling large heterogeneity of tracers and scanners of PET imaging. This study employs a simple way to integrate domain knowledge in DL for CT-free PET imaging. In contrast to conventional direct DL methods, we simplify the complex problem by a domain decomposition so that the learning of anatomy-dependent attenuation correction can be achieved robustly in a low-frequency domain while the original anatomy-independent high-frequency texture can be preserved during the processing. Even with the training from one tracer on one scanner, the effectiveness and robustness of our proposed approach are confirmed in tests of various external imaging tracers on different scanners. The robust, generalizable, and transparent DL development may enhance the potential of clinical translation., (© 2022. The Author(s).)

Myocarditis represents the entity of an inflamed myocardium and is a diagnostic challenge caused by its heterogeneous presentation. Contemporary noninvasive evaluation of patients with clinically suspected myocarditis using cardiac magnetic resonance (CMR) includes dimensions and function of the heart chambers, conventional T2-weighted imaging, late gadolinium enhancement, novel T1 and T2 mapping, and extracellular volume fraction calculation. CMR feature-tracking, texture analysis, and artificial intelligence emerge as potential modern techniques to further improve diagnosis and prognostication in this clinical setting. This review describes the evidence surrounding different CMR methods and image postprocessing methods and highlights their values for clinical decision making, monitoring, and risk stratification across stages of this condition., Competing Interests: Funding Support and Author Disclosures Dr Bucciarelli-Ducci is the CEO (part-time) of the Society for Cardiovascular Magnetic Resonance. Dr Sznitman is a shareholder and has served as a scientific advisor of RetinAI Medical (Switzerland). Dr Kwong has received research support from Bristol Myers Squibb, Alnylam Pharmaceuticals, Epirium Bio, and Bayer AG. Dr Gräni has received funding support from the Swiss National Foundation and InnoSuisse. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose., (Copyright © 2022 American College of Cardiology Foundation. Published by Elsevier Inc. All rights reserved.)

Purpose: Alzheimer's disease (AD) studies revealed that abnormal deposition of tau spreads in a specific spatial pattern, namely Braak stage. However, Braak staging is based on post mortem brains, each of which represents the cross section of the tau trajectory in disease progression, and numerous studies were reported that do not conform to that model. This study thus aimed to identify the tau trajectory and quantify the tau progression in a data-driven approach with the continuous latent space learned by variational autoencoder (VAE)., Methods: A total of 1080 [ 18 F]Flortaucipir brain positron emission tomography (PET) images were collected from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database. VAE was built to compress the hidden features from tau images in latent space. Hierarchical agglomerative clustering and minimum spanning tree (MST) were applied to organize the features and calibrate them to the tau progression, thus deriving pseudo-time. The image-level tau trajectory was inferred by continuously sampling across the calibrated latent features. We assessed the pseudo-time with regard to tau standardized uptake value ratio (SUVr) in AD-vulnerable regions, amyloid deposit, glucose metabolism, cognitive scores, and clinical diagnosis., Results: We identified four clusters that plausibly capture certain stages of AD and organized the clusters in the latent space. The inferred tau trajectory agreed with the Braak staging. According to the derived pseudo-time, tau first deposits in the parahippocampal and amygdala, and then spreads to the fusiform, inferior temporal lobe, and posterior cingulate. Prior to the regional tau deposition, amyloid accumulates first., Conclusion: The spatiotemporal trajectory of tau progression inferred in this study was consistent with Braak staging. The profile of other biomarkers in disease progression agreed well with previous findings. We addressed that this approach additionally has the potential to quantify tau progression as a continuous variable by taking a whole-brain tau image into account., (© 2022. The Author(s).)

Purpose: A critical bottleneck for the credibility of artificial intelligence (AI) is replicating the results in the diversity of clinical practice. We aimed to develop an AI that can be independently applied to recover high-quality imaging from low-dose scans on different scanners and tracers., Methods: Brain [ 18 F]FDG PET imaging of 237 patients scanned with one scanner was used for the development of AI technology. The developed algorithm was then tested on [ 18 F]FDG PET images of 45 patients scanned with three different scanners, [ 18 F]FET PET images of 18 patients scanned with two different scanners, as well as [ 18 F]Florbetapir images of 10 patients. A conditional generative adversarial network (GAN) was customized for cross-scanner and cross-tracer optimization. Three nuclear medicine physicians independently assessed the utility of the results in a clinical setting., Results: The improvement achieved by AI recovery significantly correlated with the baseline image quality indicated by structural similarity index measurement (SSIM) (r = -0.71, p < 0.05) and normalized dose acquisition (r = -0.60, p < 0.05). Our cross-scanner and cross-tracer AI methodology showed utility based on both physical and clinical image assessment (p < 0.05)., Conclusion: The deep learning development for extensible application on unknown scanners and tracers may improve the trustworthiness and clinical acceptability of AI-based dose reduction., (© 2021. The Author(s).)

Recent developments in data science in general and machine learning in particular have transformed the way experts envision the future of surgery. Surgical Data Science (SDS) is a new research field that aims to improve the quality of interventional healthcare through the capture, organization, analysis and modeling of data. While an increasing number of data-driven approaches and clinical applications have been studied in the fields of radiological and clinical data science, translational success stories are still lacking in surgery. In this publication, we shed light on the underlying reasons and provide a roadmap for future advances in the field. Based on an international workshop involving leading researchers in the field of SDS, we review current practice, key achievements and initiatives as well as available standards and tools for a number of topics relevant to the field, namely (1) infrastructure for data acquisition, storage and access in the presence of regulatory constraints, (2) data annotation and sharing and (3) data analytics. We further complement this technical perspective with (4) a review of currently available SDS products and the translational progress from academia and (5) a roadmap for faster clinical translation and exploitation of the full potential of SDS, based on an international multi-round Delphi process., Competing Interests: Declaration of Competing Interest Anand Malpani is a future employee at Mimic Technologies Inc. (Seattle, WA, US). Johannes Fallert and Lars Mündermann are employed at KARL STORZ SE & Co. KG (Tuttlingen, Germany). Hirenkumar Nakawala is employed at CMR Surgical Ltd (Cambridge, UK). Nicolas Padoy is a scientific advisor of Caresyntax (Berlin, Germany). Daniel A. Hashimoto is a consultant for Johnson & Johnson (New Brunswick, NJ, USA), Verily Life Sciences (San Francisco, CA, USA), and Activ Surgical (Boston, MA, USA). He has received research support from Olympus Corporation and the Intuitive Foundation. Carla Pugh is the founder of 10 Newtons Inc. (Madison, WI, US). Danail Stoyanov is employed at Digital Surgery Ltd (London, UK) and Odin Vision Ltd (London, UK). Teodor Grantcharov is the founder of Surgical Safety Technologies Inc. (Toronto, Ontario, Canada). Tobias Roß is employed at Quality Match GmbH (Heidelberg, Germany). All other authors do not declare any conflicts of interest., (Copyright © 2021. Published by Elsevier B.V.)

Introduction: In this retrospective cohort study, we wanted to evaluate the performance and analyze the insights of an artificial intelligence (AI) algorithm in detecting retinal fluid in spectral-domain OCT volume scans from a large cohort of patients with neovascular age-related macular degeneration (AMD) and diabetic macular edema (DME)., Methods: A total of 3,981 OCT volumes from 374 patients with AMD and 11,501 OCT volumes from 811 patients with DME were acquired with Heidelberg-Spectralis OCT device (Heidelberg Engineering Inc., Heidelberg, Germany) between 2013 and 2021. Each OCT volume was annotated for the presence or absence of intraretinal fluid (IRF) and subretinal fluid (SRF) by masked reading center graders (ground truth). The performance of an already published AI algorithm to detect IRF and SRF separately, and a combined fluid detector (IRF and/or SRF) of the same OCT volumes was evaluated. An analysis of the sources of disagreement between annotation and prediction and their relationship to central retinal thickness was performed. We computed the mean areas under the curves (AUC) and under the precision-recall curves (AP), accuracy, sensitivity, specificity, and precision., Results: The AUC for IRF was 0.92 and 0.98, for SRF 0.98 and 0.99, in the AMD and DME cohort, respectively. The AP for IRF was 0.89 and 1.00, for SRF 0.97 and 0.93, in the AMD and DME cohort, respectively. The accuracy, specificity, and sensitivity for IRF were 0.87, 0.88, 0.84, and 0.93, 0.95, 0.93, and for SRF 0.93, 0.93, 0.93, and 0.95, 0.95, 0.95 in the AMD and DME cohort, respectively. For detecting any fluid, the AUC was 0.95 and 0.98, and the accuracy, specificity, and sensitivity were 0.89, 0.93, and 0.90 and 0.95, 0.88, and 0.93, in the AMD and DME cohort, respectively. False positives were present when retinal shadow artifacts and strong retinal deformation were present. False negatives were due to small hyporeflective areas in combination with poor image quality. The combined detector correctly predicted more OCT volumes than the single detectors for IRF and SRF, 89.0% versus 81.6% in the AMD and 93.1% versus 88.6% in the DME cohort., Discussion/conclusion: The AI-based fluid detector achieves high performance for retinal fluid detection in a very large dataset dedicated to AMD and DME. Combining single detectors provides better fluid detection accuracy than considering the single detectors separately. The observed independence of the single detectors ensures that the detectors learned features particular to IRF and SRF., (© 2022 The Author(s). Published by S. Karger AG, Basel.)

Purpose: Clinically evaluate the noninferiority of a custom virtual reality (VR) perimetry system when compared to a clinically and routinely used perimeter on both healthy subjects and glaucoma patients., Methods: We use a custom-designed VR perimetry system tailored for visual field testing. The system uses Oculus Quest VR headset (Facebook Technologies, LLC, Bern, Switzerland), that includes a clicker for participant response feedback. A prospective, single center, study was conducted at the Department of Ophthalmology of the Bern University Hospital (Bern, Switzerland) for 12 months. Of the 114 participants recruited 70 subjects (36 healthy and 34 glaucoma patients with early to moderate visual field loss) were included in the study. Participants underwent perimetry tests on an Octopus 900 (Haag-Streit, Köniz, Switzerland) as well as on the custom VR perimeter. In both cases, standard dynamic strategy (DS) was used in conjunction with the G testing pattern. Collected visual fields (VFs) from both devices were then analyzed and compared., Results: High mean defect (MD) correlations between the two systems (Spearman, ρ ≥ 0.75) were obtained. The VR system was found to slightly underestimate VF defects in glaucoma subjects (1.4 dB). No significant bias was found with respect to eccentricity or subject age. On average, a similar number of stimuli presentations per VF was necessary when measuring glaucoma patients and healthy subjects., Conclusions: This study demonstrates that a clinically used perimeter and the proposed VR perimetry system have comparable performances with respect to a number of perimetry parameters in healthy and glaucoma patients with early to moderate visual field loss., Translational Relevance: This suggests that VR perimeters have the potential to assess VFs with high enough confidence, whereby alleviating challenges in current perimetry practices by providing a portable and more accessible visual field test.

The ability to quickly annotate medical imaging data plays a critical role in training deep learning frameworks for segmentation. Doing so for image volumes or video sequences is even more pressing as annotating these is particularly burdensome. To alleviate this problem, this work proposes a new method to efficiently segment medical imaging volumes or videos using point-wise annotations only. This allows annotations to be collected extremely quickly and remains applicable to numerous segmentation tasks. Our approach trains a deep learning model using an appropriate Positive/Unlabeled objective function using sparse point-wise annotations. While most methods of this kind assume that the proportion of positive samples in the data is known a-priori, we introduce a novel self-supervised method to estimate this prior efficiently by combining a Bayesian estimation framework and new stopping criteria. Our method iteratively estimates appropriate class priors and yields high segmentation quality for a variety of object types and imaging modalities. In addition, by leveraging a spatio-temporal tracking framework, we regularize our predictions by leveraging the complete data volume. We show experimentally that our approach outperforms state-of-the-art methods tailored to the same problem., Competing Interests: Declaration of Competing Interest Authors declare that they have no conflict of interest., (Copyright © 2021. Published by Elsevier B.V.)

Recently, Bayesian brain-based models emerged as a possible composite of existing theories, providing an universal explanation of tinnitus phenomena. Yet, the involvement of multiple synergistic mechanisms complicates the identification of behavioral and physiological evidence. To overcome this, an empirically tested computational model could support the evaluation of theoretical hypotheses by intrinsically encompassing different mechanisms. The aim of this work was to develop a generative computational tinnitus perception model based on the Bayesian brain concept. The behavioral responses of 46 tinnitus subjects who underwent ten consecutive residual inhibition assessments were used for model fitting. Our model was able to replicate the behavioral responses during residual inhibition in our cohort (median linear correlation coefficient of 0.79). Using the same model, we simulated two additional tinnitus phenomena: residual excitation and occurrence of tinnitus in non-tinnitus subjects after sensory deprivation. In the simulations, the trajectories of the model were consistent with previously obtained behavioral and physiological observations. Our work introduces generative computational modeling to the research field of tinnitus. It has the potential to quantitatively link experimental observations to theoretical hypotheses and to support the search for neural signatures of tinnitus by finding correlates between the latent variables of the model and measured physiological data., (Copyright © 2021 The Author(s). Published by Elsevier B.V. All rights reserved.)

Purpose: The detection and segmentation of surgical instruments has been a vital step for many applications in minimally invasive surgical robotics. Previously, the problem was tackled from a semantic segmentation perspective, yet these methods fail to provide good segmentation maps of instrument types and do not contain any information on the instance affiliation of each pixel. We propose to overcome this limitation by using a novel instance segmentation method which first masks instruments and then classifies them into their respective type., Methods: We introduce a novel method for instance segmentation where a pixel-wise mask of each instance is found prior to classification. An encoder-decoder network is used to extract instrument instances, which are then separately classified using the features of the previous stages. Furthermore, we present a method to incorporate instrument priors from surgical robots., Results: Experiments are performed on the robotic instrument segmentation dataset of the 2017 endoscopic vision challenge. We perform a fourfold cross-validation and show an improvement of over 18% to the previous state-of-the-art. Furthermore, we perform an ablation study which highlights the importance of certain design choices and observe an increase of 10% over semantic segmentation methods., Conclusions: We have presented a novel instance segmentation method for surgical instruments which outperforms previous semantic segmentation-based methods. Our method further provides a more informative output of instance level information, while retaining a precise segmentation mask. Finally, we have shown that robotic instrument priors can be used to further increase the performance.

Purpose: To assess the potential of machine learning to predict low and high treatment demand in real life in patients with neovascular age-related macular degeneration (nAMD), retinal vein occlusion (RVO), and diabetic macular edema (DME) treated according to a treat-and-extend regimen (TER)., Design: Retrospective cohort study., Participants: Three hundred seventy-seven eyes (340 patients) with nAMD and 333 eyes (285 patients) with RVO or DME treated with anti-vascular endothelial growth factor agents (VEGF) according to a predefined TER from 2014 through 2018., Methods: Eyes were grouped by disease into low, moderate, and high treatment demands, defined by the average treatment interval (low, ≥10 weeks; high, ≤5 weeks; moderate, remaining eyes). Two random forest models were trained to predict the probability of the long-term treatment demand of a new patient. Both models use morphological features automatically extracted from the OCT volumes at baseline and after 2 consecutive visits, as well as patient demographic information. Evaluation of the models included a 10-fold cross-validation ensuring that no patient was present in both the training set (nAMD, approximately 339; RVO and DME, approximately 300) and test set (nAMD, approximately 38; RVO and DME, approximately 33)., Main Outcome Measures: Mean area under the receiver operating characteristic curve (AUC) of both models; contribution to the prediction and statistical significance of the input features., Results: Based on the first 3 visits, it was possible to predict low and high treatment demand in nAMD eyes and in RVO and DME eyes with similar accuracy. The distribution of low, high, and moderate demanders was 127, 42, and 208, respectively, for nAMD and 61, 50, and 222, respectively, for RVO and DME. The nAMD-trained models yielded mean AUCs of 0.79 and 0.79 over the 10-fold crossovers for low and high demand, respectively. Models for RVO and DME showed similar results, with a mean AUC of 0.76 and 0.78 for low and high demand, respectively. Even more importantly, this study revealed that it is possible to predict low demand reasonably well at the first visit, before the first injection., Conclusions: Machine learning classifiers can predict treatment demand and may assist in establishing patient-specific treatment plans in the near future., (Copyright © 2021 American Academy of Ophthalmology. Published by Elsevier Inc. All rights reserved.)

In this paper we analyse the performance of machine learning methods in predicting patient information such as age or sex solely from retinal imaging modalities in a heterogeneous clinical population. Our dataset consists of N = 135,667 fundus images and N = 85,536 volumetric OCT scans. Deep learning models were trained to predict the patient's age and sex from fundus images, OCT cross sections and OCT volumes. For sex prediction, a ROC AUC of 0.80 was achieved for fundus images, 0.84 for OCT cross sections and 0.90 for OCT volumes. Age prediction mean absolute errors of 6.328 years for fundus, 5.625 years for OCT cross sections and 4.541 for OCT volumes were observed. We assess the performance of OCT scans containing different biomarkers and note a peak performance of AUC = 0.88 for OCT cross sections and 0.95 for volumes when there is no pathology on scans. Performance drops in case of drusen, fibrovascular pigment epitheliuum detachment and geographic atrophy present. We conclude that deep learning based methods are capable of classifying the patient's sex and age from color fundus photography and OCT for a broad spectrum of patients irrespective of underlying disease or image quality. Non-random sex prediction using fundus images seems only possible if the eye fovea and optic disc are visible.

Current studies of cell signaling dynamics that use live cell fluorescent biosensors routinely yield thousands of single-cell, heterogeneous, multi-dimensional trajectories. Typically, the extraction of relevant information from time series data relies on predefined, human-interpretable features. Without a priori knowledge of the system, the predefined features may fail to cover the entire spectrum of dynamics. Here we present CODEX, a data-driven approach based on convolutional neural networks (CNNs) that identifies patterns in time series. It does not require a priori information about the biological system and the insights into the data are built through explanations of the CNNs' predictions. CODEX provides several views of the data: visualization of all the single-cell trajectories in a low-dimensional space, identification of prototypic trajectories, and extraction of distinctive motifs. We demonstrate how CODEX can provide new insights into ERK and Akt signaling in response to various growth factors, and we recapitulate findings in p53 and TGFβ-SMAD2 signaling., (©2021 The Authors. Published under the terms of the CC BY 4.0 license.)

Background: In thermal ablation of liver tumors, complete coverage of the tumor volume by the ablation volume with a sufficient ablation margin is the most important factor for treatment success. Evaluation of ablation completeness is commonly performed by visual inspection in 2D and is prone to inter-reader variability. This work aimed to introduce a standardized approach for evaluation of ablation completeness after CT-guided thermal ablation of liver tumors, using volumetric quantitative ablation margins (QAM)., Methods: A QAM computation metric based on volumetric segmentations of tumor and ablation areas and signed Euclidean surface distance maps was developed, including a novel algorithm to address QAM computation in subcapsular tumors. The code for QAM computation was verified in artificial examples of tumor and ablation spheres simulating varying scenarios of ablation margins. The applicability of the QAM metric was investigated in representative cases extracted from a prospective database of colorectal liver metastases (CRLM) treated with stereotactic microwave ablation (SMWA)., Results: Applicability of the proposed QAM metric was confirmed in artificial and clinical example cases. Numerical and visual options of data presentation displaying substrata of QAM distributions were proposed. For subcapsular tumors, the underestimation of tumor coverage by the ablation volume when applying an unadjusted QAM method was confirmed, supporting the benefits of using the proposed algorithm for QAM computation in these cases. The computational code for developed QAM was made publicly available, encouraging the use of a standard and objective metric in reporting ablation completeness and margins., Conclusion: The proposed volumetric approach for QAM computation including a novel algorithm to address subcapsular liver tumors enables precision and reproducibility in the assessment of ablation margins. The quantitative feedback on ablation completeness opens possibilities for intra-operative decision making and for refined analyses on predictability and consistency of local tumor control after thermal ablation of liver tumors., Competing Interests: SW is co-founder and shareholder of CAScination, the manufacturer of one of the navigation systems applied for stereotactic microwave ablation of colorectal liver metastases in the clinical example cases analyzed in this study. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2021 Sandu, Paolucci, Ruiter, Sznitman, de Jong, Freedman, Weber and Tinguely.)

Introduction: The simplest and cheapest method for cervical cancer screening is visual inspection after application of acetic acid (VIA). However, this method has limitations for correctly identifying precancerous cervical lesions (sensitivity) and women free from these lesions (specificity). We will assess alternative screening methods that could improve sensitivity and specificity in women living with humanimmunodeficiency virus (WLHIV) in Southern Africa., Methods and Analysis: We will conduct a paired, prospective, screening test accuracy study among consecutive, eligible women aged 18-65 years receiving treatment for HIV/AIDS at Kanyama Hospital, Lusaka, Zambia. We will assess a portable magnification device (Gynocular, Gynius Plus AB, Sweden) based on the Swede score assessment of the cervix, test for high-risk subtypes of human papillomavirus (HR-HPV, GeneXpert, Cepheid, USA) and VIA. All study participants will receive all three tests and the reference standard at baseline and at six-month follow-up. The reference standard is histological assessment of two to four biopsies of the transformation zone. The primary histological endpoint is cervical intraepithelial neoplasia grade two and above (CIN2+). Women who are VIA-positive or have histologically confirmed CIN2+ lesions will be treated as per national guidelines. We plan to enrol 450 women. Primary outcome measures for test accuracy include sensitivity and specificity of each stand-alone test. In the secondary analyses, we will evaluate the combination of tests. Pre-planned additional studies include use of cervigrams to test an automated visual assessment tool using image pattern recognition, cost-analysis and associations with trichomoniasis., Ethics and Dissemination: Ethical approval was obtained from the University of Zambia Biomedical Research Ethics Committee, Zambian National Health Regulatory Authority, Zambia Medicines Regulatory Authority, Swissethics and the International Agency for Research on Cancer Ethics Committee. Results of the study will be submitted for publication in a peer-reviewed journal., Trial Registration Number: NCT03931083; Pre-results., Competing Interests: Competing interests: None declared., (© Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.)

Purpose: To clinically validate the noninferiority of the sequentially optimized reconstruction strategy (SORS) when compared to the dynamic strategy (DS)., Methods: SORS is a novel perimetry testing strategy that evaluates a subset of test locations of a visual field (VF) test pattern and estimates the untested locations by linear approximation. When testing fewer locations, SORS has been shown in computer simulations to bring improvements in speed over conventional perimetry tests, while maintaining acquisition at high-quality acquisition. To validate SORS, a prospective clinical study was conducted at the Department of Ophthalmology of Bern University Hospital, over 12 months. Eighty-three subjects (32 healthy and 51 glaucoma patients with early to moderate visual field loss) of 114 participants were included in the study. The subjects underwent perimetry tests on an Octopus 900 (Haag-Streit, Köniz, Switzerland) using the G pattern with both DS and SORS. The acquired sensitivity thresholds (ST) by both tests were analyzed and compared., Results: DS-acquired VFs were used as a reference. High correlations between individual STs ( r ≥ 0 . 74), as well as between mean defect values ( r ≥ 0 . 88) given by DS and SORS were obtained. The mean absolute error of SORS was under 3 dB with a 70% reduction in acquisition time. SORS overestimated healthy VFs while slightly underestimating glaucomatous VFs. Qualitatively, SORS acquisition yielded VF with detectable defect patterns, albeit some isolated and small defects were occasionally missed., Conclusions: This clinical study showed that for healthy and glaucomatous patients, SORS-acquired VFs sufficiently correlated with the DS-acquired VFs with up to 70% reduction in acquisition time., Translational Relevance: This clinical study suggests that the novel perimetry strategy SORS could be used in routine clinical practice with comparable utility to the current standard DS, whereby providing a shorter and more comfortable perimetry experience., Competing Interests: Disclosure: Ş.S. Kucur, None; S. Häckel, None; J. Stapelfeldt, None; J. Odermatt, None; M.E. Iliev, None; M. Abegg, None; R. Sznitman, None; R. Höhn, None, (Copyright 2020 The Authors.)

Deep learning methods have proven extremely effective at performing a variety of medical image analysis tasks. With their potential use in clinical routine, their lack of transparency has however been one of their few weak points, raising concerns regarding their behavior and failure modes. While most research to infer model behavior has focused on indirect strategies that estimate prediction uncertainties and visualize model support in the input image space, the ability to explicitly query a prediction model regarding its image content offers a more direct way to determine the behavior of trained models. To this end, we present a novel Visual Question Answering approach that allows an image to be queried by means of a written question. Experiments on a variety of medical and natural image datasets show that by fusing image and question features in a novel way, the proposed approach achieves an equal or higher accuracy compared to current methods.

In this work we evaluated a postprocessing, customized automatic retinal OCT B-scan enhancement software for noise reduction, contrast enhancement and improved depth quality applicable to Heidelberg Engineering Spectralis OCT devices. A trained deep neural network was used to process images from an OCT dataset with ground truth biomarker gradings. Performance was assessed by the evaluation of two expert graders who evaluated image quality for B-scan with a clear preference for enhanced over original images. Objective measures such as SNR and noise estimation showed a significant improvement in quality. Presence grading of seven biomarkers IRF, SRF, ERM, Drusen, RPD, GA and iRORA resulted in similar intergrader agreement. Intergrader agreement was also compared with improvement in IRF and RPD, and disagreement in high variance biomarkers such as GA and iRORA.