Search

We present a new method to measure sub-microcurie activities of photon-emitting radionuclides in organs and lesions of small animals in vivo. Our technique, named the collimator-less likelihood fit, combines a very high sensitivity collimatorless detector with a Monte Carlo-based likelihood fit in order to estimate the activities in previously segmented regions of interest along with their uncertainties. This is done directly from the photon projections in our collimatorless detector and from the region of interest segmentation provided by an x-ray computed tomography scan. We have extensively validated our approach with 225Ac experimentally in spherical phantoms and mouse phantoms, and also numerically with simulations of a realistic mouse anatomy. Our method yields statistically unbiased results with uncertainties smaller than 20% for activities as low as ~111Bq (3nCi) and for exposures under 30 minutes. We demonstrate that our method yields more robust recovery coefficients when compared to SPECT imaging with a commercial pre-clinical scanner, specially at very low activities. Thus, our technique is complementary to traditional SPECT/CT imaging since it provides a more accurate and precise organ and tumor dosimetry, with a more limited spatial information. Finally, our technique is specially significant in extremely low-activity scenarios when SPECT/CT imaging is simply not viable.

Quantitative PET attenuation correction (AC) for cardiac PET/CT and PET/MR is a challenging problem. We propose and evaluate an AC approach that uses coincidences from a relatively weak and physically fixed sparse external source, in combination with that from the patient, to reconstruct μ -maps based on physics principles alone. The low 30 cm3 volume of the source makes it easy to fill and place, and the method does not use prior image data or attenuation map assumptions. Our supplemental transmission aided maximum likelihood reconstruction of attenuation and activity (sTX-MLAA) algorithm contains an attenuation map update that maximizes the likelihood of terms representing coincidences originating from tracer in the patient and a weighted expression of counts segmented from the external source alone. Both external source and patient scatter and randoms are fully corrected. We evaluated performance of sTX-MLAA compared to reference standard CT-based AC with FDG PET/CT phantom studies; including modeling a patient with myocardial inflammation. Through an ROI analysis we measured ≤ 5 % bias in activity concentrations for PET images generated with sTX-MLAA and a TX source strength ≥ 12.7 MBq, relative to CT-AC. PET background variability (from noise and sparse sampling) was substantially reduced with sTX-MLAA compared to using counts segmented from the transmission source alone for AC. Results suggest that sTX-MLAA will enable quantitative PET during cardiac PET/CT and PET/MR of human patients.

Image reconstruction of low-count positron emission tomography (PET) data is challenging. Kernel methods address the challenge by incorporating image prior information in the forward model of iterative PET image reconstruction. The kernelized expectation-maximization (KEM) algorithm has been developed and demonstrated to be effective and easy to implement. A common approach for a further improvement of the kernel method would be adding an explicit regularization, which however leads to a complex optimization problem. In this paper, we propose an implicit regularization for the kernel method by using a deep coefficient prior, which represents the kernel coefficient image in the PET forward model using a convolutional neural-network. To solve the maximum-likelihood neural network-based reconstruction problem, we apply the principle of optimization transfer to derive a neural KEM algorithm. Each iteration of the algorithm consists of two separate steps: a KEM step for image update from the projection data and a deep-learning step in the image domain for updating the kernel coefficient image using the neural network. This optimization algorithm is guaranteed to monotonically increase the data likelihood. The results from computer simulations and real patient data have demonstrated that the neural KEM can outperform existing KEM and deep image prior methods.

Tensor fields are useful for modeling the structure of biological tissues. The challenge to measure tensor fields involves acquiring sufficient data of scalar measurements that are physically achievable and reconstructing tensors from as few projections as possible for efficient applications in medical imaging. In this paper, we present a filtered back-projection algorithm for the reconstruction of a symmetric second-rank tensor field from directional X-ray projections about three axes. The tensor field is decomposed into a solenoidal and irrotational component, each of three unknowns. Using the Fourier projection theorem, a filtered back-projection algorithm is derived to reconstruct the solenoidal and irrotational components from projections acquired around three axes. A simple illustrative phantom consisting of two spherical shells and a 3D digital cardiac diffusion image obtained from diffusion tensor MRI of an excised human heart are used to simulate directional X-ray projections. The simulations validate the mathematical derivations and demonstrate reasonable noise properties of the algorithm. The decomposition of the tensor field into solenoidal and irrotational components provides insight into the development of algorithms for reconstructing tensor fields with sufficient samples in terms of the type of directional projections and the necessary orbits for the acquisition of the projections of the tensor field.

Positron emission tomography is widely used in clinical and preclinical applications. Positronium lifetime carries information about the tissue microenvironment where positrons are emitted, but such information has not been captured because of two technical challenges. One challenge is the low sensitivity in detecting triple coincidence events. This problem has been mitigated by the recent developments of PET scanners with long (1-2 m) axial field of view. The other challenge is the low spatial resolution of the positronium lifetime images formed by existing methods that is determined by the time-of-flight (TOF) resolution (200-500 ps) of existing PET scanners. This paper solves the second challenge by developing a new image reconstruction method to generate high-resolution positronium lifetime images using existing TOF PET. Simulation studies demonstrate that the proposed method can reconstruct positronium lifetime images at much better spatial resolution than the limit set by the TOF resolution of the PET scanner. The proposed method opens up the possibility of performing positronium lifetime imaging using existing TOF PET scanners. The lifetime information can be used to understand the tissue microenvironment in vivo which could facilitate the study of disease mechanism and selection of proper treatments.

Respiratory motion is one of the main sources of motion artifacts in positron emission tomography (PET) imaging. The emission image and patient motion can be estimated simultaneously from respiratory gated data through a joint estimation framework. However, conventional motion estimation methods based on registration of a pair of images are sensitive to noise. The goal of this study is to develop a robust joint estimation method that incorporates a deep learning (DL)-based image registration approach for motion estimation. We propose a joint estimation framework by incorporating a learned image registration network into a regularized PET image reconstruction. The joint estimation was formulated as a constrained optimization problem with moving gated images related to a fixed image via the deep neural network. The constrained optimization problem is solved by the alternating direction method of multipliers (ADMM) algorithm. The effectiveness of the algorithm was demonstrated using simulated and real data. We compared the proposed DL-ADMM joint estimation algorithm with a monotonic iterative joint estimation. Motion compensated reconstructions using pre-calculated deformation fields by DL-based (DL-MC recon) and iterative (iterative-MC recon) image registration were also included for comparison. Our simulation study shows that the proposed DL-ADMM joint estimation method reduces bias compared to the ungated image without increasing noise and outperforms the competing methods. In the real data study, our proposed method also generated higher lesion contrast and sharper liver boundaries compared to the ungated image and had lower noise than the reference gated image.

Direct reconstruction methods have been developed to estimate parametric images directly from the measured PET sinograms by combining the PET imaging model and tracer kinetics in an integrated framework. Due to limited counts received, signal-to-noise-ratio (SNR) and resolution of parametric images produced by direct reconstruction frameworks are still limited. Recently supervised deep learning methods have been successfully applied to medical imaging denoising/reconstruction when large number of high-quality training labels are available. For static PET imaging, high-quality training labels can be acquired by extending the scanning time. However, this is not feasible for dynamic PET imaging, where the scanning time is already long enough. In this work, we proposed an unsupervised deep learning framework for direct parametric reconstruction from dynamic PET, which was tested on the Patlak model and the relative equilibrium Logan model. The training objective function was based on the PET statistical model. The patient's anatomical prior image, which is readily available from PET/CT or PET/MR scans, was supplied as the network input to provide a manifold constraint, and also utilized to construct a kernel layer to perform non-local feature denoising. The linear kinetic model was embedded in the network structure as a 1 ×1 ×1 convolution layer. Evaluations based on dynamic datasets of 18F-FDG and 11C-PiB tracers show that the proposed framework can outperform the traditional and the kernel method-based direct reconstruction methods.

Lung cancer is by far the leading cause of cancer death in the US. Recent studies have demonstrated the effectiveness of screening using low dose CT (LDCT) in reducing lung cancer related mortality. While lung nodules are detected with a high rate of sensitivity, this exam has a low specificity rate and it is still difficult to separate benign and malignant lesions. The ISBI 2018 Lung Nodule Malignancy Prediction Challenge, developed by a team from the Quantitative Imaging Network of the National Cancer Institute, was focused on the prediction of lung nodule malignancy from two sequential LDCT screening exams using automated (non-manual) algorithms. We curated a cohort of 100 subjects who participated in the National Lung Screening Trial and had established pathological diagnoses. Data from 30 subjects were randomly selected for training and the remaining was used for testing. Participants were evaluated based on the area under the receiver operating characteristic curve (AUC) of nodule-wise malignancy scores generated by their algorithms on the test set. The challenge had 17 participants, with 11 teams submitting reports with method description, mandated by the challenge rules. Participants used quantitative methods, resulting in a reporting test AUC ranging from 0.698 to 0.913. The top five contestants used deep learning approaches, reporting an AUC between 0.87 - 0.91. The team's predictor did not achieve significant differences from each other nor from a volume change estimate (p =.05 with Bonferroni-Holm's correction).

Chronic obstructive pulmonary disease (COPD) is a common lung disease, and quantitative CT-based bronchial phenotypes are of increasing interest as a means of exploring COPD sub-phenotypes, establishing disease progression, and evaluating intervention outcomes. Reliable, fully automated, and accurate segmentation of pulmonary airway trees is critical to such exploration. We present a novel approach of multi-parametric freeze-and-grow (FG) propagation which starts with a conservative segmentation parameter and captures finer details through iterative parameter relaxation. First, a CT intensity-based FG algorithm is developed and applied for airway tree segmentation. A more efficient version is produced using deep learning methods generating airway lumen likelihood maps from CT images, which are input to the FG algorithm. Both CT intensity- and deep learning-based algorithms are fully automated, and their performance, in terms of repeat scan reproducibility, accuracy, and leakages, is evaluated and compared with results from several state-of-the-art methods including an industry-standard one, where segmentation results were manually reviewed and corrected. Both new algorithms show a reproducibility of 95% or higher for total lung capacity (TLC) repeat CT scans. Experiments on TLC CT scans from different imaging sites at standard and low radiation dosages show that both new algorithms outperform the other methods in terms of leakages and branch-level accuracy. Considering the performance and execution times, the deep learning-based FG algorithm is a fully automated option for large multi-site studies.

This work introduces a 4D flow magnetic resonance imaging (MRI) pressure reconstruction method which employs weighted least-squares (WLS) for pressure integration. Pressure gradients are calculated from the velocity fields, and velocity errors are estimated from the velocity divergence for incompressible flow. Pressure gradient errors are estimated by propagating the velocity errors through Navier-Stokes momentum equation. A weight matrix is generated based on the pressure gradient errors, then employed for pressure reconstruction. The pressure reconstruction method was demonstrated and analyzed using synthetic velocity fields as well as Poiseuille flow measured using in vitro 4D flow MRI. Performance of the proposed WLS method was compared to the method of solving the pressure Poisson equation which has been the primary method used in the previous studies. Error analysis indicated that the proposed method is more robust to velocity measurement errors. Improvement on pressure results was found to be more significant for the cases with spatially-varying velocity error level, with reductions in error ranging from 50% to over 200%. Finally, the method was applied to flow in patient-specific cerebral aneurysms. Validation was performed with in vitro flow data collected using Particle Tracking Velocimetry (PTV) and in vivo flow measurement obtained using 4D flow MRI. Pressure calculated by WLS, as opposed to the Poisson equation, was more consistent with the flow structures and showed better agreement between the in vivo and in vitro data. These results suggest the utility of WLS method to obtain reliable pressure field from clinical flow measurement data.

Electromagnetic brain imaging is the reconstruction of brain activity from non-invasive recordings of the magnetic fields and electric potentials. An enduring challenge in this imaging modality is estimating the number, location, and time course of sources, especially for the reconstruction of distributed brain sources with complex spatial extent. Here, we introduce a novel robust empirical Bayesian algorithm that enables better reconstruction of distributed brain source activity with two key ideas: kernel smoothing and hyperparameter tiling. Since the proposed algorithm builds upon many of the performance features of the sparse source reconstruction algorithm - Champagne and we refer to this algorithm as Smooth Champagne. Smooth Champagne is robust to the effects of high levels of noise, interference, and highly correlated brain source activity. Simulations demonstrate excellent performance of Smooth Champagne when compared to benchmark algorithms in accurately determining the spatial extent of distributed source activity. Smooth Champagne also accurately reconstructs real MEG and EEG data.

Recently, deep neural networks have been widely and successfully applied in computer vision tasks and have attracted growing interest in medical imaging. One barrier for the application of deep neural networks to medical imaging is the need for large amounts of prior training pairs, which is not always feasible in clinical practice. This is especially true for medical image reconstruction problems, where raw data are needed. Inspired by the deep image prior framework, in this paper, we proposed a personalized network training method where no prior training pairs are needed, but only the patient's own prior information. The network is updated during the iterative reconstruction process using the patient-specific prior information and measured data. We formulated the maximum-likelihood estimation as a constrained optimization problem and solved it using the alternating direction method of multipliers algorithm. Magnetic resonance imaging guided positron emission tomography reconstruction was employed as an example to demonstrate the effectiveness of the proposed framework. Quantification results based on simulation and real data show that the proposed reconstruction framework can outperform Gaussian post-smoothing and anatomically guided reconstructions using the kernel method or the neural-network penalty.

Current clinical practice relies on clinical history to determine the time since stroke (TSS) onset. Imaging-based determination of acute stroke onset time could provide critical information to clinicians in deciding stroke treatment options, such as thrombolysis. The patients with unknown or unwitnessed TSS are usually excluded from thrombolysis, even if their symptoms began within the therapeutic window. In this paper, we demonstrate a machine learning approach for TSS classification using routinely acquired imaging sequences. We develop imaging features from the magnetic resonance (MR) images and train machine learning models to classify the TSS. We also propose a deep-learning model to extract hidden representations for the MR perfusion-weighted images and demonstrate classification improvement by incorporating these additional deep features. The cross-validation results show that our best classifier achieved an area under the curve of 0.765, with a sensitivity of 0.788 and a negative predictive value of 0.609, outperforming existing methods. We show that the features generated by our deep-learning algorithm correlate with the MR imaging features, and validate the robustness of the model on imaging parameter variations (e.g., year of imaging). This paper advances magnetic resonance imaging analysis one-step-closer to an operational decision support tool for stroke treatment guidance.

In this paper, we study a central problem in multimodal neuroimaging analysis, i.e., identification of significantly correlated brain regions between multiple imaging modalities. We propose a spatially varying correlation model and the associated inference procedure, which improves substantially over the common alternative solutions of voxel-wise and region-wise analysis. Compared with voxel-wise analysis, our method aggregates voxels with similar correlations into regions, takes into account spatial continuity of correlations at nearby voxels, and enjoys a much higher detection power. Compared with region-wise analysis, our method does not rely on any pre-specified brain region map, but instead finds homogenous correlation regions adaptively given the data. We applied our method to a multimodal positron emission tomography study, and found brain regions with significant correlation between tau and glucose metabolism that voxel-wise or region-wise analysis failed to identify. Our findings conform and lend additional support to prior hypotheses about how the two pathological proteins of Alzheimers disease, tau and amyloid, interact with glucose metabolism in the aging human brain.

We present a method of generating spatial maps of kinetic parameters from dynamic sequences of images collected in hyperpolarized carbon-13 magnetic resonance imaging (MRI) experiments. The technique exploits spatial correlations in the dynamic traces via regularization in the space of parameter maps. Similar techniques have proven successful in other dynamic imaging problems, such as dynamic contrast enhanced MRI. In this paper, we apply these techniques for the first time to hyperpolarized MRI problems, which are particularly challenging due to limited signal-to-noise ratio (SNR). We formulate the reconstruction as an optimization problem and present an efficient iterative algorithm for solving it based on the alternation direction method of multipliers. We demonstrate that this technique improves the qualitative appearance of parameter maps estimated from low SNR dynamic image sequences, first in simulation then on a number of data sets collected in vivo. The improvement this method provides is particularly pronounced at low SNR levels.

A flat-field correction method is proposed for multiple measured hyperspectral microscopy imaging in this paper. As the most crucial preprocessing process in quantitative microscopic analysis, flat-field correction solves the uneven illumination caused by vignetting in microscopic images, and guarantees the precision of spatial and spectral information in hyperspectral microscopic imaging. In order to carry out flat-field correction and extract uneven illumination among groups of hyperspectral microscopic data containing hundreds of bands simultaneously, two properties of vignetting have been exploited: i) low-rank property is reflected by little information contained in vignetting; ii) local smoothness can be observed as a gradual change in brightness of vignetting, which is typically equivalent to the sparseness in spatial frequency domain. Combining the two properties above, a novel Low Rank Smooth Flat-field Correction (LRSFC) model modified from common orthogonal basis extraction is proposed, while an optimization is solved based on alternating direction multiplier method (ADMM), obtaining a unique flat-field term with low-rank and smooth properties. Qualitative and quantitative experimental assessments indicate that LRSFC does not add extra cell texture to the extracted flat-field term, whose performance appears prior to other state-of-the-art flat-field correction methods. [ABSTRACT FROM AUTHOR]

We investigated the imaging performance of a fast convergent ordered-subsets algorithm with subiteration-dependent preconditioners (SDPs) for positron emission tomography (PET) image reconstruction. In particular, we considered the use of SDP with the block sequential regularized expectation maximization (BSREM) approach with the relative difference prior (RDP) regularizer due to its prior clinical adaptation by vendors. Because the RDP regularization promotes smoothness in the reconstructed image, the directions of the gradients in smooth areas more accurately point toward the objective function’s minimizer than those in variable areas. Motivated by this observation, two SDPs have been designed to increase iteration step-sizes in the smooth areas and reduce iteration step-sizes in the variable areas relative to a conventional expectation maximization preconditioner. The momentum technique used for convergence acceleration can be viewed as a special case of SDP. We have proved the global convergence of SDP-BSREM algorithms by assuming certain characteristics of the preconditioner. By means of numerical experiments using both simulated and clinical PET data, we have shown that the SDP-BSREM algorithms substantially improve the convergence rate, as compared to conventional BSREM and a vendor’s implementation as Q.Clear. Specifically, SDP-BSREM algorithms converge 35%-50% faster in reaching the same objective function value than conventional BSREM and commercial Q.Clear algorithms. Moreover, we showed in phantoms with hot, cold and background regions that the SDP-BSREM algorithms approached the values of a highly converged reference image faster than conventional BSREM and commercial Q.Clear algorithms. [ABSTRACT FROM AUTHOR]

Detecting Out-of-Distribution (OoD) data is one of the greatest challenges in safe and robust deployment of machine learning algorithms in medicine. When the algorithms encounter cases that deviate from the distribution of the training data, they often produce incorrect and over-confident predictions. OoD detection algorithms aim to catch erroneous predictions in advance by analysing the data distribution and detecting potential instances of failure. Moreover, flagging OoD cases may support human readers in identifying incidental findings. Due to the increased interest in OoD algorithms, benchmarks for different domains have recently been established. In the medical imaging domain, for which reliable predictions are often essential, an open benchmark has been missing. We introduce the Medical-Out-Of-Distribution-Analysis-Challenge (MOOD) as an open, fair, and unbiased benchmark for OoD methods in the medical imaging domain. The analysis of the submitted algorithms shows that performance has a strong positive correlation with the perceived difficulty, and that all algorithms show a high variance for different anomalies, making it yet hard to recommend them for clinical practice. We also see a strong correlation between challenge ranking and performance on a simple toy test set, indicating that this might be a valuable addition as a proxy dataset during anomaly detection algorithm development. [ABSTRACT FROM AUTHOR]

Positron emission tomography (PET) is a functional imaging modality widely used in oncology, cardiology, and neuroscience. It is highly sensitive, but suffers from relatively poor spatial resolution, as compared with anatomical imaging modalities, such as magnetic resonance imaging (MRI). With the recent development of combined PET/MR systems, we can improve the PET image quality by incorporating MR information into image reconstruction. Previously, kernel learning has been successfully embedded into static and dynamic PET image reconstruction using either PET temporal or MRI information. Here, we combine both PET temporal and MRI information adaptively to improve the quality of direct Patlak reconstruction. We examined different approaches to combine the PET and MRI information in kernel learning to address the issue of potential mismatches between MRI and PET signals. Computer simulations and hybrid real-patient data acquired on a simultaneous PET/MR scanner were used to evaluate the proposed methods. Results show that the method that combines PET temporal information and MRI spatial information adaptively based on the structure similarity index has the best performance in terms of noise reduction and resolution improvement.

By exploiting cross-information among multiple imaging data, multimodal fusion has often been used to better understand brain diseases. However, most current fusion approaches are blind, without adopting any prior information. There is increasing interest to uncover the neurocognitive mapping of specific clinical measurements on enriched brain imaging data; hence, a supervised, goal-directed model that employs prior information as a reference to guide multimodal data fusion is much needed and becomes a natural option. Here, we proposed a fusion with reference model called "multi-site canonical correlation analysis with reference + joint-independent component analysis" (MCCAR+jICA), which can precisely identify co-varying multimodal imaging patterns closely related to the reference, such as cognitive scores. In a three-way fusion simulation, the proposed method was compared with its alternatives on multiple facets; MCCAR+jICA outperforms others with higher estimation precision and high accuracy on identifying a target component with the right correspondence. In human imaging data, working memory performance was utilized as a reference to investigate the co-varying working memory-associated brain patterns among three modalities and how they are impaired in schizophrenia. Two independent cohorts (294 and 83 subjects respectively) were used. Similar brain maps were identified between the two cohorts along with substantial overlaps in the central executive network in fMRI, salience network in sMRI, and major white matter tracts in dMRI. These regions have been linked with working memory deficits in schizophrenia in multiple reports and MCCAR+jICA further verified them in a repeatable, joint manner, demonstrating the ability of the proposed method to identify potential neuromarkers for mental disorders.

Tomographic image reconstruction for low-dose computed tomography (CT) is increasingly challenging as dose continues to reduce in clinical applications. Pre-log domain methods and post-log domain methods have been proposed individually and each method has its own disadvantage. While having the potential to improve image quality for low-dose data by using an accurate imaging model, pre-log domain methods suffer slow convergence in practice due to the nonlinear transformation from the image to measurements. In contrast, post-log domain methods have fast convergence speed but the resulting image quality is suboptimal for low dose CT data because the log transformation is extremely unreliable for low-count measurements and undefined for negative values. This paper proposes a hybrid method that integrates the pre-log model and post-log model together to overcome the disadvantages of individual pre-log and post-log methods. We divide a set of CT data into high-count and low-count regions. The post-log weighted least squares model is used for measurements in the high-count region and the pre-log shifted Poisson model for measurements in the low-count region. The hybrid likelihood function can be optimized using an existing iterative algorithm. Computer simulations and phantom experiments show that the proposed hybrid method can achieve faster early convergence than the pre-log shifted Poisson likelihood method and better signal-to-noise performance than the post-log weighted least squares method.

Limited-angle CT is a challenging problem in real applications. Incomplete projection data will lead to severe artifacts and distortions in reconstruction images. To tackle this problem, we propose a novel reconstruction framework termed Deep Iterative Optimization-based Residual-learning (DIOR) for limited-angle CT. Instead of directly deploying the regularization term on image space, the DIOR combines iterative optimization and deep learning based on the residual domain, significantly improving the convergence property and generalization ability. Specifically, the asymmetric convolutional modules are adopted to strengthen the feature extraction capacity in smooth regions for deep priors. Besides, in our DIOR method, the information contained in low-frequency and high-frequency components is also evaluated by perceptual loss to improve the performance in tissue preservation. Both simulated and clinical datasets are performed to validate the performance of DIOR. Compared with existing competitive algorithms, quantitative and qualitative results show that the proposed method brings a promising improvement in artifact removal, detail restoration and edge preservation. [ABSTRACT FROM AUTHOR]

The objective of this paper is to develop a noncontact, noninvasive system for detecting and monitoring subcutaneous infection occurring at the tissue and osseointegrated prosthesis interface. It is known that the local pH of tissue can change due to infection. Therefore, the sensing system integrates two parts, namely, pH-sensitive thin films that can be coated onto prosthesis surfaces prior to them being implanted and an electrical capacitance tomography (ECT) algorithm that can reconstruct the spatial permittivity distribution of a region of space in a noncontact fashion. First, a thin film pH sensor was fabricated by spray coating, and tests confirmed that the film exhibited changes in its permittivity due to pH. Second, the ECT forward and inverse problems were implemented. Third, an aluminum rod was employed as a representative phantom of an osseointegrated prosthesis and then spray coated with the pH sensor. Finally, the film-coated phantom was immersed in different pH buffers, dried, and subjected to ECT interrogation and spatial permittivity reconstruction. The results validated that ECT was able to detect and localize permittivity variations correlated to pH changes.

An accurate system matrix is essential in positron emission tomography (PET) for reconstructing high quality images. To reduce storage size and image reconstruction time, we factor the system matrix into a product of a geometry projection matrix and a sinogram blurring matrix. The geometric projection matrix is computed analytically and the sinogram blurring matrix is estimated from point source measurements. Previously, we have estimated a 2-D blurring matrix for a preclinical PET scanner. The 2-D blurring matrix only considers blurring effects within a transaxial sinogram and does not compensate for inter-sinogram blurring effects. For PET scanners with a long axial field of view, inter-sinogram blurring can be a major problem influencing the image quality in the axial direction. Hence, the estimation of a 4-D blurring matrix is desirable to further improve the image quality. The 4-D blurring matrix estimation is an ill-conditioned problem due to the large number of unknowns. Here, we propose a rank-one approximation for each blurring kernel image formed by a row vector of the sinogram blurring matrix to improve the stability of the 4-D blurring matrix estimation. The proposed method is applied to the simulated data as well as the real data obtained from an Inveon microPET scanner. The results show that the newly estimated 4-D blurring matrix can improve the image quality over those obtained with a 2-D blurring matrix and requires less point source scans to achieve similar image quality compared with an unconstrained 4-D blurring matrix estimation.

This paper presents a novel method based on leveraging physics-informed neural networks for magnetic resonance electrical property tomography (MREPT). MREPT is a noninvasive technique that can retrieve the spatial distribution of electrical properties (EPs) of scanned tissues from measured transmit radiofrequency (RF) in magnetic resonance imaging (MRI) systems. The reconstruction of EP values in MREPT is achieved by solving a partial differential equation derived from Maxwell's equations that lacks a direct solution. Most conventional MREPT methods suffer from artifacts caused by the invalidation of the assumption applied for simplification of the problem and numerical errors caused by numerical differentiation. Existing deep learning-based (DL-based) MREPT methods comprise data-driven methods that need to collect massive datasets for training or model-driven methods that are only validated in trivial cases. Hence we proposed a model-driven method that learns mapping from a measured RF, its spatial gradient and Laplacian to EPs using fully connected networks (FCNNs). The spatial gradient of EP can be computed through the automatic differentiation of FCNNs and the chain rule. FCNNs are optimized using the residual of the central physical equation of convection-reaction MREPT as the loss function ( L) . To alleviate the ill condition of the problem, we added multiconstraints, including the similarity constraint between permittivity and conductivity and the l 1 norm of spatial gradients of permittivity and conductivity, to the L . We demonstrate the proposed method with a three-dimensional realistic head model, a digital phantom simulation, and a practical phantom experiment at a 9.4T animal MRI system.

Accurate segmentation of anatomical structures in Computed Tomography (CT) images is crucial for clinical diagnosis, treatment planning, and disease monitoring. The present deep learning segmentation methods are hindered by factors such as data scale and model size. Inspired by how doctors identify tissues, we propose a novel approach, the Prior Category Network (PCNet), that boosts segmentation performance by leveraging prior knowledge between different categories of anatomical structures. Our PCNet comprises three key components: prior category prompt (PCP), hierarchy category system (HCS), and hierarchy category loss (HCL). PCP utilizes Contrastive Language-Image Pretraining (CLIP), along with attention modules, to systematically define the relationships between anatomical categories as identified by clinicians. HCS guides the segmentation model in distinguishing between specific organs, anatomical structures, and functional systems through hierarchical relationships. HCL serves as a consistency constraint, fortifying the directional guidance provided by HCS to enhance the segmentation model's accuracy and robustness. We conducted extensive experiments to validate the effectiveness of our approach, and the results indicate that PCNet can generate a high-performance, universal model for CT segmentation. The PCNet framework also demonstrates a significant transferability on multiple downstream tasks. The ablation experiments show that the methodology employed in constructing the HCS is of critical importance. The prompt and HCS can be accessed at https://github.com/PKU-MIPET/PCNet.

Owing to the success of transformer models, recent works study their applicability in 3D medical segmentation tasks. Within the transformer models, the self-attention mechanism is one of the main building blocks that strives to capture long-range dependencies, compared to the local convolutional-based design. However, the self-attention operation has quadratic complexity which proves to be a computational bottleneck, especially in volumetric medical imaging, where the inputs are 3D with numerous slices. In this paper, we propose a 3D medical image segmentation approach, named UNETR++, that offers both high-quality segmentation masks as well as efficiency in terms of parameters, compute cost, and inference speed. The core of our design is the introduction of a novel efficient paired attention (EPA) block that efficiently learns spatial and channel-wise discriminative features using a pair of inter-dependent branches based on spatial and channel attention. Our spatial attention formulation is efficient and has linear complexity with respect to the input. To enable communication between spatial and channel-focused branches, we share the weights of query and key mapping functions that provide a complimentary benefit (paired attention), while also reducing the complexity. Our extensive evaluations on five benchmarks, Synapse, BTCV, ACDC, BraTS, and Decathlon-Lung, reveal the effectiveness of our contributions in terms of both efficiency and accuracy. On Synapse, our UNETR++ sets a new state-of-the-art with a Dice Score of 87.2%, while significantly reducing parameters and FLOPs by over 71%, compared to the best method in the literature. Our code and models are available at: https://tinyurl.com/2p87x5xn.

Functional magnetic resonance imaging (fMRI) is a commonly used technique to measure neural activation. Its application has been particularly important in identifying underlying neurodegenerative conditions such as Parkinson's, Alzheimer's, and Autism. Recent analysis of fMRI data models the brain as a graph and extracts features by graph neural networks (GNNs). However, the unique characteristics of fMRI data require a special design of GNN. Tailoring GNN to generate effective and domain-explainable features remains challenging. In this paper, we propose a contrastive dual-attention block and a differentiable graph pooling method called ContrastPool to better utilize GNN for brain networks, meeting fMRI-specific requirements. We apply our method to 5 resting-state fMRI brain network datasets of 3 diseases and demonstrate its superiority over state-of-the-art baselines. Our case study confirms that the patterns extracted by our method match the domain knowledge in neuroscience literature, and disclose direct and interesting insights. Our contributions underscore the potential of ContrastPool for advancing the understanding of brain networks and neurodegenerative conditions. The source code is available at https://github.com/AngusMonroe/ContrastPool.

Intraoperative imaging techniques for reconstructing deformable tissues in vivo are pivotal for advanced surgical systems. Existing methods either compromise on rendering quality or are excessively computationally intensive, often demanding dozens of hours to perform, which significantly hinders their practical application. In this paper, we introduce Fast Orthogonal Plane (Forplane), a novel, efficient framework based on neural radiance fields (NeRF) for the reconstruction of deformable tissues. We conceptualize surgical procedures as 4D volumes, and break them down into static and dynamic fields comprised of orthogonal neural planes. This factorization discretizes the four-dimensional space, leading to a decreased memory usage and faster optimization. A spatiotemporal importance sampling scheme is introduced to improve performance in regions with tool occlusion as well as large motions and accelerate training. An efficient ray marching method is applied to skip sampling among empty regions, significantly improving inference speed. Forplane accommodates both binocular and monocular endoscopy videos, demonstrating its extensive applicability and flexibility. Our experiments, carried out on two in vivo datasets, the EndoNeRF and Hamlyn datasets, demonstrate the effectiveness of our framework. In all cases, Forplane substantially accelerates both the optimization process (by over 100 times) and the inference process (by over 15 times) while maintaining or even improving the quality across a variety of non-rigid deformations. This significant performance improvement promises to be a valuable asset for future intraoperative surgical applications. The code of our project is now available at https://github.com/Loping151/ForPlane.

Identifying the progression stages of Alzheimer's disease (AD) can be considered as an imbalanced multi-class classification problem in machine learning. It is challenging due to the class imbalance issue and the heterogeneity of the disease. Recently, graph convolutional networks (GCNs) have been successfully applied in AD classification. However, these works did not handle the class imbalance issue in classification. Besides, they ignore the heterogeneity of the disease. To this end, we propose a novel cost-sensitive weighted contrastive learning method based on graph convolutional networks (CSWCL-GCNs) for imbalanced AD staging using resting-state functional magnetic resonance imaging (rs-fMRI). The proposed method is developed on a multi-view graph constructed by the functional connectivity (FC) and high-order functional connectivity (HOFC) features of the subjects. A novel cost-sensitive weighted contrastive learning procedure is proposed to capture discriminative information from the minority classes, encouraging the samples in the minority class to provide adequate supervision. Considering the heterogeneity of the disease, the weights of the negative pairs are introduced into contrastive learning and they are computed based on the distance to class prototypes, which are automatically learned from the training data. Meanwhile, the cost-sensitive mechanism is further introduced into contrastive learning to handle the class imbalance issue. The proposed CSWCL-GCN is evaluated on 720 subjects (including 184 NCs, 40 SMC patients, 208 EMCI patients, 172 LMCI patients and 116 AD patients) from the ADNI (Alzheimer's Disease Neuroimaging Initiative). Experimental results show that the proposed CSWCL-GCN outperforms state-of-the-art methods on the ADNI database.

Self-supervised learning (SSL) has emerged as a powerful technique for improving the efficiency and effectiveness of deep learning models. Contrastive methods are a prominent family of SSL that extract similar representations of two augmented views of an image while pushing away others in the representation space as negatives. However, the state-of-the-art contrastive methods require large batch sizes and augmentations designed for natural images that are impractical for 3D medical images. To address these limitations, we propose a new longitudinal SSL method, 3DTINC, based on non-contrastive learning. It is designed to learn perturbation-invariant features for 3D optical coherence tomography (OCT) volumes, using augmentations specifically designed for OCT. We introduce a new non-contrastive similarity loss term that learns temporal information implicitly from intra-patient scans acquired at different times. Our experiments show that this temporal information is crucial for predicting progression of retinal diseases, such as age-related macular degeneration (AMD). After pretraining with 3DTINC, we evaluated the learned representations and the prognostic models on two large-scale longitudinal datasets of retinal OCTs where we predict the conversion to wet-AMD within a six-month interval. Our results demonstrate that each component of our contributions is crucial for learning meaningful representations useful in predicting disease progression from longitudinal volumetric scans.

Individual brains vary greatly in morphology, connectivity and organization. Individualized brain parcellation is capable of precisely localizing subject-specific functional regions. However, most individualization approaches have examined single modalities of data and have not generalized to nonhuman primates. The present study proposed a novel multimodal connectivity-based individual parcellation (MCIP) method, which optimizes within-region homogeneity, spatial continuity and similarity to a reference atlas with the fusion of personal functional and anatomical connectivity. Comprehensive evaluation demonstrated that MCIP outperformed state-of-the-art multimodal individualization methods in terms of functional and anatomical homogeneity, predictability of cognitive measures, heritability, reproducibility and generalizability across species. Comparative investigation showed a higher topographic variability in humans than that in macaques. Therefore, MCIP provides improved accurate and reliable mapping of brain functional regions over existing methods at an individual level across species, and could facilitate comparative and translational neuroscience research.

Early detection and treatment of breast cancer can significantly reduce patient mortality, and mammogram is an effective method for early screening. Computer-aided diagnosis (CAD) of mammography based on deep learning can assist radiologists in making more objective and accurate judgments. However, existing methods often depend on datasets with manual segmentation annotations. In addition, due to the large image sizes and small lesion proportions, many methods that do not use region of interest (ROI) mostly rely on multi-scale and multi-feature fusion models. These shortcomings increase the labor, money, and computational overhead of applying the model. Therefore, a deep location soft-embedding-based network with regional scoring (DLSEN-RS) is proposed. DLSEN-RS is an end-to-end mammography image classification method containing only one feature extractor and relies on positional embedding (PE) and aggregation pooling (AP) modules to locate lesion areas without bounding boxes, transfer learning, or multi-stage training. In particular, the introduced PE and AP modules exhibit versatility across various CNN models and improve the model's tumor localization and diagnostic accuracy for mammography images. Experiments are conducted on published INbreast and CBIS-DDSM datasets, and compared to previous state-of-the-art mammographic image classification methods, DLSEN-RS performed satisfactorily.

Multimodal neuroimaging provides complementary information critical for accurate early diagnosis of Alzheimer's disease (AD). However, the inherent variability between multimodal neuroimages hinders the effective fusion of multimodal features. Moreover, achieving reliable and interpretable diagnoses in the field of multimodal fusion remains challenging. To address them, we propose a novel multimodal diagnosis network based on multi-fusion and disease-induced learning (MDL-Net) to enhance early AD diagnosis by efficiently fusing multimodal data. Specifically, MDL-Net proposes a multi-fusion joint learning (MJL) module, which effectively fuses multimodal features and enhances the feature representation from global, local, and latent learning perspectives. MJL consists of three modules, global-aware learning (GAL), local-aware learning (LAL), and outer latent-space learning (LSL) modules. GAL via a self-adaptive Transformer (SAT) learns the global relationships among the modalities. LAL constructs local-aware convolution to learn the local associations. LSL module introduces latent information through outer product operation to further enhance feature representation. MDL-Net integrates the disease-induced region-aware learning (DRL) module via gradient weight to enhance interpretability, which iteratively learns weight matrices to identify AD-related brain regions. We conduct the extensive experiments on public datasets and the results confirm the superiority of our proposed method. Our code will be available at: https://github.com/qzf0320/MDL-Net.

The lack of reliable biomarkers makes predicting the conversion from intermediate to neovascular age-related macular degeneration (iAMD, nAMD) a challenging task. We develop a Deep Learning (DL) model to predict the future risk of conversion of an eye from iAMD to nAMD from its current OCT scan. Although eye clinics generate vast amounts of longitudinal OCT scans to monitor AMD progression, only a small subset can be manually labeled for supervised DL. To address this issue, we propose Morph-SSL, a novel Self-supervised Learning (SSL) method for longitudinal data. It uses pairs of unlabelled OCT scans from different visits and involves morphing the scan from the previous visit to the next. The Decoder predicts the transformation for morphing and ensures a smooth feature manifold that can generate intermediate scans between visits through linear interpolation. Next, the Morph-SSL trained features are input to a Classifier which is trained in a supervised manner to model the cumulative probability distribution of the time to conversion with a sigmoidal function. Morph-SSL was trained on unlabelled scans of 399 eyes (3570 visits). The Classifier was evaluated with a five-fold cross-validation on 2418 scans from 343 eyes with clinical labels of the conversion date. The Morph-SSL features achieved an AUC of 0.779 in predicting the conversion to nAMD within the next 6 months, outperforming the same network when trained end-to-end from scratch or pre-trained with popular SSL methods. Automated prediction of the future risk of nAMD onset can enable timely treatment and individualized AMD management.

Nuclei classification provides valuable information for histopathology image analysis. However, the large variations in the appearance of different nuclei types cause difficulties in identifying nuclei. Most neural network based methods are affected by the local receptive field of convolutions, and pay less attention to the spatial distribution of nuclei or the irregular contour shape of a nucleus. In this paper, we first propose a novel polygon-structure feature learning mechanism that transforms a nucleus contour into a sequence of points sampled in order, and employ a recurrent neural network that aggregates the sequential change in distance between key points to obtain learnable shape features. Next, we convert a histopathology image into a graph structure with nuclei as nodes, and build a graph neural network to embed the spatial distribution of nuclei into their representations. To capture the correlations between the categories of nuclei and their surrounding tissue patterns, we further introduce edge features that are defined as the background textures between adjacent nuclei. Lastly, we integrate both polygon and graph structure learning mechanisms into a whole framework that can extract intra and inter-nucleus structural characteristics for nuclei classification. Experimental results show that the proposed framework achieves significant improvements compared to the previous methods. Code and data are made available via https://github.com/lhaof/SENC.

Robust segmenting with noisy labels is an important problem in medical imaging due to the difficulty of acquiring high-quality annotations. Despite the enormous success of recent developments, these developments still require multiple networks to construct their frameworks and focus on limited application scenarios, which leads to inflexibility in practical applications. They also do not explicitly consider the coarse boundary label problem, which results in sub-optimal results. To overcome these challenges, we propose a novel Simultaneous Edge Alignment and Memory-Assisted Learning (SEAMAL) framework for noisy-label robust segmentation. It achieves single-network robust learning, which is applicable for both 2D and 3D segmentation, in both Set-HQ-knowable and Set-HQ-agnostic scenarios. Specifically, to achieve single-model noise robustness, we design a Memory-assisted Selection and Correction module (MSC) that utilizes predictive history consistency from the Prediction Memory Bank to distinguish between reliable and non-reliable labels pixel-wisely, and that updates the reliable ones at the superpixel level. To overcome the coarse boundary label problem, which is common in practice, and to better utilize shape-relevant information at the boundary, we propose an Edge Detection Branch (EDB) that explicitly learns the boundary via an edge detection layer with only slight additional computational cost, and we improve the sharpness and precision of the boundary with a thinning loss. Extensive experiments verify that SEAMAL outperforms previous works significantly.

Medical imaging is limited by acquisition time and scanning equipment. CT and MR volumes, reconstructed with thicker slices, are anisotropic with high in-plane resolution and low through-plane resolution. We reveal an intriguing phenomenon that due to the mentioned nature of data, performing slice-wise interpolation from the axial view can yield greater benefits than performing super-resolution from other views. Based on this observation, we propose an Inter-Intra-slice Interpolation Network ( [Formula: see text]Net), which fully explores information from high in-plane resolution and compensates for low through-plane resolution. The through-plane branch supplements the limited information contained in low through-plane resolution from high in-plane resolution and enables continual and diverse feature learning. In-plane branch transforms features to the frequency domain and enforces an equal learning opportunity for all frequency bands in a global context learning paradigm. We further propose a cross-view block to take advantage of the information from all three views online. Extensive experiments on two public datasets demonstrate the effectiveness of [Formula: see text]Net, and noticeably outperforms state-of-the-art super-resolution, video frame interpolation and slice interpolation methods by a large margin. We achieve 43.90dB in PSNR, with at least 1.14dB improvement under the upscale factor of ×2 on MSD dataset with faster inference. Code is available at https://github.com/DeepMed-Lab-ECNU/Medical-Image-Reconstruction.

In Ultrasound Localization Microscopy (ULM), achieving high-resolution images relies on the precise localization of contrast agent particles across a series of beamformed frames. However, our study uncovers an enormous potential: The process of delay-and-sum beamforming leads to an irreversible reduction of Radio-Frequency (RF) channel data, while its implications for localization remain largely unexplored. The rich contextual information embedded within RF wavefronts, including their hyperbolic shape and phase, offers great promise for guiding Deep Neural Networks (DNNs) in challenging localization scenarios. To fully exploit this data, we propose to directly localize scatterers in RF channel data. Our approach involves a custom super-resolution DNN using learned feature channel shuffling, non-maximum suppression, and a semi-global convolutional block for reliable and accurate wavefront localization. Additionally, we introduce a geometric point transformation that facilitates seamless mapping to the B-mode coordinate space. To understand the impact of beamforming on ULM, we validate the effectiveness of our method by conducting an extensive comparison with State-Of-The-Art (SOTA) techniques. We present the inaugural in vivo results from a wavefront-localizing DNN, highlighting its real-world practicality. Our findings show that RF-ULM bridges the domain shift between synthetic and real datasets, offering a considerable advantage in terms of precision and complexity. To enable the broader research community to benefit from our findings, our code and the associated SOTA methods are made available at https://github.com/hahnec/rf-ulm.

Adapting a medical image segmentation model to a new domain is important for improving its cross-domain transferability, and due to the expensive annotation process, Unsupervised Domain Adaptation (UDA) is appealing where only unlabeled images are needed for the adaptation. Existing UDA methods are mainly based on image or feature alignment with adversarial training for regularization, and they are limited by insufficient supervision in the target domain. In this paper, we propose an enhanced Filtered Pseudo Label (FPL+)-based UDA method for 3D medical image segmentation. It first uses cross-domain data augmentation to translate labeled images in the source domain to a dual-domain training set consisting of a pseudo source-domain set and a pseudo target-domain set. To leverage the dual-domain augmented images to train a pseudo label generator, domain-specific batch normalization layers are used to deal with the domain shift while learning the domain-invariant structure features, generating high-quality pseudo labels for target-domain images. We then combine labeled source-domain images and target-domain images with pseudo labels to train a final segmentor, where image-level weighting based on uncertainty estimation and pixel-level weighting based on dual-domain consensus are proposed to mitigate the adverse effect of noisy pseudo labels. Experiments on three public multi-modal datasets for Vestibular Schwannoma, brain tumor and whole heart segmentation show that our method surpassed ten state-of-the-art UDA methods, and it even achieved better results than fully supervised learning in the target domain in some cases.

To improve the spatial resolution of power Doppler (PD) imaging, we explored null subtraction imaging (NSI) as an alternative beamforming technique to delay-and-sum (DAS). NSI is a nonlinear beamforming approach that uses three different apodizations on receive and incoherently sums the beamformed envelopes. NSI uses a null in the beam pattern to improve the lateral resolution, which we apply here for improving PD spatial resolution both with and without contrast microbubbles. In this study, we used NSI with three types of singular value decomposition (SVD)-based clutter filters and noise equalization to generate high-resolution PD images. An element sensitivity correction scheme was also proposed as a crucial component of NSI-based PD imaging. First, a microbubble trace experiment was performed to evaluate the resolution improvement of NSI-based PD over traditional DAS-based PD. Then, both contrast-enhanced and contrast free ultrasound PD images were generated from the scan of a rat brain. The cross-sectional profile of the microbubble traces and microvessels were plotted. FWHM was also estimated to provide a quantitative metric. Furthermore, iso-frequency curves were calculated to provide a resolution evaluation metric over the global field of view. Up to six-fold resolution improvement was demonstrated by the FWHM estimate and four-fold resolution improvement was demonstrated by the iso-frequency curve from the NSI-based PD microvessel images compared to microvessel images generated by traditional DAS-based beamforming. A resolvability of [Formula: see text] was measured from the NSI-based PD microvessel image. The computational cost of NSI-based PD was only increased by 40 percent over the DAS-based PD.

Proximal femoral fracture segmentation in computed tomography (CT) is essential in the preoperative planning of orthopedic surgeons. Recently, numerous deep learning-based approaches have been proposed for segmenting various structures within CT scans. Nevertheless, distinguishing various attributes between fracture fragments and soft tissue regions in CT scans frequently poses challenges, which have received comparatively limited research attention. Besides, the cornerstone of contemporary deep learning methodologies is the availability of annotated data, while detailed CT annotations remain scarce. To address the challenge, we propose a novel weakly-supervised framework, namely Rough Turbo Net (RT-Net), for the segmentation of proximal femoral fractures. We emphasize the utilization of human resources to produce rough annotations on a substantial scale, as opposed to relying on limited fine-grained annotations that demand a substantial time to create. In RT-Net, rough annotations pose fractured-region constraints, which have demonstrated significant efficacy in enhancing the accuracy of the network. Conversely, the fine annotations can provide more details for recognizing edges and soft tissues. Besides, we design a spatial adaptive attention module (SAAM) that adapts to the spatial distribution of the fracture regions and align feature in each decoder. Moreover, we propose a fine-edge loss which is applied through an edge discrimination network to penalize the absence or imprecision edge features. Extensive quantitative and qualitative experiments demonstrate the superiority of RT-Net to state-of-the-art approaches. Furthermore, additional experiments show that RT-Net has the capability to produce pseudo labels for raw CT images that can further improve fracture segmentation performance and has the potential to improve segmentation performance on public datasets. The code is available at: https://github.com/zyairelu/RT-Net.

Accurate motion estimation at high acceleration factors enables rapid motion-compensated reconstruction in Magnetic Resonance Imaging (MRI) without compromising the diagnostic image quality. In this work, we introduce an attention-aware deep learning-based framework that can perform non-rigid pairwise registration for fully sampled and accelerated MRI. We extract local visual representations to build similarity maps between the registered image pairs at multiple resolution levels and additionally leverage long-range contextual information using a transformer-based module to alleviate ambiguities in the presence of artifacts caused by undersampling. We combine local and global dependencies to perform simultaneous coarse and fine motion estimation. The proposed method was evaluated on in-house acquired fully sampled and accelerated data of 101 patients and 62 healthy subjects undergoing cardiac and thoracic MRI. The impact of motion estimation accuracy on the downstream task of motion-compensated reconstruction was analyzed. We demonstrate that our model derives reliable and consistent motion fields across different sampling trajectories (Cartesian and radial) and acceleration factors of up to 16x for cardiac motion and 30x for respiratory motion and achieves superior image quality in motion-compensated reconstruction qualitatively and quantitatively compared to conventional and recent deep learning-based approaches. The code is publicly available at https://github.com/lab-midas/GMARAFT.

The quantification of stenosis severity from X-ray catheter angiography is a challenging task. Indeed, this requires to fully understand the lesion's geometry by analyzing dynamics of the contrast material, only relying on visual observation by clinicians. To support decision making for cardiac intervention, we propose a hybrid CNN-Transformer model for the assessment of angiography-based non-invasive fractional flow-reserve (FFR) and instantaneous wave-free ratio (iFR) of intermediate coronary stenosis. Our approach predicts whether a coronary artery stenosis is hemodynamically significant and provides direct FFR and iFR estimates. This is achieved through a combination of regression and classification branches that forces the model to focus on the cut-off region of FFR (around 0.8 FFR value), which is highly critical for decision-making. We also propose a spatio-temporal factorization mechanisms that redesigns the transformer's self-attention mechanism to capture both local spatial and temporal interactions between vessel geometry, blood flow dynamics, and lesion morphology. The proposed method achieves state-of-the-art performance on a dataset of 778 exams from 389 patients. Unlike existing methods, our approach employs a single angiography view and does not require knowledge of the key frame; supervision at training time is provided by a classification loss (based on a threshold of the FFR/iFR values) and a regression loss for direct estimation. Finally, the analysis of model interpretability and calibration shows that, in spite of the complexity of angiographic imaging data, our method can robustly identify the location of the stenosis and correlate prediction uncertainty to the provided output scores.

Multi-frequency electrical impedance tomography (mfEIT) offers a nondestructive imaging technology that reconstructs the distribution of electrical characteristics within a subject based on the impedance spectral differences among biological tissues. However, the technology faces challenges in imaging multi-class lesion targets when the conductivity of background tissues is frequency-dependent. To address these issues, we propose a spatial-frequency cross-fusion network (SFCF-Net) imaging algorithm, built on a multi-path fusion structure. This algorithm uses multi-path structures and hyper-dense connections to capture both spatial and frequency correlations between multi-frequency conductivity images, which achieves differential imaging for lesion targets of multiple categories through cross-fusion of information. According to both simulation and physical experiment results, the proposed SFCF-Net algorithm shows an excellent performance in terms of lesion imaging and category discrimination compared to the weighted frequency-difference, U-Net, and MMV-Net algorithms. The proposed algorithm enhances the ability of mfEIT to simultaneously obtain both structural and spectral information from the tissue being examined and improves the accuracy and reliability of mfEIT, opening new avenues for its application in clinical diagnostics and treatment monitoring.

Ophthalmic diseases such as central serous chorioretinopathy (CSC) significantly impair the vision of millions of people globally. Precise segmentation of choroid and macular edema is critical for diagnosing and treating these conditions. However, existing 3D medical image segmentation methods often fall short due to the heterogeneous nature and blurry features of these conditions, compounded by medical image clarity issues and noise interference arising from equipment and environmental limitations. To address these challenges, we propose the Spectrum Analysis Synergy Axial-Spatial Network (SASAN), an approach that innovatively integrates spectrum features using the Fast Fourier Transform (FFT). SASAN incorporates two key modules: the Frequency Integrated Neural Enhancer (FINE), which mitigates noise interference, and the Axial-Spatial Elementum Multiplier (ASEM), which enhances feature extraction. Additionally, we introduce the Self-Adaptive Multi-Aspect Loss ( L SM ), which balances image regions, distribution, and boundaries, adaptively updating weights during training. We compiled and meticulously annotated the Choroid and Macular Edema OCT Mega Dataset (CMED-18k), currently the world's largest dataset of its kind. Comparative analysis against 13 baselines shows our method surpasses these benchmarks, achieving the highest Dice scores and lowest HD95 in the CMED and OIMHS datasets. Our code is publicly available at https://github.com/IMOP-lab/SASAN-Pytorch.

Accurate myocardial segmentation is crucial in the diagnosis and treatment of myocardial infarction (MI), especially in Late Gadolinium Enhancement (LGE) cardiac magnetic resonance (CMR) images, where the infarcted myocardium exhibits a greater brightness. However, segmentation annotations for LGE images are usually not available. Although knowledge gained from CMR images of other modalities with ample annotations, such as balanced-Steady State Free Precession (bSSFP), can be transferred to the LGE images, the difference in image distribution between the two modalities (i.e., domain shift) usually results in a significant degradation in model performance. To alleviate this, an end-to-end Variational autoencoder based feature Alignment Module Combining Explicit and Implicit features (VAMCEI) is proposed. We first re-derive the Kullback-Leibler (KL) divergence between the posterior distributions of the two domains as a measure of the global distribution distance. Second, we calculate the prototype contrastive loss between the two domains, bringing closer the prototypes of the same category across domains and pushing away the prototypes of different categories within or across domains. Finally, a domain discriminator is added to the output space, which indirectly aligns the feature distribution and forces the extracted features to be more favorable for segmentation. In addition, by combining CycleGAN and VAMCEI, we propose a more refined multi-stage unsupervised domain adaptation (UDA) framework for myocardial structure segmentation. We conduct extensive experiments on the MSCMRSeg 2019, MyoPS 2020 and MM-WHS 2017 datasets. The experimental results demonstrate that our framework achieves superior performances than state-of-the-art methods.

Automated classification of breast cancer subtypes from digital pathology images has been an extremely challenging task due to the complicated spatial patterns of cells in the tissue micro-environment. While newly proposed graph transformers are able to capture more long-range dependencies to enhance accuracy, they largely ignore the topological connectivity between graph nodes, which is nevertheless critical to extract more representative features to address this difficult task. In this paper, we propose a novel connectivity-aware graph transformer (CGT) for phenotyping the topology connectivity of the tissue graph constructed from digital pathology images for breast cancer classification. Our CGT seamlessly integrates connectivity embedding to node feature at every graph transformer layer by using local connectivity aggregation, in order to yield more comprehensive graph representations to distinguish different breast cancer subtypes. In light of the realistic intercellular communication mode, we then encode the spatial distance between two arbitrary nodes as connectivity bias in self-attention calculation, thereby allowing the CGT to distinctively harness the connectivity embedding based on the distance of two nodes. We extensively evaluate the proposed CGT on a large cohort of breast carcinoma digital pathology images stained by Haematoxylin & Eosin. Experimental results demonstrate the effectiveness of our CGT, which outperforms state-of-the-art methods by a large margin. Codes are released on https://github.com/wang-kang-6/CGT.

Aneurysmal subarachnoid hemorrhage is a medical emergency of brain that has high mortality and poor prognosis. Causal effect estimation of treatment strategies on patient outcomes is crucial for aneurysmal subarachnoid hemorrhage treatment decision-making. However, most existing studies on treatment decision-making support of this disease are unable to simultaneously compare the potential outcomes of different treatments for a patient. Furthermore, these studies fail to harmoniously integrate the imaging data with non-imaging clinical data, both of which are useful in clinical scenarios. In this paper, we estimate the causal effect of various treatments on patients with aneurysmal subarachnoid hemorrhage by integrating plain CT with non-imaging clinical data, which is represented using structured tabular data. Specifically, we first propose a novel scheme that uses multi-modality confounders distillation architecture to predict the treatment outcome and treatment assignment simultaneously. With these distilled confounder features, we design an imaging and non-imaging interaction representation learning strategy to use the complementary information extracted from different modalities to balance the feature distribution of different treatment groups. We have conducted extensive experiments using a clinical dataset of 656 subarachnoid hemorrhage cases, which was collected from the Hospital Authority Data Collaboration Laboratory in Hong Kong. Our method shows consistent improvements on the evaluation metrics of treatment effect estimation, achieving state-of-the-art results over strong competitors. Code is released at https://github.com/med-air/TOP-aSAH.

Vision-Language Pre-training (VLP) that utilizes the multi-modal information to promote the training efficiency and effectiveness, has achieved great success in vision recognition of natural domains and shown promise in medical imaging diagnosis for the Chest X-Rays (CXRs). However, current works mainly pay attention to the exploration on single dataset of CXRs, which locks the potential of this powerful paradigm on larger hybrid of multi-source CXRs datasets. We identify that although blending samples from the diverse sources offers the advantages to improve the model generalization, it is still challenging to maintain the consistent superiority for the task of each source due to the existing heterogeneity among sources. To handle this dilemma, we design a Conquer-and-Divide pre-training framework, termed as UniChest, aiming to make full use of the collaboration benefit of multiple sources of CXRs while reducing the negative influence of the source heterogeneity. Specially, the "Conquer" stage in UniChest encourages the model to sufficiently capture multi-source common patterns, and the "Divide" stage helps squeeze personalized patterns into different small experts (query networks). We conduct thorough experiments on many benchmarks, e.g., ChestX-ray14, CheXpert, Vindr-CXR, Shenzhen, Open-I and SIIM-ACR Pneumothorax, verifying the effectiveness of UniChest over a range of baselines, and release our codes and pre-training models at https://github.com/Elfenreigen/UniChest.