Search

Acute coronary syndromes caused by plaque erosion might be potentially managed conservatively without stenting. Currently, the diagnosis of plaque erosion requires expertise in optical coherence tomographic (OCT) image interpretation. In addition, the current deep learning (DL) approaches for OCT image interpretation are based on a single frame, without integrating the information from adjacent frames.The aim of this study was to develop a novel DL model to facilitate an accurate diagnosis of plaque erosion.A novel "Transformer"-based DL model was developed that integrates information from adjacent frames emulating the cardiologists who review consecutive OCT frames to make a diagnosis and compared with the standard convolutional neural network (CNN) DL model. A total of 237,021 cross-sectional OCT images from 581 patients were used for training and internal validation, and 65,394 images from 292 patients from another dataset were used for external validation. Model performances were evaluated using the area under the receiver-operating characteristic curve (AUC).For the frame-level diagnosis of plaque erosion, the Transformer model showed superior performance than the CNN model, with an AUC of 0.94 compared with 0.85 in the external validation. For the lesion-level diagnosis, the Transformer model showed improved diagnostic performance compared with the CNN model, with an AUC of 0.91 compared with 0.84 in the external validation.This newly developed Transformer model will help cardiologists diagnose plaque erosion with high accuracy in patients with acute coronary syndromes.

High-angle annular dark-field (HAADF) scanning transmission electron microscopy (STEM) can be acquired together with energy dispersive X-ray (EDX) spectroscopy to give complementary information on the nanoparticles being imaged. Recent deep learning approaches show potential for accurate 3D tomographic reconstruction for these applications, but a large number of high-quality electron micrographs are usually required for supervised training, which may be difficult to collect due to the damage on the particles from the electron beam. To overcome these limitations and enable tomographic reconstruction even in low-dose sparse-view conditions, here we present an unsupervised deep learning method for HAADF-STEM-EDX tomography. Specifically, to improve the EDX image quality from low-dose condition, a HAADF-constrained unsupervised denoising approach is proposed. Additionally, to enable extreme sparse-view tomographic reconstruction, an unsupervised view enrichment scheme is proposed in the projection domain. Extensive experiments with different types of quantum dots show that the proposed method offers a high-quality reconstruction even with only

Optical coherence tomography (OCT) has been widely adopted in research on coronary atherosclerosis and adopted clinically to optimize percutaneous coronary intervention. In this Review, Jang and colleagues summarize this rapidly progressing field, with the aim of standardizing the use of OCT in coronary atherosclerosis.Since optical coherence tomography (OCT) was first performed in humans two decades ago, this imaging modality has been widely adopted in research on coronary atherosclerosis and adopted clinically for the optimization of percutaneous coronary intervention. In the past 10 years, substantial advances have been made in the understanding of in vivo vascular biology using OCT. Identification by OCT of culprit plaque pathology could potentially lead to a major shift in the management of patients with acute coronary syndromes. Detection by OCT of healed coronary plaque has been important in our understanding of the mechanisms involved in plaque destabilization and healing with the rapid progression of atherosclerosis. Accurate detection by OCT of sequelae from percutaneous coronary interventions that might be missed by angiography could improve clinical outcomes. In addition, OCT has become an essential diagnostic modality for myocardial infarction with non-obstructive coronary arteries. Insight into neoatherosclerosis from OCT could improve our understanding of the mechanisms of very late stent thrombosis. The appropriate use of OCT depends on accurate interpretation and understanding of the clinical significance of OCT findings. In this Review, we summarize the state of the art in cardiac OCT and facilitate the uniform use of this modality in coronary atherosclerosis. Contributions have been made by clinicians and investigators worldwide with extensive experience in OCT, with the aim that this document will serve as a standard reference for future research and clinical application.

Healed coronary plaques, morphologically characterized by a layered phenotype, are signs of previous plaque destabilization and healing. Recent optical coherence tomography (OCT) studies demonstrated that layered plaque is associated with higher levels of local and systemic inflammation and rapid plaque progression. However, the diagnosis of layered plaque needs expertise in OCT image analysis and is susceptible to inter-observer variability. We developed a deep learning (DL) model for an accurate diagnosis of layered plaque. A Visual Transformer (ViT)-based DL model that integrates information from adjacent frames emulating the cardiologists who review consecutive OCT frames to make a diagnosis was developed and compared with the standard convolutional neural network (CNN) model. A total of 237,021 cross-sectional OCT images from 581 patients collected from 8 sites were used for training and internal validation, and 65,394 images from 292 patients collected from another site were used for external validation. In the five-fold cross-validation, the ViT-based model provided better performance (area under the curve [AUC]: 0.860; 95% confidence interval [CI]: 0.855–0.866) than the standard CNN-based model (AUC: 0.799; 95% CI: 0.792–0.805). The ViT-based model (AUC: 0.845; 95% CI: 0.837–0.853) also surpassed the standard CNN-based model (AUC: 0.791; 95% CI: 0.782–0.800) in the external validation. The ViT-based DL model can accurately diagnose a layered plaque, which could help risk stratification for cardiac events.

Recently, deep learning approaches have become the main research frontier for biological image reconstruction and enhancement problems thanks to their high performance, along with their ultra-fast inference times. However, due to the difficulty of obtaining matched reference data for supervised learning, there has been increasing interest in unsupervised learning approaches that do not need paired reference data. In particular, self-supervised learning and generative models have been successfully used for various biological imaging applications. In this paper, we overview these approaches from a coherent perspective in the context of classical inverse problems, and discuss their applications to biological imaging, including electron, fluorescence and deconvolution microscopy, optical diffraction tomography and functional neuroimaging.

Recent proposals of deep learning-based beamformers for ultrasound imaging (US) have attracted significant attention as computational efficient alternatives to adaptive and compressive beamformers. Moreover, deep beamformers are versatile in that image post-processing algorithms can be readily combined. Unfortunately, with the existing technology, a large number of beamformers need to be trained and stored for different probes, organs, depth ranges, operating frequency, and desired target 'styles', demanding significant resources such as training data, etc. To address this problem, here we propose a switchable and tunable deep beamformer that can switch between various types of outputs such as DAS, MVBF, DMAS, GCF, etc., and also adjust noise removal levels at the inference phase, by using a simple switch or tunable nozzle. This novel mechanism is implemented through Adaptive Instance Normalization (AdaIN) layers, so that distinct outputs can be generated using a single generator by merely changing the AdaIN codes. Experimental results using B-mode focused ultrasound confirm the flexibility and efficacy of the proposed method for various applications.

Metal artifact reduction (MAR) is one of the most important research topics in computed tomography (CT). With the advance of deep learning approaches for image reconstruction, various deep learning methods have been suggested for metal artifact reduction, among which supervised learning methods are most popular. However, matched metal-artifact-free and metal artifact corrupted image pairs are difficult to obtain in real CT acquisition. Recently, a promising unsupervised learning for MAR was proposed using feature disentanglement, but the resulting network architecture is so complicated that it is difficult to handle large size clinical images. To address this, here we propose a simple and effective unsupervised learning method for MAR. The proposed method is based on a novel β -cycleGAN architecture derived from the optimal transport theory for appropriate feature space disentanglement. Moreover, by adding the convolutional block attention module (CBAM) layers in the generator, we show that the metal artifacts can be more focused so that it can be effectively removed. Experimental results confirm that we can achieve improved metal artifact reduction that preserves the detailed texture of the original image.

Plaque erosion, a distinct histopathological and clinical entity, accounts for over 30% of acute coronary syndromes (ACS). Optical coherence tomography allows in vivo diagnosis of plaque erosion. Local flow perturbation with activation of Toll-like receptor 2 and CD8+ T cells and subsequent desquamation of endothelium and neutrophil extracellular trap formation contribute to mechanisms of plaque erosion. Compared with ACS patients with plaque rupture, those with plaque erosion are younger, have fewer traditional cardiovascular risk factors, have lower plaque burden, and are more likely to present with non-ST-segment elevation ACS. Early evidence suggests that in patients with ACS caused by plaque erosion, antithrombotic therapy without stenting may be a safe and effective option. Future randomized trials are needed to validate these findings. Clinical studies to develop noninvasive point-of-care biomarkers that distinguish plaque rupture from erosion, and to test novel therapies that target molecular pathways involved in plaque erosion are needed.

Multispectral satellite imaging sensors acquire various spectral band images and have a unique spectroscopic property in each band. Unfortunately, image artifacts from imaging sensor noise often affect the quality of scenes and have a negative impact on applications for satellite imagery. Recently, deep learning approaches have been extensively explored to remove noise in satellite imagery. Most deep learning denoising methods, however, follow a supervised learning scheme, which requires matched noisy image and clean image pairs that are difficult to collect in real situations. In this article, we propose a novel unsupervised multispectral denoising method for satellite imagery using a wavelet directional cycle-consistent adversarial network (WavCycleGAN). The proposed method is based on an unsupervised learning scheme using adversarial loss and cycle-consistency loss to overcome the lack of paired data. Moreover, in contrast to the standard image-domain cycleGAN, we introduce a wavelet directional learning scheme for effective denoising without sacrificing high-frequency components such as edges and detailed information. Experimental results for the removal of vertical stripes and wave noise in satellite imaging sensors demonstrate that the proposed method effectively removes noise and preserves important high-frequency features of satellite images.

Chest X-ray (CXR) images are commonly used to show the internal structure of the human body without invasive intervention. The quality of CXR is an important factor as it affects the accuracy of a clinical diagnosis. Unfortunately, it is difficult to always get good quality CXR scans due to noises and scatters.Recently, wavelet directional CycleGAN (WavCycleGAN) has shown promising results in image restoration tasks by removing noise and artifacts without sacrificing high-frequency components of the input image. Unfortunately, WavCycleGAN directly reconstructs wavelet directional images that require a wavelet transform in both the training and test phases, resulting in additional processing steps and unnatural artifacts originating from the wavelet domain image. In addition, WavCycleGAN can only process artifact-related subbands, so it is difficult to apply WavCycleGAN when different levels of artifacts are present in all subbands. To address this, here we present a novel unsupervised CXR image restoration scheme with similar or even better artifact removal performance than WavCycleGAN in spite of wavelet transform being only applied in the training phase.We introduce a novel wavelet subband discriminator which can be combined with CycleGAN or switchable CycleGAN, where wavelet transform is applied only in the training phase for discriminators to match the distribution of wavelet subband components. In our framework, the image restoration network can be still applied in the image domain to prevent unnatural artifacts of the wavelet domain image with the help of the image-domain cycle-consistency loss. In addition, using wavelet subband discriminator makes it possible to remove artifacts in all subbands by utilizing frequency-specific wavelet subband discriminators.Through extensive experiments for noise and scatter removal in CXRs, we confirm that our method provides competitive performance compared to existing approaches without additional processing steps in the test phase. Furthermore, we show that our wavelet subband discriminator combined with the switchable CycleGAN can provide the flexibility by generating different levels of artifact removal.The proposed wavelet subband discriminator can be combined with the existing CycleGAN or switchable CycleGAN structures to construct an efficient unsupervised CXR image reconstruction. The advantage of our wavelet subband discriminator-based CXR image restoration is that, unlike traditional WavCycleGAN, it does not require any additional processing steps in the testing phase and does not generate unnatural artifacts originating from the wavelet domain image. We believe that our wavelet subband discriminator can be applied to various CXR image applications.

Although deep learning-based computer-aided diagnosis systems have recently achieved expert-level performance, developing a robust model requires large, high-quality data with annotations that are expensive to obtain. This situation poses a conundrum that annually-collected chest x-rays cannot be utilized due to the absence of labels, especially in deprived areas. In this study, we present a framework named distillation for self-supervision and self-train learning (DISTL) inspired by the learning process of the radiologists, which can improve the performance of vision transformer simultaneously with self-supervision and self-training through knowledge distillation. In external validation from three hospitals for diagnosis of tuberculosis, pneumothorax, and COVID-19, DISTL offers gradually improved performance as the amount of unlabeled data increase, even better than the fully supervised model with the same amount of labeled data. We additionally show that the model obtained with DISTL is robust to various real-world nuisances, offering better applicability in clinical setting.

Recently, deep learning approaches have been successfully used for ultrasound (US) image artifact removal. However, paired high-quality images for supervised training are difficult to obtain in many practical situations. Inspired by the recent theory of unsupervised learning using optimal transport driven CycleGAN (OT-CycleGAN), here, we investigate the applicability of unsupervised deep learning for US artifact removal problems without matched reference data. Two types of OT-CycleGAN approaches are employed: one with the partial knowledge of the image degradation physics and the other with the lack of such knowledge. Various US artifact removal problems are then addressed using the two types of OT-CycleGAN. Experimental results for various unsupervised US artifact removal tasks confirmed that our unsupervised learning method delivers results comparable to supervised learning in many practical applications.

Optical diffraction tomography measures the three-dimensional refractive index map of a specimen and visualizes biochemical phenomena at the nanoscale in a non-destructive manner. One major drawback of optical diffraction tomography is poor axial resolution due to limited access to the three-dimensional optical transfer function. This missing cone problem has been addressed through regularization algorithms that use a priori information, such as non-negativity and sample smoothness. However, the iterative nature of these algorithms and their parameter dependency make real-time visualization impossible. In this article, we propose and experimentally demonstrate a deep neural network, which we term DeepRegularizer, that rapidly improves the resolution of a three-dimensional refractive index map. Trained with pairs of datasets (a raw refractive index tomogram and a resolution-enhanced refractive index tomogram via the iterative total variation algorithm), the three-dimensional U-net-based convolutional neural network learns a transformation between the two tomogram domains. The feasibility and generalizability of our network are demonstrated using bacterial cells and a human leukaemic cell line, and by validating the model across different samples. DeepRegularizer offers more than an order of magnitude faster regularization performance compared to the conventional iterative method. We envision that the proposed data-driven approach can bypass the high time complexity of various image reconstructions in other imaging modalities.

(1) To compare low-contrast detectability of a deep learning–based denoising algorithm (DLA) with ADMIRE and FBP, and (2) to compare image quality parameters of DLA with those of reconstruction methods from two different CT vendors (ADMIRE, IMR, and FBP). Using abdominal CT images of 100 patients reconstructed via ADMIRE and FBP, we trained DLA by feeding FBP images as input and ADMIRE images as the ground truth. To measure the low-contrast detectability, the randomized repeat scans of Catphan® phantom were performed under various conditions of radiation exposures. Twelve radiologists evaluated the presence/absence of a target on a five-point confidence scale. The multi-reader multi-case area under the receiver operating characteristic curve (AUC) was calculated, and non-inferiority tests were performed. Using American College of Radiology CT accreditation phantom, contrast-to-noise ratio, target transfer function, noise magnitude, and detectability index (d’) of DLA, ADMIRE, IMR, and FBPs were computed. The AUC of DLA in low-contrast detectability was non-inferior to that of ADMIRE (p < .001) and superior to that of FBP (p < .001). DLA improved the image quality in terms of all physical measurements compared to FBPs from both CT vendors and showed profiles of physical measurements similar to those of ADMIRE. The low-contrast detectability of the proposed deep learning–based denoising algorithm was non-inferior to that of ADMIRE and superior to that of FBP. The DLA could successfully improve image quality compared with FBP while showing the similar physical profiles of ADMIRE. • Low-contrast detectability in the images denoised using the deep learning algorithm was non-inferior to that in the images reconstructed using standard algorithms. • The proposed deep learning algorithm showed similar profiles of physical measurements to advanced iterative reconstruction algorithm (ADMIRE).

Objective. To unify the style of computed tomography (CT) images from multiple sources, we propose a novel multi-domain image translation network to convert CT images from different scan parameters and manufacturers by simply changing a routing vector. Approach. Unlike the existing multi-domain translation techniques, our method is based on a shared encoder and a routable decoder architecture to maximize the expressivity and conditioning power of the network. Main results. Experimental results show that the proposed CT image conversion can minimize the variation of image characteristics caused by imaging parameters, reconstruction algorithms, and hardware designs. Quantitative results and clinical evaluation from radiologists also show that our method can provide accurate translation results. Significance. Quantitative evaluation of CT images from multi-site or longitudinal studies has been a difficult problem due to the image variation depending on CT scan parameters and manufacturers. The proposed method can be utilized to address this for the quantitative analysis of multi-domain CT images.

Volumetric imaging by fluorescence microscopy is often limited by anisotropic spatial resolution, in which the axial resolution is inferior to the lateral resolution. To address this problem, we present a deep-learning-enabled unsupervised super-resolution technique that enhances anisotropic images in volumetric fluorescence microscopy. In contrast to the existing deep learning approaches that require matched high-resolution target images, our method greatly reduces the effort to be put into practice as the training of a network requires only a single 3D image stack, without a priori knowledge of the image formation process, registration of training data, or separate acquisition of target data. This is achieved based on the optimal transport-driven cycle-consistent generative adversarial network that learns from an unpaired matching between high-resolution 2D images in the lateral image plane and low-resolution 2D images in other planes. Using fluorescence confocal microscopy and light-sheet microscopy, we demonstrate that the trained network not only enhances axial resolution but also restores suppressed visual details between the imaging planes and removes imaging artifacts.

Ultrasound (US) imaging is a mature technology that has widespread applications especially in the healthcare sector. Despite its widespread use and popularity, it has an inherent disadvantage that ultrasound images are prone to speckle and other kinds of noise. The image quality in the low-cost ultrasound imaging systems is degraded due to the presence of such noise and low resolution of such ultrasound systems. Herein, we propose a method for image enhancement where, the overall quality of the US images is improved by simultaneous enhancement of US image resolution and noise suppression. To avoid over-smoothing and preserving structural/texture information, we devise texture compensation in our proposed method to retain the useful anatomical features. Moreover, we also utilize US image formation physics knowledge to generate augmentation datasets which can improve the training of our proposed method. Our experimental results showcase the performance of the proposed network as well as the effectiveness of the utilization of US physics knowledge to generate augmentation datasets.

Energy-dispersive X-ray spectroscopy (EDX) is often performed simultaneously with high-angle annular dark-field scanning transmission electron microscopy (STEM) for nanoscale physico-chemical analysis. However, high-quality STEM-EDX tomographic imaging is still challenging due to fundamental limitations such as sample degradation with prolonged scan time and the low probability of X-ray generation. To address this, we propose an unsupervised deep learning method for high-quality 3D EDX tomography of core–shell nanocrystals, which can be usually permanently dammaged by prolonged electron beam. The proposed deep learning STEM-EDX tomography method was used to accurately reconstruct Au nanoparticles and InP/ZnSe/ZnS core–shell quantum dots, used in commercial display devices. Furthermore, the shape and thickness uniformity of the reconstructed ZnSe/ZnS shell closely correlates with optical properties of the quantum dots, such as quantum efficiency and chemical stability. Advanced electron microscopy and spectroscopy techniques can reveal useful structural and chemical details at the nanoscale. An unsupervised deep learning approach helps to reconstruct 3D images and observe the relationship between optical and structural properties of semiconductor nanocrystals, of interest in optoelectronic applications.

Time-resolved MR angiography (tMRA) has been widely used for dynamic contrast enhanced MRI (DCE-MRI) due to its highly accelerated acquisition. In tMRA, the periphery of the ${\textit k}$ -space data are sparsely sampled so that neighbouring frames can be merged to construct one temporal frame. However, this view-sharing scheme fundamentally limits the temporal resolution, and it is not possible to change the view-sharing number to achieve different spatio-temporal resolution trade-offs. Although many deep learning approaches have been recently proposed for MR reconstruction from sparse samples, the existing approaches usually require matched fully sampled ${\textit k}$ -space reference data for supervised training, which is not suitable for tMRA due to the lack of high spatio-temporal resolution ground-truth images. To address this problem, here we propose a novel unpaired training scheme for deep learning using optimal transport driven cycle-consistent generative adversarial network (cycleGAN). In contrast to the conventional cycleGAN with two pairs of generator and discriminator, the new architecture requires just a single pair of generator and discriminator, which makes the training much simpler but still improves the performance. Reconstruction results using in vivo tMRA and simulation data set confirm that the proposed method can immediately generate high quality reconstruction results at various choices of view-sharing numbers, allowing us to exploit better trade-off between spatial and temporal resolution in time-resolved MR angiography.

Optical diffraction tomography (ODT) produces a three-dimensional distribution of the refractive index (RI) by measuring scattering fields at various angles. Although the distribution of the RI is highly informative, due to the missing cone problem stemming from the limited-angle acquisition of holograms, reconstructions have very poor resolution along the axial direction compared to the horizontal imaging plane. To solve this issue, we present a novel unsupervised deep learning framework that learns the probability distribution of missing projection views through an optimal transport-driven CycleGAN. The experimental results show that missing cone artifacts in ODT data can be significantly resolved by the proposed method.

Deep-learning-based tomographic imaging is an important application of artificial intelligence and a new frontier of machine learning. Deep learning has been widely used in computer vision and image analysis, which deal with existing images, improve these images, and produce features from them. Since 2016, deep learning techniques have been actively researched for tomographic imaging, especially in the context of biomedicine, with impressive results and great potential. Tomographic reconstruction produces images of multi-dimensional structures from externally measured ‘encoded’ data in the form of various tomographic transforms (integrals, harmonics, echoes and so on). In this Review, we provide a general background, highlight representative results with an emphasis on medical imaging, and discuss key issues that need to be addressed in this emerging field. In particular, tomographic imaging is an integral part of modern medicine, and will play a key role in personalized, preventive and precision medicine and make it intelligent, inexpensive and indiscriminate. The popularity of deep learning is leading to new areas in biomedical applications. Wang and colleagues summarize in this Review the recent development and future directions of deep neural networks for superior image quality in the tomographic imaging field.

Conebeam CT using a circular trajectory is quite often used for various applications due to its relative simple geometry. For conebeam geometry, Feldkamp, Davis and Kress algorithm is regarded as the standard reconstruction method, but this algorithm suffers from so-called conebeam artifacts as the cone angle increases. Various model-based iterative reconstruction methods have been developed to reduce the cone-beam artifacts, but these algorithms usually require multiple applications of computational expensive forward and backprojections. In this paper, we develop a novel deep learning approach for accurate conebeam artifact removal. In particular, our deep network, designed on the differentiated backprojection domain, performs a data-driven inversion of an ill-posed deconvolution problem associated with the Hilbert transform. The reconstruction results along the coronal and sagittal directions are then combined using a spectral blending technique to minimize the spectral leakage. Experimental results show that our method outperforms the existing iterative methods despite significantly reduced runtime complexity., Comment: This paper is accepted for IEEE Trans. Medical Imaging

The nineteen papers in this special section focus on domain enriched learning for medical imaging. In recent years, learning based methods have emerged to complement traditional model and feature based methods for a variety of medical imaging problems such as image formation, classification and segmentation, quality enhancement etc. In the case of deep neural networks, many solutions have achieved unprecedented performance gains and have defined a new state of the art. Despite the progress, compelling open challenges remain. One such key challenge is that many learning frameworks (notably deep learning) are purely data-driven approaches and their performance depends strongly on the quantity and quality of training image data available. When training is limited or noisy, the performance drops sharply. Deep neural networks based approaches additionally face the challenge of often not being straightforward to interpret. Fortunately, exciting recent progress has emerged in enriching learning frameworks with domain knowledge and signal structure. As a couple of representative examples: in image reconstruction problems, this may involve using statistical/structural image priors; for image segmentation, shape and anatomical knowledge (conveyed by an expert) may be leveraged, etc. This special issue brings together contributions that combine signal, image priors and other flavors of domain knowledge with machine learning methods for solving many diverse medical imaging problems.

Since optical coherence tomography (OCT) was first performed in humans two decades ago, this imaging modality has been widely adopted in research on coronary atherosclerosis and adopted clinically for the optimization of percutaneous coronary intervention. In the past 10 years, substantial advances have been made in the understanding of in vivo vascular biology using OCT. Identification by OCT of culprit plaque pathology could potentially lead to a major shift in the management of patients with acute coronary syndromes. Detection by OCT of healed coronary plaque has been important in our understanding of the mechanisms involved in plaque destabilization and healing with the rapid progression of atherosclerosis. Accurate detection by OCT of sequelae from percutaneous coronary interventions that might be missed by angiography could improve clinical outcomes. In addition, OCT has become an essential diagnostic modality for myocardial infarction with non-obstructive coronary arteries. Insight into neoatherosclerosis from OCT could improve our understanding of the mechanisms of very late stent thrombosis. The appropriate use of OCT depends on accurate interpretation and understanding of the clinical significance of OCT findings. In this Review, we summarize the state of the art in cardiac OCT and facilitate the uniform use of this modality in coronary atherosclerosis. Contributions have been made by clinicians and investigators worldwide with extensive experience in OCT, with the aim that this document will serve as a standard reference for future research and clinical application.

In ultrasound (US) imaging, various types of adaptive beamforming techniques have been investigated to improve the resolution and contrast-to-noise ratio of the delay and sum (DAS) beamformers. Unfortunately, the performance of these adaptive beamforming approaches degrade when the underlying model is not sufficiently accurate and the number of channels decreases. To address this problem, here we propose a deep learning-based beamformer to generate significantly improved images over widely varying measurement conditions and channel subsampling patterns. In particular, our deep neural network is designed to directly process full or sub-sampled radio-frequency (RF) data acquired at various subsampling rates and detector configurations so that it can generate high quality ultrasound images using a single beamformer. The origin of such input-dependent adaptivity is also theoretically analyzed. Experimental results using B-mode focused ultrasound confirm the efficacy of the proposed methods., This is a significantly extended version of the original paper in arXiv:1901.01706. This paper is accepted for IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control

Under the global pandemic of COVID-19, the use of artificial intelligence to analyze chest X-ray (CXR) image for COVID-19 diagnosis and patient triage is becoming important. Unfortunately, due to the emergent nature of the COVID-19 pandemic, a systematic collection of the CXR data set for deep neural network training is difficult. To address this problem, here we propose a patch-based convolutional neural network approach with a relatively small number of trainable parameters for COVID-19 diagnosis. The proposed method is inspired by our statistical analysis of the potential imaging biomarkers of the CXR radiographs. Experimental results show that our method achieves state-of-the-art performance and provides clinically interpretable saliency maps, which are useful for COVID-19 diagnosis and patient triage., Accepted for IEEE Trans. on Medical Imaging Special Issue on Imaging-based Diagnosis of COVID-19

A unique advantage of magnetic resonance imaging (MRI) is its mechanism for generating various image contrasts depending on tissue-specific parameters, which provides useful clinical information. Unfortunately, a complete set of MR contrasts is often difficult to obtain in a real clinical environment. Recently, there have been claims that generative models such as generative adversarial networks (GANs) can synthesize MR contrasts that are not acquired. However, the poor scalability of existing GAN-based image synthesis poses a fundamental challenge to understanding the nature of MR contrasts: which contrasts matter, and which cannot be synthesized by generative models? Here, we show that these questions can be addressed systematically by learning the joint manifold of multiple MR contrasts using collaborative generative adversarial networks. Our experimental results show that the exogenous contrast provided by contrast agents is not replaceable, but endogenous contrasts such as T1 and T2 can be synthesized from other contrasts. These findings provide important guidance for the acquisition-protocol design of MR in clinical environments. Magnetic resonance scans use different contrast agents to generate different images, each giving specific clinical information. Lee et al. use a collaborative generative model to synthesize some magnetic resonance contrasts from others, providing guidance for how clinical imaging times can be reduced.

Deconvolution microscopy has been extensively used to improve the resolution of the wide-field fluorescent microscopy, but the performance of classical approaches critically depends on the accuracy of a model and optimization algorithms. Recently, the convolutional neural network (CNN) approaches have been studied as a fast and high performance alternative. Unfortunately, the CNN approaches usually require matched high resolution images for supervised training. In this article, we present a novel unsupervised cycle-consistent generative adversarial network (cycleGAN) with a linear blur kernel, which can be used for both blind- and non-blind image deconvolution. In contrast to the conventional cycleGAN approaches that require two deep generators, the proposed cycleGAN approach needs only a single deep generator and a linear blur kernel, which significantly improves the robustness and efficiency of network training. We show that the proposed architecture is indeed a dual formulation of an optimal transport problem that uses a special form of the penalized least squares cost as a transport cost. Experimental results using simulated and real experimental data confirm the efficacy of the algorithm.