Search

Low-dose computed tomography (LDCT) has been widely used in medical diagnosis. In practice, doctors often zoom in on LDCT slices for clearer lesions and issues, while, a simple zooming operation fails to suppress low-dose artifacts, leading to distorted details. Therefore, numerous LDCT super-resolution (SR) methods have been proposed to promote the quality of zooming without the increase of the dose in CT scanning. However, there are still some drawbacks that need to be addressed in existing methods. First, the region of interest (ROI) is not emphasized due to the lack of guidance in the reconstruction process. Second, the convolutional blocks extracting fix-resolution features fail to concentrate on the essential multi-scale features. Third, a single SR head cannot suppress the residual artifacts. To address these issues, we propose an LDCT CT joint SR and denoising reconstruction network. Our proposed network consists of global dual-guidance attention fusion modules (GDAFMs) and multi-scale anastomosis blocks (MABs). The GDAFM directs the network to focus on ROI by fusing the extra mask guidance and average CT image guidance, while the MAB introduces hierarchical features through anastomosis connections to leverage multi-scale features and promote the feature representation ability. To suppress radial residual artifacts, we optimize our network using the feedback feature distillation mechanism (FFDM) which shares the backbone to learn features corresponding to the denoising task. We apply the proposed method to the 3D-IRCADB and PANCREAS datasets to evaluate its ability on LDCT image SR reconstruction. The experimental results compared with state-of-the-art methods illustrate the superiority of our approach with respect to peak signal-to-noise (PSNR), structural similarity (SSIM), and qualitative observations. Our proposed LDCT joint SR and denoising reconstruction network has been extensively evaluated through ablation, quantitative, and qualitative experiments. The results demonstrate that our method can recover noise-free and detail-sharp images, resulting in better reconstruction results. Code is available at https://github.com/neu-szy/ldct_sr_dn_w_ffdm. [ABSTRACT FROM AUTHOR]

Low-dose computer tomography (LDCT) has been widely used in medical diagnosis. Various denoising methods have been presented to remove noise in LDCT scans. However, existing methods cannot achieve satisfactory results due to the difficulties in (1) distinguishing the characteristics of structures, textures, and noise confused in the image domain, and (2) representing local details and global semantics in the hierarchical features. In this paper, we propose a novel denoising method consisting of (1) a 2D dual-domain restoration framework to reconstruct noise-free structure and texture signals separately, and (2) a 3D multi-depth reinforcement U-Net model to further recover image details with enhanced hierarchical features. In the 2D dual-domain restoration framework, the convolutional neural networks are adopted in both the image domain where the image structures are well preserved through the spatial continuity, and the sinogram domain where the textures and noise are separately represented by different wavelet coefficients and processed adaptively. In the 3D multi-depth reinforcement U-Net model, the hierarchical features from the 3D U-Net are enhanced by the cross-resolution attention module (CRAM) and dual-branch graph convolution module (DBGCM). The CRAM preserves local details by integrating adjacent low-level features with different resolutions, while the DBGCM enhances global semantics by building graphs for high-level features in intra-feature and inter-feature dimensions. Experimental results on the LUNA16 dataset and 2016 NIH-AAPM-Mayo Clinic LDCT Grand Challenge dataset illustrate the proposed method outperforms the state-of-the-art methods on removing noise from LDCT images with clear structures and textures, proving its potential in clinical practice. [ABSTRACT FROM AUTHOR]

Detection of suspicious pulmonary nodules from lung CT scans is a crucial task in computer-aided diagnosis (CAD) systems. In recent years, various deep learning-based approaches have been proposed and demonstrated significant potential for addressing this task. However, existing deep convolutional neural networks exhibit limited long-range dependency capabilities and neglect crucial contextual information, resulting in reduced performance on detecting small-size nodules in CT scans. In this work, we propose a novel end-to-end framework called LGDNet for the detection of suspicious pulmonary nodules in lung CT scans by fusing local features and global representations. To overcome the limited long-range dependency capabilities inherent in convolutional operations, a dual-branch module is designed to integrate the convolutional neural network (CNN) branch that extracts local features with the transformer branch that captures global representations. To further address the issue of misalignment between local features and global representations, an attention gate module is proposed in the up-sampling stage to selectively combine misaligned semantic data from both branches, resulting in more accurate detection of small-size nodules. Our experiments on the large-scale LIDC dataset demonstrate that the proposed LGDNet with the dual-branch module and attention gate module could significantly improve the nodule detection sensitivity by achieving a final competition performance metric (CPM) score of 89.49%, outperforming the state-of-the-art nodule detection methods, indicating its potential for clinical applications in the early diagnosis of lung diseases. [ABSTRACT FROM AUTHOR]

Recently, RGB–T saliency detection becomes gradually a hot topic due to the fact that RGB–T multi-modal data could overcome the limitation of conventional RGB data in some cases. However, existing RGB–T saliency detection methods usually fail to take both advantages of two modalities and cannot boost performance effectively. Therefore, we achieve RGB–T saliency detection via a novel method, namely cross-modal collaborative propagation (CMCP), which contains a novel saliency propagation mechanism and a novel cross-modal collaborative learning framework relied on the proposed propagation mechanism. More specifically, we firstly propose a novel saliency propagation method and then, respectively, regard two modalities as inputs to generate RGB-induced and thermal-induced propagation mechanisms. To bridge RGB–T modalities, a novel cross-modal collaborative learning framework between RGB-induced and thermal-induced propagation mechanisms is devised to optimize, respectively, two propagation results. In other words, two modalities constantly extract supervision information to help the opposite side to refine propagation result until attaining a stable state. Finally, we integrate two modalities-induced propagation results into a refined saliency map. We compare our model with the state-of-the-art RGB–T and RGB saliency detection algorithms on three benchmark datasets, and experimental results show that the proposed CMCP achieves the significant improvement. [ABSTRACT FROM AUTHOR]

In real-world long-range surveillance systems, thermal face images captured from a distance suffer from low resolution and noise, posing challenges for thermal-to-visible face image translation. Current methods assume similar resolutions and noise-free conditions between thermal and visible images, limiting their applicability. To address these issues, we propose the Generative Facial Prior Embedded Degradation Adaption Network (GDANet), which synthesizes high-quality visible images from low-quality thermal images. GDANet combines pretrained Generative Adversarial Network (GAN) blocks with a U-shaped deep neural network (DNN) to incorporate faithful facial priors, including geometry, facial textures, and colors. Additionally, an unsupervised degradation representation learning scheme is developed to capture abstract degradation representations of degraded thermal images in a representation space. This approach allows GDANet to adapt spatial features based on the degradation representation, striking a balance between fidelity and texture faithfulness using degradation-aware feature fusion (DAFF) blocks. Experimental results demonstrate that GDANet outperforms state-of-the-art methods, showing its effectiveness in handling real-world low-quality thermal images across diverse practical applications. [ABSTRACT FROM AUTHOR]

Hallucinating a photo-realistic high-resolution (HR) face image from an occluded low-resolution (LR) face image is beneficial for a series of face-related applications. However, previous efforts focused on either super-resolving HR face images from non-occluded LR counterparts or inpainting occluded HR faces. It is necessary to address all these challenges jointly for real-world face images in unconstrained environment. In this paper, we develop a novel Local-to-Global Face Hallucination Transformer (LGFH-Transformer), which simultaneously handles the occluded LR face image super-resolution (SR) and inpainting in a unified framework. Specifically, the LGFH-Transformer is built on self-attention modules which excel at modeling long-range information between image patch sequences. Meanwhile, we introduce a mask-guided convolution and gated mechanism into the building modules (i.e., multi-head attention and feed-forward network) of each Transformer block, which can bring in the complimentary strength of convolution operation to emphasize on the spatially local context. Moreover, equipped with the delicate designed local-to-global feature reasoning mechanism in the phase of encoder, we exploit facial geometry priors (i.e., facial parsing maps) as the semantic guidance during the hallucination process in the phase of decoder to reconstruct more realistic facial details. Extensive experiments demonstrate the effectiveness and advancement of LGFH-Transformer. [ABSTRACT FROM AUTHOR]

Eye diseases often affect human health. Accurate detection of the optic disc contour is one of the important steps in diagnosing and treating eye diseases. However, the structure of fundus images is complex, and the optic disc region is often disturbed by blood vessels. Considering that the optic disc is usually a saliency region in fundus images, we propose a weakly-supervised optic disc detection method based on the fully convolution neural network (FCN) combined with the weighted low-rank matrix recovery model (WLRR). Firstly, we extract the low-level features of the fundus image and cluster the pixels using the Simple Linear Iterative Clustering (SLIC) algorithm to generate the feature matrix. Secondly, the top-down semantic prior information provided by FCN and bottom-up background prior information of the optic disc region are used to jointly construct the prior information weighting matrix, which more accurately guides the decomposition of the feature matrix into a sparse matrix representing the optic disc and a low-rank matrix representing the background. Experimental results on the DRISHTI-GS dataset and IDRiD dataset show that our method can segment the optic disc region accurately, and its performance is better than existing weakly-supervised optic disc segmentation methods. Graphical abstract of optic disc segmentation [ABSTRACT FROM AUTHOR]

Computer tomography (CT) has played an essential role in the field of medical diagnosis. Considering the potential risk of exposing patients to X-ray radiations, low-dose CT (LDCT) images have been widely applied in the medical imaging field. Since reducing the radiation dose may result in increased noise and artifacts, methods that can eliminate the noise and artifacts in the LDCT image have drawn increasing attentions and produced impressive results over the past decades. However, recent proposed methods mostly suffer from noise remaining, over-smoothing structures, or false lesions derived from noise. To tackle these issues, we propose a novel degradation adaption local-to-global transformer (DALG-Transformer) for restoring the LDCT image. Specifically, the DALG-Transformer is built on self-attention modules which excel at modeling long-range information between image patch sequences. Meanwhile, an unsupervised degradation representation learning scheme is first developed in medical image processing to learn abstract degradation representations of the LDCT images, which can distinguish various degradations in the representation space rather than the pixel space. Then, we introduce a degradation-aware modulated convolution and gated mechanism into the building modules (i.e., multi-head attention and feed-forward network) of each Transformer block, which can bring in the complementary strength of convolution operation to emphasize on the spatially local context. The experimental results show that the DALG-Transformer can provide superior performance in noise removal, structure preservation, and false lesions elimination compared with five existing representative deep networks. The proposed networks may be readily applied to other image processing tasks including image reconstruction, image deblurring, and image super-resolution. [ABSTRACT FROM AUTHOR]

Multi-modal brain magnetic resonance imaging (MRI) data has been widely applied in vison-based brain tumor segmentation methods due to its complementary diagnostic information from different modalities. Since the multi-modal image data is likely to be corrupted by noise or artifacts during the practical scanning process, making it difficult to build a universal model for the subsequent segmentation and diagnosis with incomplete input data, image completion has become one of the most attractive fields in the medical image pre-processing. It can not only assist clinicians to observe the patient's lesion area more intuitively and comprehensively, but also realize the desire to save costs for patients and reduce the psychological pressure of patients during tedious pathological examinations. Recently, many deep learning-based methods have been proposed to complement the multi-modal image data and provided good performance. However, current methods cannot fully reflect the continuous semantic information between the adjacent slices and the structural information of the intra-slice features, resulting in limited complementation effects and efficiencies. To solve these problems, in this work, we propose a novel generative adversarial network (GAN) framework, named as random generative adversarial network (RAGAN), to complete the missing T1, T1ce, and FLAIR data from the given T2 modal data in real brain MRI, which consists of the following parts: (1) For the generator, we use T2 modal images and multi-modal classification labels from the same sample for cyclically supervised training of image generation, so as to realize the restoration of arbitrary modal images. (2) For the discriminator, a multi-branch network is proposed where the primary branch is designed to judge whether the certain generated modal image is similar to the target modal image, while the auxiliary branch is to judge whether its essential visual features are similar to those of the target modal image. We conduct qualitative and quantitative experimental validations on the BraTs2018 dataset, generating 10,686 MRI data in each missing modality. Real brain tumor morphology images were compared with synthetic brain tumor morphology images using PSNR and SSIM as evaluation metrics. Experiments demonstrate that the brightness, resolution, location, and morphology of brain tissue under different modalities are well reconstructed. Meanwhile, we also use the segmentation network as a further validation experiment. Blend synthetic and real images into a segmentation network. Our segmentation network adopts the classic segmentation network UNet. The segmentation result is 77.58%. In order to prove the value of our proposed method, we use the better segmentation network RES_UNet with depth supervision as the segmentation model, and the segmentation accuracy rate is 88.76%. Although our method does not significantly outperform other algorithms, the DICE value is 2% higher than the current state-of-the-art data completion algorithm TC-MGAN. [ABSTRACT FROM AUTHOR]

This paper established an eight-layer convolu-tional neural network to automatically recognize the characteristic phases of the NiCrAlY coating, the coating/substrate interface, and the oxide layer. Using this neural network, the Cr-rich phase, the coating/substrate interface, and the oxide layer, as the features of the NiCrAlY coating, were successfully identified and retrieved at different constant oxidation temperatures. Based on this achievement, the variations of the Cr-rich phase distribution and the changes of the oxide layer thickness calculated by the network were obtained, which are all consistent with the trend of the oxidation kinetic curves at different temperatures; the preliminary intelligent calculation of oxidation kinetics of the coating was carried out through the thickness of the oxide layer from the SEM images. [ABSTRACT FROM AUTHOR]

With many thyroid nodules being incidentally detected, it is important to identify as many malignant nodules as possible while excluding those that are highly likely to be benign from fine needle aspiration (FNA) biopsies or surgeries. This paper presents a computer-aided diagnosis (CAD) system for classifying thyroid nodules in ultrasound images. We use deep learning approach to extract features from thyroid ultrasound images. Ultrasound images are pre-processed to calibrate their scale and remove the artifacts. A pre-trained GoogLeNet model is then fine-tuned using the pre-processed image samples which leads to superior feature extraction. The extracted features of the thyroid ultrasound images are sent to a Cost-sensitive Random Forest classifier to classify the images into 'malignant' and 'benign' cases. The experimental results show the proposed fine-tuned GoogLeNet model achieves excellent classification performance, attaining 98.29% classification accuracy, 99.10% sensitivity and 93.90% specificity for the images in an open access database (Pedraza et al. 16), while 96.34% classification accuracy, 86% sensitivity and 99% specificity for the images in our local health region database. [ABSTRACT FROM AUTHOR]

6DoF pose estimation has received much attention in recent years. A key challenge is the difficulty of estimating object pose when the target texture is weak. In this work, we present the cross-modal Transformer (CMT-6D), a Transformer-based network suitable for highly accurate workpiece-level object 6D pose estimation from a single RGBD image. Our main insight is to make the surface texture information of RGB images with the geometric feature information of point clouds complement each other through a cross-modal Transformer, enabling accurate estimation of the pose of weakly textured targets. Specifically, the whole framework consists of two parallel Transformer branches, named Point Transformer and Image Transformer. Both parallel transformer networks use a pyramid structured encoder and a multi-layer perceptron structured decoder to extract geometric features of point clouds and texture features of RGB images, respectively. Then, a cross-modal key query strategy is proposed for information exchange between parallel channels. In addition, at the output representation stage, we design a simple and effective 3D keypoint selection algorithm to solve the problem that keypoints are likely to appear in the non-significant region. Finally, to improve the accuracy of attitude estimation and meet real-time requirements, a lightweight pose iterative network based on target feature regression is proposed to correct the initial attitude estimation error. Extensive experiments demonstrate the effectiveness and superiority of our method on LineMOD, Occlusion LineMOD, T-Less, and YCB-Video datasets. We demonstrate that our method can improve the 6D pose estimation performance by comparing with the state-of-the-art. Ablation research and visualization validate the design of CMT-6D. [ABSTRACT FROM AUTHOR]