Search

Detection of suspicious pulmonary nodules from lung CT scans is a crucial task in computer-aided diagnosis (CAD) systems. In recent years, various deep learning-based approaches have been proposed and demonstrated significant potential for addressing this task. However, existing deep convolutional neural networks exhibit limited long-range dependency capabilities and neglect crucial contextual information, resulting in reduced performance on detecting small-size nodules in CT scans. In this work, we propose a novel end-to-end framework called LGDNet for the detection of suspicious pulmonary nodules in lung CT scans by fusing local features and global representations. To overcome the limited long-range dependency capabilities inherent in convolutional operations, a dual-branch module is designed to integrate the convolutional neural network (CNN) branch that extracts local features with the transformer branch that captures global representations. To further address the issue of misalignment between local features and global representations, an attention gate module is proposed in the up-sampling stage to selectively combine misaligned semantic data from both branches, resulting in more accurate detection of small-size nodules. Our experiments on the large-scale LIDC dataset demonstrate that the proposed LGDNet with the dual-branch module and attention gate module could significantly improve the nodule detection sensitivity by achieving a final competition performance metric (CPM) score of 89.49%, outperforming the state-of-the-art nodule detection methods, indicating its potential for clinical applications in the early diagnosis of lung diseases. [ABSTRACT FROM AUTHOR]

Recently, RGB–T saliency detection becomes gradually a hot topic due to the fact that RGB–T multi-modal data could overcome the limitation of conventional RGB data in some cases. However, existing RGB–T saliency detection methods usually fail to take both advantages of two modalities and cannot boost performance effectively. Therefore, we achieve RGB–T saliency detection via a novel method, namely cross-modal collaborative propagation (CMCP), which contains a novel saliency propagation mechanism and a novel cross-modal collaborative learning framework relied on the proposed propagation mechanism. More specifically, we firstly propose a novel saliency propagation method and then, respectively, regard two modalities as inputs to generate RGB-induced and thermal-induced propagation mechanisms. To bridge RGB–T modalities, a novel cross-modal collaborative learning framework between RGB-induced and thermal-induced propagation mechanisms is devised to optimize, respectively, two propagation results. In other words, two modalities constantly extract supervision information to help the opposite side to refine propagation result until attaining a stable state. Finally, we integrate two modalities-induced propagation results into a refined saliency map. We compare our model with the state-of-the-art RGB–T and RGB saliency detection algorithms on three benchmark datasets, and experimental results show that the proposed CMCP achieves the significant improvement. [ABSTRACT FROM AUTHOR]

Hallucinating a photo-realistic high-resolution (HR) face image from an occluded low-resolution (LR) face image is beneficial for a series of face-related applications. However, previous efforts focused on either super-resolving HR face images from non-occluded LR counterparts or inpainting occluded HR faces. It is necessary to address all these challenges jointly for real-world face images in unconstrained environment. In this paper, we develop a novel Local-to-Global Face Hallucination Transformer (LGFH-Transformer), which simultaneously handles the occluded LR face image super-resolution (SR) and inpainting in a unified framework. Specifically, the LGFH-Transformer is built on self-attention modules which excel at modeling long-range information between image patch sequences. Meanwhile, we introduce a mask-guided convolution and gated mechanism into the building modules (i.e., multi-head attention and feed-forward network) of each Transformer block, which can bring in the complimentary strength of convolution operation to emphasize on the spatially local context. Moreover, equipped with the delicate designed local-to-global feature reasoning mechanism in the phase of encoder, we exploit facial geometry priors (i.e., facial parsing maps) as the semantic guidance during the hallucination process in the phase of decoder to reconstruct more realistic facial details. Extensive experiments demonstrate the effectiveness and advancement of LGFH-Transformer. [ABSTRACT FROM AUTHOR]

Eye diseases often affect human health. Accurate detection of the optic disc contour is one of the important steps in diagnosing and treating eye diseases. However, the structure of fundus images is complex, and the optic disc region is often disturbed by blood vessels. Considering that the optic disc is usually a saliency region in fundus images, we propose a weakly-supervised optic disc detection method based on the fully convolution neural network (FCN) combined with the weighted low-rank matrix recovery model (WLRR). Firstly, we extract the low-level features of the fundus image and cluster the pixels using the Simple Linear Iterative Clustering (SLIC) algorithm to generate the feature matrix. Secondly, the top-down semantic prior information provided by FCN and bottom-up background prior information of the optic disc region are used to jointly construct the prior information weighting matrix, which more accurately guides the decomposition of the feature matrix into a sparse matrix representing the optic disc and a low-rank matrix representing the background. Experimental results on the DRISHTI-GS dataset and IDRiD dataset show that our method can segment the optic disc region accurately, and its performance is better than existing weakly-supervised optic disc segmentation methods. Graphical abstract of optic disc segmentation [ABSTRACT FROM AUTHOR]

Computer tomography (CT) has played an essential role in the field of medical diagnosis. Considering the potential risk of exposing patients to X-ray radiations, low-dose CT (LDCT) images have been widely applied in the medical imaging field. Since reducing the radiation dose may result in increased noise and artifacts, methods that can eliminate the noise and artifacts in the LDCT image have drawn increasing attentions and produced impressive results over the past decades. However, recent proposed methods mostly suffer from noise remaining, over-smoothing structures, or false lesions derived from noise. To tackle these issues, we propose a novel degradation adaption local-to-global transformer (DALG-Transformer) for restoring the LDCT image. Specifically, the DALG-Transformer is built on self-attention modules which excel at modeling long-range information between image patch sequences. Meanwhile, an unsupervised degradation representation learning scheme is first developed in medical image processing to learn abstract degradation representations of the LDCT images, which can distinguish various degradations in the representation space rather than the pixel space. Then, we introduce a degradation-aware modulated convolution and gated mechanism into the building modules (i.e., multi-head attention and feed-forward network) of each Transformer block, which can bring in the complementary strength of convolution operation to emphasize on the spatially local context. The experimental results show that the DALG-Transformer can provide superior performance in noise removal, structure preservation, and false lesions elimination compared with five existing representative deep networks. The proposed networks may be readily applied to other image processing tasks including image reconstruction, image deblurring, and image super-resolution. [ABSTRACT FROM AUTHOR]

One major branch of bottom-up salient object detection methods is machine learning-based methods which learn to classify salient object(positive) and background(negative) based on various learning algorithms. Generally, each input image corresponds to a training set which is determined by prior knowledge. However, we find that the class-imbalance problem (i.e., positive and negative data are seriously imbalanced in quantity) is inevitable for these methods due to the existence of various factors. Imbalanced training set might fail to make learning process succeed. To solve above problem, we propose a novel bottom-up Salient object detection algorithm based on Class-Imbalance Learning(CILS). For the input image, our goal is to ensure that a robust saliency classifier is well learned even if class-imbalance problem occurs in training set. To this end, we propose a novel over-sampling strategy which concentrates on constructing synthetic samples to make two classes be balanced. As a result, the balanced training data could be better applied to subsequent learning and classification process. Finally, the saliency map obtained by CILS is further refined by a novel optimization framework via foreground consistency and background consistency. Our proposed algorithm and other state-of-the-art algorithms are tested on three datasets. Results demonstrate adequately that our model produces better performance than other compared algorithms. Furthermore, we apply our salient object detection algorithm to underwater object recognition task and recognition accuracy could be further improved by introducing saliency cues. [ABSTRACT FROM AUTHOR]

Automatic and accurate segmentation of optic disc (OD) and optic cup (OC) in fundus images is a fundamental task in computer-aided ocular pathologies diagnosis. The complex structures, such as blood vessels and macular region, and the existence of lesions in fundus images bring great challenges to the segmentation task. Recently, the convolutional neural network-based methods have exhibited its potential in fundus image analysis. In this paper, we propose a cascaded two-stage network architecture for robust and accurate OD and OC segmentation in fundus images. In the first stage, the U-Net like framework with an improved attention mechanism and focal loss is proposed to detect accurate and reliable OD location from the full-scale resolution fundus images. Based on the outputs of the first stage, a refined segmentation network in the second stage that integrates multi-task framework and adversarial learning is further designed for OD and OC segmentation separately. The multi-task framework is conducted to predict the OD and OC masks by simultaneously estimating contours and distance maps as auxiliary tasks, which can guarantee the smoothness and shape of object in segmentation predictions. The adversarial learning technique is introduced to encourage the segmentation network to produce an output that is consistent with the true labels in space and shape distribution. We evaluate the performance of our method using two public retinal fundus image datasets (RIM-ONE-r3 and REFUGE). Extensive ablation studies and comparison experiments with existing methods demonstrate that our approach can produce competitive performance compared with state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Optic disc (OD) region provides a wealth of information for fundus images analysis. The accurate detection of OD contour is very important for the diagnosis and treatment of eye diseases. Considering that the OD region is generally a saliency area that distinguishes with the background in the fundus image, in this paper, we propose a unified OD detection approach by using the fully convolution neural network (FCNN) combined with the structured matrix decomposition (SMD) model, which can detect the OD region from original image directly excluding the localization step. First, the original fundus image is clustered into super pixels by Simple Linear Iterative Clustering (SLIC) algorithm, and its color, texture, and edge features are extracted to construct a feature matrix. Then, a hierarchical segmentation tree is established based on the spatial connectivity and feature similarity between super pixel patches. Finally, the SMD model and the high-level semantic prior knowledge provided by FCNN are used together to decompose the feature matrix into a sparse matrix representing the OD region and a low-rank matrix indicating the background. In this way, the OD region is derived from the sparse matrix obtained by decomposition. Our proposed method is evaluated on DRISHTI-GS dataset and IDRiD dataset and shows superior performance compared with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Feature selection, which aims to select an optimal feature subset to avoid the “curse of dimensionality,” is an important research topic in many real-world applications. To select informative features from a high-dimensional dataset, we propose a novel unsupervised feature selection algorithm called Double Regularized Matrix Factorization Feature Selection (DRMFFS) in this paper. DRMFFS is based on the feature selection framework of matrix factorization, but extends this framework by introducing double regularizations (i.e., graph regularization and inner product regularization). There are three major contributions to our approach. First, for the sake of preserving the useful underlying geometric structure information of the feature space of the data, we introduce the graph regularization to guide the learning of the feature selection matrix, making it more effective. Second, in order to take into account the correlations among features, an inner product regularization term is imposed on the objective function of matrix factorization. Therefore, the selected features by DRMFFS cannot only represent the original high-dimensional data well but also contain low redundancy. Third, we design an efficient iteratively update algorithm to solve our approach and also prove its convergence. Experiments on six benchmark databases demonstrate that the proposed approach outperforms the state-of-the-art approaches in terms of both the classification and clustering performance. [ABSTRACT FROM AUTHOR]

6DoF pose estimation has received much attention in recent years. A key challenge is the difficulty of estimating object pose when the target texture is weak. In this work, we present the cross-modal Transformer (CMT-6D), a Transformer-based network suitable for highly accurate workpiece-level object 6D pose estimation from a single RGBD image. Our main insight is to make the surface texture information of RGB images with the geometric feature information of point clouds complement each other through a cross-modal Transformer, enabling accurate estimation of the pose of weakly textured targets. Specifically, the whole framework consists of two parallel Transformer branches, named Point Transformer and Image Transformer. Both parallel transformer networks use a pyramid structured encoder and a multi-layer perceptron structured decoder to extract geometric features of point clouds and texture features of RGB images, respectively. Then, a cross-modal key query strategy is proposed for information exchange between parallel channels. In addition, at the output representation stage, we design a simple and effective 3D keypoint selection algorithm to solve the problem that keypoints are likely to appear in the non-significant region. Finally, to improve the accuracy of attitude estimation and meet real-time requirements, a lightweight pose iterative network based on target feature regression is proposed to correct the initial attitude estimation error. Extensive experiments demonstrate the effectiveness and superiority of our method on LineMOD, Occlusion LineMOD, T-Less, and YCB-Video datasets. We demonstrate that our method can improve the 6D pose estimation performance by comparing with the state-of-the-art. Ablation research and visualization validate the design of CMT-6D. [ABSTRACT FROM AUTHOR]