Search

Detection of suspicious pulmonary nodules from lung CT scans is a crucial task in computer-aided diagnosis (CAD) systems. In recent years, various deep learning-based approaches have been proposed and demonstrated significant potential for addressing this task. However, existing deep convolutional neural networks exhibit limited long-range dependency capabilities and neglect crucial contextual information, resulting in reduced performance on detecting small-size nodules in CT scans. In this work, we propose a novel end-to-end framework called LGDNet for the detection of suspicious pulmonary nodules in lung CT scans by fusing local features and global representations. To overcome the limited long-range dependency capabilities inherent in convolutional operations, a dual-branch module is designed to integrate the convolutional neural network (CNN) branch that extracts local features with the transformer branch that captures global representations. To further address the issue of misalignment between local features and global representations, an attention gate module is proposed in the up-sampling stage to selectively combine misaligned semantic data from both branches, resulting in more accurate detection of small-size nodules. Our experiments on the large-scale LIDC dataset demonstrate that the proposed LGDNet with the dual-branch module and attention gate module could significantly improve the nodule detection sensitivity by achieving a final competition performance metric (CPM) score of 89.49%, outperforming the state-of-the-art nodule detection methods, indicating its potential for clinical applications in the early diagnosis of lung diseases. [ABSTRACT FROM AUTHOR]

The diagnosis of liver cancer is one of the most attractive fields in clinical practice for its high mortality. Accurate segmentation of liver and tumor has been publicly accepted to be an effective method to assist doctors in determining the disease condition and planning the subsequent treatments. Recently, deep learning based methods have been widely used in tumor segmentation and provided good performance. However, current methods cannot fully reflect the differences between tumor, inside-liver tissues and outside-liver organs simultaneously, while the extraction of features reflecting axial changes of liver and tumor is always discounted by the heavy computational burden, resulting in limited learning effects and efficiencies. To solve these problems, in this paper, we propose a novel framework to segment liver and tumors in abdominal CT volumes, which consists of two parts: 1) we propose a multi-branch network where an up-sampling branch for liver region recognition and a pyramid-like convolution structure for inner-liver feature extraction are integrated into the back-bone Dense UNet structure for better extracting intra-slice features of liver and tumors; 2) we simplify the traditional 3D UNet by using the convolutional kernels with the fixed size 3 × 3 in x-y plane and apply it as a 3D counterpart for aggregating contextual information along the z-axis from the stacked, filtered CT slices, with the advantages of inhibiting the influence from neighboring pixels and alleviating the computational burden greatly. The above two parts are formulated as a unified end-to-end network so that the intra-slice feature representation and the inter-slice information aggregation can be learned and optimized jointly. Furthermore, we novely define a loss function combining a modified dice loss and a contour-detection based loss, so that the region features and contour features of the predicted segmentation of liver and tumors are jointly considered for network training and parameters optimization. Experimental results on the MICCAI 2017 Liver Tumor Segmentation Challenge dataset and 3DIRCADb dataset have demonstrated that the proposed method can provide superior performance to the state-of-the-art methods with respect to the certain benchmarks for liver and tumor segmentation. [ABSTRACT FROM AUTHOR]

Recently, RGB–T saliency detection becomes gradually a hot topic due to the fact that RGB–T multi-modal data could overcome the limitation of conventional RGB data in some cases. However, existing RGB–T saliency detection methods usually fail to take both advantages of two modalities and cannot boost performance effectively. Therefore, we achieve RGB–T saliency detection via a novel method, namely cross-modal collaborative propagation (CMCP), which contains a novel saliency propagation mechanism and a novel cross-modal collaborative learning framework relied on the proposed propagation mechanism. More specifically, we firstly propose a novel saliency propagation method and then, respectively, regard two modalities as inputs to generate RGB-induced and thermal-induced propagation mechanisms. To bridge RGB–T modalities, a novel cross-modal collaborative learning framework between RGB-induced and thermal-induced propagation mechanisms is devised to optimize, respectively, two propagation results. In other words, two modalities constantly extract supervision information to help the opposite side to refine propagation result until attaining a stable state. Finally, we integrate two modalities-induced propagation results into a refined saliency map. We compare our model with the state-of-the-art RGB–T and RGB saliency detection algorithms on three benchmark datasets, and experimental results show that the proposed CMCP achieves the significant improvement. [ABSTRACT FROM AUTHOR]

Deep learning methods, with their good performance in semantic representation of different images, have been widely used for saliency detection. Recent saliency detection methods have applied deep learning to obtain high-level features and combined them with hand-crafted low-level features to estimate saliency in the images. However, it is difficult to find the relationship between high-level and low-level features, resulting in incomplete integration framework for saliency detection. In this paper, we novely propose a saliency detection model by integrating high-level and low-level features with joint probability estimation. Firstly, the high-level features from FCN-8S network are used to estimate the probability of each superpixel as foreground or background region. Secondly, low-level features are extracted from each superpixels and clustered via affinity propagation (AP) clustering. The distributions of vectors from different clusters are consequently utilized to calculate the conditional probability of each superpixel as salient object under different assumptions. Thirdly, the joint probability of each superpixel as salient object in foreground or background is computed to compose the saliency map of the whole image. To further improve the uniformity of saliency in the same object region, the structured random forest (SRF) method is used to detect the contour of the image and the saliency of superpixels in homogeneous regions are uniformly merged. The advantage of high-level features in representing semantic regions and that of low-level features in differentiating local details in the image are unified and restrained by the joint probability estimation in the proposed model. Experimental results demonstrate that the proposed method provide better saliency detection performance than the state-of-the-art methods on 5 public databases. [ABSTRACT FROM AUTHOR]

• Blind reconstruction of lung CT image suffers from degradations with unknown models. • Reconstruction and degradation estimation can be obtained in one unified framework. • Residual self-attention module improves learning noise information from CT images. • Using transformer cyclically achieves super-resolution and blur estimation meantime. Background and objective: Computer tomography (CT) imaging technology has played significant roles in the diagnosis and treatment of various lung diseases, but the degradations in CT images usually cause the loss of detailed structural information and interrupt the judgement from clinicians. Therefore, reconstructing noise-free, high-resolution CT images with sharp details from degraded ones is of great importance for the computer-assisted diagnosis (CAD) system. However, current image reconstruction methods suffer from unknown parameters of multiple degradations in actual clinical images. Methods: To solve these problems, we propose a unified framework, so called Posterior Information Learning Network (PILN), for blind reconstruction of lung CT images. The framework consists of two stages: Firstly, a noise level learning (NLL) network is proposed to quantify the Gaussian and artifact noise degradations into different levels. Inception-residual modules are designed to extract multi-scale deep features from the noisy image, and residual self-attention structures are proposed to refine deep features to essential representations of noise. Secondly, by taking the estimated noise levels as prior information, a cyclic collaborative super-resolution (CyCoSR) network is proposed to iteratively reconstruct the high-resolution CT image and estimate the blur kernel. Two convolutional modules are designed based on cross-attention transformer structure, named as Reconstructor and Parser. The high-resolution image is restored from the degraded image by the Reconstructor under the guidance of the predicted blur kernel, while the blur kernel is estimated by the Parser according to the reconstructed image and the degraded one. The NLL and CyCoSR networks are formulated as an end-to-end framework to handle multiple degradations simultaneously. Results: The proposed PILN is applied to the Cancer Imaging Archive (TCIA) dataset and the Lung Nodule Analysis 2016 Challenge (LUNA16) dataset to evaluate its ability in reconstructing lung CT images. Compared with the state-of-the-art image reconstruction algorithms, it can provide high-resolution images with less noise and sharper details with respect to quantitative benchmarks. Conclusions: Extensive experimental results demonstrate that our proposed PILN can achieve better performance on blind reconstruction of lung CT images, providing noise-free, detail-sharp and high-resolution images without knowing the parameters of multiple degradation sources. [ABSTRACT FROM AUTHOR]

Hallucinating a photo-realistic frontal face image from a low-resolution (LR) non-frontal face image is beneficial for a series of face-related applications. However, previous efforts either focus on super-resolving high-resolution (HR) face images from nearly frontal LR counterparts or frontalizing non-frontal HR faces. It is necessary to address all these challenges jointly for real-world face images in unconstrained environment. In this paper, we develop a novel Cross-view Information Interaction and Feedback Network (CVIFNet), which simultaneously handles the non-frontal LR face image super-resolution (SR) and frontalization in a unified framework and interacts them with each other to further improve their performance. Specifically, the CVIFNet is composed of two feedback sub-networks for frontal and profile face images. Considering the reliable correspondence between frontal and non-frontal face images can be crucial and contribute to face hallucination in a different manner, we design a cross-view information interaction module (CVIM) to aggregate HR representations of different views produced by the SR and frontalization processes to generate finer face hallucination results. Besides, since 3D rendered facial priors contain rich hierarchical features, such as low-level (e.g., sharp edge and illumination) and perception level (e.g., identity) information, we design an identity-preserving consistency loss based on 3D rendered facial priors, which can ensure that the high-frequency details of frontal face hallucination result are consistent with the profile. Extensive experiments demonstrate the effectiveness and advancement of CVIFNet. • A novel unified framework, named Cross-view Information Interaction and Feedback Network, is proposed to jointly tackle face SR and face frontalization problems. • A novel feedback mechanism is proposed to provide the high-level reconstruction information in top-down feedback flows through feedback connections. • A novel 3D rendered facial priors based identity-preserving consistency loss is proposed to supervise the reconstruction of facial components spatial information. [ABSTRACT FROM AUTHOR]

Coronavirus Disease 2019 (COVID-19) has spread globally since the first case was reported in December 2019, becoming a world-wide existential health crisis with over 90 million total confirmed cases. Segmentation of lung infection from computed tomography (CT) scans via deep learning method has a great potential in assisting the diagnosis and healthcare for COVID-19. However, current deep learning methods for segmenting infection regions from lung CT images suffer from three problems: (1) Low differentiation of semantic features between the COVID-19 infection regions, other pneumonia regions and normal lung tissues; (2) High variation of visual characteristics between different COVID-19 cases or stages; (3) High difficulty in constraining the irregular boundaries of the COVID-19 infection regions. To solve these problems, a multi-input directional UNet (MID-UNet) is proposed to segment COVID-19 infections in lung CT images. For the input part of the network, we firstly propose an image blurry descriptor to reflect the texture characteristic of the infections. Then the original CT image, the image enhanced by the adaptive histogram equalization, the image filtered by the non-local means filter and the blurry feature map are adopted together as the input of the proposed network. For the structure of the network, we propose the directional convolution block (DCB) which consist of 4 directional convolution kernels. DCBs are applied on the short-cut connections to refine the extracted features before they are transferred to the de-convolution parts. Furthermore, we propose a contour loss based on local curvature histogram then combine it with the binary cross entropy (BCE) loss and the intersection over union (IOU) loss for better segmentation boundary constraint. Experimental results on the COVID-19-CT-Seg dataset demonstrate that our proposed MID-UNet provides superior performance over the state-of-the-art methods on segmenting COVID-19 infections from CT images. • We propose a multidimensional input to reflect the GGOs or infiltration features. • We propose a directional convolution block to represent the fibrotic-streak features. • We propose a novel region contour loss function to restric irregular boundaries. [ABSTRACT FROM AUTHOR]

This paper proposes a new texture enhancement method which uses an image decomposition that allows different visual characteristics of textures to be represented by separate components in contrast with previous methods which either enhance texture indirectly or represent all texture information using a single image component. Our method is intended to be used as a preprocessing step prior to the use of texture-based image segmentation algorithms. Our method uses a modification of morphological component analysis (MCA) which allows texture to be separated into multiple morphological components each representing a different visual characteristic of texture. We select four such texture characteristics and propose new dictionaries to extract these components using MCA. We then propose procedures for modifying each texture component and recombining them to produce a texture-enhanced image. We applied our method as a preprocessing step prior to a number of texture-based segmentation methods and compared the accuracy of the results, finding that our method produced results superior to comparator methods for all segmentation algorithms tested. We also demonstrate by example the main mechanism by which our method produces superior results, namely that it causes the clusters of local texture features of each distinct image texture to mutually diverge within the multidimensional feature space to a vastly superior degree versus the comparator enhancement methods. [ABSTRACT FROM AUTHOR]

Pulmonary nodule detection in chest computed tomography (CT) is of great significance for the early diagnosis of lung cancer. Therefore, it has attracted more and more researchers to propose various computer-assisted pulmonary nodule detection methods. However, these methods still could not provide convincing results because the nodules are easily confused with calcifications, vessels, or other benign lumps. In this paper, we propose a novel deep convolutional neural network (DCNN) framework for detecting pulmonary nodules in the chest CT image. The framework consists of three cascaded networks: First, a U-net network integrating inception structure and dense skip connection is proposed to segment the region of lung parenchyma from the chest CT image. The inception structure is used to replace the first convolution layer for better feature extraction with respect to multiple receptive fields, while the dense skip connection could reuse these features and transfer them through the network. Secondly, a modified U-net network where all the convolution layers are replaced by dilated convolution is proposed to detect the "suspicious nodules" in the image. The dilated convolution can increase the receptive fields to improve the ability of the network in learning global information of the image. Thirdly, a modified U-net adapting multi-scale pooling and multi-resolution convolution connection is proposed to find the true pulmonary nodule in the image with multiple candidate regions. During the detection, the result of the former step is used as the input of the latter step to follow the "coarse-to-fine" detection process. Moreover, the focal loss, perceptual loss and dice loss were used together to replace the cross-entropy loss to solve the problem of imbalance distribution of positive and negative samples. We apply our method on two public datasets to evaluate its ability in pulmonary nodule detection. Experimental results illustrate that the proposed method outperform the state-of-the-art methods with respect to accuracy, sensitivity and specificity. [ABSTRACT FROM AUTHOR]

Computed tomography (CT) imaging technology has been widely used to assist medical diagnosis in recent years. However, noise during the process of imaging, and data compression during the process of storage and transmission always interrupt the image quality, resulting in unreliable performance of the post-processing steps in the computer assisted diagnosis system (CADs), such as medical image segmentation, feature extraction, and medical image classification. Since the degradation of medical images typically appears as noise and low-resolution blurring, in this paper, we propose a uniform deep convolutional neural network (DCNN) framework to handle the de-noising and super-resolution of the CT image at the same time. The framework consists of two steps: Firstly, a dense-inception network integrating an inception structure and dense skip connection is proposed to estimate the noise level. The inception structure is used to extract the noise and blurring features with respect to multiple receptive fields, while the dense skip connection can reuse those extracted features and transfer them across the network. Secondly, a modified residual-dense network combined with joint loss is proposed to reconstruct the high-resolution image with low noise. The inception block is applied on each skip connection of the dense-residual network so that the structure features of the image are transferred through the network more than the noise and blurring features. Moreover, both the perceptual loss and the mean square error (MSE) loss are used to restrain the network, leading to better performance in the reconstruction of image edges and details. Our proposed network integrates the degradation estimation, noise removal, and image super-resolution in one uniform framework to enhance medical image quality. We apply our method to the Cancer Imaging Archive (TCIA) public dataset to evaluate its ability in medical image quality enhancement. The experimental results demonstrate that the proposed method outperforms the state-of-the-art methods on de-noising and super-resolution by providing higher peak signal to noise ratio (PSNR) and structure similarity index (SSIM) values. [ABSTRACT FROM AUTHOR]

Recently, single image super-resolution (SISR) has been widely applied in the fields of multimedia and computer vision communities and obtained remarkable performance. However, most current methods ignore to utilize multi-granularity features of low-resolution (LR) image to further improve the SISR performance. And the channel and spatial features obtained from original LR images are treated equally, resulting in unnecessary computations for abundant uninformative features, thereby hindering the representational ability of super-resolution (SR) models. In this paper, we present a novel Multi-Granularity Pyramid Attention Network (MGPAN) which fully exploits the multi-granularity perception and attention mechanisms to improve the quality of reconstructed images. We design a multi-branch dilated convolution layer with varied kernels corresponding to receptive fields of different sizes to modulate multi-granularity features for adaptively capturing more important information. Moreover, a novel spatial pyramid pooling attention (SPPA) module is constructed to integrate the channel-wise and multi-scale spatial information, which is beneficial to compute the response values from the multi-scale regions of each neuron, and then establish the accurate mapping from low to high dimensional solution space. Besides, for long-short-term information preservation and information flow enhancement, we adopt the short, long, and global skip connection structures to concatenate and fuse the states of each module, which can improve the SR network performance effectively. Extensive experiments on several standard benchmark datasets show that the proposed MGPAN can provide state-of-the-art or even better performance in both quantitative and qualitative measurements. [ABSTRACT FROM AUTHOR]

Eye diseases often affect human health. Accurate detection of the optic disc contour is one of the important steps in diagnosing and treating eye diseases. However, the structure of fundus images is complex, and the optic disc region is often disturbed by blood vessels. Considering that the optic disc is usually a saliency region in fundus images, we propose a weakly-supervised optic disc detection method based on the fully convolution neural network (FCN) combined with the weighted low-rank matrix recovery model (WLRR). Firstly, we extract the low-level features of the fundus image and cluster the pixels using the Simple Linear Iterative Clustering (SLIC) algorithm to generate the feature matrix. Secondly, the top-down semantic prior information provided by FCN and bottom-up background prior information of the optic disc region are used to jointly construct the prior information weighting matrix, which more accurately guides the decomposition of the feature matrix into a sparse matrix representing the optic disc and a low-rank matrix representing the background. Experimental results on the DRISHTI-GS dataset and IDRiD dataset show that our method can segment the optic disc region accurately, and its performance is better than existing weakly-supervised optic disc segmentation methods. Graphical abstract of optic disc segmentation [ABSTRACT FROM AUTHOR]

Glaucoma is a group of eye conditions, which can seriously damage optic nerves because of an elevated intraocular pressure. Nowadays, glaucoma has become one of the principal causes of blindness that results in irreversible visual loss. Early screening and treatment of glaucoma can prevent further progression of optic nerve degeneration effectively. The vertical cup-to-disc ratio (CDR) is a commonly used measurement for the detection of glaucoma, and therefore accurate segmentation of optic disc (OD) and optic cup (OC) regions in retinal fundus images is of great significance. In this paper, we present a prior shape constraint-based two-layer level set method for OD and OC segmentation in fundus images. This method uses two different layers of one level set function to represent the OD and OC boundaries. In this method, the distance regularized term is designed to guarantee that the distance between the OD and OC varies smoothly. By introducing the prior shape constraint term energy, the final segmentation results of OD and OC are always in the shape of approximate ellipses. In addition, the proposed method has the property of dealing with the intensity inhomogeneity of fundus images through the local fitting energy embedded. Experiments on images from the Baidu Research database demonstrate that the proposed method has superior performance in terms of accuracy and effectiveness. [ABSTRACT FROM AUTHOR]

In recent years, large-scale person re-identification has attracted a lot of attention from video surveillance. Usual approaches addressing this task either learn the effective feature embeddings or design the learning architectures to obtain discriminative metrics. Most of them only focus on improving the accuracy of recognition but neglect retrieval efficiency. To improve the accuracy and efficiency of person re-identification simultaneously, an accurate and fast method is proposed based on the bag of visual words (BoVW) model, which has widely been applied in image retrieval. A bag of local features is developed to simplify feature representation for person re-identification. Cross-view dictionary learning is used to eliminate the redundancy of these local features. These local features consist of scale invariant feature transform and local maximal occurrence representation (LOMO) that are invariant in scale and color, respectively. Finally, integrated BoVW histograms are obtained, which encode the images by k-means clustering. Experiments conducted on the CUHK03, Market1501, and MARS datasets show that the proposed method performs favorably against existing approaches. © 2018 SPIE and IS&T [DOI: 10.1117/1.JEI.27.5.053041] [ABSTRACT FROM AUTHOR]

• A center-connected graph convolutional network enhanced with salient image features (SIFE-CGCN) was proposed by integrating the image semantics to improve the recognition performance of similar actions. • A center-connected skeleton topology was developed to enhance the learning capability of the GCN on the potential cooperative dependencies of all joints. • The DTW-based metric was developed to measure the action similarity and build the similar action dataset. The proposed model achieves state-of-the-art performance on three large-scale datasets. Skeleton-based human action recognition has drawn more and more attention due to its easy implementation and stable application in intelligent human-robot interaction. However, most existing studies only used the skeleton data but completely ignored other image semantic information to build action recognition models, which would confuse the recognition of similar actions because of the ambiguity between skeleton data. Here, a center-connected graph convolutional network enhanced with salient image features (SIFE-CGCN) is proposed to address the problem of similar action recognition. First, a center-connected network (CGCN) is constructed to capture the small differences between similar actions through exploring the possible collaboration between all joints. Subsequently, a metric of movement changes is employed to optimally select the salient image from an action video, and then the EfficientNet is used to achieve the action semantic classification of the salient images. Finally, the recognition results of CGCN are strengthened with the classification results of salient images to further improve the recognition accuracy for similar actions. Additionally, a metric is proposed to measure the action similarity with the skeleton data, and then a similar action dataset is built. Extensive experiments on the datasets of similar action and NTU RGB + D 60/120 were conducted to verify the performance of the proposed methods. Experimental results validated the effectiveness of salient image feature enhancement and showed that the proposed SIFE-CGCN achieved the state-of-the-art performance on the similar action and NTU RGB + D 60/120 datasets. [ABSTRACT FROM AUTHOR]

6DoF pose estimation has received much attention in recent years. A key challenge is the difficulty of estimating object pose when the target texture is weak. In this work, we present the cross-modal Transformer (CMT-6D), a Transformer-based network suitable for highly accurate workpiece-level object 6D pose estimation from a single RGBD image. Our main insight is to make the surface texture information of RGB images with the geometric feature information of point clouds complement each other through a cross-modal Transformer, enabling accurate estimation of the pose of weakly textured targets. Specifically, the whole framework consists of two parallel Transformer branches, named Point Transformer and Image Transformer. Both parallel transformer networks use a pyramid structured encoder and a multi-layer perceptron structured decoder to extract geometric features of point clouds and texture features of RGB images, respectively. Then, a cross-modal key query strategy is proposed for information exchange between parallel channels. In addition, at the output representation stage, we design a simple and effective 3D keypoint selection algorithm to solve the problem that keypoints are likely to appear in the non-significant region. Finally, to improve the accuracy of attitude estimation and meet real-time requirements, a lightweight pose iterative network based on target feature regression is proposed to correct the initial attitude estimation error. Extensive experiments demonstrate the effectiveness and superiority of our method on LineMOD, Occlusion LineMOD, T-Less, and YCB-Video datasets. We demonstrate that our method can improve the 6D pose estimation performance by comparing with the state-of-the-art. Ablation research and visualization validate the design of CMT-6D. [ABSTRACT FROM AUTHOR]

Brain tumor segmentation using Magnetic Resonance (MR) Imaging technology plays a significant role in computer-aided brain tumor diagnosis. However, when applying classic segmentation methods, limitations such as inhomogeneous intensity, complex physiological structure and blurred tissues boundaries in brain MR images usually lead to unsatisfactory results. To address these issues, this paper proposes an automatic sparse constrained level set method to realize the brain tumor segmentation in MR images. By studying brain tumor images, this method finds out common characteristics of brain tumors' shape and constructs a sparse representation model. By considering this model as a prior constraint, an energy function based on level set method is constructed. In experiments, the proposed method can achieve an average accuracy of 96.20% for the MR images from the dataset Brats2017 and performs better than the others. With lower false positive rate and stronger robustness, the experimental results show that the proposed method can segment brain tumor from MR image accurately and stably. • Automatically locate brain tumors and generate initial contours in MR images. • Construct a sparse shape constraint and incorporate it into the energy function. • Improve the performance of level set method in segmenting complex brain tumor. [ABSTRACT FROM AUTHOR]

A novel and efficient metric learning strategy for person re‐identification is proposed. Person re‐identification is formulated as a multi‐domain learning problem. The assumption that the feature distributions from different camera views are the same is overthrown in this Letter. ID‐based transfer component analysis (IDB‐TCA) is proposed to learn a shared subspace, in which the differences in the feature distribution between source domain and target domain are significantly reduced. Experimental evaluation on the CUHK01 dataset demonstrates that metric learning with IDB‐TCA embedded outperforms state‐of‐art metric methods for person re‐identification. [ABSTRACT FROM AUTHOR]

Recently, deep-learning based face super-resolution methods have succeeded in hallucinating high-resolution face images from low-resolution inputs. However, since the given face images have tiny resolution and arbitrary characteristics which need to be reconstructed at high magnification factors, existing methods still suffer from large space of possible mapping functions, resulting in the limited performance in producing sharp textures. In this paper, we propose a novel CNN-based Dual Closed-loop Network (DCLNet) to minimize the possible mapping space. To that end, we design two dual learning networks to form the dual closed-loop structure with a primary face super-resolution network, which can provide the primary branch with additional prior constraint to guide the essential facial features restoring. Our work represents the first attempt to introduce multiple dual learning networks into face super-resolution model to constrain the possible mapping space. In addition, a progressive facial prior estimation framework and a new prior-guided feature enhancement module are presented to integrate the facial prior knowledge and guide the face image super-resolution. By generating multiple facial components maps for the activation of essential facial parts, our enhancement module can address the difficulty in learning and integrating strong priors into a face super-resolution model. In this way, the collaboration between the face super-resolution and alignment processes can be enhanced with effect. Extensive experiments are implemented on CelebA and Helen dataset, showing that our proposed method can provide state-of-the-art or even better performance in both quantitative and qualitative measurements. [ABSTRACT FROM AUTHOR]

Recently, single image super-resolution (SISR) has been widely applied in the fields of underwater robot vision and obtained remarkable performance. However, most current methods generally suffered from the problem of a heavy burden on computational resources with large model sizes, which limited their real-world underwater robotic applications. In this paper, we introduce and tackle the super resolution (SR) problem for underwater robot vision and provide an efficient solution for near real-time applications. We present a novel lightweight multi-stage information distillation network, named MSIDN, for better balancing performance against applicability, which aggregates the local distilled features from different stages for more powerful feature representation. Moreover, a novel recursive residual feature distillation (RRFD) module is constructed to progressively extract useful features with a modest number of parameters in each stage. We also propose a channel interaction & distillation (CI&D) module that employs channel split operation on the preceding features to produce two-part features and utilizes the inter channel-wise interaction information between them to generate the distilled features, which can effectively extract the useful information of current stage without extra parameters. Besides, we present USR-2K dataset, a collection of over 1.6K samples for large-scale underwater image SR training, and a testset with an additional 400 samples for benchmark evaluation. Extensive experiments on several standard benchmark datasets show that the proposed MSIDN can provide state-of-the-art or even better performance in both quantitative and qualitative measurements. • A information distillation network is proposed for underwater image super-resolution. • A recursive residual module is constructed for informative features distillation. • A channel-wise interaction mechanism is proposed to generate distilled features. • A novel underwater image super-resolution dataset is presented. [ABSTRACT FROM AUTHOR]