Search

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Recent research on image super-resolution (SR) task has greatly progressed with the development of convolutional neural networks (CNNs). Most previous studies with single-scale feature enhance expressiveness by increasing network depth. However, most of them do not adequately extract and utilize multi-scale features. In this paper, we propose a novel residual aggregation U-shaped network (RAU), which fully utilizes multi-scale features to help reconstruct high-quality images. First, we use progressive downsampling structure to obtain multi-scale features and capture context information, and use progressive upsampling structure to fuse multi-scale features and fill detail texture. Second, we introduce auxiliary supervision in the middle layer to provide additional regularization and accelerate the convergence speed. Third, we propose a lightweight model for our network, and we replace the traditional convolution with the Ghost module in multiple locations of network. Extensive experiments on the challenge datasets confirmed the effectiveness of the proposed network. Our algorithm can restore high-quality high-resolution (HR) images and outperform other methods by a large margin. [ABSTRACT FROM AUTHOR]

Despite notable advancements achieved on Hyperspectral (HS) pansharpening tasks through deep learning techniques, previous methods are inherently constrained by convolution or self-attention intrinsic defects, leading to limited performance. In this paper, we proposed an Attention-Interactive Dual-Branch Convolutional Neural Network (AIDB-Net) for HS pansharpening. Our model purely consists of convolutional layers and simultaneously inherits the strengths of both convolution and self-attention, especially the modeling of short- and long-range dependencies. Specially, we first extract, tokenize, and align the hyperspectral image (HSI) and panchromatic image (PAN) by Overlapping Patch Embedding Blocks. Then, we specialize a novel Spectral-Spatial Interactive Attention which is able to globally interact and fuse the cross-modality features. The resultant token-global similarity scores can guide the refinement and renewal of the textural details and spectral characteristics within HSI features. By deeply combined these two paradigms, our AIDB-Net significantly improve the pansharpening performance. Moreover, with the acceleration by the convolution inductive bias, our interactive attention can be trained without large scale dataset and achieves competitive time cost with its counterparts. Compared with the state-of-the-art methods, our AIDB-Net makes 5.2%, 3.1%, and 2.2% improvement on PSNR metric on three public datasets, respectively. Comprehensive experiments quantitatively and qualitatively demonstrate the effectiveness and superiority of our AIDB-Net. [ABSTRACT FROM AUTHOR]

Copyright of Cyber Security & Data Governance is the property of Editorial Office of Information Technology & Network Security and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Recently, deep learning methods have been widely applied to single image super-resolution reconstruction tasks and achieved great improvement in both quantitative and qualitative evaluations. However, most of the existing convolutional neural network-based methods commonly reduce the number of layers or channels to obtain lightweight model. These strategies may reduce the representation ability of informative features and degrade the network performance. To address this issue, we propose a sparse lightweight attention network (SLAN), a novel SISR algorithm to keep informative features between layers. Specially, a sparse attention feature fusion module with lightweight attention and sparse extracting modules is developed to expand feature extracting receptive field and enhance the informative feature extracting ability. To take advantage of the multi-level features and keep fewer multi-adds, cross fusion is used and demonstrated usefully. Extensive experimental results on public datasets demonstrate the superior performance of our proposed SLAN. The average PSNR(dB)/SSIM values of SLAN are about 0.04/0.0004, 0.06/0.0009, and 0.09/0.0018 larger than the competitors under scaling factors of × 2, × 3 and × 4, respectively. Our SLAN benefits from its model size and low computation cost and may be deployed on mobile platforms. [ABSTRACT FROM AUTHOR]

The feature extraction ability of some existing super-resolution networks is relatively weak. And these networks do not further process the extracted features. These problems make the networks often show limited performance, resulting in blurred details and unclear edges of the reconstructed images. Therefore, further research is needed to resolve these problems. In this paper, we propose a novel dual-branch feature learning super resolution network (DBSR). The core of DBSR is the dual-branch feature learning (DB) block. In order to enhance the ability of feature extraction, the block adopts a multi-level and dual-branch structure. At the same time, some components for further processing features are introduced in each branch to maximize the learning ability of the block. The reconstructed images of DBSR are clearer than other networks in line and contour, and better results are obtained in peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). For example, when the scaling factor is 2, the PSNR/SSIM on each test dataset is 38.25dB/0.9614, 34.11dB/0.9218, 32.35dB/0.9018, 32.94dB/0.9354 and 39.46dB/0.9783 respectively. The experimental results demonstrate that DBSR achieves better accuracy and visually pleasing than the current excellent methods. [ABSTRACT FROM AUTHOR]

Single image super-resolution (SISR) is a fundamental image processing task, which aims to generate a high-resolution (HR) image from its low-resolution (LR) counterpart. Deep convolution neural networks (CNNs) have significantly improved the performance of SISR, and dominated the current research on SISR techniques. However, the performance improvement depends heavily on the size of the networks. In general, the deeper the networks, the better the performance, which limits their use to devices with limited computing and memory resources. The challenge is to find the optimal balance between network model complexity and SISR performance. In this paper, we propose a new lightweight SISR algorithm based on CNNs, which uses multi-path progressive feature fusion and attention mechanism. Our main contributions are as follows: (1) We propose a multi-path progressive feature fusion block (PFF), which can use the feature from the previous path to gradually guide the feature learning of the next path step by step in multiple paths. (2) We propose a multi-path feature attention mechanism (FAM), which can adaptively weigh the multi-path feature channels to be concatenated, improve the utilization of feature information and feature representation capability. The experimental results show that whether it is an objective measurement or a subjective measurement, our method is better than other similar state of the art methods, and has a better model complexity and performance balance. [ABSTRACT FROM AUTHOR]

Image super-resolution (SR) reconstruction technology can improve the quality of low-resolution (LR) images. There are many available deep learning networks different from traditional machine learning algorithms. However, these networks are usually prone to poor performance on complex computation, vanishing gradients, and loss of useful information. In this work, we propose a sub-pixel convolutional neural network (SPCNN) for image SR reconstruction. First, to reduce the strong correlation, the RGB mode was translated into YCbCr mode, and the Y channel data was chosen as the input LR image. Meanwhile, the LR image was chosen as the network input to reduce computation instead of the interpolation reconstructed image as used in the super-resolution convolutional neural network (SRCNN). Then, two convolution layers were built to obtain more features, and four non-linear mapping layers were used to achieve different level features. Furthermore, the residual network was introduced to transfer the feature information from the lower layer to the higher layer to avoid the gradient explosion or vanishing gradient phenomenon. Finally, the sub-pixel convolution layer based on up-sampling was designed to reduce the reconstruction time. Experiments on three different data sets proved that the proposed SPCNN performs superiorly to the Bicubic, sparsity constraint super-resolution (SCSR), anchored neighborhood regression (ANR), and SRCNN methods on reconstruction precision and time consumption. [ABSTRACT FROM AUTHOR]

Despite notable advancements achieved on Hyperspectral (HS) pansharpening tasks through deep learning techniques, previous methods are inherently constrained by convolution or self-attention intrinsic defects, leading to limited performance. In this paper, we proposed an Attention-Interactive Dual-Branch Convolutional Neural Network (AIDB-Net) for HS pansharpening. Our model purely consists of convolutional layers and simultaneously inherits the strengths of both convolution and self-attention, especially the modeling of short- and long-range dependencies. Specially, we first extract, tokenize, and align the hyperspectral image (HSI) and panchromatic image (PAN) by Overlapping Patch Embedding Blocks. Then, we specialize a novel Spectral-Spatial Interactive Attention which is able to globally interact and fuse the cross-modality features. The resultant token-global similarity scores can guide the refinement and renewal of the textural details and spectral characteristics within HSI features. By deeply combined these two paradigms, our AIDB-Net significantly improve the pansharpening performance. Moreover, with the acceleration by the convolution inductive bias, our interactive attention can be trained without large scale dataset and achieves competitive time cost with its counterparts. Compared with the state-of-the-art methods, our AIDB-Net makes 5.2%, 3.1%, and 2.2% improvement on PSNR metric on three public datasets, respectively. Comprehensive experiments quantitatively and qualitatively demonstrate the effectiveness and superiority of our AIDB-Net.

Copyright of Journal of Computer-Aided Design & Computer Graphics / Jisuanji Fuzhu Sheji Yu Tuxingxue Xuebao is the property of Gai Kan Bian Wei Hui and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Sentinel-2 satellite remote sensing images have been widely used in various fields, such as change detection and resource monitoring. However, Sentinel-2 provides multispectral bands with inconsistent spatial resolutions (i.e., 60 m for three bands, 20 m for six bands, and 10 m for four bands), which has greatly limited the application values, especially for cooperative analysis or application on different bands at a unified resolution. In this article, we proposed a multistage progressive interactive fusion network to generate all 10-m high-resolution bands. Specifically, a refined multistage spatial resolution enhancement model in a multistage way is developed to progressively improve the low-resolution bands to preserve the spectral information of the enhanced bands. Moreover, an information interaction module is proposed for the three branches of high- (10 m), medium- (20 m), and low-resolution (60 m) bands to achieve effective information interaction. The experimental results show that our method is superior to other existing state-of-the-art methods, and it can be applied to the reconstruction of the high-resolution vegetation index.

High-fidelity information, such as 4K quality videos and photographs, is increasing as high-speed internet access becomes more widespread and less expensive. Even though camera sensors’ performance is constantly improving, artificially enhanced photos and videos created by intelligent image processing algorithms have significantly improved image fidelity in recent years. Single image super-resolution is a class of algorithms that produces a high-resolution image from a given low-resolution image. Since the advent of deep learning a decade ago, this field has made significant strides. This paper presents a comprehensive review of the deep learning assisted single image super-resolution domain including generative adversarial network (GAN) models that discusses the prominent architectures, models used, and their merits and demerits. The reason behind covering the GAN models is that it is been known to perform better than the conventional deep learning methods given the resources and the time. For real-world applications with noise and other issues that can cause low-fidelity super resolution (SR) images, we examine another solution based on GAN model. This GAN model-based technique popularly known as blind super resolution is more resilient. We examined the various super-resolution techniques by varying image scaling factors (i.e., 2x, 3x, 4x) to measure PSNR and SSIM metrics for the different datasets. PSNR across the different datasets covered in the experimental Section shows an average of 14–17 % decrease in the score as we move up the image resolution scale from 2x to 4x. This is observed across all the datasets and for every model mentioned in the experimental Section of the paper. The results also show that blind super-resolution outperforms the conventional deep learning methods and the more complex GAN models. GAN models are complex and preferred when the upscale factor is high, while residual and dense models are recommended for smaller upscaling factors. This paper also discusses the applications of image super-resolution, and finally, the paper is concluded with challenges and future directions.

Acute lower respiratory infection is a leading cause of death in developing countries. Hence, progress has been made for early detection and treatment. There is still a need for improved diagnostic and therapeutic strategies, particularly in resource-limited settings. Chest X-ray and computed tomography (CT) have the potential to serve as effective screening tools for lower respiratory infections, but the use of artificial intelligence (AI) in these areas is limited. To address this gap, we present a computer-aided diagnostic system for chest X-ray and CT images of several common pulmonary diseases, including COVID-19, viral pneumonia, bacterial pneumonia, tuberculosis, lung opacity, and various types of carcinoma. The proposed system depends on super-resolution (SR) techniques to enhance image details. Deep learning (DL) techniques are used for both SR reconstruction and classification, with the InceptionResNetv2 model used as a feature extractor in conjunction with a multi-class support vector machine (MCSVM) classifier. In this paper, we compare the proposed model performance to those of other classification models, such as Resnet101 and Inceptionv3, and evaluate the effectiveness of using both softmax and MCSVM classifiers. The proposed system was tested on three publicly available datasets of CT and X-ray images and it achieved a classification accuracy of 98.028% using a combination of SR and InceptionResNetv2. Overall, our system has the potential to serve as a valuable screening tool for lower respiratory disorders and assist clinicians in interpreting chest X-ray and CT images. In resource-limited settings, it can also provide a valuable diagnostic support. [ABSTRACT FROM AUTHOR]

Due to its widespread usage in many applications, numerous deep learning algorithms have been proposed to overcome Light Field's trade-off (LF). The sensor's low resolution limits angular and spatial resolution, which causes this trade-off. The proposed method should be able to model the non-local properties of the 4D LF data fully to mitigate this problem. Therefore, this paper proposes a different approach to increase spatial and angular information interaction for LF image super-resolution (SR). We achieved this by processing the LF Sub-Aperture Images (SAI) independently to extract the spatial information and the LF Macro-Pixel Image (MPI) to extract the angular information. The MPI or Lenslet LF image is characterized by its ability to integrate more complementary information between different viewpoints (SAIs). In particular, we extract initial features and then process MAI and SAIs alternately to incorporate angular and spatial information. Finally, the interacted features are added to the initial extracted features to reconstruct the final output. We trained the proposed network to minimize the sum of absolute errors between low-resolution (LR) input and high-resolution (HR) output images. Experimental results prove the high performance of our proposed method over the state-of-the-art methods on LFSR for small baseline LF images. [ABSTRACT FROM AUTHOR]

As convolutional neural networks (CNNs) have been commonly applied to ill-posed single image super-resolution (SISR) task, most previous CNN-based methods made significant progress in terms of both high signal-to-noise ratios (PSNR) and structural similarity (SSIM). However, with the layers in those networks going deeper and deeper, they require more and more computing power, fail to consider distilling the feature maps. In this paper, we propose a lightweight global-locally connected distillation network, GLCDNet. Specifically, we propose a wide activation shrink-expand convolutional block whose filter channels will first shrink then expand to aggregate more information. This information will concatenate with feature maps of the previous blocks to further explore shallow information. Thus, the block will exploit statistics within most feature channels while refining useful information of features. Furthermore, together with the global-local connection method, our network is robust to benchmark datasets with high processing speed. Comparative results demonstrate that our GLCDNet achieves superior performance while keeping the parameters and speed balanced. [ABSTRACT FROM AUTHOR]

• A lightweight attention mechanism based on strip convolution in parallel called MSA. • We generalize MSA to other networks, and outperform them in performance. • A lightweight network for image SR called multi-scale strip-shaped convolution attention network. • MSAN achieves SOTA performance with fewer model parameter counts. Lightweight convolutional neural networks for Single Image Super-Resolution (SISR) have exhibited remarkable performance improvements in recent years. These models achieve excellent performance by relying on attention mechanisms that incorporate square-shaped convolutions to enhance feature representation. However, these approaches still suffer from redundancy which comes from square-shaped convolutional kernels and overlooks the utilization of multi-scale information. In this paper, we propose a novel attention mechanism called Multi-scale Strip-shaped convolution Attention (MSA), which utilizes three sets of differently sized depth-wise separable stripe convolution kernels in parallel to replace the redundant square-shaped convolution attention and extract multi-scale features. We also generalize MSA to other lightweight neural network models, and experimental results show that MSA outperforms other convolutional based attention mechanisms. Building upon MSA, we propose an Efficient Feature Extraction Block (EFEB), a lightweight block for SISR. Finally, based on EFEB, we propose a lightweight image super-resolution neural network named Multi-scale Strip-shaped convolution Attention Network (MSAN). Experiments demonstrate that MSAN outperforms existing state-of-the-art lightweight SR methods with fewer parameters and lower computational complexity. [ABSTRACT FROM AUTHOR]

Deep convolutional neural networks (DCNNs) have been employed in single image super-resolution (SISR) tasks and achieved considerable performance. However, most of the existing approaches simply deepen and widen the network structure to increase the receptive field of the convolution kernel and adopt the equal processing methods in the channel and the spatial dimension, which brings about the rise of computational complexity and the loss of important information. To address these issues, we propose dual-attention guided multi-scale network (DAMSN) for SISR, which intends to obtain better performance with relatively fewer parameters. Exactly, we first introduce a non-local residual block (NLRB) to capture the abundant information of both low-level features and high-level features. Furthermore, we design a multi-scale dual attention block (MSDAB) to diminish the redundant information and extract features from different scales powerfully. Inside the MSDAB, dual attention module (DAM) focuses on high-frequency information extraction by combining the channel and spatial mechanism. Followed by DAM, multi-scale residual module (MSRM) is utilized to learn the informative feature via multi-sized convolution kernels adaptively. Extensive experiments demonstrate that our proposed DAMSN is superior to other state-of-the-art approaches in terms of both quantitative metrics and subjective visual quality. [ABSTRACT FROM AUTHOR]

In this paper, we propose a tree-like branching network for image super-resolution. Specifically, the network consists of information divide-and-conquer groups (IDCG) to preserve the low-frequency structure of images as well as restore high-frequency information. The kernel of IDCG contains several essential components: (a) a simple attention module and an effective residual attention module to maintain low-frequency structures and restore high-frequency information, (b) a novel local merge cell alleviates information redundancy that flexibly and adaptively fuses multiple informative features from different states. Lastly, a multi-scale aggregation unit is designed to improve the final reconstruction. Through a series of experiments, we prove that our method is more effective than previous state-of-the-art results in both quantitative and qualitative evaluation. [ABSTRACT FROM AUTHOR]

Super-resolution (SR) and image enhancement (IE) are two common and important tasks in image processing. Due to poor equipment, illumination, and photographic skills, many low-resolution and low-quality (in terms of color, and contrasts) photos are captured in our daily life. To make the photos more visually pleasing, we usually enlarge and retouch them in two separate steps. In this paper, we propose a new Joint Image Super-resolution and Enhancement Network (JSENet), which is arguably the first end-to-end method based on deep learning for joint SR and IE. Compared with simply conducting these two tasks in parallel or sequentially, JSENet employs the bilateral learning framework to integrate the two tasks together seamlessly. Meanwhile, two lightweight modules are designed for restoring details and generating color transformation coefficients respectively, which ensures that our JSENet can be deployed in real-world applications. Extensive experiments are conducted to demonstrate that JSENet outperforms the state-of-the-art SR-IE methods. [ABSTRACT FROM AUTHOR]

Acute lower respiratory infection is a leading cause of death in developing countries. Hence, progress has been made for early detection and treatment. There is still a need for improved diagnostic and therapeutic strategies, particularly in resource-limited settings. Chest X-ray and computed tomography (CT) have the potential to serve as effective screening tools for lower respiratory infections, but the use of artificial intelligence (AI) in these areas is limited. To address this gap, we present a computer-aided diagnostic system for chest X-ray and CT images of several common pulmonary diseases, including COVID-19, viral pneumonia, bacterial pneumonia, tuberculosis, lung opacity, and various types of carcinoma. The proposed system depends on super-resolution (SR) techniques to enhance image details. Deep learning (DL) techniques are used for both SR reconstruction and classification, with the InceptionResNetv2 model used as a feature extractor in conjunction with a multi-class support vector machine (MCSVM) classifier. In this paper, we compare the proposed model performance to those of other classification models, such as Resnet101 and Inceptionv3, and evaluate the effectiveness of using both softmax and MCSVM classifiers. The proposed system was tested on three publicly available datasets of CT and X-ray images and it achieved a classification accuracy of 98.028% using a combination of SR and InceptionResNetv2. Overall, our system has the potential to serve as a valuable screening tool for lower respiratory disorders and assist clinicians in interpreting chest X-ray and CT images. In resource-limited settings, it can also provide a valuable diagnostic support.

With the development of deep learning technology, a variety of image SR approaches based on convolutional neural network (CNN) are developed to learn the mapping from low resolution (LR) to high resolution (HR). However, the feature information from raw images could not be distinguished very clearly by most of these existing methods, resulting in declining performance. In order to achieve a good image resolution, a very deep network is often used to integrate all the feature information from LR image. Obviously, deep network without fully exploring the available information achieves a very large computation complexity but cannot always ensure high image quality. To address these problems, a stratified attention dense network (SADN) is proposed in this paper to reconstruct higher quality HR images. In SADN, a stratified dense group (SDG) architecture is proposed to fully explore the feature information in LR images, including local and global information. Particularly, the attention dense module (ADM) is proposed to distinguish the extracted feature information so as to enhance the discrimination of network. The extensive experiments on benchmark datasets verify the effectiveness of the proposed method. Comparison with other state-of-the-art methods shows the superiority of the proposed SADN. [ABSTRACT FROM AUTHOR]

Image Super-Resolution (SR) provides a promising technique to enhance the image quality of low-resolution sensors for a wide range of optical applications. It is noted that the costs for capturing high-resolution images in various spectral ranges are significantly different, thus it is reasonable to utilize low-cost channel images (e.g., visible/depth images) as guidance to boost the accuracy of SR results of the expensive channel (e.g., thermal images) significantly. In this paper, we attempt to leverage complementary information from low-cost channels (visible/depth) to boost image quality of an expensive channel (thermal) using fewer parameters. To this end, we first build a multi-modal imaging system and present an effective method to generate pixel-wise aligned visible and thermal images via virtual 3D viewpoint rendering. Then, we design a feature-level multispectral fusion residual network model to perform high-accuracy SR of thermal images by adaptively integrating co-occurrence features presented in multispectral images. Experimental results demonstrate that this novel approach can effectively alleviate the ill-posed inverse problem of image SR by taking into account complementary information from an additional low-cost channel, significantly outperforming state-of-the-art SR approaches in terms of both accuracy and efficiency. [ABSTRACT FROM AUTHOR]

Image super-resolution reconstruction is the process of using low-resolution images to reconstruct the corresponding high-resolution images. At present, image super-resolution technology has been successfully applied in the fields of computer vision and image processing. In recent years, due to deep learning's ability of self-learning from a large amount of data, it has been widely used in the field of image super-resolution. This article introduces the background of image super-resolution reconstruction, and summarizes the deep learning based image super-resolution model in detail, and then elaborates the image super-resolution technology in satellite remote sensing images, medical imaging, video surveillance, and industrial inspection tasks application. Finally, this article summarizes the current research status and future development directions of image super-resolution algorithms. [ABSTRACT FROM AUTHOR]

The digital elevation model (DEM) is an important basic data tool applied in geoscience applications. Because of its high cost and long development cycle of enhancing hardware performance, designing the related models and algorithms to improve the resolution of DEM is of considerable significance. At present, there is little research on DEM super-resolution based on deep learning, and the results of the reconstructed DEMs obtained by existing methods are inaccurate. Therefore, deepening of the network layers is utilized to improve the accuracy of a reconstructed DEM. This paper designs a neutral network model with 30 convolutional layers to learn the feature mapping relationship between a low- and high-resolution DEM. To avoid the problem of network degradation caused by increasing the number of convolutional layers, residual learning is introduced to accelerate the convergence speed of the model, thereby preferably realizing the DEM super-resolution process. The results show that DEM super-resolution based on a deep residual network is better than that obtained using a neural network with fewer convolutional layers, and the reconstructed result of the DEM based on a deep residual network is remarkably improved in terms of the peak signal to noise ratio and visual effect.

Deep convolutional neural networks have recently achieved dramatic success in super-resolution (SR) performance in the past few years. However, the parameters of the mapping functions of these networks require an external dataset for training. In this paper, we propose a convolutional network for image super-resolution reconstruction that can be trained using an internal dataset constructed using a single image. The proposed single image convolutional neural network (SICNN) is designed with two branches. First, a large scale-feature branch trains the feature mappings that are from the low resolution (LR) image patches to the high- resolution image (HR) patches. The LR image patches are the enlarged image patches via bicubic interpolation. Second, the small scale-feature branch trains the feature mappings that are from the down-sampling image patches to the enlarged image patches. In contrast to the existing SR networks, the SICNN enjoys two desirable properties: 1) it does not require external datasets to conduct training, and 2) it enlarges an SR image at an arbitrary scale while restoring the clear edges and textures. The results of evaluations on a wide variety of images show that the proposed SICNN achieves advantages over the state-of-the-art methods in terms of both numerical results and visual quality.

Due to its widespread usage in many applications, numerous deep learning algorithms have been proposed to overcome Light Field’s trade-off (LF). The sensor’s low resolution limits angular and spatial resolution, which causes this trade-off. The proposed method should be able to model the non-local properties of the 4D LF data fully to mitigate this problem. Therefore, this paper proposes a different approach to increase spatial and angular information interaction for LF image super-resolution (SR). We achieved this by processing the LF Sub-Aperture Images (SAI) independently to extract the spatial information and the LF Macro-Pixel Image (MPI) to extract the angular information. The MPI or Lenslet LF image is characterized by its ability to integrate more complementary information between different viewpoints (SAIs). In particular, we extract initial features and then process MAI and SAIs alternately to incorporate angular and spatial information. Finally, the interacted features are added to the initial extracted features to reconstruct the final output. We trained the proposed network to minimize the sum of absolute errors between low-resolution (LR) input and high-resolution (HR) output images. Experimental results prove the high performance of our proposed method over the state-of-the-art methods on LFSR for small baseline LF images.

In this article, we propose a novel method for image restoration in sensor network. Deep learning based image Super-Resolution (SR) is such process which takes low-resolution image as input and outputs high-resolution one by an end-to-end trainable architecture. It has shown rapid development in this field by constantly deepening and widening the SR model or combining with other specially designed structures. Recently non-local attention mechanism is proposed and introduced into SR field which focuses on the self-similarity of single feature or difference between channels, but those methods don’t use the mechanism to explore the relationship between features in different layers. In this article, we design a Back Projection Attention Module to learn the cross-relation between different types of image degradation and high-resolution components. Then we construct our network by stacking the iterative up-sampling and down-sampling process and concatenating all outputs of up-sampling process to finally improve the ability of characteristic expression. Extend experiments on four commonly used datasets show the proposed model can improve the performance in quantitative and qualitative measurements. [ABSTRACT FROM AUTHOR]

In this paper, we present an efficient method for single image super-resolution based on the dense network. First, considering that the dense architectures reuse all the previous features in each layer and cause a huge memory overhead, we simplify the dense architecture and only reuse the hierarchical features in the reconstruct layer; therefore, the redundancy in dense architecture is reduced. Subsequently, since the information of the degraded low-resolution image is much less than the potential high-resolution image, we propose a non-degenerate layer to address the degeneracy problem caused by the loss of input information. In the non-degenerate layer, we introduce a skip connection to the linear transformation to eliminate the singularity and utilize an invertible nonlinear activation function to avoid the dead zone. In addition, the direct reconstruction scheme is adopted for efficient models, and the structural similarity is utilized for better human perception. The experiments show promising results in terms of quantitative and qualitative results, which indicates that our method is effective and superior.

Deep convolutional neural networks have significantly improved the peak signal-to-noise ratio of Super-Resolution (SR). However, image viewer applications commonly allow users to zoom the images to arbitrary magnification scales, thus far imposing a large number of required training scales at a tremendous computational cost. To obtain a more computationally efficient model for arbitrary-scale SR, this paper employs a Laplacian pyramid method to reconstruct any-scale high-resolution (HR) images using the high-frequency image details in a Laplacian Frequency Representation. For SR of small-scales (between 1 and 2), images are constructed by interpolation from a sparse set of precalculated Laplacian pyramid levels. SR of larger scales is computed by recursion from small scales, which significantly reduces the computational cost. For a full comparison, fixed- and any-scale experiments are conducted using various benchmarks. At fixed scales, ASDN outperforms predefined upsampling methods (e.g., SRCNN, VDSR, DRRN) by about 1 dB in PSNR. At any-scale, ASDN generally exceeds Meta-SR on many scales. [ABSTRACT FROM AUTHOR]

Super-resolution is generally defined as a process to obtain high-resolution images form inputs of low-resolution observations, which has attracted quantity of attention from researchers of image-processing community. In this paper, we aim to analyze, compare, and contrast technical problems, methods, and the performance of super-resolution research, especially real-time super-resolution methods based on deep learning structures. Specifically, we first summarize fundamental problems, perform algorithm categorization, and analyze possible application scenarios that should be considered. Since increasing attention has been drawn in utilizing convolutional neural networks (CNN) or generative adversarial networks (GAN) to predict high-frequency details lost in low- resolution images, we provide a general overview on background technologies and pay special attention to super-resolution methods built on deep learning architectures for real-time super-resolution, which not only produce desirable reconstruction results, but also enlarge possible application scenarios of super resolution to systems like cell phones, drones, and embedding systems. Afterwards, benchmark datasets with descriptions are enumerated, and performance of most representative super-resolution approaches is provided to offer a fair and comparative view on performance of current approaches. Finally, we conclude the paper and suggest ways to improve usage of deep learning methods on real-time image super-resolution. [ABSTRACT FROM AUTHOR]

A novel super-resolution (SR) method is proposed in this paper to reconstruct high-resolution (HR) remote sensing images. Different scenes of remote sensing images have great disparities in structural complexity. Nevertheless, most existing SR methods ignore these differences, which increases the difficulty to train an SR network. Therefore, we first propose a preclassification strategy and adopt different SR networks to process the remote sensing images with different structural complexity. Furthermore, the main edge of low-resolution images are extracted as the shallow features and fused with the deep features extracted by the network to solve the blurry edge problem in remote sensing images. Finally, an edge loss function and a cycle consistent loss function are added to guide the training process to keep the edge details and main structures in a reconstructed image. A large number of comparative experiments on two typical remote sensing images datasets (WHURS and AID) illustrate that our approach achieves better performance than state-of-the-art approaches in both quantitative indicators and visual qualities. The peak signal-to-noise ratio (PSNR) value and the structural similarity (SSIM) value using the proposed method are improved by 0.5353 dB and 0.0262, respectively, over the average values of five typical deep learning methods on the ×4 AID testing set. Our method obtains satisfactory reconstructed images for the subsequent applications based on HR remote sensing images.

Abstract Real-time image and video processing is a challenging problem in smart surveillance applications. It is necessary to trade off between high frame rate and high resolution to meet the limited bandwidth requirement in many specific applications. Thus, image super-resolution become one commonly used techniques in surveillance platform. The existing image super-resolution methods have demonstrated that making full use of image prior can improve the algorithm performance. However, the previous deep-learning-based image super-resolution methods rarely take image prior into account. Therefore, how to make full use of image prior is one of the unsolved problems for deep-network-based single image super-resolution methods. In this paper, we establish the relationship between the traditional sparse-representation-based single-image super-resolution methods and the deep-learning-based ones and use transfer learning to make our proposed deep network take the image prior into account. Another unresolved problem of the deep-learning-based single-image super-resolution method is how to avoid neurons compromise to different image contents. In this paper, the image patches are anchored to the dictionary atoms to group into various categories. As a result, each neuron will work on the same types of image patches that have similar details, which makes the network more accurate to recover high-frequency details. By solving these two problems, we propose an anchored neighborhood deep network for single-image super-resolution. Experimental results show that our proposed method outperforms many state-of-the-art single-image super-resolution methods.

A new multi-scale deep learning (MDL) framework is proposed and exploited for conducting image interpolation in this paper. The core of the framework is a seeding network that needs to be designed for the targeted task. For image interpolation, a novel attention-aware inception network (AIN) is developed as the seeding network; it has two key stages: 1) feature extraction based on the low-resolution input image; and 2) feature-to-image mapping to enlarge image’s size or resolution. Note that the designed seeding network, AIN, needs to be trained with a matched training dataset at each scale. For that, multi-scale image patches are generated using our proposed pyramid cut, which outperforms the conventional image pyramid method by completely avoiding aliasing issue. After training, the trained AINs are then combined for processing the input image in the testing stage. Extensive experimental simulation results obtained from seven image datasets (comprising 359 images in total) have clearly shown that the proposed MAIN consistently delivers highly accurate interpolated images. [ABSTRACT FROM AUTHOR]

We propose a novel dual-domain convolutional neural network framework to improve structural information of routine 3 T images. We introduce a parameter-efficient butterfly network that involves two complementary domains: a spatial domain and a frequency domain. The butterfly network allows the interaction of these two domains in learning the complex mapping from 3 T to 7 T images. We verified the efficacy of the dual-domain strategy and butterfly network using 3 T and 7 T image pairs. Experimental results demonstrate that the proposed framework generates synthetic 7 T-like images and achieves performance superior to state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Image & Graphics is the property of Editorial Office of Journal of Image & Graphics and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

• An effective contrast learning method is proposed to distinguish different degradation information of a target image. • The high-order channel attention block (HCA) is introduced. • Covariance normalization is introduced to improve the ability of feature representation. Blind super-resolution of images based on contrast learning has achieved better performance, which can distinguish between different degraded information. However, there is no significant improvement in the processing of image super-resolution. To further improve the performance of image blind super-resolution, a new degenerate higher-order attention network (DHAN) is designed for image super-resolution, in which a higher-order channel attention mechanism is introduced to the framework. Meanwhile, it also introduces normalized variance and global covariance pooling, which can provide higher-order feature information and more discriminative feature information. Comparing the experimental results, it can be found that the proposed higher-order attention network (DHAN) is able to distinguish various complex degradation patterns and thus obtain accurate degradation information. Numerous experiments have shown that higher-order attention network (DHAN) achieves good visual results in the image blind super-resolution task. [Display omitted] [ABSTRACT FROM AUTHOR]

Convolutional neural network (CNN) based methods have recently achieved extraordinary performance in single image super-resolution (SISR) tasks. However, most existing CNN-based approaches increase the model's depth by stacking massive kernel convolutions, bringing expensive computational costs and limiting their application in mobile devices with limited resources. Furthermore, large kernel convolutions are rarely used in lightweight super-resolution designs. To alleviate the above problems, we propose a multi-scale convolutional attention network (MCAN), a lightweight and efficient network for SISR. Specifically, a multi-scale convolutional attention (MCA) is designed to aggregate the spatial information of different large receptive fields. Since the contextual information of the image has a strong local correlation, we design a local feature enhancement unit (LFEU) to further enhance the local feature extraction. Extensive experimental results illustrate that our proposed MCAN can achieve better performance with lower model complexity compared with other state-of-the-art lightweight methods. • A lightweight and efficient network for single image super-resolution. • Exploring the use of large kernel convolutions in lightweight super-resolution. • Using depth-wise asymmetric convolution to reduce redundant computations. • Adopting a visual transformer-style design to achieve an excellent feature extraction capability. [ABSTRACT FROM AUTHOR]

The rapid worldwide spread of Coronavirus Disease 2019 (COVID-19) has resulted in a global pandemic. Correct facemask wearing is valuable for infectious disease control, but the effectiveness of facemasks has been diminished, mostly due to improper wearing. However, there have not been any published reports on the automatic identification of facemask-wearing conditions. In this study, we develop a new facemask-wearing condition identification method by combining image super-resolution and classification networks (SRCNet), which quantifies a three-category classification problem based on unconstrained 2D facial images. The proposed algorithm contains four main steps: Image pre-processing, facial detection and cropping, image super-resolution, and facemask-wearing condition identification. Our method was trained and evaluated on the public dataset Medical Masks Dataset containing 3835 images with 671 images of no facemask-wearing, 134 images of incorrect facemask-wearing, and 3030 images of correct facemask-wearing. Finally, the proposed SRCNet achieved 98.70% accuracy and outperformed traditional end-to-end image classification methods using deep learning without image super-resolution by over 1.5% in kappa. Our findings indicate that the proposed SRCNet can achieve high-accuracy identification of facemask-wearing conditions, thus having potential applications in epidemic prevention involving COVID-19.

Deep convolutional neural networks (CNNs) are successful in single-image super-resolution. Traditional CNNs are limited to exploit multi-scale contextual information for image reconstruction due to the fixed convolutional kernel in their building modules. To restore various scales of image details, we enhance the multi-scale inference capability of CNNs by introducing competition among multi-scale convolutional filters, and build up a shallow network under limited computational resources. The proposed network has the following two advantages: (1) the multi-scale convolutional kernel provides the multi-context for image super-resolution, and (2) the maximum competitive strategy adaptively chooses the optimal scale of information for image reconstruction. Our experimental results on image super-resolution show that the performance of the proposed network outperforms the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Incorporation of prior knowledge about organ shape and location is key to improve performance of image analysis approaches. In particular, priors can be useful in cases where images are corrupted and contain artefacts due to limitations in image acquisition. The highly constrained nature of anatomical objects can be well captured with learning-based techniques. However, in most recent and promising techniques such as CNN-based segmentation it is not obvious how to incorporate such prior knowledge. State-of-the-art methods operate as pixel-wise classifiers where the training objectives do not incorporate the structure and inter-dependencies of the output. To overcome this limitation, we propose a generic training strategy that incorporates anatomical prior knowledge into CNNs through a new regularisation model, which is trained end-to-end. The new framework encourages models to follow the global anatomical properties of the underlying anatomy (e.g. shape, label structure) via learnt non-linear representations of the shape. We show that the proposed approach can be easily adapted to different analysis tasks (e.g. image enhancement, segmentation) and improve the prediction accuracy of the state-of-the-art models. The applicability of our approach is shown on multi-modal cardiac data sets and public benchmarks. In addition, we demonstrate how the learnt deep models of 3-D shapes can be interpreted and used as biomarkers for classification of cardiac pathologies. [ABSTRACT FROM PUBLISHER]

Recently, single image super-resolution (SR) models based on deep convolutional neural network (CNN) have achieved significant advances in accuracy and speed. However, these models are not efficient enough for the image SR task. Firstly, we find that generic deep CNNs learn the low frequency features in all layers, which is redundant and leads to a slow training speed. Secondly, rectified linear unit (ReLU) only activate the positive response of the neuron, while the negative response is also worth being activated. In this paper, we propose a novel joint residual network (JRN) with three subnetworks, in which two shallow subnetworks aim to learn the low frequency information and one deep subnetwork aims to learn the high frequency information. In order to activate the negative part of the neurons and to preserve the sparsity of activation function, we propose a paired ReLUs activation scheme: one of the ReLUs is for positive activation and the other is for negative activation. The above two innovations lead to a much faster training, as well as a more efficient local structure. The proposed JRN achieves the same accuracy of a generic CNN with only 10.5% training iterations. The experiments on a wide range of images show that JRN is superior to the state-of-the-art methods both in accuracy and computational efficiency. [ABSTRACT FROM AUTHOR]

Transformer has become one of the main architectures in deep learning, showing impressive performance in various vision tasks, especially for image super-resolution (SR). However, due to the usage of high-resolution input images, most current Transformer-based image super-resolution models have a large number of parameters and high computational complexity. Moreover, some components employed in the Transformer may be redundant for SR tasks, which may limit the SR performance. In this work, we propose an efficient and concise model for image super-resolution tasks termed Residual Separation Hybrid Attention Network (RSHAN), which aims to solve the problems of redundant components and insufficient ability to extract high-frequency information of Transformer. Specifically, we present the Residual Separation Hybrid Attention Module (RSHAM) which fuses the local features extracted by the convolutional neural network (CNN) branch and the long-range dependencies extracted by Transformers to improve the performance of RSHAN. Extensive experiments on numerous benchmark datasets show that the proposed method outperforms state-of-the-art SR methods by up to 0.11 dB in peak signal-to-noise ratio (PSNR) metric, while the computational complexity and the inference time is reduced by 5% and 10%, respectively. [ABSTRACT FROM AUTHOR]