Search

High-resolution temperature forecasting plays a crucial role in various applications such as climate impact assessment, hydrological modeling, and localized weather forecasting. The existing low-resolution forecast data may not accurately capture the fine-grained temperature patterns required for localized predictions. These forecasts may contain biases that need to be corrected for accurate results. Therefore, there is a need for an effective framework that can downscale low-resolution forecast data and correct biases to generate high-resolution temperature forecasts. The paper proposes a machine learning-based spatial downscaling and bias-correction framework for high-resolution temperature forecasting. The framework utilizes low-resolution forecast data from the European Centre for Medium-Range Weather Forecasts and real-time 1km analysis product data from the National Meteorological Administration, to generate high-resolution 1km forecast temperature data. The framework consists of four modules: data acquisition module, data preprocessing module, downscaling and correction model module, and post-processing and visualization module. Through experiments, it demonstrated that the framework has superior performance and potential in meteorological data downscaling and correction and can be used to achieve real-time high-resolution temperature forecasting, which has important significance for various applications such as climate impact assessment, hydrological modeling, and localized weather forecasting. [ABSTRACT FROM AUTHOR]

In recent years, real-time vehicle congestion detection has become a hot research topic in the field of transportation due to the frequent occurrence of highway traffic jams. Vehicle congestion detection generally adopts a vehicle counting algorithm based on object detection, but it is not effective in scenarios with large changes in vehicle scale, dense vehicles, background clutter, and severe occlusion. A vehicle object counting network based on a feature pyramid split attention mechanism is proposed for accurate vehicle counting and the generation of high-quality vehicle density maps in highly congested scenarios. The network extracts rich contextual features by using blocks at different scales, and then obtains a multi-scale feature mapping in the channel direction using kernel convolution of different sizes, and uses the channel attention module at different scales separately to allow the network to focus on features at different scales to obtain an attention vector in the channel direction to reduce mis-estimation of background information. Experiments on the vehicle datasets TRANCOS, CARPK, and HS-Vehicle show that the proposed method outperforms most existing counting methods based on detection or density estimation. The relative improvement in MAE metrics is 90.5% for the CARPK dataset compared to Fast R-CNN and 73.0% for the HS-Vehicle dataset compared to CSRNet. In addition, the method is also extended to count other objects, such as pedestrians in the ShanghaiTech dataset, and the proposed method effectively reduces the misrecognition rate and achieves higher counting performance compared to the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

For the present, many technical problems exist in the convolution neural network for facial expression recognition, such as the complexity of convolutional face feature extraction, the difficulty in accurately recognizing the subtle feature changes of facial expressions, and the low automatic recognition rate of facial expressions. In this paper, a hybrid attention mechanism based on space and channel—Height Performance Module Implement(HPMI) attention mechanism is proposed to realize automatic facial expression recognition. The addition of this attention mechanism can enhance the weight of key features and make the model focused on the features which are useful for expression classification in the training process. An HPMI module based on spatial and channel-based mixed attention mechanism is embedded in VGG-16 network. This can effectively alleviate the overfitting phenomenon of the network, strengthen the useful information, suppress the useless information, promote the information flow between the key information of the image and the network model. At the same time, it can solve the problem of the inconsistency between the input dimension and the output dimension. The accuracy of the method in this paper is 98.97% and 88.44% on CK + and RAF-DB expression datasets. Experimental comparison shows that by embedding the HPMI module, the model can further enhance the learning of spatial and channel feature weight. Our experiments include 2 datasets for expression recognition and show an average improvement of 3.94% in the accuracy. [ABSTRACT FROM AUTHOR]

Since deep learning is introduced to medical image segmentation, UNet and its variants have been extensively applied in medical image analysis. This paper proposes a multi-scale channel attention UNet (MSCA-UNet) to raise the accuracy of the segmentation in medical ultrasound images. Specifically, a multi-scale module is constructed to connect and to enhance the feature maps with different scales extracted by convolution. Subsequently, A channel attention mechanism is designed to compress feature maps through learnable depth separable convolutions. Eventually, we have explored the global feature module to establish the dependency between multi-level features. The proposed method is thoroughly evaluated and compared with the existing methods on four medical ultrasound image datasets. The experiments indicate that our method outperforms the SOTA method in accuracy on four medical ultrasound image datasets. Compared with UNet network, the parameters of our model have decreased 29.82%. In addition, visual comparisons further demonstrate the proposed method. [ABSTRACT FROM AUTHOR]

Human Keypoints Detection is a relatively basic task in computer vision; it is the pre-task of human action recognition, behavior analysis and human–computer interaction. Since most abnormal actions occur at night, how to effectively extract skeleton sequence data in a low-light or completely dark environment poses a huge challenge for its identification. This paper proposes to use far infrared images to detection key points of the human body, which can solve the problem of human pose estimation under challenging weather conditions such as total darkness, smoke, inclement weather and glare. However, far-infrared images have some shortcomings, such as low resolution, noise and thermal characteristics; the skeleton data need to be provided in real time for the next stage of task. Based on the above reasons, this paper proposes a lightweight multi-stage attention network (LMANet) to detect the key points of human at night. This new network structure adds context information through the large receptive field, which helps to assist the detection of neighboring key points through this information, but for the sake of lightweight consideration, this article only extends the network to two stages. In addition, this article uses the attention module to effectively select channels with a large amount of information and highlight the features of key points, while eliminating background interference. In order to detect key points of the human in various complex environments, we use techniques such as difficult sample mining which improves the accuracy of key points with low confidence. Our network has been verified on two visible light datasets, fully demonstrating excellent performance. This paper successfully introduces far-infrared images into the field of pose estimation, because there is no public dataset for far-infrared pose estimation. In this paper, 700 images are selected for annotation from multiple public far-infrared object detection, segmentation and action recognition datasets; our algorithm is verified on this dataset; the effect is very good. After the paper is published, we will publish our key points of the human body annotated documents. [ABSTRACT FROM AUTHOR]

The detection of surface defects is crucial to industrial manufacturing. In recent years, numerous detection methods based on computer vision have been successfully applied in the industry. However, industrial defect detection is still full of challenges. In one aspect, most of the industrial defects are extremely small. In another aspect, even though the intra-class defects have numerous similar elements, their outward appearances differ significantly. In this paper, we propose a dual-structure attention-based multi-level feature fusion network (DaMFFN) to address these two issues. In the first attention-based multi-level feature extraction structure, we introduce novel attention pooling to preserve more detailed information about the defective features of the tiny defect by giving certain regions varying weights. In the second attention-based multi-level feature fusion structure, we propose channel attention to capture the defect feature with the greatest potential for discrimination rather than all possible defect features, which is employed to prevent the incorrect detection of intra-class defects. The experiments demonstrate that the detection performance of the DaMFFN is better than other methods in five surface defect datasets. [ABSTRACT FROM AUTHOR]

The application of attention mechanisms, especially channel attention, has achieved huge success in the field of computer vision. However, existing methods mainly focus on more sophisticated attention modules for better performance, but ignore global and local contexts in the frequency domain. This work focuses on the channel relationship and proposes a novel architectural unit called Frequency Global and Local (FGL) context block. It adaptively recalibrates global-local channel-wise feature responses by explicitly modeling interdependencies between channels in the frequency domain. The proposed lightweight FGL module is efficient well generalizable across different datasets. Meanwhile, the FGL context block significantly improves the performance of existing convolutional neural networks (CNNs) at a slight computational cost. Our FGL module is extensively evaluated with applications of image classification, object detection, and semantic segmentation with the backbones of ResNets, MobileNetV2, and MobileNeXt. The experimental results indicate that our module is more efficient than its counterparts. Our model is open-sourced at https://github.com/YunDuanFei/FGL. [ABSTRACT FROM AUTHOR]

To enhance the performance of object detection algorithm, this paper proposes segmentation attention feature pyramid network (SAFPN) to address the issue of semantic information loss. Compared to prior works, SAFPN discards the original 1 × 1 convolutions and achieves feature dimension reduction through a segmentation and accumulation architecture, thereby preserving the semantic information of high-dimensional features completely. To capture fine-grained semantic details, it integrates channel attention and spatial attention mechanisms to enhance the network's focus on important information. Extensive experimental validation demonstrates that SAFPN achieves favorable results on multiple public datasets, and can better complete the target detection task. [ABSTRACT FROM AUTHOR]

Super-resolution (SR) is a powerful technique for enhancing the quality of remote sensing imagery, which in turn can improve the accuracy of various computer vision tasks, such as object detection, classification, and segmentation. Deep convolutional neural networks (CNNs) have demonstrated significant progress in this field, and attention mechanisms are widely adopted in deep CNNs as they allow the models to assign weights to important areas within the feature map. In this paper, we propose the channel-wise gated attention (CGA) module, which integrates attention across the feature map channels and scales the resulting feature map through a gating parameter, leading to performance improvements. Furthermore, we present an SR framework that employs multiple attention blocks, with the CGA module serving as the core of each block, to enhance the spatial resolution of remote sensing imagery. Our proposed network, the channel-wise gated attention Network (CGA-Net), outperforms other attention-based deep SR models for 4 × - and 8 × -upsampling on two remote sensing datasets: Satellite Imagery Multi-Vehicles Dataset (SIMD), consisting of 5000 high-resolution remote sensing images, and DOTA, a large-scale satellite imagery dataset. We conduct several experiments to evaluate the effectiveness of our SR framework for object detection on the SIMD dataset. The code and trained weights for the proposed framework can be found at this link: . [ABSTRACT FROM AUTHOR]

As one representative object of bad weather, rain streaks can have a bad influence on image capture. Worse still, computer vision tasks can be badly influenced when working in rainy scenes. Therefore, it makes sense to conduct research on rain removal. This paper introduces a new deraining network (DDRNet). Specifically, rain streaks contain both frequency and spatial information, so we propose a rain removal module (RRM) consisting of a rain channel attention module (RCAM) and a rain spatial attention module (RSAM) for extracting rain information. Furthermore, we design a detail-recovery module (DRM) to extract the background feature so that the original background image details deleted by mistake during the deraining process can be made up by it. Moreover, a two-branch aggregation (TBA) mechanism is adopted to promote the process of information flow that effectively enhances the execution of our DDRNet. The evaluation of our DDRNet on various benchmark datasets has shown the superiority of our algorithm. In addition, a range of low-level experiments, such as object detection and semantic segmentation, provide further evidence of the effectiveness of the proposed DDRNet. [ABSTRACT FROM AUTHOR]

The hyperspectral imaging, capturing discriminative information across a series of spectrum bands, leads to building a robust face recognition system. Motivated by the success of deep convolutional network and transferring learning, this paper proposed an end-to-end hyperspectral face recognition model based on a light Convolutional Neural Network (CNN) and transfer learning. To boost the performance of hyperspectral face recognition, the Max-Feature-Map (MFM) activation function and fine-tuning (structure regularization(L2SP) or attention feature distillation (AFD)) are introduced to optimize the deep network, which will learn the fine feature representation across different bands. Especially, this method incorporates regularization and AFD cooperation into the transfer learning strategy on the visible face data. By feeding back hyperspectral images to the pretrained light CNN network, we can design an end-to end model that can leverage the generalization ability for hyperspectral face images. Finally, the entire model is trained and verified on the PolyU-HSFD, CMU, and UWA hyperspectral face datasets using the associated standard evaluation protocols. Experimental results demonstrate that the improved Light CNN network can get good representations of hyperspectral face features and the joint training with a combination of L2SP and AFD exhibits better recognition performance than the state-of-the-art methods based on other deep networks. [ABSTRACT FROM AUTHOR]

Magnetic resonance (MR) images are widely used for clinical diagnosis, whereas some surrounding factors always limit the resolution, so under-sampled data is usually generated during imaging. Since high-resolution (HR) MR images contribute to the clinic diagnosis, reconstructing HR MR images from these under-sampled data is pretty important. Recently, deep learning (DL) methods for HR reconstruction of MR images have achieved impressive performance. However, it is difficult to collect enough data for training DL models in practice due to medical data privacy regulations. Fortunately, federated learning (FL) is proposed to eliminate this issue by local/distributed training and encryption. In this paper, we propose a multi-scale channel attention network (MSCAN) for MR image super-resolution (SR) and integrate it into an FL framework named FedAve to make use of data from multiple institutions and avoid privacy risk. Specifically, to utilize multi-scale information in MR images, we introduce a multi-scale feature block (MSFB), in which multi-scale features are extracted and attention among features at different scales is captured to re-weight these multi-scale features. Then, a spatial gradient profile loss is integrated into MSCAN to facilitate the recovery of textures in MR images. Last, we incorporate MSCAN into FedAve to simulate the scenery of collaborated training among multiple institutions. Ablation studies show the effectiveness of the multi-scale features, the multi-scale channel attention, and the texture loss. Comparative experiments with some state-of-the-art (SOTA) methods indicate that the proposed MSCAN is superior to the compared methods and the model with FL has close results to the one trained by centralized data. [ABSTRACT FROM AUTHOR]

Food image segmentation which aims to distinguish various ingredients is crucial for food safety, as estimating calories and other nutrients is important for human health and sustainable development. However, the performances of current image segmentation methods are inferior on food image datasets due to the significant diversity of appearances and distinctive conditions between ingredients and daily props, while these methods have insufficient capabilities for feature extraction of food images. In addition, utilizing attention mechanisms to obtain contextual detail information and long-range dependencies leads to a quadratic computational complexity. In this paper, we propose a Cross Spatial Attention (CSA) module to extract richer spatial features from food images, with lower time and space complexity. Specifically, the CSA module aggregates the contextual information by cross-calculation of horizontal and vertical dimensions. And experiments demonstrate that, by taking a two-step cross-calculation, each pixel could eventually capture global long-range dependencies. Furthermore, our method integrates a Channel Attention (CA) module to selectively highlight interdependent channel information by integrating relevant features across all feature maps. Then the outputs of these two attention modules are aggregated to enhance the representation of the image feature. Convincing performance improvement is achieved on the FoodSeg103, UECFoodPix and ADE20K. Moreover, the proposed network achieved better trade-off between accuracy and efficiency. [ABSTRACT FROM AUTHOR]

Prostate diseases often occur in men. For further clinical treatment and diagnosis, we need to do accurate segmentation on prostate. There are already many methods that concentrate on solving the problem of automatic prostate MR image segmentation. However, the design of some hyperparameters of these methods is migrated from the models that are used for nature images which do not consider the difference between medical image and nature image. Besides, there is trend that researchers are likely to use deeper and more complicated networks to achieve high accuracy. The improvement is limited with surging parameters, computations, training time, and inference time. In this paper, we propose an efficient attention residual U-Net to segment the prostate MR image. We analyze the property of prostate MR image and fine-tune the architecture of U-Net. To accelerate the convergence of our method, residual connection and channel attention are added to our network. A set of experiments suggest our method can achieve a similar accuracy of state of the art with less parameters, less computations, shorter training time, and shorter inference time. [ABSTRACT FROM AUTHOR]

Automation in searching and making decisions about the threat and non-threat objects from the baggage X-ray images could ease the security inspection systems rather than rely on human choices. Although various object detection and threat object identification methods have been proposed, at the present time, there are still many open-ended problems in Automatic Threat Detection from Baggage X-ray images. This paper proposes a novel framework, Multi-Channel Correlated Detail Preserving Network (MC-CDPNet), for segmenting threat objects based on LinkNet. The Correlated Detail Preserving block in the proposed network represents the target with location and channel based attention on different color space feature maps. The additional pooling model in encoder section also improves the ability to extract the prominent features. The performance of the proposed MC-CDPNet is equivalently successful on segmenting threat objects from the state-of-the-art datasets, GDXray, and SIXray, in comparison to many other existing methods. [ABSTRACT FROM AUTHOR]

Deep convolutional neural networks (DCNNs) have achieved outstanding results in facial expression recognition (FER). However, their runtime memory and computational resource requirements make it challenging to deploy them on resource-constrained devices, such as mobile devices. In this paper, we propose a novel lightweight attention DCNN (LA-Net) for robust FER, which uses squeeze-and-excitation (SE) modules and the network slimming strategy. First, we combine the SE modules with the CNN network, which assigns a certain weight to each feature channel. This enables LA-Net to focus on learning the prominent facial features, reduce redundant information, and finally extract discriminative features from facial images. Then, we use the network slimming method to further reduce the model's size, which results in a thin and compact network that uses less runtime memory and computational operations with minimal accuracy loss. The proposed LA-Net model can achieve 95.52%, 87.00% and 100% test accuracy on KDEF, RAF-DB and FERG-DB FER datasets, respectively. The experimental results show that the proposed method achieves better or comparable results than state-of-the-art FER methods and significantly reduces the computational cost and the number of parameters, with better generalization capability and robustness. [ABSTRACT FROM AUTHOR]

Vehicle re-identification (Re-ID) aims to find the image of the same vehicle in different cameras. One of the reasons that this task remains challenging is that different vehicles of the same type and color look very similar in appearance. In recent years, attention mechanisms have been widely used in vehicle Re-ID, including some mechanisms that use the relationships between nodes to infer attention. However, such methods are vulnerable to interference by some noise information in the image, especially partial information from other vehicles. For this reason, we propose in this paper a global reference attention mechanism for attention learning by utilizing the relationships between nodes and the global reference node, where the global reference node is built by all nodes in the image. At the same time, we propose a Global Reference Attention Network (GRA-Net) based on the mechanism to mine a large number of discriminative features useful for vehicle re-identification, thus easing the difficulty of distinguishing between different vehicles of similar appearance. Specifically, to extract more discriminative features, we adopt a multi-branch neural network and embed different global reference attention modules in each branch to compose our GRA-Net. Extensive experiments valiyear the effectiveness of GRA-Net and show that our approach achieves state-of-the-art performance on two massive vehicle Re-ID datasets. [ABSTRACT FROM AUTHOR]

We propose an image super-resolution method (SR) using a deeply-recursive convolutional network (DRCN). Single-image super-resolution reconstruction technology is to reconstruct fuzzy low-resolution images into clearer high-resolution images. It is a research hotspot in the field of computer vision and image processing. In recent years, the attention mechanism has been successfully applied in image super-resolution reconstruction. However, the existing methods use the channel attention mechanism and the spatial attention mechanism separately, or simply superimpose them, which cannot effectively unify the adjustment effects of both, and the performance is limited. This paper proposes a method that can merge channel attention and spatial attention into pixel attention, which achieves more precise adjustment of feature map information. The pixel attention convolutional neural network method built on this basis can improve the quality of image texture detail reconstruction. We have been tested on five widely used standard datasets, the experimental results show that the method is superior to most current representative reconstruction methods, especially in terms of high-definition picture texture restoration. [ABSTRACT FROM AUTHOR]

Image deraining is a highly ill-posed problem. Although significant progress has been made due to the use of deep convolutional neural networks, this problem still remains challenging, especially for the details restoration and generalization to real rain images. In this paper, we propose a deep residual channel attention network (DeRCAN) for deraining. The channel attention mechanism is able to capture the inherent properties of the feature space and thus facilitates more accurate estimations of structures and details for image deraining. In addition, we further propose an unsupervised learning approach to better solve real rain images based on the proposed network. Extensive qualitative and quantitative evaluation results on both synthetic and real-world images demonstrate that the proposed DeRCAN performs favorably against state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Person re-identification (re-id) expects to find a picture according to a few clues that refer to the same pedestrian from an image database caught from different locations. Previous studies show that the exploration of local features may help for capturing more robust feature representations. However, part-level characteristics are demonstrated useful but easily confused with comparable attributes in corresponding locations. For this problem, we propose an effective relation-aware multi-granularity hierarchical network (RMHNet) to explore the value of correlation between part-level and global features. In this paper, with a concise structure, RMHNet conducts a uniform partitioning strategy to relocate images into horizontal and vertical stripes. We incorporate the relationship between part-level semantic representation with the entire image to mine structural relationship information of the probe image for potential critical identification clues. Extensive experiments demonstrated the remarkable advantage of our RMHNet to state-of-the-art methods on several public benchmarks includes 84.2% rank-1 accuracy on MSMT17 (Wei et al., in: Internaltional conference on computer vision and pattern recogintion (CVPR), pp 79–88, 2018. https://doi.org/10.1109/CVPR.2018.00016) datasets, which is currently the largest publicly available person re-id dataset [ABSTRACT FROM AUTHOR]

There are still many pressing problems in pedestrian detection, such as difficulty in detection due to severe pedestrian occlusion, difficulty in detecting small objects and low detection speed. In this paper, we propose A Fast and Efficient Pedestrian Detector with Center and Scale Prediction (FE-CSP). We combine channel attention with spatial attention, replace the traditional convolution with deformable convolution, and embed the backbone network to propose CSANet (Channel and Spatial Attention Network), which efficiently extracts the semantic features of the object, and then propose a feature pyramid network to replace the traditional concatenation to perform multi-scale feature detection, which effectively improves the detection speed. By conducting experiments on CityPersons, our method achieves 10.1%, 13.7% and 47.4% M R - 2 at a speed of 0.21 s/img on the reasonable setting, small setting and heavy setting, respectively. On Caltech, our method achieves 5.2% M R - 2 at a speed of 0.06 s/img on the Reasonable setting, further demonstrating the superiority and generalization ability of the proposed method. [ABSTRACT FROM AUTHOR]

Fine-grained visual classification aims to identify images belonging to multiple subcategories within the same category. Most existing methods use a single network to extract image features or learn fine-grained features by localizing and scaling key regions. Due to the limited number of components, this may miss valuable clues or cause performance degradation. This paper proposes an efficient approach to address this problem. First, we propose to learn as many global features as possible in images via a dual-baseline network. Second, considering the importance of the attention mechanism for image classification, we exploit the gated interaction of channels between global feature maps to generate attention to discover key discriminant regions of images. In the same way, the interactive channel attention and position attention of the global feature map are used to focus on the key discriminant regions of the image. In the above attention, interactive gated attention is generated by the gating vector mapped by the multi-layer perceptron MLP. Similarly, for channel attention and position attention, we perform attention based on global feature semantic information enhancement. The proposed model performs well on three datasets: CUB-200-2011, Stanford Cars, and FGVC aircraft. [ABSTRACT FROM AUTHOR]

Existing person re-identification methods are difficult to generalize to unseen datasets because of the domain gaps. The key to solving this problem is to extract domain-invariant and discriminative features. In this paper, we propose a normalization and enhancement (NE) module that can effectively suppress domain gaps and enhance pedestrian features without any target domain data, thereby enhancing the generalization ability of the model. NE module consists of a residual connection and a channel attention (CA) block. The residual connection is designed to suppress the domain gaps while retaining discriminative information. The proposed CA block embeds spatial information into the channel dimension and models the dependencies between channels to enhance pedestrian features. In addition, the NE loss is designed to constrain the training of NE module to extract domain-invariant features with excellent distribution. Ablation experiments were conducted on Market-1501, DukeMTMC-reID, CUHK03-NP and MSMT17. Experimental results confirmed the superiority of the proposed method, the mAP reached a maximum of 47.3% in cross-domain scenarios. [ABSTRACT FROM AUTHOR]

Up-sampling operations are frequently utilized to recover the spatial resolution of feature maps in neural networks for segmentation task. However, current up-sampling methods, such as bilinear interpolation or deconvolution, do not fully consider the relationship of feature maps, which have negative impact on learning discriminative features for semantic segmentation. In this paper, we propose a pixel and channel enhanced up-sampling (PCE) module for low-resolution feature maps, aiming to use the relationship of adjacent pixels and channels for learning discriminative high-resolution feature maps. Specifically, the proposed up-sampling module includes two main operations: (1) increasing spatial resolution of feature maps with pixel shuffle and (2) recalibrating channel-wise high-resolution feature response. Our proposed up-sampling module could be integrated into CNN and Transformer segmentation architectures. Extensive experiments on three different modality datasets of biomedical images, including computed tomography (CT), magnetic resonance imaging (MRI) and micro-optical sectioning tomography images (MOST) demonstrate the proposed method could effectively improve the performance of representative segmentation models. [ABSTRACT FROM AUTHOR]

With the rapid development of deep learning, no-reference image quality assessment (NR-IQA) based on convolutional neural network (CNN) plays an important role in image processing. Currently, most CNN-based NR-IQA methods focus primarily on the global features of images while ignoring detail-rich local features and channel dependencies. In fact, there are subtle differences in detail between distorted and reference images, as well as differences in the contribution of different channels to IQA. Furthermore, multi-scale feature extraction can be used to fuse the detailed information from images with different resolutions, and the combination of global and local features is critical in extracting image features. As a result, in this paper, a multi-scale residual CNN with an attention mechanism (MsRCANet) is proposed for NR-IQA. Specifically, a multi-scale residual block is first used to extract features from distorted images. Then, the residual learning with active weighted mapping strategy and channel attention mechanism is used to further process image features to obtain more abundant information. Finally, the fusion strategy and full connection layer are used to evaluate image quality. The experimental results on four synthetic databases and three in-the-wild IQA databases, as well as cross-database validation results, show that the proposed method has good generalization ability and can be compared with the most advanced methods. [ABSTRACT FROM AUTHOR]

Remote sensing images change detection (RSICD) is a task to identify desired significant differences between multi-temporal images acquired at different times. From the existing methods, most of them solved this issue with a Siamese network, focusing on how to utilize the comparison between two image features to generate an initial difference map. However, Siamese network-based methods have three drawbacks: (1) complex architecture; (2) rough change map; (3) cumbersome detecting procedure: including feature extraction and feature comparison. To overcome the above drawbacks, we devoted our work to design a general framework which has a simple architecture, integrated detecting procedure, and good capacity of detecting subtle changes. In this paper, we proposed a channel self-attention network based on the generative adversarial network for change detection of remote sensing images. The network used an encoder–decoder network to directly produce a change map from two input images. It was better to detect small punctate and slim linear changes than Siamese-based networks. By regarding RSICD as an image translation problem, we used a Generative Adversarial Network to detect changes. In addition, a channel self-attention module was proposed to further improve the performance of this network. Experimental results on three public remote sensing RGB-image datasets, including change detection dataset, Wuhan University building change detection dataset and LEVIR building Change Detection dataset demonstrated that our method outperformed other state-of-the-art methods. In terms of the F1 score, the proposed method achieved maximum improvements of 5.1%, 3.1%, and 1.7% on the above datasets, respectively. Models and codes will be available at https://github.com/wangle53/CSA-CDGAN. [ABSTRACT FROM AUTHOR]

How to effectively utilize inter-frame redundancies is the key to improve the accuracy and speed of video super-resolution reconstruction methods. Previous methods usually process every frame in the whole video in the same way, and do not make full use of redundant information between frames, resulting in low accuracy or long reconstruction time. In this paper, we propose the idea of reconstructing key frames and non-key frames respectively, and give a video super-resolution reconstruction method based on deep back projection and motion feature fusion. Key-frame reconstruction subnet can obtain key frame features and reconstruction results with high accuracy. For non-key frames, key frame features can be reused by fusing them and motion features, so as to obtain accurate non-key frame features and reconstruction results quickly. Experiments on several public datasets show that the proposed method performs better than the state-of-the-art methods, and has good robustness. [ABSTRACT FROM AUTHOR]

Siamese networks have been widely used in visual tracking. However, it is difficult to deal with complex appearance variations when the discriminative background information is ignored and an offline training strategy is adopted. In this paper, we present a novel backbone network based on CNN model and attention mechanism in the Siamese framework. The attention mechanism is composed of a channel attention module and a spatial attention module. The channel attention module uses the learned global information to selectively focus on the convolution features, which enhances a network representation ability. Besides, the spatial attention module obtains more contextual information and semantic features of target candidates. The designed attention mechanism-based backbone is lightweight and has a real-time tracking performance. We utilize GOT-10K as a training set to offline adjust trained model parameters. The extensive experimental evaluations on OTB2015, VOT2016, VOT2018, GOT-10k and UAV123 datasets demonstrate that the proposed algorithm has excellent performances against state-of-the-art trackers. [ABSTRACT FROM AUTHOR]

Hazy images obstruct the visibility of image content, which can negatively affect vision-based decision-making in multimedia systems and applications. Recently, convolutional neural networks (CNN) are proven with great benefit to remove single image haze, which has aroused research attention. However, in practice, previous works fail to fully exploit multi-scale features and restore the faithful image details from the hazy inputs, resulting in sub-optimal performance. In this paper, we propose a novel and high-efficiency deep hourglass-structured fusion model to address this issue, which also indicates the applicability of the modified hourglass architecture to remove haze. Unlike the conventional multi-scale learning schemes, top-down and bottom-up feature fusions are repeated, so each of the coarse-to-fine scale representations receives data of parallel ones, which allows for more flexible information exchange and aggregation at various scales. To be specific, we develop residual dense module as the backbone unit, while introducing the channel-wise attention mechanism to further enhance the representation ability of the network. As proved by extensive assessments demonstrate, our designed model outclasses existing ones and achieves the advanced performance on benchmark datasets and real hazy images. We have released source codes on GitHub: https://github.com/cxtalk/Hourglass-DehazeNet. [ABSTRACT FROM AUTHOR]

In recent years, the convolutional neural networks (CNNs) for single image super-resolution (SISR) are becoming more and more complex, and it is more challenging to improve the SISR performance. In contrast, the reference image guided super-resolution (RefSR) is an effective strategy to boost the SR (super-resolution) performance. In RefSR, the introduced high-resolution (HR) references can facilitate the high-frequency residual prediction process. According to the best of our knowledge, the existing CNN-based RefSR methods treat the features from the references and the low-resolution (LR) input equally by simply concatenating them together. However, the HR references and the LR inputs contribute differently to the final SR results. Therefore, we propose a progressive channel attention network (PCANet) for RefSR. There are two technical contributions in this paper. First, we propose a novel channel attention module (CAM), which estimates the channel weighting parameter by weightedly averaging the spatial features instead of using global averaging. Second, considering that the residual prediction process can be improved when the LR input is enriched with more details, we perform super-resolution progressively, which can take advantage of the reference images in multi-scales. Extensive quantitative and qualitative evaluations on three benchmark datasets, which represent three typical scenarios for RefSR, demonstrate that our method is superior to the state-of-the-art SISR and RefSR methods in terms of PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity). [ABSTRACT FROM AUTHOR]

Transformers have achieved remarkable results in high-level vision tasks, but their application in low-level computer vision tasks such as image denoising remains largely unexplored. In this paper, we propose a novel channel attention residual enhanced Swin Transformer denoising network (CARSTDn), which is an efficient and effective Transformer-based architecture. CARSTDn consists of three modules: shallow feature extraction, deep feature extraction, and image reconstruction modules. The deep feature extraction module is the core of CARSTDn, and it employs a channel attention residual Swin Transformer block (CARSTB). Our benchmarking results demonstrate that CARSTDn outperforms existing state-of-the-art methods, showcasing its superiority. We hope that our work will inspire further research into the use of Transformer-based architectures for image denoising tasks. [ABSTRACT FROM AUTHOR]

No-reference (NR) image quality assessment (IQA) is an important task of computer vision. Most NR-IQA methods via deep neural networks do not reach desirable IQA performance and have bulky models which make them difficult to be used in the practical scenarios. This paper proposes a lightweight transformer and multi-head prediction network for NR-IQA. The proposed method consists of two lightweight modules: feature extraction and multi-head prediction. The module of feature extraction exploits lightweight transformer blocks to learn features at different scales for measuring different image distortions. The module of multi-head prediction uses three weighted prediction blocks and an FC layer to aggregate the learned features for predicting image quality score. The weighted prediction block can measure the importance of different elements of input feature at the same scale. Since the importance of feature elements at the same scale and the importance of the features at different scales are both considered, the module of multi-head prediction can provide more accurate prediction results. Extensive experiments on the standard IQA datasets are conducted. The results show that the proposed method outperforms some baseline NR-IQA methods in IQA performance on the large image datasets. For the model complexity, the proposed method is also superior to several recent NR-IQA methods. [ABSTRACT FROM AUTHOR]

Skin lesion segmentation is an important process in skin diagnosis, but still a challenging problem due to the variety of shapes, colours, and boundaries of melanoma. In this paper, we propose a novel and efficient U-shaped network named EAM-CPFNet, which combines with edge attention module (EAM) and context pyramid fusion (CPF) to improve the performance of the skin lesion segmentation. First, we design a plug-and-play module named edge attention module (EAM), which is used to highlight the edge information learned in the encoder. Secondly, we integrate two pyramid modules collectively named context pyramid fusion (CPF) for context information fusion. One is multiple global pyramid guidance (GPG) modules, which replace the skip connections between the encoder and the decoder to capture global context information, and the other is scale-aware pyramid fusion (SAPF) module, which is designed to dynamically fuse multi-scale context information in high-level features by utilizing spatial and channel attention mechanisms. Furthermore, we introduce full-scale skip connections to enhance different levels of global context information. We evaluate the proposed method on the publicly available ISIC2018 dataset, and the experimental results demonstrate that our proposed method is very competitive compared with other state-of-the-art methods for the skin lesion segmentation. [ABSTRACT FROM AUTHOR]

In recent years, video saliency object detection has received more and more attention, and many excellent algorithms have been proposed. In the paper, we propose a new idea of video saliency object detection, named MAED-Net. Our method is mainly divided into two modules: spatial module and temporal module. In spatial module: we use a set of parallel dilated convolutions, and add channel attention to each dilated convolutions. Multi-scale mimics the characteristics of the human retina. Attention is to imitate the human attention mechanism. We combine multi-scale information with attention information, which constitutes the pyramid multi-scale channel attention. Multi-scale channel attention allows us to obtain more precise saliency clues, laying a solid foundation for the next part of the temporal. In temporal module: we use a set of Encoder-decoder ConvLSTM with different dilated rates, and we use dense connection and skip connection to blend information of different scales. We evaluate our results on four datasets and compare with twelve algorithms. The experimental results show that our algorithm achieved the state-of-the-arts. [ABSTRACT FROM AUTHOR]

The performance of vision systems can be affected when used in severe weather conditions such as heavy rain or snow. Rain streak removal is an ill posed problem as they can vary in shape, size, and density across the image. In this paper, a single-image deraining network named Factorized Multi-scale Multi-resolution Residual Network (FMMRNet), which follows a U-Net backbone, is proposed. As rain streaks affect non-local regions of the image, larger receptive fields are beneficial to capture these non-local dependencies. We propose the use of multi-scale grouped convolutions to integrate information from global and local scales. In the proposed FMMRNet, multi-scale convolutions are factorized so that they can have large effective kernel sizes while reducing the computational complexity. During training, intermediate multi-resolution outputs are produced for loss computation which improves gradient flow in the deeper layers of the network and promotes better learning. A channel-wise attention mechanism is included to recalibrate feature maps before fusion instead of direct concatenation at each stage of the decoder. A higher-level reconstruction loss called perceptual loss is included for effective training to improve the visual quality of the derained images. The performance of the proposed FMMRNet is quantitatively and qualitatively compared on public benchmark datasets, and it outperforms the state-of-the-art deraining methods. Furthermore, to show the practical applicability of the proposed network, we demonstrate that when FMMRNet is used as a preprocessing step for object-detection methods such as Faster-RCNN and YOLO, it improves their performance on images degraded by rain streaks by 12.6% and 75.2% respectively. [ABSTRACT FROM AUTHOR]

Due to the influence of color, boundaries, and shapes of melanoma, the segmentation of the lesion area is still a challenging problem. In this paper, we propose a method to connect two attention modules (channel attention and spatial attention) serially and embed them into the skip connection of the encoder–decoder network. Different from previous work on image segmentation through attention mechanism, we have made a different combination of the existing popular spatial attention module and channel attention module. The experimental results show that the sequence combinations with channel attention in front and spatial attention in the rear are more likely to aggregate global and local information as well as information between channels than other combinations. Our method achieved an average Jaccard Index of 0.7692 on the ISIC2017 dataset. At the same time, we also compared with some advanced methods of image segmentation, the experimental results show our proposed method has a competitive performance. [ABSTRACT FROM AUTHOR]

Although the salient object detection method based on the fully convolutional neural network has achieved better performance, how to learn effective feature representations in complex scenes to obtain more accurate saliency maps is still a challenge. In order to cope with the above-mentioned challenges, an addition or cascade structure is generally used to fuse feature information between multiple levels. However, these methods are susceptible to the influence of messy background information. The network may regard non-salient objects with similar salient appearances as target predictions, and the prediction results may be incomplete due to different appearance areas of salient objects. We design a network composed of multiple attention mechanisms to selectively integrate deep and shallow feature information, and more effectively deal with the transfer and fusion of features. In this paper, we propose a multi-attention embedded network (MAENet), which introduces attention mechanisms to give different feature information with different weights for handling the transfer and aggregation of features at different levels. The multi-attention feature aggregation (MAFA) module is proposed, which uses the channel attention mechanism to give different weights to the features to be fused, and then uses the spatial attention mechanism to selectively aggregate shallow edge information and deep abstract semantic features to avoid excessive redundant information which affects the saliency mapping, as well as suppressing non-salient areas with "salient" appearance. In addition, The multi-scale feature extraction (MFE) module and the self-attention (SA) module are also proposed for obtaining sufficiently rich and useful multi-scale context information and enhancing the function of the top layer. Finally, the attentional residual refinement (ARR) module is utilized to refine the saliency map after each feature fusion and further improve the input function. MAENet can accurately segment salient objects and provide clear local details. Experimental results on five benchmark datasets show that the proposed method achieves the favorable performance against 14 state-of-the-art methods on popular evaluation metrics. [ABSTRACT FROM AUTHOR]

Image dehazing aims to recover a clean image from a hazy image, which is a challengingly longstanding problem. In this paper, we propose an Ensemble Multi-scale Residual Attention Network (EMRA-Net) to directly generate a clean image, which include two parts: a three-scale residual attention CNN (TRA-CNN), and an ensemble attention CNN (EA-CNN). In TRA-CNN, we employ wavelet transform to obtain the downsampled images, instead of using common spatial downsampling methods, such as nearest downsampling and strided-convolution. With the help of wavelet transform, we can avoid the loss of image texture details. Moreover, in each scale-branch, Res2Net modules are connected in series to make full use of the hierarchical features from the original hazy images, and channel attention mechanism is introduced to focus channel-dimension information. Finally, an EA-CNN is proposed to fuse coarse images generated from TRA-CNN into a refined clean image. Extensive experiments on the benchmark synthetic hazy datasets and the real-world hazy dataset prove that proposed EMRA-Net is superior to previous state-of-the-art methods both in subjective visual perception and objective image quality assessment metrics. [ABSTRACT FROM AUTHOR]

Pedestrian trajectory prediction is a crucial technology for agents to assist human beings, which remains highly challenging due to the complex interactions between pedestrians and the environment. However, previous works based on pedestrian relative position modeling have the problem of ignoring environmental information and global pedestrian perception, which inevitably leads to a significant deviation from reality. To address these challenges, we introduce a Channel Spatio-temporal Convolutional Network (CSTCN) for predicting pedestrian trajectories. The CSTCN explicitly models pedestrian interactions with perceptual information to capture the temporal and spatial characteristics of pedestrians. Meanwhile, we use Group-SE to model the sensitivity of pedestrians to multi-channel data, which facilitates predictions based on historically observed trajectories. We evaluated our proposed method on the ETH and UCY datasets. The experimental results demonstrate that our method outperforms other state-of-the-art methods by 11.4% in Average Displacement Error (ADE) and 6.7% in Final Displacement Error (FDE). [ABSTRACT FROM AUTHOR]

Due to the instability of the hyperspectral imaging system and the atmospheric interference, hyperspectral images (HSIs) often suffer from losing the image information of areas with different shapes, which significantly degrades the data quality and further limits the effectiveness of methods for subsequent tasks. Although mainstream deep learning-based methods have achieved promising inpainting performance, the complicated ground object distributions increase the difficulty of HSIs inpainting in practice. In addition, spectral redundancy and complex texture details are two main challenges for deep neural network-based inpainting methods. To address the above issues, we propose a Global-Local Constraints-based Spectral Adaptive Network (GLCSA-Net) for HSI inpainting. To reduce the redundancy of spectral information, a multi-frequency channel attention module is designed to strengthen the essential channels and suppress the less significant ones, which calculates adaptive weight coefficients by converting feature maps to the frequency domain. Furthermore, we propose to constrain the generation of missing areas from both global and local perspectives, by fully leveraging the HSI texture information, so that the overall structure information and regional texture consistency of the original HSI can be maintained. The proposed method has been extensively evaluated on the Indian Pines and FCH datasets. The promising results indicate that GLCSA-Net outperforms the state-of-the-art methods in quantitative and qualitative assessments. [ABSTRACT FROM AUTHOR]

Crowd counting methods have become increasingly mature. However, the problem of dramatic scale variation still exists. For this reason, we propose an efficient separated and fused pyramid attention network, which can extract multiscale features on channels and space and greatly alleviate the problem of dramatic scale variation. First, in order to extract the rich features on the channel, we design a separated and fused channel attention module, which is composed of two 3x3 convolution layers, a separated attention module, and a SE module. Second, we design a spatial contextual feature fusion module to fully extract multiscale features in spatial dimensions. Finally, we conduct comparison experiments with state-of-the-art methods on several challenging datasets, including the ShanghaiTech, UCF_CC_50, and WorldExpo'10 datasets. The experimental results show our method outperforms most of the state-of-the-art methods. We conduct ablation experiments on the ShanghaiTech Part A and Part B datasets to verify the importance of each submodule. [ABSTRACT FROM AUTHOR]

Image matching has a wide range of applications in computer vision. Existing image matching is mostly used for homologous images. In the matching of visible images and infrared images, the imaging principles for visible and infrared images differ significantly, resulting in substantial differences between the images. The matching network used for homologous images cannot achieve satisfactory results when applied to visible and infrared images. This inadequacy stems primarily from the absence of feature extraction networks. This dearth results in the inability to perform effective feature representation for visible and infrared images. In addition, deep learning is a data-driven method. The scarcity of visible-infrared image matching datasets hampers the learning process of network models, making it impossible for the network model to learn the best parameters and achieve the best performance. Regarding the above issues, we propose a visible-infrared image matching network based on the attention mechanism. The matching network adopts a Siamese structure, and the two branches use the same CNN. We add an attention module after the last layer of the CNN to improve the feature extraction ability of the network for visible and infrared images. We extend the dataset by reorganizing and re-labeling the existing sequence of visible-infrared datasets to obtain sufficiently rich training and testing data. To improve the quality of the dataset, we use the focal loss to solve the dataset's positive and negative sample imbalance problems during the training process. Compared with other methods, experimental results show that our method achieves better matching results. [ABSTRACT FROM AUTHOR]

Aiming at the problem of noise and insufficient feature extraction in event camera-based target recognition task, we proposes an event target recognition method based on DRUNet and multi-scale attention. Firstly, DRUNet is added as a filter to reduce the event noise during the conversion of the event stream into event tensor; secondly, a multi-scale convolutional layer is used instead of a single convolutional layer to extract feature information at different scales, and a depth-separable convolution is utilized to replace part of the standard convolution in the network structure to reduce the number of network parameters without losing the performance of the network; thirdly, multi-scale features are performed on different channel fusion and connecting the channel attention module to enhance the network’s representation of effective features; then the classifier is redesigned to reduce feature loss and improve recognition accuracy by compressing the semantic information layer-by-layer and step-by-step; finally, the Adam optimizer based on the gradient centered algorithm is used for training to improve the network’s generalization ability and training speed. On the N-Caltech101 and N-Cars datasets, the recognition accuracy of the model is 87.2 % and 96.3 % , respectively, which is significantly higher than other algorithms. [ABSTRACT FROM AUTHOR]

Video-based group emotion recognition is an important research area in computer vision and is of great significance for the intelligent understanding of videos and human–computer interactions. Previous studies have adopted the traditional two-stage shallow pipeline to extract visual or audio features and train classifiers. A single feature or two are insufficient to comprehensively represent video information. In addition, sparse expression of emotions has not been addressed effectively. Therefore, in this study, we propose a novel deep convolutional neural networks (CNNs) architecture for video-based group emotion recognition that fuses multimodal feature information such as vision, audio, optical flow, and face. To address the problem of sparse emotional expressions in videos, we constructed an improved keyframe extraction algorithm for a visual stream to extract keyframes with more emotional features. A subnetwork incorporating spatial and channel attention was designed to automatically concentrate on the regions and channels carrying distinctive information in each keyframe to more accurately represent the emotional features of the visual stream. The proposed model was used to conduct extensive experiments on a video group affect dataset. It outperformed other video-based group emotion recognition methods. [ABSTRACT FROM AUTHOR]

Low-light image enhancement (LLIE) can improve the visibility of low-light images. Low-light images exhibit a series of visual degradation, such as decreased visibility, low contrast, and biased colour, which hinder computer vision tasks. It is challenging to simultaneously handle these degradations. In this study, we propose a geometrical sparse representation (GSR) guided low-light enhancement model to fulfil the LLIE task. The proposed model consists of two branches to simultaneously learn the enhanced mapping of the low-light image and its GSR map. The main branch estimates the enhanced mapping of the low-light images through a generative adversarial network (GAN). Specifically, we introduce a novel structural model named the channel attention dense block (CADB) to the main branch for both efficiency and effectiveness. Moreover, there are always undesirable structural detail losses in the enhanced results. To remedy this shortcoming, we utilize a guided branch to learn the enhanced GSR map of the input low-light image and integrate it into the main branch as a structure prior. In addition, we propose a GSR-space GAN loss with GSR-space pixel loss to help generative networks concentrate more on geometrical structures. We conducted experiments on several benchmark datasets. The experimental results show that the proposed approach favourably performs against state-of-the-art, low-light image enhancement algorithms. [ABSTRACT FROM AUTHOR]

The narrowed area of coronary arteries is a common cardiovascular disease (CVD). Accurate computer-assisted treatment can find the location of the narrowed area in time and reduce the patient's life risk. This article designed a deeper residual U-Net and added the Efficient Channel Attention (ECA) module to segment Computed Tomography Angiography (CTA) images. The network is called ECA-UNet. ECA module can implement cross-channel interaction without dimensionality reduction. During our experiment, choosing an adaptive kernel size for the ECA module can effectively improve the network's performance. Experimental results proved that after selecting the adaptive cross-channel interaction coverage, the Intersection over Union (Iou) score of ECA-UNet on the testing set reaches 94.29%. Compared with other models in medical image segmentation, there is a pronounced improvement. For the segmented results, use the Ray-casting algorithm to reconstruct the three-dimensional coronary arteries, the narrowed area of the coronary artery can be clearly and intuitively observed from the reconstructed model. [ABSTRACT FROM AUTHOR]

Remote sensing is an important technical tool for rapid detection of illegal mining behavior. Due to the complex features of open-pit coal mines, there are few studies about automatic extraction of open-pit coal mines. Based on Convolutional Neural Network and Dense Block, we propose a lightweight densely connected network-AD-Net for the extraction of open-pit coal mining areas from Sentinel-2 remote sensing images, and construct three sample libraries of open-pit coal mining areas in north-central Xinzhou City, Shanxi Province. The AD-Net model consists of two convolutional layers, two pooling layers, a channel attention module, and a Dense Block. The two convolutional layers greatly reduce the complexity of the model, and the Dense Block enhances the feature propagation while reducing the parameter computation. The application is designed in different modules that runs independently on different machines and communicate with each other. Furthermore, we create and build a unique remote sensing image service system that connects a remote datacentre and its associated edge networks, employing the edge-cloud architecture. While the datacentre acts as the cloud platform and is in charge of storing and processing the original remote sensing images, the edge network is largely utilised for caching, predicting, and disseminating the processed images. First, we find out the optimal optimizer and the optimal size of the input image by extensive experiments, and then we compare the extraction effect of AD-Net with AlexNet, VGG-16, GoogLeNet, Xception, ResNet50, and DenseNet121 models in the study area. The experimental results show that the combination of NIR, red, green, and blue band synthesis is more suitable for the extraction of the open-pit coal mine, and the OA and Kappa of AD-Net reach 0.959 and 0.918 respectively, which is better than other models and well balances the classification accuracy and running speed. With this design of edge-cloud, the proposed system not only evenly distributes the strain of processing activities across the edges but also achieves data efficiency among them, reducing the cost of data transmission and improving the latency. [ABSTRACT FROM AUTHOR]

Automatic anomaly detection is a crucial task in video surveillance system intensively used for public safety and others. The present system adopts a spatial branch and a temporal branch in a unified network that exploits both spatial and temporal information effectively. The network has a residual autoencoder architecture, consisting of a deep convolutional neural network-based encoder and a multi-stage channel attention-based decoder, trained in an unsupervised manner. The temporal shift method is used for exploiting the temporal feature, whereas the contextual dependency is extracted by channel attention modules. System performance is evaluated using three standard benchmark datasets. Result suggests that our network outperforms the state-of-the-art methods, achieving 97.4% for UCSD Ped2, 86.7% for CUHK Avenue, and 73.6% for ShanghaiTech dataset in term of Area Under Curve, respectively. [ABSTRACT FROM AUTHOR]

Single image rain removal requires a large number of channel features and image texture in the process of rain removal. In this work, we propose a Channel Memory Network (CMN) for single-image rain removal, which is a multi-stage rain removing network structure similar to recurrent neural network. One Channel Memory Block (CMB) is also employed by CMN to extract rain streaks texture feature efficiently. CMB is able to focus on features on the channel and optionally selects useful information. In addition, Channel Attention Block (CAB) is adopted in skip connection to enhance the features from previous modules. Gated Recurrent Unit (GRU) reduces the loss in the process of parameters sharing and selectively transfers the feature to the next stage. The network structure shares a lot of parameters so that more rain streaks information can be used efficiently in the process of feature transmission. A quantity of experiments show that the performance of the proposed method is better than that of the state-of-the-art methods on five synthetic datasets. [ABSTRACT FROM AUTHOR]

The classification of images based on the principles of human vision is a major task in the field of computer vision. It is a common method to use multi-scale information and attention mechanism to obtain better classification performance. The methods based on multi-scale can obtain more accurate feature description by fusing different levels of information, and the methods based on attention can make the deep learning models focus on more valuable information in the image. However, the current methods usually treat the acquisition of multi-scale feature maps and the acquisition of attention weights as two separate steps in sequence. Since human eyes usually use these two methods at the same time when observing objects, we propose a multi-scale attention (MSA) module. The proposed MSA module directly extracts the attention information of different scales from a feature map, that is, the multi-scale and attention methods are simultaneously completed in one step. In the MSA module, we obtain different scales of channel and spatial attention by controlling the size of the convolution kernel for cross-channel and cross-space information interaction. Our module can be easily integrated into different convolutional neural networks to form Multi-scale attention networks (MSANet) architectures. We demonstrate the performance of MSANet on CIFAR-10 and CIFAR-100 data sets. In particular, the accuracy of our ResNet-110 based model on CIFAR-10 is 94.39%. Compared with the benchmark convolution model, our proposed multi-scale attention module can bring a roughly 3% increase in accuracy rate on CIFAR-100. Experimental results show that the proposed multi-scale attention module is superior in image classification. [ABSTRACT FROM AUTHOR]