Search

Image super-resolution (SR) techniques aim to enhance the clarity and realism of images. Recently, a wide range of excellent SR algorithms with powerful characterization capabilities have emerged and are widely used. However, there are still challenges and room for improvement in designing a lighter and more edge-friendly SR networks for hardware devices. In this paper, we propose a lightweight enhanced feature refinement network (EFRN) based on depthwise separatable convolution for SR reconstruction. The core network components consist of multiple enhanced feature refinement blocks (EFRB), which fully fuse channel features to extract more accurate low-frequency information based on the attention of different channels. In addition, a lightweight residual block (LRB) and a lightweight dual attention block (LDAB) are designed to enhance network information extraction with minimal parameter cost. We improve the feature refinement by using 1 × 1 convolution instead of a channel selection operation to reduce the dimensionality of the features and extract the refined features more efficiently. Finally, to achieve better reconstruction performance, the depth and number of channels of the network are expanded while keeping the total number of parameters at a low level. Extensive experiments have been conducted to demonstrate the superiority of our EFRN over other mainstream SR algorithms in terms of reconstruction results and the number of parameters. [ABSTRACT FROM AUTHOR]

Since the advent of the registration method based on deep learning, it has demonstrated a time efficiency advantage several orders of magnitude higher than traditional methods. However, the current deep networks have not fully explored the potential to capture spatial relationships comprehensively. Faced with the complex anatomical structures and detailed deformations in the brain, existing methods often run into difficulties. Therefore, addressing this challenging issue, we propose an unsupervised registration network for 3D brain magnetic resonance imaging (MRI). This framework adopts a hybrid structure of CNN transformer, gradually refining the deformation field using a pyramid structure and multi-level strategies, introducing a local salient position transformer to focus on local details of the deformation, and a hierarchical feature fusion module to merge features of multi-layer deformation fields further. These strategies help preserve high-level semantic information in the brain, thereby improving the quality of the deformation field and achieving deformable registration more effectively. We evaluated the proposed method on two publicly available brain MRI datasets, and the results show that our method outperforms all advanced competing methods in terms of accuracy. [ABSTRACT FROM AUTHOR]

Semantic segmentation is currently a hot topic in remote sensing image processing. There are extensive applications in land planning and surveying. Many current studies combine Convolutional Neural Networks (CNNs), which extract local information, with Transformers, which capture global information, to obtain richer information. However, the fused feature information is not sufficiently enriched and it often lacks detailed refinement. To address this issue, we propose a novel method called the Multi-View Feature Fusion and Rich Information Refinement Network (MFRNet). Our model is equipped with the Multi-View Feature Fusion Block (MAFF) to merge various types of information, including local, non-local, channel, and positional information. Within MAFF, we introduce two innovative methods. The Sliding Heterogeneous Multi-Head Attention (SHMA) extracts local, non-local, and positional information using a sliding window, while the Multi-Scale Hierarchical Compressed Channel Attention (MSCA) leverages bar-shaped pooling kernels and stepwise compression to obtain reliable channel information. Additionally, we introduce the Efficient Feature Refinement Module (EFRM), which enhances segmentation accuracy by interacting the results of the Long-Range Information Perception Branch and the Local Semantic Information Perception Branch. We evaluate our model on the ISPRS Vaihingen and Potsdam datasets. We conducted extensive comparison experiments with state-of-the-art models and verified that MFRNet outperforms other models. [ABSTRACT FROM AUTHOR]

Maximizing the advantages of different views and mitigating their respective disadvantages in fine-grained segmentation tasks are an important challenge in the field of point cloud multi-view fusion. Traditional multi-view fusion methods ignore two fatal problems: 1. the loss of depth and quantization information due to mapping and voxelization operations, resulting in "anomalies" in the extracted features; 2. how to pay attention to the large differences in object sizes among different views during point cloud learning, and fine-tune the fusion efficiency in order to improve the performance of network. In this paper, we propose a new algorithm that uses channel self-attention to fuse range-point-voxel, abbreviated as RPV-CASNet. RPV-CASNet integrates the three different views: range, point and voxel in a more subtle way through an interactive structure (range-point-voxel cross-adaptive layer known as RPVLayer for short), to take full advantage of the differences among them. The RPVLayer contains two key designs: the Feature Refinement Module (FRM) and the Multi-Fine-Grained Feature Self-Attention Module(MFGFSAM). Specifically, the FRM allows for a re-inference representation of points with entrained anomalous features, correcting the features. The MFGFSAM addresses two challenges: efficiently aggregating tokens from distant regions and preserving multiscale features within a single attention layer. In addition, we design a Dynamic Feature Pyramid Extractor (DFPE) for network deployment, which is used to extract rich features from spherical range images. Our method achieves impressive mIoU scores of 69.8% and 77.1% on the SemanticKITTI and nuScenes datasets, respectively. [ABSTRACT FROM AUTHOR]

With the development of convolutional neural networks (CNNs), salient object detection methods have made great progress in performance. Most methods are designed with complex structures to aggregate the multi-level feature maps, to reach the goal of filtering noise and obtaining rich information. However, there is no differentiation when dealing with the multi-level features, and only a uniform treatment is used in general. Based on the above considerations, in this paper, we propose a multi-branch feature fusion and refinement network (MFFRNet), which is a framework for treating low-level features and high-level features differently, and effectively fuses the information of multi-level features to make the results more accurate. We propose a detail optimization module (DOM) designed for the rich detail information in low-level features and a pyramid feature extraction module (PFEM) designed for the rich semantic information in high-level features, as well as a feature optimization module (FOM) for refining the fused feature of multiple levels. Extensive experiments are conducted on six benchmark datasets, and the results show that our approach outperforms the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Deep neural networks are vulnerable to adversarial attacks, bringing high risk to numerous security-critical applications. Existing adversarial defense algorithms primarily concentrate on optimizing adversarial training strategies to improve the robustness of neural networks, but ignore that the misguided decisions are essentially made by the activation values. Besides, such conventional strategies normally result in a great decline in clean accuracy. To address the above issues, we propose a novel RSA algorithm to counteract adversarial perturbations while maintaining clean accuracy. Specifically, RSA comprises three distinct modules: feature refinement, activation suppression, and alignment modules. First, the feature refinement module refines malicious activation values in the feature space. Subsequently, the feature activation suppression module mitigates redundant activation values induced by adversarial perturbations across both channel and spatial dimensions. Finally, to avoid an excessive performance drop on clean samples, RSA incorporates a consistency constraint and a knowledge distillation constraint for feature alignment. Extensive experiments on five public datasets and three backbone networks demonstrate that our proposed algorithm achieves consistently superior performance in both adversarial robustness and clean accuracy over the state-of-the-art. [ABSTRACT FROM AUTHOR]

Computer-aided systems can assist radiologists in improving the efficiency of COVID-19 diagnosis. Current work adopts complex structures and the correlations between convolutional neural network (CNN) and Transformer haven't been explored. We propose a novel model named bridging CNN and Transformer via online feature distillation (BCT-OFD). First, lightweight Mpvit-tiny and MobileNetV3-small are chosen as the teacher and student networks, respectively. Sufficient pathological knowledge is smoothly transferred from the teacher to the student using OFD. Then, a adaptive feature fusion module is designed to efficiently fuse the heterogeneous CNN and Transformer features. The implicit correlations between the two networks are fully mined to generate more discriminative fused features. And coordinate attention is adopted to make further feature refinement. The accuracy on three public avaliable datasets reach 97.76%, 98.12% and 96.96%, respectively. It validates that BCT-OFD outperforms state-of-the-art baselines in terms of effectiveness and generalization ability. Notably, BCT-OFD is relatively more lightweight and easier to deploy on those resource-constrained devices, making it the bridge that links theory to application as well as narrowing the gap between them. This study provides an innovative approach in the field of COVID-19 image recognition, offering valuable insights for further improvements in performance. [ABSTRACT FROM AUTHOR]

Speech data gathered from real-world environments typically contain noise, a significant element that undermines the performance of deep neural network-based speaker verification (SV) systems. To mitigate performance degradation due to noise and develop noise-robust SV systems, several researchers have integrated speech enhancement (SE) and SV systems. We previously proposed the extended U-Net (ExU-Net), which achieved state-of-the-art performance in SV in noisy environments by jointly training SE and SV systems. In the SE field, some studies have shown that recognizing noise components within speech can improve the system’s performance. Inspired by these approaches, we propose a noise-aware ExU-Net (NA-ExU-Net) that acknowledges noise information in the SE process based on the ExU-Net architecture. The proposed system comprises a Split Encoder and a feature refinement module (FRM). The Split Encoder handles the speech and noise separately by dividing the encoder blocks, whereas FRM is designed to inhibit the propagation of irrelevant data via skip connections. To validate the effectiveness of our proposed framework in noisy conditions, we evaluated the models on the VoxCeleb1 test set with added noise from the MUSAN corpus. The experimental results demonstrate that NA-ExU-Net outperforms the ExU-Net and other baseline systems under all evaluation conditions. Furthermore, evaluations in out-of-domain noise environments indicate that NA-ExU-Net significantly surpasses existing frameworks, highlighting its robustness and generalization capabilities. The codes utilized in our experiments can be accessed at https://github.com/chan-yeong0519/NA-ExU-Net.

Siamese network-based trackers have demonstrated competitive performance in the domain of single object tracking. However, their effectiveness is significantly hindered when the target undergoes challenges such as deformation and illumination changes, due to the fixed template features from the first frame. To address this issue, we propose a novel tracking framework called SiamTAR. This framework adaptively updates the current frame’s template by fusing template features from the first frame with updated template features and tracking box features from the previous frame, thereby effectively improving tracking accuracy. Additionally, to reduce the tracker’s attention to redundant information such as similar shapes, colors, and textures near the target, we designed a feature refinement module. This module integrates three attention mechanisms through two parallel branches to capture critical target information, allowing the tracker to ignore some redundant information. To tackle issues of tracking box drift and inaccurate scale estimation during online tracking, we introduce a relocation mechanism. This mechanism corrects the tracking box position by merging the output tracking box features with the template features. Extensive experiments on multiple datasets validate the superior tracking performance of SiamTAR. Specifically, on the GOT-10K dataset, SiamTAR surpasses the current leading Siamese tracker, SiamPW-RBO, by 1.5% in AO and 7.5% in $SR_{0.75}$ metrics, achieving a tracking speed of 26.23 FPS. Source code is available at https://github.com/rkj12345/SiamTAR.

Abstract Deep neural networks are vulnerable to adversarial attacks, bringing high risk to numerous security-critical applications. Existing adversarial defense algorithms primarily concentrate on optimizing adversarial training strategies to improve the robustness of neural networks, but ignore that the misguided decisions are essentially made by the activation values. Besides, such conventional strategies normally result in a great decline in clean accuracy. To address the above issues, we propose a novel RSA algorithm to counteract adversarial perturbations while maintaining clean accuracy. Specifically, RSA comprises three distinct modules: feature refinement, activation suppression, and alignment modules. First, the feature refinement module refines malicious activation values in the feature space. Subsequently, the feature activation suppression module mitigates redundant activation values induced by adversarial perturbations across both channel and spatial dimensions. Finally, to avoid an excessive performance drop on clean samples, RSA incorporates a consistency constraint and a knowledge distillation constraint for feature alignment. Extensive experiments on five public datasets and three backbone networks demonstrate that our proposed algorithm achieves consistently superior performance in both adversarial robustness and clean accuracy over the state-of-the-art.

Remote sensing image (RSI) target detection methods based on traditional multiscale feature fusion (MSFF) have achieved great success. However, the traditional MSFF method significantly increases the computational cost during model training and inference, and the simple fusion operation may lead to the semantic confusion of the feature map, which cannot realize the refined extraction of features by the model. In order to reduce the computational effort associated with the MSFF operation and to enable the features in the feature map to present an accurate, fine-grained distribution, we propose a single-stage detection model (RS-YOLO). Our main additions to RS-YOLO are a computationally smaller and faster Quick and Small E-ELEN (QS-E-ELEN) module and a feature refinement extraction (FRE) module. In the QS-E-ELEN module, We utilize QSBlock, jump-join, and convolution operations to fuse features on different scales and reduce the computational effort of the model by exploiting the similarity of the RSI feature map channels. In order for the model to better utilize the enhanced features, we designed the FRE module to make the location of the enhanced features more accurate and fine. By conducting experiments on the popular NWPU-VHR- 10 and SSDD datasets, we derive results showing that RS-YOLO outperforms most mainstream models in terms of the tradeoff between accuracy and speed. Specifically, in terms of accuracy, it improves 1.6$\%$ and 1.7$\%$ compared to the current state-of-the-art models, respectively. At the same time, RS-YOLO reduces the number of parameters and computational effort.

The current computer-aided diagnosis for cervical cancer screening encounters issues with missing detailed information during colposcopic image segmentation and incomplete edge delineation. To overcome these challenges, this study introduces the RUC-U2Net architecture, which enhances image segmentation through feature refinement and upsampling connections. Two variants are developed: RUC-U2Net and the lightweight RUC+-U2Net. Initially, a feature refinement module that leverages an attention mechanism is proposed to improve detail capture by the model’s fundamental unit during downsampling. Subsequently, the integration of diagonal attention in connecting peer-level encoders and decoders supplements finer semantic details to the decoder’s feature maps, addressing the problem of incomplete edge segmentation. Finally, the application of the Focal Tversky loss function allows the model to concentrate on difficult samples, mitigating the challenges posed by imbalanced distributions of positive and negative samples in training datasets. Experimental evaluations on three publicly available datasets demonstrate that the proposed models significantly outperform existing methods across seven performance metrics, evidencing their superior segmentation accuracy.

Image super-resolution (SR) techniques aim to enhance the clarity and realism of images. Recently, a wide range of excellent SR algorithms with powerful characterization capabilities have emerged and are widely used. However, there are still challenges and room for improvement in designing a lighter and more edge-friendly SR networks for hardware devices. In this paper, we propose a lightweight enhanced feature refinement network (EFRN) based on depthwise separatable convolution for SR reconstruction. The core network components consist of multiple enhanced feature refinement blocks (EFRB), which fully fuse channel features to extract more accurate low-frequency information based on the attention of different channels. In addition, a lightweight residual block (LRB) and a lightweight dual attention block (LDAB) are designed to enhance network information extraction with minimal parameter cost. We improve the feature refinement by using 1 × 1 convolution instead of a channel selection operation to reduce the dimensionality of the features and extract the refined features more efficiently. Finally, to achieve better reconstruction performance, the depth and number of channels of the network are expanded while keeping the total number of parameters at a low level. Extensive experiments have been conducted to demonstrate the superiority of our EFRN over other mainstream SR algorithms in terms of reconstruction results and the number of parameters.

RGB-D salient object detection (RGB-D SOD) has currently attracted much attention for its prospect of broad application. On the basis of the "encoder-decoder" paradigm of the fully convolutional network (FCN), many FCN-based strategies have emerged and achieved huge progress, but underestimated the potential of level-specific characteristics of multi-modal features. In this paper, we propose the adaptive guided fusion network (AGFNet) to further mine the potential information between the depth image and the RGB image, and design an adaptive fusion and coarse-to-fine decoding strategy to achieve high-precision detection of salient objects. Specifically, we first use a two-stream encoder to extract the multi-level features of the RGB image and depth image but refrain from the previous practice of using depth features for each layer. Second, a simple but effective way named multi-modal selective fusion strategy is designed to fuse the multi-level features. Third, for enhancement of contextual information of each level adaptively, an adaptive cross fusion module (ACFM) fuses the features at all levels and outputs a coarse saliency map. Finally, a guided attention refinement module (GARM) utilizes the coarse saliency map to guide the final features from ACFM to realize the enhancement and obtain a refined saliency map. Our method is compared with other state-of-the-art RGB-SOD methods through extensive experiments, and the results demonstrate the superiority of our proposed AGFNet. The source code of this project is available at https://github.com/HaodongSun809/my_AGFNet.git. [ABSTRACT FROM AUTHOR]

Accurate calibration between LiDAR and camera sensors is crucial for autonomous driving systems to perceive and understand the environment effectively. Typically, LiDAR–camera extrinsic calibration requires feature alignment and overlapping fields of view. Aligning features from different modalities can be challenging due to noise influence. Therefore, this paper proposes a targetless extrinsic calibration method for monocular cameras and LiDAR sensors that have a non-overlapping field of view. The proposed solution uses pose transformation to establish data association across different modalities. This conversion turns the calibration problem into an optimization problem within a visual SLAM system without requiring overlapping views. To improve performance, line features serve as constraints in visual SLAM. Accurate positions of line segments are obtained by utilizing an extended photometric error optimization method. Moreover, a strategy is proposed for selecting appropriate calibration methods from among several alternative optimization schemes. This adaptive calibration method selection strategy ensures robust calibration performance in urban autonomous driving scenarios with varying lighting and environmental textures while avoiding failures and excessive bias that may result from relying on a single approach. [ABSTRACT FROM AUTHOR]

We propose a Context-aware Feature Transformer Network (CaFTNet), a novel network for human pose estimation. To address the issue of limited modeling of global dependencies in convolutional neural networks, we design the Transformerneck to strengthen the expressive power of features. Transformerneck directly substitutes 3 × 3 convolution in the bottleneck of HRNet with a Contextual Transformer (CoT) block while reducing the complexity of the network. Specifically, the CoT first produces keys with static contextual information through 3 × 3 convolution. Then, relying on query and contextualization keys, dynamic contexts are generated through two concatenated 1 × 1 convolutions. Static and dynamic contexts are eventually fused as an output. Additionally, for multi-scale networks, in order to further refine the features of the fusion output, we propose an Attention Feature Aggregation Module (AFAM). Technically, given an intermediate input, the AFAM successively deduces attention maps along the channel and spatial dimensions. Then, an adaptive refinement module (ARM) is exploited to activate the obtained attention maps. Finally, the input undergoes adaptive feature refinement through multiplication with the activated attention maps. Through the above procedures, our lightweight network provides powerful clues for the detection of keypoints. Experiments are performed on the COCO and MPII datasets. The model achieves a 76.2 AP on the COCO val2017 dataset. Compared to other methods with a CNN as the backbone, CaFTNet has a 72.9% reduced number of parameters. On the MPII dataset, our method uses only 60.7% of the number of parameters, acquiring similar results to other methods with a CNN as the backbone. [ABSTRACT FROM AUTHOR]

Occluded person re-identification (ReID) aims to retrieve the same pedestrian from partially occluded pedestrian images across non-overlapping cameras. Current state-of-the-art methods generally use auxiliary models to obtain non-occluded regions, which not only result in more complex models, but also cannot effectively handle the more generalized ReID task. To this end, a Multi-Branch Attention Network (MBA-Net) is proposed to achieve multi-level refinement of features through an end-to-end multi-branch framework with attention mechanisms. Specifically, we first achieve preliminary feature refinement through a backbone network with a non-local attention mechanism. Then, a two-level multi-branch architecture in MBA-Net is proposed with two-level features refinement to obtain aware local discriminative features from the self-attention branch, non-occluded local complementary features from the cross-attention branch, and global features from the global branch. Finally, we can obtain retrieval features that are robust to occlusion by concatenating all the above features. Experimental results show that our MBA-Net achieves state-of-the-art performance on an occluded person ReID dataset Occluded-Duke and simultaneously achieves competitive performance on two general person ReID datasets Market-1501 and DukeMTMC-ReID. [ABSTRACT FROM AUTHOR]

Recently, red-green-blue (RGB) and thermal (RGB-T) data have attracted considerable interest for semantic segmentation because they provide robust imaging under the complex lighting conditions of urban roads. Most existing RGB-T semantic segmentation methods adopt an encoder-decoder structure, and repeated upsampling causes semantic information loss during decoding. Moreover, using simple cross-modality fusion neither completely mines complementary information from different modalities nor removes noise from the extracted features. To address these problems, we developed a dual-decoding hierarchical fusion network (DHFNet) to extract RGB and thermal information for RGB-T Semantic Segmentation. DHFNet uses a novel two-layer decoder and implements boundary refinement and boundary-guided foreground/background enhancement modules. The modules process features from different levels to achieve the global guidance and local refinement of the segmentation prediction. In addition, an adaptive attention-filtering fusion module filters and extracts complementary information from the RGB and thermal modalities. Further, we introduce a graph convolutional network and an atrous spatial pyramid pooling module to obtain multiscale features and deepen the extracted semantic information. Experimental results on two benchmark datasets showed that the proposed DHFNet performed well relative to state-of-the-art semantic segmentation methods in terms of different evaluation metrics. [ABSTRACT FROM AUTHOR]

Semantic segmentation is currently a hot topic in remote sensing image processing. There are extensive applications in land planning and surveying. Many current studies combine Convolutional Neural Networks (CNNs), which extract local information, with Transformers, which capture global information, to obtain richer information. However, the fused feature information is not sufficiently enriched and it often lacks detailed refinement. To address this issue, we propose a novel method called the Multi-View Feature Fusion and Rich Information Refinement Network (MFRNet). Our model is equipped with the Multi-View Feature Fusion Block (MAFF) to merge various types of information, including local, non-local, channel, and positional information. Within MAFF, we introduce two innovative methods. The Sliding Heterogeneous Multi-Head Attention (SHMA) extracts local, non-local, and positional information using a sliding window, while the Multi-Scale Hierarchical Compressed Channel Attention (MSCA) leverages bar-shaped pooling kernels and stepwise compression to obtain reliable channel information. Additionally, we introduce the Efficient Feature Refinement Module (EFRM), which enhances segmentation accuracy by interacting the results of the Long-Range Information Perception Branch and the Local Semantic Information Perception Branch. We evaluate our model on the ISPRS Vaihingen and Potsdam datasets. We conducted extensive comparison experiments with state-of-the-art models and verified that MFRNet outperforms other models.

Object detection is a fundamental aspect of computer vision, with numerous generic object detectors proposed by various researchers. The proposed work presents a novel single-stage rotation detector that can detect oriented and multi-scale objects accurately from diverse scenarios. This detector addresses the challenges faced by current rotation detectors, such as the detection of arbitrary orientations, objects that are densely arranged, and the issue of loss discontinuity. First, the detector also adopts a progressive regression form (coarse-to-fine-grained approach) that uses both horizontal anchors (speed and higher recall) and rotating anchors (oriented objects) in cluttered backgrounds. Second, the proposed detector includes a feature refinement module that helps minimize the problems related to feature angulation and reduces the number of bounding boxes generated. Finally, to address the issue of loss discontinuity, the proposed detector utilizes a newly formulated adjustable loss function that can be extended to both single-stage and two-stage detectors. The proposed detector shows outstanding performance on benchmark datasets and significantly outperforms other state-of-the-art methods in terms of speed and accuracy.

Change detection (CD) plays a critical role in extracting ground changes from bi-temporal remote sensing (RS) images and is instrumental in understanding surface dynamics. In recent years, deep learning has made significant breakthroughs in CD. However, typical CD methods that employ the Siamese network for temporal feature extraction lack feature alignment ability for bi-temporal heterogeneous RS images, resulting in inadequate temporal discriminative capability. Moreover, deep learning-based CD methods are still susceptible to the problem of minor changes missing due to scale variation with deeper network layers. In this article, we propose a bi-temporal feature alignment and refinement network (FARNet). To improve the discriminative capability of the Siamese network, an adversarial learning-based temporal discriminatory loss function is designed to align temporal-level features and eliminate bi-temporal domain shift, and a cosine similarity-based loss function is employed to measure feature distance at the Pixel-level. To address the problem of minor changes missing, we adopt a dilated convolution-based Siamese network to prevent feature map size reduction, and a multi-level feature detail supplement (MFDS) module is designed to supplement the deep layer features with shallow layer features. Additionally, we construct a change map refinement (CMR) module that refines the coarse change map to the finegrained change map. Furthermore, we design a cross-temporal feature interaction (CFI) module to learn more fine-grained change features by combining features across temporal. Comprehensive experimental results on two popular CD datasets demonstrate the effectiveness and efficiency of FARNet compared with state-of-theart (SOTA) methods. [ABSTRACT FROM AUTHOR]

3D shape recognition is a critical research topic in the field of computer vision, attracting substantial attention. Existing approaches mainly focus on extracting distinctive 3D shape features; however, they often neglect the model's robustness and lack refinement in deep features. To address these limitations, we propose the point-view fusion attention network that aims to extract a concise, informative, and robust 3D shape descriptor. Initially, our approach combines multi-view features with point cloud features to obtain accurate and distinguishable fusion features. To effectively handle these fusion features, we design a dual-attention convolutional network which consists of a channel attention module and a spatial attention module. This dual-attention mechanism greatly enhances the generalization ability and robustness of 3D recognition models. Notably, we introduce a strip-pooling layer in the channel attention module to refine the features, resulting in improved fusion features that are more compact. Finally, a classification process is performed on the refined features to assign appropriate 3D shape labels. Our extensive experiments on the ModelNet10 and ModelNet40 datasets for 3D shape recognition and retrieval demonstrate the remarkable accuracy and robustness of the proposed method. [ABSTRACT FROM AUTHOR]

As an emerging computer vision task, crowd localization has received increasing attention due to its ability to produce more accurate spatially predictions. However, continuous scale variations in complex crowd scenes lead to tiny individuals at the edges, so that existing methods cannot achieve precise crowd localization. Aiming at alleviating the above problems, we propose a novel Dilated Convolution-based Feature Refinement Network (DFRNet) to enhance the representation learning capability. Specifically, the DFRNet is built with three branches that can capture the information of each individual in crowd scenes more precisely. More specifically, we introduce a Feature Perception Module to model long-range contextual information at different scales by adopting multiple dilated convolutions, thus providing sufficient feature information to perceive tiny individuals at the edge of images. Afterwards, a Feature Refinement Module is deployed at multiple stages of the three branches to facilitate the mutual refinement of feature information at different scales, thus further improving the expression capability of multi-scale contextual information. By incorporating the above modules, DFRNet can locate individuals in complex scenes more precisely. Extensive experiments on multiple datasets demonstrate that the proposed method has more advanced performance compared to existing methods and can be more accurately adapted to complex crowd scenes. [ABSTRACT FROM AUTHOR]

Video object detection (VOD) is a sophisticated visual task. It is a consensus that is used to find effective supportive information from correlation frames to boost the performance of the model in VOD tasks. In this paper, we not only improve the method of finding supportive information from correlation frames but also strengthen the quality of the features extracted from the correlation frames to further strengthen the fusion of correlation frames so that the model can achieve better performance. The feature refinement module FRM in our model refines the features through the key–value encoding dictionary based on the even-order Taylor series, and the refined features are used to guide the fusion of features at different stages. In the stage of correlation frame fusion, the generative MLP is applied in the feature aggregation module DFAM to fuse the refined features extracted from the correlation frames. Experiments adequately demonstrate the effectiveness of our proposed approach. Our YOLOX-based model can achieve 83.3% AP50 on the ImageNet VID dataset. [ABSTRACT FROM AUTHOR]

In modern energy systems, substations are the core of electricity transmission and distribution. However, similar appearance and small size pose significant challenges for automatic identification of electrical devices. To address these issues, we collect and annotate the substation rotated device dataset (SRDD). Further, feature fusion and feature refinement network (F3RNet) are constructed based on the classic structure pattern of backbone-neck-head. Considering the similar appearance of electrical devices, the deconvolution fusion module (DFM) is designed to enhance the expression of feature information. The balanced feature pyramid (BFP) is embedded to aggregate the global feature. The feature refinement is constructed to adjust the original feature maps by considering the feature alignment between the anchors and devices. It can generate more accurate feature vectors. To address the problem of sample imbalance between electrical devices, the gradient harmonized mechanism (GHM) loss is utilized to adjust the weight of each sample. The ablation experiments are conducted on the SRDD dataset. F3RNet achieves the best detection performance compared with classical object detection networks. Also, it is verified that the features from global feature maps can effectively recognize the similar and small devices.

Robust multi-scale object detection is challenging as it requires both spatial details and semantic knowledge to deal with problems including high scale variation and cluttered background. Appropriate fusion of high-resolution features with deep semantic features is the key issue to achieve better performance. Different approaches have been developed to extract and combine deep features with shallow layer spatial features, such as feature pyramid network. However, high-resolution feature maps contain noisy and distractive features. Directly combines shallow features with semantic features might degrade detection accuracy. Besides, contextual information is also important for multi-scale object detection. In this work, we present a feature refinement scheme to tackle the feature fusion problem. The proposed feature refinement module increases feature resolution and refine feature maps progressively with the guidance from deep features. Meanwhile, we propose a context extraction method to capture global and local contextual information. The method utilizes a multi-level cross-pooling unit to extract global context and a cascaded context module to extract local context. The proposed object detection framework has been evaluated on PASCAL VOC and MS COCO datasets. Experimental results demonstrate that the proposed method performs favorably against state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Fabric defect detection plays an irreplaceable role in the quality control of the textile manufacturing industry, but it is still a challenging task due to the diversity and complexity of defects and environmental factors. Visual saliency models imitating the human vision system can quickly determine the defect regions from the complex texture background. However, most visual saliency-based methods still suffer from incomplete predictions owing to the variability of fabric defects and low contrast with the background. In this paper, we develop a context-aware attention cascaded feedback network for fabric defect detection to achieve more accurate predictions, in which a parallel context extractor is designed to characterize the multi-scale contextual information. Moreover, a top-down attention cascaded feedback module was devised adaptively to select the important multi-scale complementary information and then transmit it to an adjacent shallower layer to compensate for the inconsistency of information among layers for accurate location. Finally, a multi-level loss function is applied to guide our model for generating more accurate prediction results via optimizing multiple side-output predictions. Experimental results on the two fabric datasets built under six widely used evaluation metrics demonstrate that our proposed framework outperforms state-of-the-art models remarkably. [ABSTRACT FROM AUTHOR]

Change detection (CD) plays an important role in Earth surface analysis. Current CD methods have achieved good performance in large flat areas, but CD of detailed parts is still a great challenge, and the loss of detail causes many faults around the change boundaries and on small objects. By analyzing the feature map of the widely used U-Net architecture in existing methods, we ascribe the detail loss to the depletion of detailed features during the top-to-down delivery in the U-Net architecture. The feature refine CD (FRCD) model is proposed in which the detection results are predicted directly from the multiscale features instead of the U-Net architecture. By direct prediction, the representation ability of details is enhanced, and thus the detection accuracy (Acc) of boundaries and small objects improves. Moreover, the normal upsampling in direct prediction is replaced with the deformable upsampling, which delivers detailed information from the low-level to the high-level via the deformable convolution, allowing the results to further fit boundaries in the FRCD model. Experimental results on two datasets confirm the effectiveness of FRCD compared to the state-of-the-art methods, and the CD results of boundaries and small objects are improved significantly by the proposed method. Code will be available after the acceptance of the letter in https://github.com/ijnokml/cdfr. [ABSTRACT FROM AUTHOR]

Strip surface defects have large intraclass and small interclass differences, resulting in the available detection techniques having either a low accuracy or very poor real-time performance. In order to improve the ability for capturing steel surface defects, the context fusion structure introduces the local information of the shallow layer and the semantic information of the deep layer into multiscale feature maps. In addition, for filtering the semantic conflicts and redundancies arising from context fusion, a feature refinement module is introduced in our method, which further improves the detection accuracy. Our experimental results show that this significantly improved the performance. In particular, our method achieved 79.5% mAP and 71 FPS on the public NEU-DET dataset. This means that our method had a higher detection accuracy compared to other techniques. [ABSTRACT FROM AUTHOR]

Localization of anatomical landmarks is essential for clinical diagnosis, treatment planning, and research. This paper proposes a novel deep network named feature aggregation and refinement network (FARNet) for automatically detecting anatomical landmarks. FARNet employs an encoder-decoder structure architecture. To alleviate the problem of limited training data in the medical domain, we adopt a backbone network pre-trained on natural images as the encoder. The decoder includes a multi-scale feature aggregation module for multi-scale feature fusion and a feature refinement module for high-resolution heatmap regression. Coarse-to-fine supervisions are applied to the two modules to facilitate end-to-end training. We further propose a novel loss function named Exponential Weighted Center loss for accurate heatmap regression, which focuses on the losses from the pixels near landmarks and suppresses the ones from far away. We evaluate FARNet on three publicly available anatomical landmark detection datasets, including cephalometric, hand, and spine radiographs. Our network achieves state-of-the-art performances on all three datasets. Code is available at https://github.com/JuvenileInWind/FARNet. [ABSTRACT FROM AUTHOR]

Accurate calibration between LiDAR and camera sensors is crucial for autonomous driving systems to perceive and understand the environment effectively. Typically, LiDAR–camera extrinsic calibration requires feature alignment and overlapping fields of view. Aligning features from different modalities can be challenging due to noise influence. Therefore, this paper proposes a targetless extrinsic calibration method for monocular cameras and LiDAR sensors that have a non-overlapping field of view. The proposed solution uses pose transformation to establish data association across different modalities. This conversion turns the calibration problem into an optimization problem within a visual SLAM system without requiring overlapping views. To improve performance, line features serve as constraints in visual SLAM. Accurate positions of line segments are obtained by utilizing an extended photometric error optimization method. Moreover, a strategy is proposed for selecting appropriate calibration methods from among several alternative optimization schemes. This adaptive calibration method selection strategy ensures robust calibration performance in urban autonomous driving scenarios with varying lighting and environmental textures while avoiding failures and excessive bias that may result from relying on a single approach.

Different from the object detection which has made great progress in natural imagery, transmission line images acquired by UAVs have their own challenges in detecting defects in critical parts, such as object scale variation and small defect targets. In this paper, we construct an effective architecture, called FOLO, to improve the accuracy of defect detection in critical parts of transmission lines. To capture the critical part defect object features, a local contextual feature pyramid network (LCFPN) is proposed to refine the local contextual information and perform multi-scale learning. In LCFPN, we introduce a channel feature refinement block (CFRB) and multiple spatial feature refinement block (SFRBs) to further improve the ability of the network to focus on local features. Besides, a local adaptive feature network (LAFN) is designed, which makes it possible to locate adaptive components with defects in critical areas of different shapes. Since existing transmission line datasets have a single category, we create a new defect detection dataset containing insulators, anti-vibration hammers and bird nests, named IVB. Experimental results on IVB show that the proposed FOLO yields promising performance against other approaches. [ABSTRACT FROM AUTHOR]

Precisely predicting the drug–drug interaction (DDI) is an important application and host research topic in drug discovery, especially for avoiding the adverse effect when using drug combination treatment for patients. Nowadays, machine learning and deep learning methods have achieved great success in DDI prediction. However, we notice that most of the works ignore the importance of the relation type when building the DDI prediction models. In this work, we propose a novel R |$^2$| -DDI framework, which introduces a relation-aware feature refinement module for drug representation learning. The relation feature is integrated into drug representation and refined in the framework. With the refinement features, we also incorporate the consistency training method to regularize the multi-branch predictions for better generalization. Through extensive experiments and studies, we demonstrate our R |$^2$| -DDI approach can significantly improve the DDI prediction performance over multiple real-world datasets and settings, and our method shows better generalization ability with the help of the feature refinement design. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) perform well in tasks of segmenting buildings from remote sensing images. However, the intraclass heterogeneity of buildings is high in images, while the interclass homogeneity between buildings and other nonbuilding objects is low. This leads to an inaccurate distinction between buildings and complex backgrounds. To overcome this challenge, we propose an Attentional Feature Learning Network (AFL-Net) that can accurately extract buildings from remote sensing images. We designed an attentional multiscale feature fusion (AMFF) module and a shape feature refinement (SFR) module to improve building recognition accuracy in complex environments. The AMFF module adaptively adjusts the weights of multi-scale features through the attention mechanism, which enhances the global perception and ensures the integrity of building segmentation results. The SFR module captures the shape features of the buildings, which enhances the network capability for identifying the area between building edges and surrounding nonbuilding objects and reduces the over-segmentation of buildings. An ablation study was conducted with both qualitative and quantitative analyses, verifying the effectiveness of the AMFF and SFR modules. The proposed AFL-Net achieved 91.37, 82.10, 73.27, and 79.81% intersection over union (IoU) values on the WHU Building Aerial Imagery, Inria Aerial Image Labeling, Massachusetts Buildings, and Building Instances of Typical Cities in China datasets, respectively. Thus, the AFL-Net offers the prospect of application for successful extraction of buildings from remote sensing images. [ABSTRACT FROM AUTHOR]

Fabric defect detection plays a critical role for measuring quality control in the textile manufacturing industry. Deep learning-based saliency models can quickly spot the most interesting regions that attract human attention from the complex background, which have been successfully applied in fabric defect detection. However, most of the previous methods mainly adopted multi-level feature aggregation yet ignored the complementary relationship among different features, and thus resulted in poor representation capability for the tiny and slender defects. To remedy these issues, we propose a novel saliency-based fabric defect detection network, which can exploit the complementary information between different layers to enhance the representation features ability and discrimination of defects. Specifically, a multiscale feature aggregation unit (MFAU) is proposed to effectively characterize the multi-scale contextual features. Besides, a feature fusion refinement module (FFR) composed of an attention fusion unit (AFU) and an auxiliary refinement unit (ARU) is designed to exploit complementary important information and further refine the input features for enhancing the discriminative ability of defect features. Finally, a multi-level deep supervision (MDS) is adopted to guide the model to generate more accurate saliency maps. Under different evaluation metrics, our proposed method outperforms most state-of-the-art methods on our developed fabric datasets. [ABSTRACT FROM AUTHOR]

Both clinical images and metadata are the foundation of clinical diagnosis, effectively fusing these two resources is a major difficulty in the detection of skin cancer. Even though existing fusion methods produced better fusion outcomes, they only carried out single-level fusion prior to making decisions and used distinct feature extraction for each modal data. The ability of inter-modal synergy is diminished by this fusion strategy, resulting in coarse fusion features. To enhance the multidimensional representation of images, we suggest a Self-contrastive Feature Guidance Based Multidimensional Collaborative Network (SGMC Net). Specifically, we split the fusion method into three steps: spatial dimension fusion, channel dimension fusion, and adaptive corrective outputting to establish multidimensional collaboration between metadata and image features in the feature extraction process. Accordingly, we build three blocks: channel fusion block, spatial fusion block, and feature rectification block. On this basis, we propose a Self-contrastive Feature Guidance method that utilizes the contrast loss between shallow and deep features of the image as a supervisory signal in a non-enhanced manner to optimize shallow features. Finally, extensive experiments were conducted on PAD-UFES-20 and Der7pt dataset, our method achieved an accuracy of 83.3% beyond other state-of-the-art models. We further validated the effectiveness of the feature guidance method, showing a 5.2% improvement in accuracy for SGMC18. • Implementing multidimensional collaboration between metadata and image features. • Designed a non-enhanced manner to promote convergence based on contrast learning. • Introduced a spatial-wise fusion method between metadata and image feature. • SGMC achieves SOTA on PAD-UFES20 dataset. [ABSTRACT FROM AUTHOR]

Modeling actor interaction relations is crucial for group activity recognition. Previous approaches often adopt a fixed paradigm that involves calculating an affinity matrix to model these interaction relations, yielding significant performance. On the one hand, the affinity matrix introduces an inductive bias that actor interaction relations should be dynamically computed based on the input actor features. On the other hand, MLPs with static parameterization, in which parameters are fixed after training, can represent arbitrary functions. Therefore, it is an open question whether inductive bias is necessary for modeling actor interaction relations. To explore the impact of this inductive bias, we propose an affinity matrix-free paradigm that directly uses the MLP with static parameterization to model actor interaction relations. We term this approach MLP-AIR. This paradigm overcomes the limitations of the inductive bias and enhances the capture of implicit actor interaction relations. Specifically, MLP-AIR consists of two sub-modules: the MLP-based Interaction relation modeling module (MLP-I) and the MLP-based Relation refining module (MLP-R). MLP-I is used to model the spatial–temporal interaction relations by emphasizing cross-actor and cross-frame feature learning. Meanwhile, MLP-R is used to refine the relation between different channels of each relation feature, thereby enhancing the expression ability of the features. MLP-AIR is a plug-and-play module. To evaluate our module, we applied MLP-AIR to replicate three representative methods. We conducted extensive experiments on two widely used benchmarks—the Volleyball and Collective Activity datasets. The experiments demonstrate that MLP-AIR achieves favorable results. The code is available at https://github.com/Xuguoliang12/MLP-AIR. • We propose an MLP-based module for interaction relation modeling implicitly. • Our module is designed by MLP entirely, simplifying the module's structure. • Our module is a universal module that can be used for other methods. [ABSTRACT FROM AUTHOR]

• We propose a pyramid network with progressive guidance structures. • Global and local information of features is supplemented by the pyramid structure. • The MEFM can reduce redundant information of depth maps and aggregate multiscale context information. • The FRM captures and integrates low-level spatial information and high-level semantic information to complete the feature maps. In RGB-D (red–green–blue and depth) scene semantic segmentation, depth maps provide rich spatial information to RGB images to achieve high performance. However, properly aggregating depth information and reducing noise and information loss during feature encoding after fusion are challenging aspects in scene semantic segmentation. To overcome these problems, we propose a pyramid gradual-guidance network for RGB-D indoor scene semantic segmentation. First, the quality of depth information is improved by a modality-enhancement fusion module and RGB image fusion. Then, the representation of semantic information is improved by multiscale operations. The two resulting adjacent features are used in a feature refinement module with an attention mechanism to extract semantic information. The features from adjacent modules are successively used to form an encoding pyramid, which can substantially reduce information loss and thereby ensure information integrity. Finally, we gradually integrate features at the same scale obtained from the encoding pyramid during decoding to obtain high-quality semantic segmentation. Experimental results obtained from two commonly used indoor scene datasets demonstrate that the proposed pyramid gradual-guidance network attains the highest level of performance in semantic segmentation, as compared to other existing methods. [ABSTRACT FROM AUTHOR]

As the largest target in remote sensing images, buildings have important application value in urban planning and old city reconstruction. However, most networks have poor recognition ability on high resolution images, resulting in blurred boundaries in the segmented building maps. Then, the similarity between buildings and backgrounds will lead to inter-class indistinction. Finally, the diversity of buildings brings difficulties to segmentation, which requires the network to have better generalization ability. To address these problems, we propose Fusion Multi-scale Attention Mechanism Network (FMAM-Net). Firstly, we design Feature Refine Compensation Module(FRCM) to improve the boundary ambiguity problem, including Feature Refinement Module(FRM) and Feature Compensation Module(FCM). FRM utilizes the densely connected architecture to refine features and increase recognition capabilities. FCM introduces low-level features to make up for the lack of boundary information in high-level features. Secondly, to handle inter-class indistinction, we design Tandem Attention Module(TAM) and Parallel Attention Module(PAM). TAM is designed to sequentially filter some features from channels and spaces for adaptive feature refinement. PAM combines context information and uses high-level features to guide low-level features to select more distinguishable features. Finally, based on the binary cross entropy loss function, we add an evaluation index to reduce the error caused by determining the optimization direction only through cross entropy. On the Inria Aerial Image Labeling Dataset, FMAM-Net achieves mean IoU of 85.34%, which is 5.58% higher than AMUNet and 3.77% higher than our baseline(U-Net ResNet-34). On the WHU Dataset, IoU reached the maximum value of 91.06% on FMAM-Net, 1.67% higher than SARB-UNet and 0.2% higher than MAP-Net. The visualization results show that FMAM-Net improves the fuzzy boundary of building segmentation and reduces the inter-class indistinction.

As a traditional computer vision task, monocular depth estimation plays an essential role in novel view 3D reconstruction and augmented reality. Convolutional neural network (CNN)-based models have achieved good performance for this task. However, in the depth map recovered by some existing deep learning-based methods, local details are still lost. To generate convincing depth maps with rich local details, this study proposes an efficient multilevel pyramid network for monocular depth estimation based on feature refinement and adaptive fusion. Specifically, a multilevel spatial feature generation scheme is developed to extract rich features from the spatial branch. Then, a feature refinement module that combines and enhances these multilevel contextual and spatial information is designed to derive detailed information. In addition, we design an adaptive fusion block for improving the capability of fully connected features. The performance evaluation results on public RGBD datasets indicate that the proposed approach can recover reasonable depth outputs with better details and outperform several depth recovery algorithms from a qualitative and quantitative perspective. [ABSTRACT FROM AUTHOR]

The refined segmentation of nuclei and the cytoplasm is the most challenging task in the automation of cervical cell screening. The U-Shape network structure has demonstrated great superiority in the field of biomedical imaging. However, the classical U-Net network cannot effectively utilize mixed domain information and contextual information, and fails to achieve satisfactory results in this task. To address the above problems, a module based on global dependency and local attention (GDLA) for contextual information modeling and features refinement, is proposed in this study. It consists of three components computed in parallel, which are the global dependency module, the spatial attention module, and the channel attention module. The global dependency module models global contextual information to capture a priori knowledge of cervical cells, such as the positional dependence of the nuclei and cytoplasm, and the closure and uniqueness of the nuclei. The spatial attention module combines contextual information to extract cell boundary information and refine target boundaries. The channel and spatial attention modules are used to provide adaption of the input information, and make it easy to identify subtle but dominant differences of similar objects. Comparative and ablation experiments are conducted on the Herlev dataset, and the experimental results demonstrate the effectiveness of the proposed method, which surpasses the most popular existing channel attention, hybrid attention, and context networks in terms of the nuclei and cytoplasm segmentation metrics, achieving better segmentation performance than most previous advanced methods. [ABSTRACT FROM AUTHOR]

Learning discriminative and rich features is an important research task for person re-identification. Previous studies have attempted to capture global and local features at the same time and layer of the model in a non-interactive manner, which are called synchronous learning. However, synchronous learning leads to high similarity, and further defects in model performance. To this end, we propose asynchronous learning based on the human visual perception mechanism. Asynchronous learning emphasizes the time asynchrony and space asynchrony of feature learning and achieves mutual promotion and cyclical interaction for feature learning. Furthermore, we design a dynamic progressive refinement module to improve local features with the guidance of global features. The dynamic property allows this module to adaptively adjust the network parameters according to the input image, in both the training and testing stage. The progressive property narrows the semantic gap between the global and local features, which is due to the guidance of global features. Finally, we have conducted several experiments on four datasets, including Market1501, CUHK03, DukeMTMC-ReID, and MSMT17. The experimental results show that asynchronous learning can effectively improve feature discrimination and achieve strong performance. [ABSTRACT FROM AUTHOR]

In visual tracking tasks, researchers usually focus on increasing the complexity of the model or only discretely focusing on the changes in the object itself to achieve accurate recognition and tracking of the moving object. However, they often overlook the significant contribution of video-level linear temporal information fusion and continuous spatiotemporal mapping to tracking tasks. This oversight may lead to poor tracking performance or insufficient real-time ability of the model in complex scenes. Therefore, this paper proposes a real-time tracker, namely Continuous Temporal Information Fusion Tracker (CTIFTrack). The key of CTIFTrack lies in its well-designed Temporal Information Fusion (TIF) module, which cleverly performs a linear fusion of the temporal information between the (t - 1) -th and the t -th frames and completes the spatiotemporal mapping. This enables the tracker to better understand the overall spatiotemporal information and contextual spatiotemporal correlations within the video, thereby having a positive impact on the tracking task. In addition, this paper also proposes the Object Template Feature Refinement (OTFR) module, which effectively captures the global information and local details of the object, and further improves the tracker's understanding of the object features. Extensive experiments are conducted on seven benchmarks, such as LaSOT, GOT-10K, UAV123, NFS, TrackingNet, VOT2018 and OTB-100. The experimental results validate the significant contribution of the TIF module and OTFR module to the tracking task, as well as the effectiveness of CTIFTrack. It is worth noting that while maintaining excellent tracking performance, CTIFTrack also shows outstanding real-time tracking speed. On the Nvidia Tesla T4-16GB GPU, the F P S of CTIFTrack reaches 71.98. The code and demo materials will be available at https://github.com/vpsg-research/CTIFTrack. • A video-level linear temporal information fusion method is proposed for tracking task. • A continuous spatiotemporal mapping module is proposed for video-level features. • A multi-receptive field object template feature refinement module is proposed. • The proposed model has excellent tracking ability in the scene of object occlusion. • Our model has remarkable robustness and generalization ability. [ABSTRACT FROM AUTHOR]

Discriminative correlation filters (DCF) have achieved much success in visual tracking. However, most of them simply rely on the features extracted by the last layer of the backbone, while ignoring the low-level rich structural information. In addition, they normally minimize the tailored objective functions to predict the target model in a direct way, which introduces inductive bias and limits the expressivity of the trackers. In view of this, a pyramidal feature fusion module is proposed in this paper to integrate the low-resolution, semantically strong features with high-resolution, semantically weak features. Then, an asymmetric Transformer structure is applied to predict the weights of the model. Finally, a feature refinement module is employed to optimize the search features. Extensive experiments on 5 mainstream datasets demonstrate the superiority of our tracker, where it has achieved better feature expression and more precise target localization. [ABSTRACT FROM AUTHOR]

Sandstorm images are characterized by color casts and reduced contrast due to the presence of suspended sand particles, which significantly impacts the performance of high-level computer vision tasks. Recently, numerous deep learning-based methods have been proposed for sandstorm image enhancement. However, most of them are either ineffective or have excessive parameters. In this paper, we introduce a lightweight Color-aware Transformer (CAT) for sandstorm image enhancement. Specifically, we propose the Color Restoration Module (CRM), which integrates Channel Self-attention Transformer (CST) and Vision Transformer (ViT) to effectively correct color distortion by utilizing global channel information. Additionally, the Feature Refinement Module (FRM), composed of cascaded attention blocks, is designed to dynamically capture the most valuable information to refine the features. By leveraging the aforementioned modules, our network is capable of processing an image with 256 × 256 resolution in just 0.06 s, while maintaining a compact architecture of only 2.1M parameters. Through extensive experiments on both synthetic and real-world sandstorm datasets, we demonstrate that our CAT outperforms several state-of-the-art methods in terms of image quality and computational efficiency. • A lightweight network for sandstorm image enhancement is proposed, named CAT. • The CRM and FRM are used to eliminate color deviation and recover the details. • Our network has only 2.1M parameters. [ABSTRACT FROM AUTHOR]

Semantic segmentation is an important yet unsolved problem in aerial scenes understanding. One of the major challenges is the intense variations of scenes and object scales. In this article, we propose a novel multi-scale aware-relation network (MANet) to tackle this problem in remote sensing. Inspired by the process of human perception of multi-scale (MS) information, we explore discriminative and diverse MS representations. For discriminative MS representations, we propose an inter-class and intra-class region refinement (IIRR) method to reduce feature redundancy caused by fusion. IIRR utilizes the refinement maps with intra-class and inter-class scale variation to guide MS fine-grained features. Then, we propose multi-scale collaborative learning (MCL) to enhance the diversity of MS feature representations. The MCL constrains the diversity of MS feature network parameters to obtain diverse information. Also, the segmentation results are rectified according to the dispersion of the multilevel network predictions. In this way, MANet can learn MS features by collaboratively exploiting the correlation among different scales. Extensive experiments on image and video datasets, which have large-scale variations, have demonstrated the effectiveness of our proposed MANet. [ABSTRACT FROM AUTHOR]

Recently, deep convolution neural networks (DCNNs) have obtained remarkable performance in exploring single image super-resolution (SISR). However, most of the existing CNN-based SISR methods only focus on increasing the width and depth of the network to improve SR performance, which makes them face a heavy computing burden. In this paper, we propose a lightweight dynamic dual attention iteration network (DDAIN) for SISR. Specifically, to better realize the attention of the channel and the convolution kernel, we design a dynamic convolution unit (DYCU) at the head of the network. It improves the SR performance by enhancing the complexity of the model without increasing the width and depth of the network. Compared with the traditional static convolution, it can extract more abundant high and low-frequency image features according to different input images. Moreover, to recover the high-frequency detail features of images with different resolutions as much as possible, we embed multiple dual residual attention (DRA) in the feature refinement unit (FRU). Finally, to alleviate the height discomfort caused by SR, we introduce iterative loss Liter to optimize the training process further. Extensive experimental results on benchmark show that the performance of the DDAIN in different degradation models exceeds some existing classical methods. [ABSTRACT FROM AUTHOR]

Semantic segmentation is one of the most commonly used techniques for road scene understanding. Recently developed deep learning-based semantic segmentation networks are typically based on the encoder-decoder structure and have made great progress in road scene understanding. However, these conventional networks still encounter difficulties in recovering spatial details. To overcome this problem, we introduce a lightweight prediction and boundary-aware refinement module that can hierarchically refine the segmentation results with spatial details. The proposed refinement module has two attention units called the upper-level prediction attention unit and the upper-level boundary attention unit. The upper-level prediction attention unit emphasizes the features in the regions that need to be refined by using predicted class probability from the upper-level, whereas the upper-level boundary attention unit focuses on the features near the semantic boundary of the upper-level segmentation result. By using the proposed prediction and boundary-aware refinement module in the decoder network, the segmentation result can gradually be improved in a top-down manner to a finer and more complete one. Experimental results on the Cityscapes and CamVid datasets demonstrate that the proposed prediction and boundary attention-based refinement module can achieve considerable performance improvement in segmentation accuracy with a marginal increase in computational complexity.

Convolutional neural networks (CNNs) perform well in tasks of segmenting buildings from remote sensing images. However, the intraclass heterogeneity of buildings is high in images, while the interclass homogeneity between buildings and other nonbuilding objects is low. This leads to an inaccurate distinction between buildings and complex backgrounds. To overcome this challenge, we propose an Attentional Feature Learning Network (AFL-Net) that can accurately extract buildings from remote sensing images. We designed an attentional multiscale feature fusion (AMFF) module and a shape feature refinement (SFR) module to improve building recognition accuracy in complex environments. The AMFF module adaptively adjusts the weights of multi-scale features through the attention mechanism, which enhances the global perception and ensures the integrity of building segmentation results. The SFR module captures the shape features of the buildings, which enhances the network capability for identifying the area between building edges and surrounding nonbuilding objects and reduces the over-segmentation of buildings. An ablation study was conducted with both qualitative and quantitative analyses, verifying the effectiveness of the AMFF and SFR modules. The proposed AFL-Net achieved 91.37, 82.10, 73.27, and 79.81% intersection over union (IoU) values on the WHU Building Aerial Imagery, Inria Aerial Image Labeling, Massachusetts Buildings, and Building Instances of Typical Cities in China datasets, respectively. Thus, the AFL-Net offers the prospect of application for successful extraction of buildings from remote sensing images.

Scene text detection plays a vital role for scene text understanding, but arbitrary-shaped text detection remains a significant challenge. To extract discriminative features, most recent state-of-the-art methods adopt heavy networks, resulting in parameter redundancy and inference inefficiency. For accurate and efficient scene text detection, in this paper we propose a novel recurrent feature refinement network (RFRN). RFRN, as a recurrent segmentation framework, contains a recurrent path augmentation that refines the previous feature maps as inner states, which not only helps improve the segmentation quality, but also fully facilitates the reuse of parameters and low computational cost. During testing, RFRN discards redundant prediction procedures for efficient inference, and achieves a good balance between speed and accuracy of inference. We conduct experiments on four challenging scene text benchmarks, CTW1500, Total-Text, ICDAR2015 and ICDAR2017-MLT, which include curved texts and multi-oriented texts with complex background. The results show that the proposed RFRN achieves competitive performance on detection accuracy while maintaining computational efficiency. [ABSTRACT FROM AUTHOR]