Search

Image super-resolution (SR) techniques aim to enhance the clarity and realism of images. Recently, a wide range of excellent SR algorithms with powerful characterization capabilities have emerged and are widely used. However, there are still challenges and room for improvement in designing a lighter and more edge-friendly SR networks for hardware devices. In this paper, we propose a lightweight enhanced feature refinement network (EFRN) based on depthwise separatable convolution for SR reconstruction. The core network components consist of multiple enhanced feature refinement blocks (EFRB), which fully fuse channel features to extract more accurate low-frequency information based on the attention of different channels. In addition, a lightweight residual block (LRB) and a lightweight dual attention block (LDAB) are designed to enhance network information extraction with minimal parameter cost. We improve the feature refinement by using 1 × 1 convolution instead of a channel selection operation to reduce the dimensionality of the features and extract the refined features more efficiently. Finally, to achieve better reconstruction performance, the depth and number of channels of the network are expanded while keeping the total number of parameters at a low level. Extensive experiments have been conducted to demonstrate the superiority of our EFRN over other mainstream SR algorithms in terms of reconstruction results and the number of parameters. [ABSTRACT FROM AUTHOR]

Since the advent of the registration method based on deep learning, it has demonstrated a time efficiency advantage several orders of magnitude higher than traditional methods. However, the current deep networks have not fully explored the potential to capture spatial relationships comprehensively. Faced with the complex anatomical structures and detailed deformations in the brain, existing methods often run into difficulties. Therefore, addressing this challenging issue, we propose an unsupervised registration network for 3D brain magnetic resonance imaging (MRI). This framework adopts a hybrid structure of CNN transformer, gradually refining the deformation field using a pyramid structure and multi-level strategies, introducing a local salient position transformer to focus on local details of the deformation, and a hierarchical feature fusion module to merge features of multi-layer deformation fields further. These strategies help preserve high-level semantic information in the brain, thereby improving the quality of the deformation field and achieving deformable registration more effectively. We evaluated the proposed method on two publicly available brain MRI datasets, and the results show that our method outperforms all advanced competing methods in terms of accuracy. [ABSTRACT FROM AUTHOR]

Maximizing the advantages of different views and mitigating their respective disadvantages in fine-grained segmentation tasks are an important challenge in the field of point cloud multi-view fusion. Traditional multi-view fusion methods ignore two fatal problems: 1. the loss of depth and quantization information due to mapping and voxelization operations, resulting in "anomalies" in the extracted features; 2. how to pay attention to the large differences in object sizes among different views during point cloud learning, and fine-tune the fusion efficiency in order to improve the performance of network. In this paper, we propose a new algorithm that uses channel self-attention to fuse range-point-voxel, abbreviated as RPV-CASNet. RPV-CASNet integrates the three different views: range, point and voxel in a more subtle way through an interactive structure (range-point-voxel cross-adaptive layer known as RPVLayer for short), to take full advantage of the differences among them. The RPVLayer contains two key designs: the Feature Refinement Module (FRM) and the Multi-Fine-Grained Feature Self-Attention Module(MFGFSAM). Specifically, the FRM allows for a re-inference representation of points with entrained anomalous features, correcting the features. The MFGFSAM addresses two challenges: efficiently aggregating tokens from distant regions and preserving multiscale features within a single attention layer. In addition, we design a Dynamic Feature Pyramid Extractor (DFPE) for network deployment, which is used to extract rich features from spherical range images. Our method achieves impressive mIoU scores of 69.8% and 77.1% on the SemanticKITTI and nuScenes datasets, respectively. [ABSTRACT FROM AUTHOR]

Semantic segmentation is currently a hot topic in remote sensing image processing. There are extensive applications in land planning and surveying. Many current studies combine Convolutional Neural Networks (CNNs), which extract local information, with Transformers, which capture global information, to obtain richer information. However, the fused feature information is not sufficiently enriched and it often lacks detailed refinement. To address this issue, we propose a novel method called the Multi-View Feature Fusion and Rich Information Refinement Network (MFRNet). Our model is equipped with the Multi-View Feature Fusion Block (MAFF) to merge various types of information, including local, non-local, channel, and positional information. Within MAFF, we introduce two innovative methods. The Sliding Heterogeneous Multi-Head Attention (SHMA) extracts local, non-local, and positional information using a sliding window, while the Multi-Scale Hierarchical Compressed Channel Attention (MSCA) leverages bar-shaped pooling kernels and stepwise compression to obtain reliable channel information. Additionally, we introduce the Efficient Feature Refinement Module (EFRM), which enhances segmentation accuracy by interacting the results of the Long-Range Information Perception Branch and the Local Semantic Information Perception Branch. We evaluate our model on the ISPRS Vaihingen and Potsdam datasets. We conducted extensive comparison experiments with state-of-the-art models and verified that MFRNet outperforms other models. [ABSTRACT FROM AUTHOR]

With the development of convolutional neural networks (CNNs), salient object detection methods have made great progress in performance. Most methods are designed with complex structures to aggregate the multi-level feature maps, to reach the goal of filtering noise and obtaining rich information. However, there is no differentiation when dealing with the multi-level features, and only a uniform treatment is used in general. Based on the above considerations, in this paper, we propose a multi-branch feature fusion and refinement network (MFFRNet), which is a framework for treating low-level features and high-level features differently, and effectively fuses the information of multi-level features to make the results more accurate. We propose a detail optimization module (DOM) designed for the rich detail information in low-level features and a pyramid feature extraction module (PFEM) designed for the rich semantic information in high-level features, as well as a feature optimization module (FOM) for refining the fused feature of multiple levels. Extensive experiments are conducted on six benchmark datasets, and the results show that our approach outperforms the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Frontiers of Computer Science & Technology is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Deep neural networks are vulnerable to adversarial attacks, bringing high risk to numerous security-critical applications. Existing adversarial defense algorithms primarily concentrate on optimizing adversarial training strategies to improve the robustness of neural networks, but ignore that the misguided decisions are essentially made by the activation values. Besides, such conventional strategies normally result in a great decline in clean accuracy. To address the above issues, we propose a novel RSA algorithm to counteract adversarial perturbations while maintaining clean accuracy. Specifically, RSA comprises three distinct modules: feature refinement, activation suppression, and alignment modules. First, the feature refinement module refines malicious activation values in the feature space. Subsequently, the feature activation suppression module mitigates redundant activation values induced by adversarial perturbations across both channel and spatial dimensions. Finally, to avoid an excessive performance drop on clean samples, RSA incorporates a consistency constraint and a knowledge distillation constraint for feature alignment. Extensive experiments on five public datasets and three backbone networks demonstrate that our proposed algorithm achieves consistently superior performance in both adversarial robustness and clean accuracy over the state-of-the-art. [ABSTRACT FROM AUTHOR]

Computer-aided systems can assist radiologists in improving the efficiency of COVID-19 diagnosis. Current work adopts complex structures and the correlations between convolutional neural network (CNN) and Transformer haven't been explored. We propose a novel model named bridging CNN and Transformer via online feature distillation (BCT-OFD). First, lightweight Mpvit-tiny and MobileNetV3-small are chosen as the teacher and student networks, respectively. Sufficient pathological knowledge is smoothly transferred from the teacher to the student using OFD. Then, a adaptive feature fusion module is designed to efficiently fuse the heterogeneous CNN and Transformer features. The implicit correlations between the two networks are fully mined to generate more discriminative fused features. And coordinate attention is adopted to make further feature refinement. The accuracy on three public avaliable datasets reach 97.76%, 98.12% and 96.96%, respectively. It validates that BCT-OFD outperforms state-of-the-art baselines in terms of effectiveness and generalization ability. Notably, BCT-OFD is relatively more lightweight and easier to deploy on those resource-constrained devices, making it the bridge that links theory to application as well as narrowing the gap between them. This study provides an innovative approach in the field of COVID-19 image recognition, offering valuable insights for further improvements in performance. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Computer-Aided Design & Computer Graphics / Jisuanji Fuzhu Sheji Yu Tuxingxue Xuebao is the property of Gai Kan Bian Wei Hui and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Unsupervised person re-identification aims to identify the same person from non-overlapping cameras under unsupervised settings. Aiming at the problem that the existing unsupervised person re-identification network cannot fully extract pedestrian features and the difference between cameras leads to pedestrian retrieval errors, we propose an unsupervised person re-identification of adversarial disentangling learning guided by refined features. A feature refinement information fusion module is designed and embedded into different layers of ResNet50 network to enhance the ability of the network to extract key information. A disentangled feature learning method is designed to minimize the mutual information between pedestrian features and camera features, and reduce the negative impact of camera differences on the network. At the same time, the adversarial disentangling loss function is designed for unsupervised joint learning. Using the Market-1501 and DukeMTMC-reID public datasets, we tested the proposed method. The mean average precision increased by 4.6% and 3.1% respectively. Compared with the baseline algorithm, it has strong robustness and meets the needs of pedestrian recognition in unsupervised background. [ABSTRACT FROM AUTHOR]

RGB-D salient object detection (RGB-D SOD) has currently attracted much attention for its prospect of broad application. On the basis of the "encoder-decoder" paradigm of the fully convolutional network (FCN), many FCN-based strategies have emerged and achieved huge progress, but underestimated the potential of level-specific characteristics of multi-modal features. In this paper, we propose the adaptive guided fusion network (AGFNet) to further mine the potential information between the depth image and the RGB image, and design an adaptive fusion and coarse-to-fine decoding strategy to achieve high-precision detection of salient objects. Specifically, we first use a two-stream encoder to extract the multi-level features of the RGB image and depth image but refrain from the previous practice of using depth features for each layer. Second, a simple but effective way named multi-modal selective fusion strategy is designed to fuse the multi-level features. Third, for enhancement of contextual information of each level adaptively, an adaptive cross fusion module (ACFM) fuses the features at all levels and outputs a coarse saliency map. Finally, a guided attention refinement module (GARM) utilizes the coarse saliency map to guide the final features from ACFM to realize the enhancement and obtain a refined saliency map. Our method is compared with other state-of-the-art RGB-SOD methods through extensive experiments, and the results demonstrate the superiority of our proposed AGFNet. The source code of this project is available at https://github.com/HaodongSun809/my_AGFNet.git. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Accurate calibration between LiDAR and camera sensors is crucial for autonomous driving systems to perceive and understand the environment effectively. Typically, LiDAR–camera extrinsic calibration requires feature alignment and overlapping fields of view. Aligning features from different modalities can be challenging due to noise influence. Therefore, this paper proposes a targetless extrinsic calibration method for monocular cameras and LiDAR sensors that have a non-overlapping field of view. The proposed solution uses pose transformation to establish data association across different modalities. This conversion turns the calibration problem into an optimization problem within a visual SLAM system without requiring overlapping views. To improve performance, line features serve as constraints in visual SLAM. Accurate positions of line segments are obtained by utilizing an extended photometric error optimization method. Moreover, a strategy is proposed for selecting appropriate calibration methods from among several alternative optimization schemes. This adaptive calibration method selection strategy ensures robust calibration performance in urban autonomous driving scenarios with varying lighting and environmental textures while avoiding failures and excessive bias that may result from relying on a single approach. [ABSTRACT FROM AUTHOR]

We propose a Context-aware Feature Transformer Network (CaFTNet), a novel network for human pose estimation. To address the issue of limited modeling of global dependencies in convolutional neural networks, we design the Transformerneck to strengthen the expressive power of features. Transformerneck directly substitutes 3 × 3 convolution in the bottleneck of HRNet with a Contextual Transformer (CoT) block while reducing the complexity of the network. Specifically, the CoT first produces keys with static contextual information through 3 × 3 convolution. Then, relying on query and contextualization keys, dynamic contexts are generated through two concatenated 1 × 1 convolutions. Static and dynamic contexts are eventually fused as an output. Additionally, for multi-scale networks, in order to further refine the features of the fusion output, we propose an Attention Feature Aggregation Module (AFAM). Technically, given an intermediate input, the AFAM successively deduces attention maps along the channel and spatial dimensions. Then, an adaptive refinement module (ARM) is exploited to activate the obtained attention maps. Finally, the input undergoes adaptive feature refinement through multiplication with the activated attention maps. Through the above procedures, our lightweight network provides powerful clues for the detection of keypoints. Experiments are performed on the COCO and MPII datasets. The model achieves a 76.2 AP on the COCO val2017 dataset. Compared to other methods with a CNN as the backbone, CaFTNet has a 72.9% reduced number of parameters. On the MPII dataset, our method uses only 60.7% of the number of parameters, acquiring similar results to other methods with a CNN as the backbone. [ABSTRACT FROM AUTHOR]

Occluded person re-identification (ReID) aims to retrieve the same pedestrian from partially occluded pedestrian images across non-overlapping cameras. Current state-of-the-art methods generally use auxiliary models to obtain non-occluded regions, which not only result in more complex models, but also cannot effectively handle the more generalized ReID task. To this end, a Multi-Branch Attention Network (MBA-Net) is proposed to achieve multi-level refinement of features through an end-to-end multi-branch framework with attention mechanisms. Specifically, we first achieve preliminary feature refinement through a backbone network with a non-local attention mechanism. Then, a two-level multi-branch architecture in MBA-Net is proposed with two-level features refinement to obtain aware local discriminative features from the self-attention branch, non-occluded local complementary features from the cross-attention branch, and global features from the global branch. Finally, we can obtain retrieval features that are robust to occlusion by concatenating all the above features. Experimental results show that our MBA-Net achieves state-of-the-art performance on an occluded person ReID dataset Occluded-Duke and simultaneously achieves competitive performance on two general person ReID datasets Market-1501 and DukeMTMC-ReID. [ABSTRACT FROM AUTHOR]

Recently, red-green-blue (RGB) and thermal (RGB-T) data have attracted considerable interest for semantic segmentation because they provide robust imaging under the complex lighting conditions of urban roads. Most existing RGB-T semantic segmentation methods adopt an encoder-decoder structure, and repeated upsampling causes semantic information loss during decoding. Moreover, using simple cross-modality fusion neither completely mines complementary information from different modalities nor removes noise from the extracted features. To address these problems, we developed a dual-decoding hierarchical fusion network (DHFNet) to extract RGB and thermal information for RGB-T Semantic Segmentation. DHFNet uses a novel two-layer decoder and implements boundary refinement and boundary-guided foreground/background enhancement modules. The modules process features from different levels to achieve the global guidance and local refinement of the segmentation prediction. In addition, an adaptive attention-filtering fusion module filters and extracts complementary information from the RGB and thermal modalities. Further, we introduce a graph convolutional network and an atrous spatial pyramid pooling module to obtain multiscale features and deepen the extracted semantic information. Experimental results on two benchmark datasets showed that the proposed DHFNet performed well relative to state-of-the-art semantic segmentation methods in terms of different evaluation metrics. [ABSTRACT FROM AUTHOR]

Change detection (CD) plays a critical role in extracting ground changes from bi-temporal remote sensing (RS) images and is instrumental in understanding surface dynamics. In recent years, deep learning has made significant breakthroughs in CD. However, typical CD methods that employ the Siamese network for temporal feature extraction lack feature alignment ability for bi-temporal heterogeneous RS images, resulting in inadequate temporal discriminative capability. Moreover, deep learning-based CD methods are still susceptible to the problem of minor changes missing due to scale variation with deeper network layers. In this article, we propose a bi-temporal feature alignment and refinement network (FARNet). To improve the discriminative capability of the Siamese network, an adversarial learning-based temporal discriminatory loss function is designed to align temporal-level features and eliminate bi-temporal domain shift, and a cosine similarity-based loss function is employed to measure feature distance at the Pixel-level. To address the problem of minor changes missing, we adopt a dilated convolution-based Siamese network to prevent feature map size reduction, and a multi-level feature detail supplement (MFDS) module is designed to supplement the deep layer features with shallow layer features. Additionally, we construct a change map refinement (CMR) module that refines the coarse change map to the finegrained change map. Furthermore, we design a cross-temporal feature interaction (CFI) module to learn more fine-grained change features by combining features across temporal. Comprehensive experimental results on two popular CD datasets demonstrate the effectiveness and efficiency of FARNet compared with state-of-theart (SOTA) methods. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

3D shape recognition is a critical research topic in the field of computer vision, attracting substantial attention. Existing approaches mainly focus on extracting distinctive 3D shape features; however, they often neglect the model's robustness and lack refinement in deep features. To address these limitations, we propose the point-view fusion attention network that aims to extract a concise, informative, and robust 3D shape descriptor. Initially, our approach combines multi-view features with point cloud features to obtain accurate and distinguishable fusion features. To effectively handle these fusion features, we design a dual-attention convolutional network which consists of a channel attention module and a spatial attention module. This dual-attention mechanism greatly enhances the generalization ability and robustness of 3D recognition models. Notably, we introduce a strip-pooling layer in the channel attention module to refine the features, resulting in improved fusion features that are more compact. Finally, a classification process is performed on the refined features to assign appropriate 3D shape labels. Our extensive experiments on the ModelNet10 and ModelNet40 datasets for 3D shape recognition and retrieval demonstrate the remarkable accuracy and robustness of the proposed method. [ABSTRACT FROM AUTHOR]

As an emerging computer vision task, crowd localization has received increasing attention due to its ability to produce more accurate spatially predictions. However, continuous scale variations in complex crowd scenes lead to tiny individuals at the edges, so that existing methods cannot achieve precise crowd localization. Aiming at alleviating the above problems, we propose a novel Dilated Convolution-based Feature Refinement Network (DFRNet) to enhance the representation learning capability. Specifically, the DFRNet is built with three branches that can capture the information of each individual in crowd scenes more precisely. More specifically, we introduce a Feature Perception Module to model long-range contextual information at different scales by adopting multiple dilated convolutions, thus providing sufficient feature information to perceive tiny individuals at the edge of images. Afterwards, a Feature Refinement Module is deployed at multiple stages of the three branches to facilitate the mutual refinement of feature information at different scales, thus further improving the expression capability of multi-scale contextual information. By incorporating the above modules, DFRNet can locate individuals in complex scenes more precisely. Extensive experiments on multiple datasets demonstrate that the proposed method has more advanced performance compared to existing methods and can be more accurately adapted to complex crowd scenes. [ABSTRACT FROM AUTHOR]

Video object detection (VOD) is a sophisticated visual task. It is a consensus that is used to find effective supportive information from correlation frames to boost the performance of the model in VOD tasks. In this paper, we not only improve the method of finding supportive information from correlation frames but also strengthen the quality of the features extracted from the correlation frames to further strengthen the fusion of correlation frames so that the model can achieve better performance. The feature refinement module FRM in our model refines the features through the key–value encoding dictionary based on the even-order Taylor series, and the refined features are used to guide the fusion of features at different stages. In the stage of correlation frame fusion, the generative MLP is applied in the feature aggregation module DFAM to fuse the refined features extracted from the correlation frames. Experiments adequately demonstrate the effectiveness of our proposed approach. Our YOLOX-based model can achieve 83.3% AP50 on the ImageNet VID dataset. [ABSTRACT FROM AUTHOR]

Robust multi-scale object detection is challenging as it requires both spatial details and semantic knowledge to deal with problems including high scale variation and cluttered background. Appropriate fusion of high-resolution features with deep semantic features is the key issue to achieve better performance. Different approaches have been developed to extract and combine deep features with shallow layer spatial features, such as feature pyramid network. However, high-resolution feature maps contain noisy and distractive features. Directly combines shallow features with semantic features might degrade detection accuracy. Besides, contextual information is also important for multi-scale object detection. In this work, we present a feature refinement scheme to tackle the feature fusion problem. The proposed feature refinement module increases feature resolution and refine feature maps progressively with the guidance from deep features. Meanwhile, we propose a context extraction method to capture global and local contextual information. The method utilizes a multi-level cross-pooling unit to extract global context and a cascaded context module to extract local context. The proposed object detection framework has been evaluated on PASCAL VOC and MS COCO datasets. Experimental results demonstrate that the proposed method performs favorably against state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Fabric defect detection plays an irreplaceable role in the quality control of the textile manufacturing industry, but it is still a challenging task due to the diversity and complexity of defects and environmental factors. Visual saliency models imitating the human vision system can quickly determine the defect regions from the complex texture background. However, most visual saliency-based methods still suffer from incomplete predictions owing to the variability of fabric defects and low contrast with the background. In this paper, we develop a context-aware attention cascaded feedback network for fabric defect detection to achieve more accurate predictions, in which a parallel context extractor is designed to characterize the multi-scale contextual information. Moreover, a top-down attention cascaded feedback module was devised adaptively to select the important multi-scale complementary information and then transmit it to an adjacent shallower layer to compensate for the inconsistency of information among layers for accurate location. Finally, a multi-level loss function is applied to guide our model for generating more accurate prediction results via optimizing multiple side-output predictions. Experimental results on the two fabric datasets built under six widely used evaluation metrics demonstrate that our proposed framework outperforms state-of-the-art models remarkably. [ABSTRACT FROM AUTHOR]

Change detection (CD) plays an important role in Earth surface analysis. Current CD methods have achieved good performance in large flat areas, but CD of detailed parts is still a great challenge, and the loss of detail causes many faults around the change boundaries and on small objects. By analyzing the feature map of the widely used U-Net architecture in existing methods, we ascribe the detail loss to the depletion of detailed features during the top-to-down delivery in the U-Net architecture. The feature refine CD (FRCD) model is proposed in which the detection results are predicted directly from the multiscale features instead of the U-Net architecture. By direct prediction, the representation ability of details is enhanced, and thus the detection accuracy (Acc) of boundaries and small objects improves. Moreover, the normal upsampling in direct prediction is replaced with the deformable upsampling, which delivers detailed information from the low-level to the high-level via the deformable convolution, allowing the results to further fit boundaries in the FRCD model. Experimental results on two datasets confirm the effectiveness of FRCD compared to the state-of-the-art methods, and the CD results of boundaries and small objects are improved significantly by the proposed method. Code will be available after the acceptance of the letter in https://github.com/ijnokml/cdfr. [ABSTRACT FROM AUTHOR]

Although the current deep learning technology has made amazing progress in the field of large and medium object detection, small object detection is still a challenging problem today due to the limited size of small object and the limitations of convolutional networks. Based on You Only Look Once version 5 (hereinafter referred to as YOLOv5) algorithm, this research proposes a YOLO-S model, which is very friendly to small objects. Firstly, on the basis of the orginal output layer with only three layers, a special output layer for small object detection is added by using the cascade network. Secondly, in order to supplement context information and suppress multi-scale feature fusion conflicts, a new supplement context information module CFM and channel and spatial feature thinning module FSM is designed. Finally, the upsampling method is replaced by deconvolution from the original linear interpolation. The dataset uses VisDrone2019, which is specially designed for small objects, to verify the effectiveness of the algorithm. The experimental results show that the mAP@0.5 of YOLO-S is 6.9 percentage points higher than that of YOLOv5. [ABSTRACT FROM AUTHOR]

Strip surface defects have large intraclass and small interclass differences, resulting in the available detection techniques having either a low accuracy or very poor real-time performance. In order to improve the ability for capturing steel surface defects, the context fusion structure introduces the local information of the shallow layer and the semantic information of the deep layer into multiscale feature maps. In addition, for filtering the semantic conflicts and redundancies arising from context fusion, a feature refinement module is introduced in our method, which further improves the detection accuracy. Our experimental results show that this significantly improved the performance. In particular, our method achieved 79.5% mAP and 71 FPS on the public NEU-DET dataset. This means that our method had a higher detection accuracy compared to other techniques. [ABSTRACT FROM AUTHOR]

Localization of anatomical landmarks is essential for clinical diagnosis, treatment planning, and research. This paper proposes a novel deep network named feature aggregation and refinement network (FARNet) for automatically detecting anatomical landmarks. FARNet employs an encoder-decoder structure architecture. To alleviate the problem of limited training data in the medical domain, we adopt a backbone network pre-trained on natural images as the encoder. The decoder includes a multi-scale feature aggregation module for multi-scale feature fusion and a feature refinement module for high-resolution heatmap regression. Coarse-to-fine supervisions are applied to the two modules to facilitate end-to-end training. We further propose a novel loss function named Exponential Weighted Center loss for accurate heatmap regression, which focuses on the losses from the pixels near landmarks and suppresses the ones from far away. We evaluate FARNet on three publicly available anatomical landmark detection datasets, including cephalometric, hand, and spine radiographs. Our network achieves state-of-the-art performances on all three datasets. Code is available at https://github.com/JuvenileInWind/FARNet. [ABSTRACT FROM AUTHOR]

Different from the object detection which has made great progress in natural imagery, transmission line images acquired by UAVs have their own challenges in detecting defects in critical parts, such as object scale variation and small defect targets. In this paper, we construct an effective architecture, called FOLO, to improve the accuracy of defect detection in critical parts of transmission lines. To capture the critical part defect object features, a local contextual feature pyramid network (LCFPN) is proposed to refine the local contextual information and perform multi-scale learning. In LCFPN, we introduce a channel feature refinement block (CFRB) and multiple spatial feature refinement block (SFRBs) to further improve the ability of the network to focus on local features. Besides, a local adaptive feature network (LAFN) is designed, which makes it possible to locate adaptive components with defects in critical areas of different shapes. Since existing transmission line datasets have a single category, we create a new defect detection dataset containing insulators, anti-vibration hammers and bird nests, named IVB. Experimental results on IVB show that the proposed FOLO yields promising performance against other approaches. [ABSTRACT FROM AUTHOR]

Precisely predicting the drug–drug interaction (DDI) is an important application and host research topic in drug discovery, especially for avoiding the adverse effect when using drug combination treatment for patients. Nowadays, machine learning and deep learning methods have achieved great success in DDI prediction. However, we notice that most of the works ignore the importance of the relation type when building the DDI prediction models. In this work, we propose a novel R |$^2$| -DDI framework, which introduces a relation-aware feature refinement module for drug representation learning. The relation feature is integrated into drug representation and refined in the framework. With the refinement features, we also incorporate the consistency training method to regularize the multi-branch predictions for better generalization. Through extensive experiments and studies, we demonstrate our R |$^2$| -DDI approach can significantly improve the DDI prediction performance over multiple real-world datasets and settings, and our method shows better generalization ability with the help of the feature refinement design. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) perform well in tasks of segmenting buildings from remote sensing images. However, the intraclass heterogeneity of buildings is high in images, while the interclass homogeneity between buildings and other nonbuilding objects is low. This leads to an inaccurate distinction between buildings and complex backgrounds. To overcome this challenge, we propose an Attentional Feature Learning Network (AFL-Net) that can accurately extract buildings from remote sensing images. We designed an attentional multiscale feature fusion (AMFF) module and a shape feature refinement (SFR) module to improve building recognition accuracy in complex environments. The AMFF module adaptively adjusts the weights of multi-scale features through the attention mechanism, which enhances the global perception and ensures the integrity of building segmentation results. The SFR module captures the shape features of the buildings, which enhances the network capability for identifying the area between building edges and surrounding nonbuilding objects and reduces the over-segmentation of buildings. An ablation study was conducted with both qualitative and quantitative analyses, verifying the effectiveness of the AMFF and SFR modules. The proposed AFL-Net achieved 91.37, 82.10, 73.27, and 79.81% intersection over union (IoU) values on the WHU Building Aerial Imagery, Inria Aerial Image Labeling, Massachusetts Buildings, and Building Instances of Typical Cities in China datasets, respectively. Thus, the AFL-Net offers the prospect of application for successful extraction of buildings from remote sensing images. [ABSTRACT FROM AUTHOR]

Fabric defect detection plays a critical role for measuring quality control in the textile manufacturing industry. Deep learning-based saliency models can quickly spot the most interesting regions that attract human attention from the complex background, which have been successfully applied in fabric defect detection. However, most of the previous methods mainly adopted multi-level feature aggregation yet ignored the complementary relationship among different features, and thus resulted in poor representation capability for the tiny and slender defects. To remedy these issues, we propose a novel saliency-based fabric defect detection network, which can exploit the complementary information between different layers to enhance the representation features ability and discrimination of defects. Specifically, a multiscale feature aggregation unit (MFAU) is proposed to effectively characterize the multi-scale contextual features. Besides, a feature fusion refinement module (FFR) composed of an attention fusion unit (AFU) and an auxiliary refinement unit (ARU) is designed to exploit complementary important information and further refine the input features for enhancing the discriminative ability of defect features. Finally, a multi-level deep supervision (MDS) is adopted to guide the model to generate more accurate saliency maps. Under different evaluation metrics, our proposed method outperforms most state-of-the-art methods on our developed fabric datasets. [ABSTRACT FROM AUTHOR]

As a traditional computer vision task, monocular depth estimation plays an essential role in novel view 3D reconstruction and augmented reality. Convolutional neural network (CNN)-based models have achieved good performance for this task. However, in the depth map recovered by some existing deep learning-based methods, local details are still lost. To generate convincing depth maps with rich local details, this study proposes an efficient multilevel pyramid network for monocular depth estimation based on feature refinement and adaptive fusion. Specifically, a multilevel spatial feature generation scheme is developed to extract rich features from the spatial branch. Then, a feature refinement module that combines and enhances these multilevel contextual and spatial information is designed to derive detailed information. In addition, we design an adaptive fusion block for improving the capability of fully connected features. The performance evaluation results on public RGBD datasets indicate that the proposed approach can recover reasonable depth outputs with better details and outperform several depth recovery algorithms from a qualitative and quantitative perspective. [ABSTRACT FROM AUTHOR]

The refined segmentation of nuclei and the cytoplasm is the most challenging task in the automation of cervical cell screening. The U-Shape network structure has demonstrated great superiority in the field of biomedical imaging. However, the classical U-Net network cannot effectively utilize mixed domain information and contextual information, and fails to achieve satisfactory results in this task. To address the above problems, a module based on global dependency and local attention (GDLA) for contextual information modeling and features refinement, is proposed in this study. It consists of three components computed in parallel, which are the global dependency module, the spatial attention module, and the channel attention module. The global dependency module models global contextual information to capture a priori knowledge of cervical cells, such as the positional dependence of the nuclei and cytoplasm, and the closure and uniqueness of the nuclei. The spatial attention module combines contextual information to extract cell boundary information and refine target boundaries. The channel and spatial attention modules are used to provide adaption of the input information, and make it easy to identify subtle but dominant differences of similar objects. Comparative and ablation experiments are conducted on the Herlev dataset, and the experimental results demonstrate the effectiveness of the proposed method, which surpasses the most popular existing channel attention, hybrid attention, and context networks in terms of the nuclei and cytoplasm segmentation metrics, achieving better segmentation performance than most previous advanced methods. [ABSTRACT FROM AUTHOR]

Learning discriminative and rich features is an important research task for person re-identification. Previous studies have attempted to capture global and local features at the same time and layer of the model in a non-interactive manner, which are called synchronous learning. However, synchronous learning leads to high similarity, and further defects in model performance. To this end, we propose asynchronous learning based on the human visual perception mechanism. Asynchronous learning emphasizes the time asynchrony and space asynchrony of feature learning and achieves mutual promotion and cyclical interaction for feature learning. Furthermore, we design a dynamic progressive refinement module to improve local features with the guidance of global features. The dynamic property allows this module to adaptively adjust the network parameters according to the input image, in both the training and testing stage. The progressive property narrows the semantic gap between the global and local features, which is due to the guidance of global features. Finally, we have conducted several experiments on four datasets, including Market1501, CUHK03, DukeMTMC-ReID, and MSMT17. The experimental results show that asynchronous learning can effectively improve feature discrimination and achieve strong performance. [ABSTRACT FROM AUTHOR]

Semantic segmentation is an important yet unsolved problem in aerial scenes understanding. One of the major challenges is the intense variations of scenes and object scales. In this article, we propose a novel multi-scale aware-relation network (MANet) to tackle this problem in remote sensing. Inspired by the process of human perception of multi-scale (MS) information, we explore discriminative and diverse MS representations. For discriminative MS representations, we propose an inter-class and intra-class region refinement (IIRR) method to reduce feature redundancy caused by fusion. IIRR utilizes the refinement maps with intra-class and inter-class scale variation to guide MS fine-grained features. Then, we propose multi-scale collaborative learning (MCL) to enhance the diversity of MS feature representations. The MCL constrains the diversity of MS feature network parameters to obtain diverse information. Also, the segmentation results are rectified according to the dispersion of the multilevel network predictions. In this way, MANet can learn MS features by collaboratively exploiting the correlation among different scales. Extensive experiments on image and video datasets, which have large-scale variations, have demonstrated the effectiveness of our proposed MANet. [ABSTRACT FROM AUTHOR]

Recently, deep convolution neural networks (DCNNs) have obtained remarkable performance in exploring single image super-resolution (SISR). However, most of the existing CNN-based SISR methods only focus on increasing the width and depth of the network to improve SR performance, which makes them face a heavy computing burden. In this paper, we propose a lightweight dynamic dual attention iteration network (DDAIN) for SISR. Specifically, to better realize the attention of the channel and the convolution kernel, we design a dynamic convolution unit (DYCU) at the head of the network. It improves the SR performance by enhancing the complexity of the model without increasing the width and depth of the network. Compared with the traditional static convolution, it can extract more abundant high and low-frequency image features according to different input images. Moreover, to recover the high-frequency detail features of images with different resolutions as much as possible, we embed multiple dual residual attention (DRA) in the feature refinement unit (FRU). Finally, to alleviate the height discomfort caused by SR, we introduce iterative loss Liter to optimize the training process further. Extensive experimental results on benchmark show that the performance of the DDAIN in different degradation models exceeds some existing classical methods. [ABSTRACT FROM AUTHOR]

Automatic segmentation of gastric tumor not only provides image-guided clinical diagnosis but also assists radiologists to read images and improve the diagnostic accuracy. However, due to the inhomogeneous intensity distribution of gastric tumors in CT scans, the ambiguous/missing boundaries, and the highly variable shapes of gastric tumors, it is quite challenging to develop an automatic solution. This study designs a novel 3D improved feature pyramidal network (3D IFPN) to automatically segment gastric tumors in computed tomography (CT) images. To meet the challenges of this extremely difficult task, the proposed 3D IFPN makes full use of the complementary information within the low and high layers of deep convolutional neural networks, which is equipped with three types of feature enhancement modules: 3D adaptive spatial feature fusion (ASFF) module, single-level feature refinement (SLFR) module, and multi-level feature refinement (MLFR) module. The 3D ASFF module adaptively suppresses the feature inconsistency in different levels and hence obtains the multi-level features with high feature invariance. Then, the SLFR module combines the adaptive features and previous multi-level features at each level to generate the multi-level refined features by skip connection and attention mechanism. The MLFR module adaptively recalibrates the channel-wise and spatial-wise responses by adding the attention operation, which improves the prediction capability of the network. Furthermore, a stage-wise deep supervision (SDS) mechanism and a hybrid loss function are also embedded to enhance the feature learning ability of the network. CT volumes dataset collected in three Chinese medical centers was used to evaluate the segmentation performance of the proposed 3D IFPN model. Experimental results indicate that our method outperforms state-of-the-art segmentation networks in gastric tumor segmentation. Moreover, to explore the generalization for other segmentation tasks, we also extend the proposed network to liver tumor segmentation in CT images of the MICCAI 2017 Liver Tumor Segmentation Challenge. [ABSTRACT FROM AUTHOR]

This article presents a new supervised multilayer subnetwork-based feature refinement and classification model for representation learning. The novelties of this algorithm are as follows: 1) different from most multilayer networks that go deeper with increased number of network layers, this work architects a model with wider subnetwork nodes; 2) the conventional classification methods adopt a separate search mechanism to derive a generalized feature space and to get the final cognition, but this work proposes a one-shot process to find the meaningful latent space and recognize the objects; and 3) the traditional feature representation and image classification approaches apply a unimodal feature coding, which suffers from lack of global knowledge. This work overcomes the pitfall through multimodal fusion that fuses various feature sources into one superstate encoding to achieve higher performance. A cross-domain experimental study on camera identification and image classification shows that the proposed method achieves superior performance compared to the existing models. [ABSTRACT FROM AUTHOR]

Recently, deep-learning methods have been successfully applied to the ship detection in the synthetic aperture radar (SAR) images. It is still a great challenge to detect multiscale SAR ships due to the broad diversity of the scales and the strong interference of the inshore background. Most prevalent approaches are based on the anchor mechanism that uses the predefined anchors to search the possible regions containing objects. However, the anchor settings have a great impact on their detection performance as well as the generalization ability. Furthermore, considering the sparsity of the ships, most anchors are redundant and will lead to the computation increase. In this article, a novel detection method named feature balancing and refinement network (FBR-Net) is proposed. First, our method eliminates the effect of anchors by adopting a general anchor-free strategy that directly learns the encoded bounding boxes. Second, we leverage the proposed attention-guided balanced pyramid to balance semantically the multiple features across different levels. It can help the detector learn more information about the small-scale ships in complex scenes. Third, considering the SAR imaging mechanism, the interference near the ship boundary with the similar scattering power probably affects the localization accuracy because of feature misalignment. To tackle the localization issue, a feature-refinement module is proposed to refine the object features and guide the semantic enhancement. Finally, extensive experiments are conducted to show the effectiveness of our FBR-Net compared with the general anchor-free baseline. The detection results on the SAR ship detection dataset (SSDD) and AIR-SARShip-1.0 dataset illustrate that our method achieves the state-of-the-art performance. [ABSTRACT FROM AUTHOR]

Considering the particularity of fragile people with reduced mobility, we design a nonintrusive platform called EVAL cane to assist and monitor the user’s walking. On the one hand, it has important walking-assistance functions, such as obstacle warning and fall detection. On the other hand, it collects the user’s long-term walking data in the background, which may be the potential physical-health-assessment data. Compared with an ordinary cane, the above two functions are implemented without any burden on the user’s life. In addition, we have considered the authentication security for that the walking data are private. To this end, we propose a novel user-identification scheme, leveraging the user walking gait data collected by EVAL cane. To the best of our knowledge, it is the first time that the gait information collected by cane is used for reinforcing monitoring system security. This scheme does not require the user to remember and enter any identity (such as a password), which is user-friendly for fragile people. In the scheme, a statistics-based rough gait feature-extraction method is put forward at first. Then, in order to improve the identification precision, we design a performance-based feature-deletion (PFD) algorithm to remove the bad features. Finally, a minimum Mahalanobis distance classifier is used. Experimental results show that the user-identification rate without the PFD algorithm can reach as high as 90.48%. In addition, the PFD algorithm further improves the performance by about 6%, reaching an excellent result of 96.43%. [ABSTRACT FROM AUTHOR]

The speckle interference seriously degrades the quality of synthetic aperture radar (SAR) image. The existing despeckling algorithms still struggle to remove noise and preserve details simultaneously. In order to enhance the noise suppression and detail restoration performance, this article specially presents a spatial and transform domain convolutional neural network (STD-CNN) model, which yields an integrated feature representation and learning framework for despeckling. In addition, an innovative feature refinement strategy is proposed to further reduce the detail loss by isolating detail features from noise features. Extensive experiments on synthetic and real SAR images demonstrate that the proposed method outperforms the existing SAR despeckling methods on both quantitative and qualitative assessments. With partial modification, the STD-CNN model can still be extended to other image restoration tasks. [ABSTRACT FROM AUTHOR]

Semantic segmentation for lightweight object parsing is a very challenging task, because both accuracy and efficiency (e.g., execution speed, memory footprint or computational complexity) should all be taken into account. However, most previous works pay too much attention to one-sided perspective, either accuracy or speed, and ignore others, which poses a great limitation to actual demands of intelligent devices. To tackle this dilemma, we propose a novel lightweight architecture named Context-Integrated and Feature-Refined Network (CIFReNet). The core components of CIFReNet are the Long-skip Refinement Module (LRM) and the Multi-scale Context Integration Module (MCIM). The LRM is designed to ease the propagation of spatial information between low-level and high-level stages. Furthermore, channel attention mechanism is introduced into the process of long-skip learning to boost the quality of low-level feature refinement. Meanwhile, the MCIM consists of three cascaded Dense Semantic Pyramid (DSP) blocks with image-level features, which is presented to encode multiple context information and enlarge the field of view. Specifically, the proposed DSP block exploits a dense feature sampling strategy to enhance the information representations without significantly increasing the computation cost. Comprehensive experiments are conducted on three benchmark datasets for object parsing including Cityscapes, CamVid, and Helen. As indicated, the proposed method reaches a better trade-off between accuracy and efficiency compared with the other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Purpose: The aim of this study was to develop a novel feature refinement MR reconstruction method from highly undersampled multichannel acquisitions for improving the image quality and preserve more detail information. Theory and Methods: The feature refinement technique, which uses a feature descriptor to pick up useful features from residual image discarded by sparsity constrains, is applied to preserve the details of the image in compressed sensing and parallel imaging in MRI (CS‐pMRI). The texture descriptor and structure descriptor recognizing different types of features are required for forming the feature descriptor. Feasibility of the feature refinement was validated using three different multicoil reconstruction methods on in vivo data. Results: Experimental results show that reconstruction methods with feature refinement improve the quality of reconstructed image and restore the image details more accurately than the original methods, which is also verified by the lower values of the root mean square error and high frequency error norm. Conclusion: A simple and effective way to preserve more useful detailed information in CS‐pMRI is proposed. This technique can effectively improve the reconstruction quality and has superior performance in terms of detail preservation compared with the original version without feature refinement. Magn Reson Med 80:211–223, 2018. © 2017 International Society for Magnetic Resonance in Medicine. [ABSTRACT FROM AUTHOR]

Image stitching aims at generating high-quality panoramas with the lowest computational cost. In this paper, we propose a parallax-tolerant image stitching method based on robust elastic warping, which could achieve accurate alignment and efficient processing simultaneously. Given a group of point matches between images, an analytical warping function is constructed to eliminate the parallax errors. Then, the input images are warped according to the computed deformations over the meshed image plane. The seamless panorama is composed by directly reprojecting the warped images. As an important complement to the proposed method, a Bayesian model of feature refinement is proposed to adaptively remove the incorrect local matches. This ensures a more robust alignment than existing approaches. Moreover, our warp is highly compatible with different transformation types. A flexible strategy of combining it with the global similarity transformation is provided as an example. The performance of the proposed approach is demonstrated using several challenging cases. [ABSTRACT FROM AUTHOR]

Most existing studies on person reidentification (Re-ID) utilize deep feature representation learning. However, images often contain occlusion situations and nondiscriminative personal information. To extract more representative features, some researchers extract implicit deep semantic information by designing complex modules, such as mask maps and human pose landmarks. However, this can introduce complex human annotation and computational work. To overcome these issues, we propose a Re-ID model called multifeature fusion network (MFFNet). Our network does not require any additional auxiliary information and incorporates two new designs: the feature refinement pooling block (FRPB) and the feed-forward conduction structure (FCS). Based on the "split-learn-merge" principle, the FRPB decomposes a person's features into coarse-grained to fine-grained representations. The FRPB learns the corresponding local detail information and merges the multigranular features into the partial person representation. To address the issue of most current methods heavily relying on accurate bounding boxes, the FCS enables character matching at different resolution scales by learning multiple semantic levels of representation. Through a series of ablation experiments, we demonstrate that the proposed strategy is effective for person Re-ID tasks. The results indicate that MFFNet achieves more competitive experimental results than the existing state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Opinion mining involves the analysis of customer opinions using product reviews and provides meaningful information including the polarity of the opinions. In opinion mining, feature extraction is important since the customers do not normally express their product opinions holistically but separately according to its individual features. However, previous research on feature-based opinion mining has not had good results due to drawbacks, such as selecting a feature considering only syntactical grammar information or treating features with similar meanings as different. To solve these problems, this paper proposes an enhanced feature extraction and refinement method called FEROM that effectively extracts correct features from review data by exploiting both grammatical properties and semantic characteristics of feature words and refines the features by recognizing and merging similar ones. A series of experiments performed on actual online review data demonstrated that FEROM is highly effective at extracting and refining features for analyzing customer review data and eventually contributes to accurate and functional opinion mining. [ABSTRACT FROM AUTHOR]

Image stitching aims at generating high-quality panoramas with the lowest computational cost. In this paper, we present an improved parallax image-stitching algorithm using feature blocks (PIFB), which achieves a more accurate alignment and faster calculation speed. First, each image is divided into feature blocks using an improved fuzzy C-Means (FCM) algorithm, and the characteristic descriptor of each feature block is extracted using scale invariant feature transform (SIFT). The feature matching block of the reference image and the target image are matched and then determined, and the image is pre-registered using the homography calculated by the feature points in the feature block. Finally, the overlapping area is optimized to avoid ghosting and shape distortion. The improved algorithm considering pre-blocking and block stitching effectively reduced the iterative process of feature point matching and homography calculation. More importantly, the problem that the calculated homography matrix was not global has been solved. Ghosting and shape warping are significantly eliminated by re-optimizing the overlap of the image. The performance of the proposed approach is demonstrated using several challenging cases. [ABSTRACT FROM AUTHOR]