Search

Temporal adaptive convolution has demonstrated superior performance over static convolution techniques in video understanding. However, it needs to be improved in long-time series modeling and multi-scale feature-map adaptation. To address these challenges, we introduce spatio-temporal hybrid adaptive convolution (STHAC), designed to enhance the spatio-temporal modeling capabilities of convolution. This is achieved by learning a set of spatio-temporal calibration filters to mitigate the spatial invariance intrinsic to static convolution methods. Specifically, STHAC learns a linear combination of N adaptive filters by parallelizing two lightweight attention branches. The resulting linearly mixed filters incorporate spatial multi-scale prior knowledge and long-range temporal dependencies. These spatio-temporal calibration filters modulate each frame's static convolutional weight parameters, thereby endowing static convolution with spatial multi-scale adaptability and long-range temporal modeling capabilities. Compared to other dynamic convolution methods, our proposed calibration filters require fewer parameters and incur lower computational complexity. Moreover, we introduce an Omni-dimensional aggregation module to augment the spatio-temporal modeling capacity of STHAC. When combined with STHAC, this aggregation module forms the spatio-temporal adaptive module (STAM) that can replace static convolution. We implement a spatio-temporal dynamic network based on STAM to validate our approach. Experimental results indicate that our model is competitive with state-of-the-art convolutional neural network architectures on action recognition benchmarks such as Kinetics-400(K400) and Something-Something V2(SSV2). [ABSTRACT FROM AUTHOR]

Semantic segmentation of high-resolution remote sensing images is important in land cover classification, road extraction, building extraction, water extraction, etc. However, high-resolution remote-sensing images have a lot of details. Due to the fixed receptive field of convolution blocks, it is impossible to model the correlation of global features. In addition, complex fusion methods cannot integrate spatial and global context information. In order to solve these problems, this paper proposes a cross-linear attention network (CLANet) to capture spatial and context information in images. The structure consists of a spatial branch and a context branch. The spatial branch is constructed by stacked convolution to better capture spatial information. The context branch models the global information based on the transformer deformation module. In addition, to effectively fuse spatial and context information, this paper also designs a feature fusion module (FFM), which uses a cross-linear attention mechanism for feature aggregation. Finally, this paper conducts many experiments on the ISPRS Vaihingen and the ISPRS Potsdam datasets. Among them, 82.28% of mIoU achieves on the ISPRS Vaihingen dataset. The experimental results show that CLANet has better performance and effect than the methods in recent years. [ABSTRACT FROM AUTHOR]

To address the problem that single-channel neural networks cannot fully extract text semantic features in traditional user portrait construction methods, this paper proposes a dual-channel user portrait model based on DPCNN-BIGRU and attention mechanism. The model first uses Bidirectional Encoder Representation from Transformers(Bert) and CK-means+ to obtain the fusion vector of semantic features and topic features, and then feeds the vector into Deep Pyramid Convolutional Neural Networks (DPCNN) and Bidirectional Gated Recurrent Unit (BiGRU). Deep features and global features of the text are obtained simultaneously, and then weights are assigned by the attention mechanism. Finally, the output features of the dual channels are fused and classified. It is tested on the Sogou user portrait datasets, and the experimental results prove that the dual-channel model outperforms the baseline model. [ABSTRACT FROM AUTHOR]

The purpose of the panchromatic sharpening of remote sensing images is to generate high-resolution multispectral images through software technology without increasing economic expenditure. The specific method is to fuse the spatial information of a high-resolution panchromatic image and the spectral information of a low-resolution multispectral image. This work proposes a novel model for generating high-quality multispectral images. This model uses the feature domain of the convolution neural network to fuse multispectral and panchromatic images so that the fused images can generate new features so that the final fused features can restore clear images. Because of the unique feature extraction ability of convolution neural networks, we use the core idea of convolution neural networks to extract global features. To extract the complementary features of the input image at a deeper level, we first designed two subnetworks with the same structure but different weights, and then used single-channel attention to optimize the fused features to improve the final fusion performance. We select the public data set widely used in this field to verify the validity of the model. The experimental results on the GaoFen-2 and SPOT6 data sets show that this method has a better effect in fusing multi-spectral and panchromatic images. Compared with the classical and the latest methods in this field, our model fusion obtained panchromatic sharpened images from both quantitative and qualitative analysis has achieved better results. In addition, to verify the transferability and generalization of our proposed model, we directly apply it to multispectral image sharpening, such as hyperspectral image sharpening. Experiments and tests have been carried out on Pavia Center and Botswana public hyperspectral data sets, and the results show that the model has also achieved good performance in hyperspectral data sets. [ABSTRACT FROM AUTHOR]

With the development of high-resolution camera technology, the shooting scene coverage has reached the square kilometer level, thousands of people can be observed at the same time, and the faces of people from a hundred meters away are clearly recognizable. The images captured by high-resolution cameras are very different from those captured by conventional cameras. In the face of many detection targets in high-resolution images, large differences in target scales due to spatial position, as well as difficulties in extracting features and poor detection results caused by target overlap and concealment phenomena, this paper proposes a multi-target detection method SARNet that combined with spatial attention optimization feature extraction. Use spatial attention to optimize the backbone network, expand the local receptive field, thereby enhance the representation ability, and enhance the feature extraction ability of small targets; the different scale features of the dilated feature pyramid network are subjected to the deformable region of interest pooling operation, which effectively improves the different scales detection accuracy. The experimental results show that the method proposed in this paper can get 51.9% mAP on the PANDA dataset, which is superior to the existing detection algorithms. At the same time, experimental verification of pedestrians and vehicles on the COCO2017 dataset fully proves the feasibility of the method in this paper. [ABSTRACT FROM AUTHOR]

High spatial-temporal resolution plays a vital role in the application of geoscience dynamic observance and prediction. However, thanks to the constraints of technology and budget, it is troublesome for one satellite detector to get high spatial-temporal resolution remote sensing images. Individuals have developed spatiotemporal image fusion technology to resolve this downside, and deep remote sensing images with spatiotemporal resolution have become a possible and efficient answer. Due to the fixed size of the receptive field of convolutional neural networks, the features extracted by convolution operations cannot capture long-range features, so the correlation of global features cannot be modeled in the deep learning process. We propose a spatiotemporal fusion model of remote sensing images to solve these problems based on a dual branch feedback mechanism and texture transformer. The model separates the network from the coarse-fine images with similar structures through the idea of double branches and reduces the dependence of images on time series. It principally merges the benefits of transformer and convolution network and employs feedback mechanism and texture transformer to extract additional spatial and temporal distinction features. The primary function of the transformer module is to learn global temporal correlations and fuse temporal features with spatial features. To completely extract additional elaborated features in several stages, we have a tendency to design a feedback mechanism module. This module chiefly refines the low-level representation through high-level info and obtains additional elaborated features when considering the temporal and spacial characteristics. We have a tendency to receive good results by comparison with four typical spatiotemporal fusion algorithms, proving our model's superiority and robustness. [ABSTRACT FROM AUTHOR]

Facial expression recognition (FER) in the wild is very challenging due to occlusion, posture, illumination, and other uncontrolled factors. Learning discriminant features for FER using Convolutional Neural Networks is a momentous task for the significant class imbalance, wrong labels, inter-class similarities, and intra-class variations. The traditional method utilizes the Cross entropy loss function to optimize the convolutional network to obtain discriminative features for classification. However, this loss function cannot effectively solve the above problems in practice and cannot contribute to obtaining highly discriminant facial features for further analysis. Center loss improves the learning efficiency by reducing the intra-class distance of similar expressions, while the improvement of inter-class similarity, class imbalance, and generalization is insufficient. In this paper, we propose a lightweight Effective Attention Feature Reconstruction loss (EAFR loss), which can further optimize the feature space and enhance the discriminability of expression. The loss model is composed of the Focal Smoothing loss (FS loss) and the Aggregation-Separation loss (AS loss). Firstly, the FS loss can improve the poor recognition performance caused by imbalanced classes and prevent paranoid knowledge learning behaviors. Meanwhile, AS loss further accurately condenses the intra-class expression features and expands the inter-class distance, which is achieved by using progressive stage max-pooling channel and position attention mechanism and lightweight asymmetric autoencoder model for feature reconstruction. Finally, the EAFR loss joins the above two loss functions to more comprehensively solve the above typical problems for FER in the wild. We validate the proposed loss function on three most commonly used large-scale wild expression datasets (RAF-DB, FERPlus, and AffectNet), and the results show that our model achieves superior performance to several state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Crop counting is a crucial step in crop yield estimation. By counting, crop growth status can be accurately detected and adjusted, improving crop yield and quality. In recent years, with the rapid development of convolutional neural networks, deep learning-based object detection methods have been widely used in crop counting. By summarizing the research related to crop counting, this paper reviews the development status of object detection and crop counting. It then compares deep learning-based object detection counting methods with other counting methods. The paper also introduces public datasets and evaluation metrics commonly used for algorithmic models and provides a more in-depth analysis of the application of object detection in crop counting. Finally, the current problems that need to be solved, such as the lack of datasets, difficulties in small object counting, occlusion in complex environments, and some future directions are summarized. We hope this review will encourage more researchers to use deep-learning object detection methods in agriculture. • The paper compares the advantages and disadvantages of the methods currently used for crop counting tasks. • It is the first summary of object detection-based methods' application in crop counting tasks, covering commonly used models, issues, and challenges. • Public datasets utilized for crop counting tasks through object detection-based methods have been summarized. [ABSTRACT FROM AUTHOR]

• We implement magnification-independent multi-classification methods on the BreakHis dataset. We investigate the relationship between model computational efficiency and recognition performance. • We design the DBLCNN network, which exploits the dependencies to efficiently guide the convolutional features to achieve better feature representation capabilities. • We redesign MobileNet to effectively resolve the conflict between recognition performance and computational utilization. Meanwhile, transfer learning is successfully applied to the DBLCNN. • Experimental results show that the DBLCNN method achieves state-of-the-art recognition performance and excellent computational efficiency on the BreakHis dataset. Breast histopathology analysis is the gold standard for diagnosing breast cancer. Convolutional neural network-based methods for breast histology image classification have emerged in recent years to make the analysis process simple and fast. Due to the limitation of hardware devices, these classification methods still face the problem of difficult balance recognition performance and computational efficiency. In this paper, we propose the Dependency-based lightweight convolutional neural network (DBLCNN) for the multi-classification task of breast histopathology images. Firstly, we design a new network in which dependencies (magnification and binary classification probability) were used to guide subspecies features for better recognition. Secondly, we redesign the backbone MobileNet to greatly reduce the model parameters and computation while ensuring excellent recognition performance. At the same time, transfer learning based on ImageNet is applied to the DBLCNN network. Extensive experiments on the BreakHis dataset have shown that the DBLCNN network has state-of-the-art effects in terms of recognition performance and computational utilization. [ABSTRACT FROM AUTHOR]

Text-based person re-identification (Re-id) is an important task in video surveillance, which consists of retrieving the corresponding person's image given a textual description from a large gallery of images. It is difficult to directly match visual contents with the textual descriptions due to the modality heterogeneity. On the one hand, the textual embedding are not discriminative enough, which originates from the high abstraction of the textual descriptions. One the other hand, Global average pooling (GAP) is commonly utilized to extract more general or smoothed features implicitly but ignores salient local features, which are more important for the cross-modal matching problem. With that in mind, a novel Dual-path CNN with Max Gated block (DCMG) is proposed to extract discriminative word embedding and make visual-textual association concern more on remarkable features of both modalities. The proposed framework is based on two deep residual CNNs jointly optimized with cross-modal projection matching (CMPM) loss and cross-modal projection classification (CMPC) loss to embed the two modalities into a joint feature space. First, the pre-trained language model, BERT, is combined with the convolutional neural network (CNN) to learn better word embedding in the text-to-image matching domain. Second, the global Max pooling (GMP) layer is applied to make the visual textual features focus more on the salient part. To further alleviate the noise of the maxed-pooled features, the gated block (GB) is proposed to produce an attention map that focuses on meaningful features of both modalities. Finally, extensive experiments are conducted on the benchmark dataset, CUHK-PEDES, in which our approach achieves the rank-1 score of 55.81% and outperforms the state-of-the-art method by 1.3%. We also evaluate our method on two generic retrieval datasets (Flickr30K, Oxford-102 Flowers) and obtain the competitive performance. Code is available at https://github.com/voriarty/Dual-path-CNN-with-Max-Gated-block-for-Text-Based-Person-Re-identification • Propose a novel Dual-path CNN with Max Gated block, which is based on two deep residual CNNs jointly optimized to embed the two modalities into a joint feature space. [ABSTRACT FROM AUTHOR]