Search

Object detection in Unmanned Aerial Vehicle (UAVs) faces a significant challenge in computer vision. Traditional methods are difficult to model object appearance feature with large scale variations and viewpoint differences, when drones fly at different altitudes and capture images from diverse shooting angles. To address this issue, we propose a Learnable Cross-scale Sparse Attention (LCSA) guided feature fusion method to improve the performance of UAV object detection. Specifically, the LCSA feature fusion module enables each point in a feature map to aggregate discriminative information from a set of points with learnable offsets in neighbor feature maps. It enhances local discriminative features of the object by facilitating semantic information interaction across multiple feature maps. The LCSA can function as a novel neck method that complements the existing neck methods and is also transplantable to different object detection frameworks. Moreover, we also employ a scale-aware loss function to integrate the normalized Wasserstein distance with CIoU in order to improve the incompatibility of IoU for objects with large scale variance. Experimental results on the SeaDroneSeev2 and VisDrone2019-DET datasets show that the proposed method achieves superior performance. At a resolution of 640*640, our method achieves 81.9% AP50 and 47.4% AP on SeaDroneSeev2, surpassing baseline 4.9% and 4.8%, achieves state-of-the-art performance. Furthermore, our method outperforms baseline by 5% AP on VisDrone2019-DET. Code will be available at https://github.com/qch777/LSACF.

Abstract High quality feature fusion module is one of the key components for multispectral pedestrian detection system in challenging situations, such as large‐scale variance and occlusion. Although attention mechanism is one of the most effective ways for feature refining, the correlation between attention and scales in feature pyramid still remains unknown. Therefore, a scale‐aware permutated attention module is proposed to enhance features of objects with different scales adaptively in the feature pyramid. Specifically, four different local and global attention sub‐modules are investigated to refine feature maps with different permutations in the Feature Pyramid Networks, improving the quality of the feature fusion. Besides, to address the high miss‐rate issue for small‐sized pedestrians, an adjacent‐branch feature aggregation module is proposed to aggregate features across different scales, taking both semantic context and spatial resolution into consideration. Both modules can benefit from each other with significant performance improvement in terms of efficiency and accuracy, when equipped with the dual‐branch CenterNet detection framework. Experiments on the KAIST and FLIR datasets demonstrate its superior performance compared with other state‐of‐the‐arts.

Due to the limitations of traditional retinal blood vessel segmentation algorithms in feature extraction, vessel breakage often occurs at the end. To address this issue, a retinal vessel segmentation algorithm based on a modified U-shaped network is proposed in this paper. This algorithm can extract multi-scale vascular features and perform segmentation in an end-to-end manner. First, in order to improve the low contrast of the original image, pre-processing methods are employed. Second, a multi-scale residual convolution module is employed to extract image features of different granularities, while residual learning improves feature utilization efficiency and reduces information loss. In addition, a selective kernel unit is incorporated into the skip connections to obtain multi-scale features with varying receptive field sizes achieved through soft attention. Subsequently, to further extract vascular features and improve processing speed, a residual attention module is constructed at the decoder stage. Finally, a weighted joint loss function is implemented to address the imbalance between positive and negative samples. The experimental results on the DRIVE, STARE, and CHASE_DB1 datasets demonstrate that MU-Net exhibits better sensitivity and a higher Matthew’s correlation coefficient (0.8197, 0.8051; STARE: 0.8264, 0.7987; CHASE_DB1: 0.8313, 0.7960) compared to several state-of-the-art methods.

Face images in the logarithmic space can be considered as a sum of the texture component and lighting map component according to Lambert Reflection. However, it is still not easy to separate these two parts, because face contour boundaries and lighting change boundaries are difficult to distinguish. In order to enhance the separation quality of these to parts, this paper proposes an illumination standardization algorithm based on extreme L0 Gaussian difference regularization constraints, assuming that illumination is massively spread all over the image but illumination change boundaries are simple, regular, and sparse enough. The proposed algorithm uses an iterative L0 Gaussian difference smoothing method, which achieves a more accurate lighting map estimation by reserving the fewest boundaries. Thus, the texture component of the original image can be restored better by simply subtracting the lighting map estimated. The experiments in this paper are organized with two steps: the first step is to observe the quality of the original texture restoration, and the second step is to test the effectiveness of our algorithm for complex face classification tasks. We choose the facial expression classification in this step. The first step experimental results show that our proposed algorithm can effectively recover face image details from extremely dark or light regions. In the second step experiment, we use a CNN classifier to test the emotion classification accuracy, making a comparison of the proposed illumination removal algorithm and the state-of-the-art illumination removal algorithm as face image preprocessing methods. The experimental results show that our algorithm works best for facial expression classification at about 5 to 7 percent accuracy higher than other algorithms. Therefore, our algorithm is proven to provide effective lighting processing technical support for the complex face classification problems which require a high degree of preservation of facial texture. The contribution of this paper is, first, that this paper proposes an enhanced TV model with an L0 boundary constraint for illumination estimation. Second, the boundary response is formulated with the Gaussian difference, which strongly responds to illumination boundaries. Third, this paper emphasizes the necessity of reserving details for preprocessing face images.

In the traditional graph embedding framework, the graph is usually built by k-NN or r-ball. Since it is difficult to manually set the parameters k and r in the high-dimensional space, sparse representation-based methods are usually introduced to automatically build the graphs. In recent years, nuclear norm-based matrix regression (NMR) has been proposed for face recognition using the low rank structural information (i.e., the image matrix-based error model). Inspired by NMR, we give a NMR-based projections (NMRP) method for feature extraction and recognition. The experiments on FERET and extended Yale B face databases show that NMR can be used to build the graph while NMRP is an effective feature extraction method.

This work was supported in part by the National Natural Science Fund of China under Grant 61233011, Grant 61374006, Grant 61473086, Grant 61463008, Grant 61703100, Grant 61773118, and Grant 61773117, in part by the Major Program of National Natural Science Foundation of China under Grant 11190015, in part by the Natural Science Foundation of Jiangsu under Grant BK20131300 and Grant BK20170692, in part by the Innovation Fund of Key Laboratory of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education, Nanjing University of Science and Technology, under Grant JYB201601, in part by the Innovation Fund of Key Laboratory of Measurement and Control of Complex Systems of Engineering, Southeast University, under Grant MCCSE2017B01, and in part by the Fundamental Research Funds for the Central Universities under Grant 2242016k30009.

Although face alignment using the Active Appearance Model (AAM) is relatively stable, it is known to be sensitive to initial values and not robust under inconstant circumstances. In order to strengthen the ability of AAM performance for face alignment, a two step approach for face alignment combining AAM and Active Shape Model (ASM) is proposed. In the first step, AAM is used to locate the inner landmarks of the face. In the second step, the extended ASM is used to locate the outer landmarks of the face under the constraint of the estimated inner landmarks by AAM. The two kinds of landmarks are then combined together to form the whole facial landmarks. The proposed approach is compared with the basic AAM and the progressive AAM methods. Experimental results show that the proposed approach gives a much more effective performance.

The problem of long-tailed visual recognition has been receiving increasing research attention. However, the long-tailed distribution problem remains underexplored for video-based visual recognition. To address this issue, in this article we propose a compositional learning based solution for video-based human action recognition. Our method, named Attentional Composition Networks (ACN), first learns verb-like and prepositionlike components, then shuffles these components to generate samples for the tail classes in the feature space to augment the data for the tail classes. Specifically, during training, we represent each action video by a graph that captures the spatial-temporal relations (edges) among detected human/object instances (nodes). Then, ACN utilizes the position information to decompose each action into a set of verb and preposition representations using the edge features in the graph. After that, the verb and preposition features from different videos are combined via an attention structure to synthesize feature representations for tail classes. This way, we can enrich the data for the tail classes and consequently improve the action recognition for these classes. To evaluate the compositional human action recognition, we further contribute a new human action recognition dataset, namely NEU-Interaction (NEU-I). Experimental results on both Something-Something V2 and the proposed NEU-I demonstrate the effectiveness of the proposed method for long-tailed, few-shot, and zero-shot problems in human action recognition. [ABSTRACT FROM AUTHOR]

We present a novel scheme of feature extraction, namely kernel Laplacian maximum margin criterion, for face recognition. The proposed method seeks to maximize the difference, rather than the ratio, of the determinant between the between-class Laplacian scatter matrix and within-class Laplacian scatter matrix in the implicit feature space via kernel trick. The proposed method not only can produce nonlinear discriminant features, but also does not need to calculate the inverse within-class Laplacian scatter matrix. Experimental results on ORL, FERET, and AR databases validate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]