Search

In recent years, many researchers have improved image captioning performance by fusing region features and grid features. However, the semantic gap between the two features is often overlooked during fusion, and the exploration of multimodal feature interaction remains insufficient. In this paper, we propose a Bidirectional Interactive Alignment Network (BIANet) to achieve more multi-feature and multi-modal fusion in both the encoder and decoder. We propose a bidirectional interactive encoder that utilizes cross-interaction to complement the advantages of both image features, enriching their visual information. In the decoder, we propose a cross-alignment module. This module enables the text features to interact in two sequences: “region feature-grid feature” and “grid feature-region feature”, resulting in two new text features. By improving the similarity between these two text features, the semantic gap between region features and grid features is indirectly alleviated. Extensive experiments on the MS COCO dataset demonstrate that the proposed model achieves competitive results on the Karpathy test split. [ABSTRACT FROM AUTHOR]

Driving action anomaly detection based on in-cab surveillance video has become the mainstream of current driving action research. However, there is a substantial redundancy of spatio-temporal information in the spatio-temporal action features extracted using only the 3D encoder, which will weaken the distinguishability between normal and anomaly videos, and there will be more difficult samples to be distinguished during the anomaly detection. To alleviate this problem, we propose a dual-stream model combining spatio-temporal and appearance features for anomaly driving action detection. Firstly, the model utilises 3D ResNet and 2D ResNet for feature extraction of spatio-temporal action and appearance information in the video. Next, to fully combine the respective advantages of the cross-dimension information and obtain fused features that can be more easily distinguished, a contrastive learning based alignment approach was used before fusion. In addition, cross-dimensional features are fused by introducing Transformer’s cross attention block. Moreover, the supervised contrastive learning objective function is employed to guides the model to distinguish between normal and anomaly driving action. Finally, calculating the feature similarity between the target sample and the normal memory center is utilized to obtain the anomaly score of the target sample. In the experiment, the proposed model DSTANet in this paper achieved the 96.81% AUC in the publicly available DAD dataset, which is higher than the popular models in recent years. Furthermore, feature distinguishability was significantly improved in the visualization experiments. The experiment code and data are publicly available at: . [ABSTRACT FROM AUTHOR]

Background: In the field of content-based image retrieval (CBIR), fused feature-based methods have demonstrated their advanced performance on the popular benchmark datasets. However, it is inevitable increase the vector dimensionality because the fused features have diversity. Therefore, achieving both a low-dimensional representation and high retrieval performance remains challenging. Methods: To address this problem, an image retrieval method based on the deep contrast-based layer is proposed, namely the deep contrast feature histogram (DCFH), to image retrieval. There are three highlights as follows: (1) texture features based on the edge orientation are calculated to build contrast-based layer; it can enhance the discriminative power of deep features; (2) a generalized mean aggregation method is introduced to effectively aggregate the representative information in the deep feature maps of convolutional neural network (CNN); (3) a multi-orientational PCA whitening method is proposed to provide a compact yet discriminative representation. Results: Comparative experiments demonstrated that our method can provide outstandingly competitive retrieval performance on popular benchmark datasets. Conclusions: This work captures visual information from both global and local perspectives, presenting an approach in line with human visual cognitive. Experiments demonstrated that our method can efficiently combine the strengths of various features to provide the robust representation, thereby improving the retrieval performance. Moreover, our method is easily to be implemented without requiring to retrain the CNN models and not the use of additional supervision. [ABSTRACT FROM AUTHOR]

Semi-supervised learning (SSL) employs unlabeled data with limited labeled samples to enhance deep networks, but imbalance degrades performance due to biased pseudo-labels skewing decision boundaries. To address this challenge, we propose two optimization conditions inspired by our theoretical analysis. These conditions focus on aligning class distributions and representations. Additionally, we introduce a plug-and-play method called Basis Transformation based distribution alignment (BTDA) that efficiently aligns class distributions while considering inter-class relationships. BTDA mitigates the negative impact of biased pseudo-labels through basis transformation, which involves a learnable transition matrix. Extensive experiments demonstrate the effectiveness of integrating existing SSL methods with BTDA in image classification tasks with class imbalance. For example, BTDA achieves accuracy improvements ranging from 2.47 to 6.66% on CIFAR10-LT and SVHN-LT datasets, and a remarkable 10.95% improvement on the tail class, even under high imbalanced rates. Despite its simplicity, BTDA achieves state-of-the-art performance in SSL with class imbalance on representative datasets. [ABSTRACT FROM AUTHOR]

Recently, CNN-based all-in-one image enhancement methods have been proposed to solve multiple image degradation tasks. However, these CNN-based methods usually have two limitations. One limitation is that they usually design a specific encoder for each image enhancement task, lacking of a unified and simple framework. The other limitation is that they can not effectively capture global image degradation information, as use of the CNN-based encoders has a limited local receptive field. In this work, we propose a TransEncoder-based All-in-one Image Enhancement Network (TAENet), with a single encoder and a decoder, for simultaneously handling multiple image enhancement tasks. Specifically, we propose a Transformer-based Encoder (TransEncoder), which introduces instance normalization to transformer for color recovery. The TransEncoder model global degradation information by using the transformer's global self-attention mechanism. Additionally, inspired by the Mie scattering model, we propose a novel depth loss function for perceiving image depth information by minimizing the depth difference between the enhanced image and the ground-truth, thus further improving model performance. Moreover, a novel contrastive loss is introduced to strengthen task-generalization performance by enhancing the model's representation capability. Experiments show that the TAENet outperforms 24 state-of-the-art methods on image dehazing, image deraining, and low-light image enhancement. [ABSTRACT FROM AUTHOR]

The early and effective diagnosis of Alzheimer's disease (AD) and mild cognitive impairment (MCI) has received increasing attention in recent years. However, currently available deep learning methods often ignore the contextual spatial information contained in structural MRI images used for early diagnosis and classification of Alzheimer's disease. This may lead us to miss important structural details by failing to adequately capture the potential connections between each slice and its neighboring slices. This lack of contextual information may cause the accuracy of the network model to suffer, which in turn affects its generalization ability and application in real-life scenarios. To explore deeper the connection between spatial context slices, this research is designed to develop a new network model to effectively detect or predict AD by digging into the deeper spatial contextual structural information. In this paper, we design a spatial context network based on 3D convolutional neural network to learn the multi-level structural features of brain MRI images for AD classification. The experimental results show that the model has good stability, accuracy and generalization ability. Our experimental method had a classification accuracy of 92.6% in the AD/CN comparison, 74.9% in the AD/MCI comparison, and 76.3% in the MCI/CN comparison. In addition, this paper demonstrates the effectiveness of the proposed network model through ablation experiments. [ABSTRACT FROM AUTHOR]

Transformer-based models have been used in image captioning to generate a natural language text for describing a given image accurately. In this paper, we propose a bi-directional enhanced network, which strengthens the correlation between image features and text features by the memory bank to improve the performance of the transformer-based encoder–decoder framework for image captioning. In addition, we fine-tune the connection method in the encoder to obtain rich image features. Specifically, during training, the memory bank is first used to store the correspondences between images and annotated texts in the dataset as additional information of image features. After processing through the encoder, we feed the visual features composed of image features and the additional information in the memory bank into the decoder to generate better caption. Subsequently, we utilize a decoder-like architecture to reconstruct visual features from the generated caption. Finally, we calculate the similarity loss between the reconstructed features and the visual features to optimize the encoder. Extensive experiments on the MSCOCO benchmark demonstrate that the proposed method has shown promising results on both the Karpathy test split and the online test server, providing evidence of its effectiveness. [ABSTRACT FROM AUTHOR]

Robust model estimation aims to estimate the parameters of a given geometric model, and then separate the outliers and inliers belonging to different model instances into different groups based on the estimated parameters. Robust model estimation is a fundamental task in computer vision and artificial intelligence, and mainly contains two components: data sampling for generating hypotheses and model selection for segmenting data. Over the past decade, a number of guided data sampling algorithms and model selection algorithms have been proposed separately. This results in that the performance of the robust model estimation method is still unsatisfactory. In this paper, we first present a comprehensive study of the above algorithms, by analyzing and comparing them. Then, we propose an efficient and effective robust model estimation method by using preference analysis and information theory principles. Specifically, we first employ our previously proposed data sampling algorithm based on preference analysis to sample data subsets for generating promising hypotheses. Then, we build a discriminative sparse affinity matrix based on the generated hypotheses by using information theory principles. Finally, we segment data by conducting a spectral clustering on the discriminative affinity matrix. Experimental results on the AdelaideRMF and the Hopkins 155 datasets show that the proposed method achieves higher segmentation accuracies than several state-of-the-art model estimation methods. [ABSTRACT FROM AUTHOR]

This work focuses on unsupervised visual domain adaptation which is still challenging in visual recognition. Most of the attention has been dedicated to seeking the domain-invariant features of cross-domain data, but they ignores the valuable discriminative information in the source domain. In this paper, we propose a Discriminative Dictionary Evolution (DDE) approach to seek discriminative features robust to domain shift. Specifically, DDE gradually adapts a discriminative dictionary learned from the source domain to the target domain through a dictionary evolving procedure, in which self-selected atoms of the dictionary are updated with ℓ 2 , 1 -norm-based regularization. DDE produces domain-invariant representations for cross-domain visual recognition meanwhile promotes the discriminativeness of the dictionary. Empirical results on real-world data sets demonstrate the advantages of the proposed approach over existing competitive methods. [ABSTRACT FROM AUTHOR]

A novel deep feature mapping method self-growing net (SG-Net) is proposed, and its combination with classical fuzzy c-means (FCM) called SG-Net-FCM is further developed. SG-Net is a feedforward learning structure for nonlinear explicit feature mapping and includes four types of layers, i.e., input, fuzzy mapping, hybrid, and output layers. The fuzzy mapping layer maps the data from input layer to a high-dimensional feature space using TSK fuzzy mapping, i.e. the fuzzy mapping of Takagi–Sugeno–Kang fuzzy system (TSK-FS). Afterward, each layer in SG-Net accepts additional inputs from all preceding layers and provides its own distinguished features by using principal component analysis to all subsequent layers. The final output of SG-Net is fed to FCM. Since SG-Net-FCM is developed based on the TSK fuzzy mapping, it is more interpretable than classical kernelized fuzzy clustering methods. The effectiveness of the proposed clustering algorithm is experimentally verified on UCI datasets. [ABSTRACT FROM AUTHOR]

The iterative closest point (ICP) algorithm is widely used for rigid registration for its simplicity and speed, but the registration is easy to fail when point sets lack of obvious structure variety, such as smooth surface and hemisphere. RGB-D information obtained from infrared camera and color camera could use color information to compensate the shapes, so we propose a precise new algorithm for RGB-D point cloud registration, which is an extension of ICP algorithm. First of all, we introduce the color information as a constraint condition to establish correct correspondences between point clouds. Secondly, to reduce the impact of noises and outliers, we use maximum correntropy criterion (MCC) to increase the robustness and accuracy. Thirdly, we add both color information and correntropy into our objective function model and solve it with ICP algorithm. Finally, the compared experiments on simulation and real datasets prove that our algorithm can align two smooth surfaces more accurate and robust than other point set registration algorithms. [ABSTRACT FROM AUTHOR]

Mini-quadrotor is difficult to control in the air due to its small size and light weight. This paper presents the flight and hover control system for a mini-quadrotor, including design and simulation of calculations and controllers. Firstly, the attitude and position of the mini-quadrotor are obtained by distributed multi-sensors. Since attitude calculation of aircraft needs a number of combined rotations and vectors transformed by rotation, quaternions are applied to express the attitude model. About error compensation of gyroscope and accelerometer, IMU_Updata algorithm of Mahony filter are applied and improved to realize data fusion [1]. In order to realize accurate hovering at certain position, UWB (UltraWideband) are applied to gain positional information of mini-quadrotor and correct the antenna delay caused sensor error by base station positioning. The discrete Kalman filter of original data is used to achieve the optimized estimation of the airborne position. Px4flow optical flow sensor is able to gets velocity information and avoid the noise problem, which is caused by differential of position data. Then, the mathematical model of a mini-quadrotor's flight and hover control system can be established. Herein, integral items are solved by the integral separation and integral limiting to mitigate the serious overshoot and oscillation of the system caused by the cascade PID. Finally, the simulation of the attitude controller and position controller are applied with the MATLAB Simulink library. The simulation result shows that the designed attitude controller and position controller can enable the mini-quadrotor to fly smoothly, move in all directions and hover. [ABSTRACT FROM AUTHOR]

Feature extraction is a key problem in face recognition systems. This paper tackles this problem by combining the strength of image descriptor with dimensionality reduction technology. So, this paper proposes a new efficient face recognition method-local descriptor margin projections (LDMP). Firstly, we propose a novel local descriptor for face image representation. At this step, an effective and simple metric approach named gray value accumulating distance (GAD) is firstly proposed. And then a novel local descriptor based on GAD is presented to capture the local structure information between central pixel and its neighbors effectively. Secondly, we propose a dimensionality reduction algorithm named maximum margin learning projections (MMLP) which can obtain the low-dimensional and discriminative feature. Finally, experimental results on the Yale, Extended Yale B, PIE, AR and LFW face databases show the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Illumination variation is one of the most challenging problems for robust face recognition. In this paper, after investigating the ratio relationship between two neighboring pixels in a digital image, we proposed two illumination-insensitive features, i.e., the non-directional local reflectance normalization (NDLRN) and the fused multi-directional local reflectance normalization (fMDLRN), which not only effectively reduce illumination difference among facial images under different illumination conditions, but also preserve the facial details. Experimental results show that NDLRN and fMDLRN can significantly alleviate the adverse effect of complex illumination on face recognition. [ABSTRACT FROM AUTHOR]

Recently, transfer learning has been successfully applied in early diagnosis of Alzheimer's Disease (AD) based on multi-domain data. However, most of existing methods only use data from a single auxiliary domain, and thus cannot utilize the intrinsic useful correlation information from multiple domains. Accordingly, in this paper, we consider the joint learning of tasks in multi-auxiliary domains and the target domain, and propose a novel Multi-Domain Transfer Learning (MDTL) framework for early diagnosis of AD. Specifically, the proposed MDTL framework consists of two key components: 1) a multi-domain transfer feature selection (MDTFS) model that selects the most informative feature subset from multi-domain data, and 2) a multi-domain transfer classification (MDTC) model that can identify disease status for early AD detection. We evaluate our method on 807 subjects from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database using baseline magnetic resonance imaging (MRI) data. The experimental results show that the proposed MDTL method can effectively utilize multi-auxiliary domain data for improving the learning performance in the target domain, compared with several state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Maximum margin criterion (MMC) is a promising feature extraction method proposed recently to enhance the well-known linear discriminant analysis. However, due to the maximization of L2-norm-based distances between different classes, the features extracted by MMC are not robust enough in the sense that in the case of multi-class, the faraway class pair may skew the solution from the desired one, thus leading the nearby class pair to overlap. Aiming at addressing this problem to enhance the performance of MMC, in this paper, we present a novel algorithm called complete robust maximum margin criterion (CRMMC) which includes three key components. To deemphasize the impact of faraway class pair, we maximize the L1-norm-based distance between different classes. To eliminate possible correlations between features, we incorporate an orthonormality constraint into CRMMC. To fully exploit discriminant information contained in the whole feature space, we decompose CRMMC into two orthogonal complementary subspaces, from which the discriminant features are extracted. In such a way, CRMMC can iteratively extract features by solving two related constrained optimization problems. To solve the resulting mathematical models, we further develop an effective algorithm by properly combining polarity function and optimal projected gradient method. Extensive experiments on both synthesized and benchmark datasets verify the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

This paper analyses the intrinsic relationship between the BP network learning ability and generalization ability and other influencing factors when the overfit occurs, and introduces the multiple correlation coefficient to describe the complexity of samples; it follows the calculation uncertainty principle and the minimum principle of neural network structural design, provides an analogy of the general uncertainty relation in the information transfer process, and ascertains the uncertainty relation between the training relative error of the training sample set, which reflects the network learning ability, and the test relative error of the test sample set, which represents the network generalization ability; through the simulation of BP network overfit numerical modeling test with different types of functions, it is ascertained that the overfit parameter q in the relation generally has a span of 7 × 10−3 to 7 × 10−2; the uncertainty relation then helps to obtain the formula for calculating the number of hidden nodes of a network with good generalization ability under the condition that multiple correlation coefficient is used to describe sample complexity and the given approximation error requirement is satisfied; the rationality of this formula is verified; this paper also points out that applying the BP network to the training process of the given sample set is the best method for stopping training that improves the generalization ability. [ABSTRACT FROM AUTHOR]

For recognizing small targets, fire-like objects in fire images, and detecting fires across various scenes, we propose a fire detection method based on feature fusion and channel attention. Most existing fire detection methods have specific application scenarios with poor speed or accuracy. To address the issues of poor accuracy when directly applying existing object detection models and the reduced detection speed when improving models for fire targets, our approach aims to balance accurate fire localization with real-time processing. In the backbone of the model, deformable convolution is used to capture rich image information, and channel attention is employed to enhance features. The feature fusion in the neck achieves better localization of small fire targets. The visualized heatmap results indicate the effectiveness of our improved measures. By simultaneously employing multiple improvement measures, our method achieved satisfactory fire detection performance. Experimental results on a self-annotated dataset demonstrate that the best AP@50 of the model can reach 63.9%, the fastest detection speed can reach 114 FPS, and the F1-score is stable at around 63%. Our method strikes a good balance between detection speed and accuracy. [ABSTRACT FROM AUTHOR]