Search

Feature‐based image matching is a critical technique in photogrammetry and computer vision. Recently, various advanced image matching methods have been proposed. The effectiveness of these methods is significantly challenged in the case of multisource planetary images which often have to deal with unique surface morphologies, observed by different sensors and under different illumination and viewing conditions. This study investigates and evaluates the performances of 13 feature detectors across diverse images from Moon and Mars, captured by different sensor systems under different radiometric and geometric conditions. Also, the performances of 12 feature descriptors are assessed. A ranking for combinations of detectors and descriptors is determined. The results reveal that phase congruency‐based algorithms achieve favourable performance in both feature detection and description. On the other hand methods based on deep learning may obtain better results if training data of high quality were available. Finally, we summarise the capabilities and limitations of multisource remote sensing image matching methods and provide discussions and prospects for future research. [ABSTRACT FROM AUTHOR]

Graph Neural Networks (GNNs) have gained popularity in image matching methods, proving useful for various computer vision tasks like Structure from Motion (SfM) and 3D reconstruction. A well-known example is SuperGlue. Lightweight variants, such as LightGlue, have been developed with a focus on stacking fewer GNN layers compared to SuperGlue. This paper proposes the h-GNN, a lightweight image matching model, with improvements in the two processing modules, the GNN and matching modules. After image features are detected and described as keypoint nodes of a base graph, the GNN module, which primarily aims at increasing the h-GNN's depth, creates successive hierarchies of compressed-size graphs from the base graph through a clustering technique termed SC+PCA. SC+PCA combines Principal Component Analysis (PCA) with Spectral Clustering (SC) to enrich nodes with local and global information during graph clustering. A dual non-contrastive clustering loss is used to optimize graph clustering. Additionally, four message-passing mechanisms have been proposed to only update node representations within a graph cluster at the same hierarchical level or to update node representations across graph clusters at different hierarchical levels. The matching module performs iterative pairwise matching on the enriched node representations to obtain a scoring matrix. This matrix comprises scores indicating potential correct matches between the image keypoint nodes. The score matrix is refined with a 'dustbin' to further suppress unmatched features. There is a reprojection loss used to optimize keypoint match positions. The Sinkhorn algorithm generates a final partial assignment from the refined score matrix. Experimental results demonstrate the performance of the proposed h-GNN against competing state-of-the-art (SOTA) GNN-based methods on several image matching tasks under homography, estimation, indoor and outdoor camera pose estimation, and 3D reconstruction on multiple datasets. Experiments also demonstrate improved computational memory and runtime, approximately 38.1% and 26.14% lower than SuperGlue, and an average of about 6.8% and 7.1% lower than LightGlue. Future research will explore the effects of integrating more recent simplicial message-passing mechanisms, which concurrently update both node and edge representations, into our proposed model. [ABSTRACT FROM AUTHOR]

Image matching is an important task in computer vision. The detector-free dense matching method is an important research direction of image matching due to its high accuracy and robustness. The classical detector-free image matching methods utilize convolutional neural networks to extract features and then match them. Due to the lack of scale equivariance in CNNs, this method often exhibits poor matching performance when the images to be matched undergo significant scale variations. However, large-scale variations are very common in practical problems. To solve the above problem, we propose SeLFM, a method that combines scale equivariance and the global modeling capability of transformer. The two main advantages of this method are scale-equivariant CNNs can extract scale-equivariant features, while transformer also brings global modeling capability. Experiments prove that this modification improves the performance of the matcher in matching image pairs with large-scale variations and does not affect the general matching performance of the matcher. The code will be open-sourced at this link: https://github.com/LiaoYun0x0/SeLFM/tree/main [ABSTRACT FROM AUTHOR]

Ship identification plays a critical role in maritime trade and military activities. Current research largely relies on deep learning-based methods, which demand extensive datasets and high-end hardware, often necessitating GPUs. This requirement significantly limits their practical application. Addressing this challenge, this paper introduces an enhanced bag-of-visual-words ( BoVW) model based on classical computer vision techniques for rapid ship identification. The proposed method initially employs SIFT and SURF techniques to extract local features from ship images, followed by rapid matching and fusion of these features. A graph-theoretic approach is then used to determine the regions of interest (ROI) within the image, reducing background interference. Subsequently, clustering algorithms transform features within the ROIs into visual words and construct a visual dictionary. Each image is described using histograms of visual words. The method also employs a spatial pyramid kernel to represent spatial relationships between image features and uses support vector machines (SVM) for supervised learning classification. Key parameters in the model include the size of the visual dictionary and the resolution level. Extensive experiments were conducted to explore these parameters. When the visual dictionary size was set to 300 and the resolution level to 2, the model achieved an accuracy and precision exceeding 96%,validating the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

In recent years, with the rapid development of unmanned aerial vehicle (UAV) technology, multi-view 3D reconstruction has once again become a hot spot in computer vision. Incremental Structure From Motion (SFM) is currently the most prevalent reconstruction pipeline, but it still faces challenges in reconstruction efficiency, accuracy, and feature matching. In this paper, we use deep learning algorithms for feature matching to obtain more accurate matching point pairs. Moreover, we adopted the improved Gauss–Newton (GN) method, which not only avoids numerical divergence but also accelerates the speed of bundle adjustment (BA). Then, the sparse point cloud reconstructed by SFM and the original image are used as the input of the depth estimation network to predict the depth map of each image. Finally, the depth map is fused to complete the reconstruction of dense point clouds. After experimental verification, the reconstructed dense point clouds have rich details and clear textures, and the integrity, overall accuracy, and reconstruction efficiency of the point clouds have been improved. [ABSTRACT FROM AUTHOR]

Computer vision technology is being applied at an unprecedented speed in various fields such as 3D scene reconstruction, object detection and recognition, video content tracking, pose estimation, and motion estimation. To address the issues of low accuracy and high time complexity in traditional image feature point matching, a fast image-matching algorithm based on nonlinear filtering is proposed. By applying nonlinear diffusion filtering to scene images, details and edge information can be effectively extracted. The feature descriptors of the feature points are transformed into binary form, occupying less storage space and thus reducing matching time. The adaptive RANSAC algorithm is utilized to eliminate mismatched feature points, thereby improving matching accuracy. Our experimental results on the Mikolajcyzk image dataset comparing the SIFT algorithm with SURF-, BRISK-, and ORB-improved algorithms based on the SIFT algorithm conclude that the fast image-matching algorithm based on nonlinear filtering reduces matching time by three-quarters, with an overall average accuracy of over 7% higher than other algorithms. These experiments demonstrate that the fast image-matching algorithm based on nonlinear filtering has better robustness and real-time performance. [ABSTRACT FROM AUTHOR]

Rigid point cloud registration is a crucial upstream task in computer vision, whose goal is to align two misaligned point clouds using a rigid transformation. Existing methods, which directly utilize extracted point features for computing point relationships, are likely to result in a wrong-matching relationship since two or more similar feature points in the source point cloud easily correspond to the same point in the target point cloud. To this end, this paper proposes a two-view point cloud registration network that better alleviates the problem of similar feature points from both the feature and geometry levels. Specifically, at the feature level, a residual correction unit is proposed to learn feature-aware coefficients from raw 3D point clouds to adaptively increase or decrease the difference between features. An attention mechanism is established in two-point clouds to capture the implicit feature relationship between the two-point clouds, gathering the information of another point cloud to enrich the feature information of its own point cloud. Second, at the geometry level, a dual-view graph topology fusion module is described. All points in the graph structure are no longer independent but connected by their geometry structure. Therefore, each point can aggregate neighbor information in the same point cloud and in different point clouds through the constructed single graph and interactive graph, so that each point can enhance the difference between points through its geometry structure. In order to fuse the information of a single graph and an interactive graph, a cross-attention module is proposed to supplement contextual information to obtain point features that are more suitable for matching. Experimental results demonstrate that our method achieves excellent results on complete and partial noisy point clouds. [ABSTRACT FROM AUTHOR]

Recovering three-dimensional structure from images is one of the important researches in computer vision. The quality of feature matching is one of the keys to obtaining more accurate results. However, as different objects or different surfaces of objects have similar images with the same elements and different typography, the camera pose estimation will be wrong and the task will fail. This paper proposes a new mismatch elimination algorithm based on global topology consistency. We first formulate the matching task as a mathematical model based on the global constraints, then convert the feature matching into grid matching, calculate the confidence of the grids according to the changes in the angle and displacement between correspondence grid vectors, and remove the mismatches with low confidence. The experiments have demonstrated that our proposed method performs better than the state-of-the-art feature matching methods to accomplish outlier match rejection in the task of similar image matching and could be helped to obtain the correct camera pose to reconstruct more complete and more accurate object models. [ABSTRACT FROM AUTHOR]

Bolted connections have become the most widely used connection method in steel structures. Over the long-term service of the bolts, loosening damage and other defects will inevitably occur due to various factors. To ensure the stability of bolted connections, an efficient and precise method for identifying loosened bolts in a given structure is proposed based on computer vision technology. The main idea of this method is to combine deep learning with image processing techniques to recognize and label the loosening angle from bolt connection images. A rectangular steel plate was taken as the test research object, and three grade 4.8 ordinary bolts were selected for study. The analysis was conducted under two conditions: manual loosening and simulated loosening. The results showed that the method proposed in this article could accurately locate the position of the bolts and identify the loosening angle, with an error value of about ±0.1°, which proves the accuracy and feasibility of this method, meeting the needs of structural health monitoring. [ABSTRACT FROM AUTHOR]

In the automatic sorting process, overlapping translucent and flexible workpieces on the conveyor belt, blurring the imaging edge features of translucent and flexible workpieces is a challenge to locate the upper and lower workpieces spatially, we propose a method for locating translucent and flexible workpieces spatially under the overlapping environment in conjunction with the most common automatic sorting of translucent and flexible workpieces such as infusion tube drip buckets. Firstly, we propose a rectangular surface light source based on 650 nm band and monocular CCD for imaging translucent workpieces such as infusion tube drip buckets and optimize the imaging parameters. Secondly, we study a feature matching recognition algorithm for flexible workpieces that are prone to deformation, construct a mapping relationship between the position of overlapping layers and imaging quality of translucent and flexible workpieces such as infusion tube drip buckets based on clarity and information entropy, and establish The mapping relationship between the position of the overlapping layers and the imaging quality of translucent and flexible workpieces such as infusion tube drip buckets is constructed based on clarity and information entropy, and a local spatial coordinate conversion model is established. Finally, the spatial positioning coordinates of overlapping and non-overlapping translucent and flexible workpieces in the local coordinate system are identified, and the results show that the imaging method and theory can be effectively applied to the identification of overlapping and spatial positioning coordinates in the automatic sorting of translucent workpieces such as infusion tube drip buckets. [ABSTRACT FROM AUTHOR]

In recent years, social platforms have become integrated in a variety of economic, political and cultural domains. Social media have become the primary outlets for many citizens to consume news and information, and, at the same time, to produce and share online a large amount of data and meta-data. This paper presents an innovative system able to analyze visual information shared by citizens on social media during extreme events for contributing to the situational awareness and supporting people in charge of coordinating the emergency management. The system analyzes all posts containing images shared by users by taking into account: (a) the event class and (b) the GPS coordinates of the geographical area affected by the event. Then, a Single Shot Multibox Detector (SSD) network is applied to select only the posted images correctly related to the event class and an advanced image processing procedure is used to verify if these images are correlated with the geographical area where the emergency event is ongoing. Several experiments have been carried out to evaluate the performance of the proposed system in the context of different emergency situations caused by earthquakes, floods and terrorist attacks. [ABSTRACT FROM AUTHOR]

Feature matching is a long-standing fundamental and critical problem in computer vision and photogrammetry. The indirect matching strategy has become a popular choice because of its high precision and generality, but it finds only a limited number of correct matches, and the mismatch removal phase does not utilize the critical feature descriptors. To this end, this paper proposes a novel and effective feature matching method, named Progressive Smoothness Consensus (PSC). Our PSC designs an objective function to directly construct correct matches from two feature point sets. To optimize the objective, we introduce a stepwise strategy, where a small but reliable match set with the smooth function is used as initialization, and then the correct match set is iteratively enlarged and optimized by match expansion and smooth function estimation, respectively. In addition, the local geometric constraint is added to the compact representation with a Fourier basis, thus improving the estimation precision. We perform the match expansion as a Bayesian formulation to exploit both the spatial distribution and feature description information, thus finding feasible matches to expand the match set. Extensive experiments on feature matching, homography & fundamental matrix estimation, and image registration are conducted, which demonstrate the advantages of our PSC against state-of-the-art methods in terms of generality and effectiveness. Our code is publicly available at https://github.com/XiaYifan1999/Robust-feature-matching-via-Progressive-Smoothness-Consensus. [ABSTRACT FROM AUTHOR]

This paper presented a remote sensing image mosaic algorithm for pavement diseases. There was a dynamic sensed of autonomy in its three components (3 c). The algorithm solved a problem in the detection process, which needed to parameterize the pavement disease information with high-resolution panoramic images. This paper determined the autonomous consciousness of the UAV system in a three-dimensional structure based on target detection and stereo active vision, used an incremental search strategy to triangulate SIFT point features based on geometric constraints, combined the PNP-RANSAC algorithm to find the best matching surface for eliminating the wrong matching and used trigonometric function theory to fuse the images. It realized the fast and accurate splicing of long-thread data sets of diseased pavement. The average feature search time is 43. 45% of that without the incremental search strategy, and the stitching accuracy is over 88%. The experimental data show that the three-component autonomous awareness-based UAV pavement remote sensing image stitching system can effectively cope with long-lane pavement distress images. [ABSTRACT FROM AUTHOR]

Feature matching for 3D point clouds is a fundamental yet challenging problem in remote sensing and 3D computer vision. However, due to a number of nuisances, the initial feature correspondences generated by matching local keypoint descriptors may contain many outliers (incorrect correspondences). To remove outliers, this paper presents a robust method called progressive consistency voting (PCV). PCV aims at assigning a reliable confidence score to each correspondence such that reasonable correspondences can be achieved by simply finding top-scored ones. To compute the confidence score, we suggest fully utilizing the geometric consistency cue between correspondences and propose a voting-based scheme. In addition, we progressively mine convincing voters from the initial correspondence set and optimize the scoring result by considering top-scored correspondences at the last iteration. Experiments on several standard datasets verify that PCV outperforms five state-of-the-art methods under almost all tested conditions and is robust to noise, data decimation, clutter, occlusion, and data modality change. We also apply PCV to point cloud registration and show that it can significantly improve the registration performance. [ABSTRACT FROM AUTHOR]

Objectives: Incremental structure from motion（SfM） has become the widely used workflow for aerial triangulation（AT） of unmanned aerial vehicle （UAV） images. Recently, extensive research has been conducted to improve the efficiency, precision and scalability of SfM-based AT for UAV images. Meanwhile, deep learning-based methods have also been exploited for the geometry processing in the fields of photogrammetry and computer vision, which have been verified with large potential in the AT of UAV images. This paper aims to give a review of recent work in the SfM-based AT for UAV images. Methods:Firstly, the workflow of SfM-based AT is briefly presented in terms of feature matching and geometry solving,in which the former aims to obtain enough and accurate correspondences, and the latter attempts to solve unknown parameters. Secondly, literature review is given for feature matching and geometry solving. For feature matching, classical hand-crafted and recent learning-based methods are presented from the aspects of feature extraction, feature matching and outlier removal. For geometry solving, the principle of SfM-based AT is firstly given with some well-known and widely-used open-source SfM software. Efficiency improvement and large-scale processing are then summarized, which focus on improving the capability of SfM to process large-scale UAV images. Finally, further search is concluded from four aspects, including the change of data acquisition modes, the scalability for large-scale scenes, the development of communication and hardware, and the fusion of deep learning-based methods. Results: The review demonstrates that the existing research promotes the development of SfM-based AT towards the direction of high efficiency, high precision and high robustness, and also promotes the development of both commercial and open-source software packages. Conclusions: Considering the characteristics of UAV images, the efficiency, precision and robustness of SfMbased AT and its application need further improvement and exploitation. This paper could give an extensive conclusion and be a useful reference to the related researchers [ABSTRACT FROM AUTHOR]

Automatic matching of multimodal remote sensing images remains a vital yet challenging task, particularly for remote sensing and computer vision applications. Most traditional methods mainly focus on key point detection and description of the original image, thus ignoring the deep semantic feature information such as semantic road features, with the result that the traditional method can not effectively resist nonlinear grayscale distortion, and has low matching efficiency and poor accuracy. Motivated by this, this paper proposes a novel automatic matching method named LURF via learned unstructured road features for the multimodal images. There are four main contributions in LURF. To begin with, the semantic road features were extracted from multimodal images based on segmentation model CRESIv2. Next, based on semantic road features, a stable and reliable intersection point detector has been proposed to detect unstructured key points. Moreover, a local entropy descriptor has been designed to describe key points with the local skeleton feature. Finally, a global optimization strategy is adopted to achieve the correct matching. The extensive experimental results demonstrate that the proposed LURF outperforms other state-of-the-art methods in terms of both accuracy and efficiency on different multimodal image data sets. [ABSTRACT FROM AUTHOR]

• A novel method for the individual identification of lying cows was developed. • When herd members change, there is need to collect some images of new cows lying without retraining the model. • The models deployed in an embedded system achieved real-time identification. The lying time of cows is a key indicator of their health and comfort. The ability to automatically recognize the lying posture of cows while simultaneously realizing individual cow identification can play an important role in improving cow welfare, increasing milk yield, detecting cow diseases in a timely manner and enabling precision dairy farming management. In this paper, a method of individual identification for lying dairy cows in a barn based on YOLOX and a feature extraction network named CowbodyNet is proposed. When new cows join the herd, there is no need to collect a large number of images to retrain the model. It is very convenient to collect several images of newly added cows lying and store them in the database. First, the low-light images collected at night are enhanced by the multiscale retinex with chromaticity preservation (MSRCP) algorithm to improve the image quality. Then, the YOLOX target detection algorithm is applied to detect and segment cows in the lying posture. Following this, the segmented images of lying cows are input into CowbodyNet to generate feature vectors, which are used to construct a feature vector database. Subsequently, the Euclidean distances between the feature vector of a cow to be identified and the feature vectors in the database are calculated to determine the identification result. The proposed method achieves 94.43% lying cow identification accuracy on a data set containing top-view images of 72 cows. Finally, the individual cow detection and identification model is successfully deployed on the Jetson Xavier NX embedded platform. The results demonstrate the effectiveness and practicability of the proposed cow identification method. This study provides effective technical support for the application of identifying individual lying cows. [ABSTRACT FROM AUTHOR]

Image keypoint detection and feature matching are fundamental steps in computer vision tasks. However, variations in environment, time, and viewpoint pose a challenge to the stability of image keypoint detection and matching. Most traditional and deep learning-based methods cannot accurately and efficiently extract highly repeatable keypoints and robust match pairs in low-luminance environments. Therefore, we propose a two-step 'detection + matching' framework, which consists of deep neural networks in each step. Firstly, we design a self-supervised robust keypoint detection network, which utilizes multi-scale, multi-angle, and multi-luminance transformation techniques to create pseudo-labeled datasets to improve the model's keypoint detection repeatability and luminance invariance. Secondly, we propose a descriptor-free cross-fusion matching network, which uses the cross-fusion attention mechanism to establish connections between keypoint-centered image patches and converts the feature-matching task into an image patch assignment task to improve the accuracy and efficiency of matching. Thirdly, the proposed framework is used to replace traditional SIFT in SfM. Experimental results on testing datasets show that the self-supervised robust keypoint detection network achieves higher keypoint repeatability in low luminance environments compared to SIFT, ORB, LIFT, and Superpoint. The descriptor-free cross-fusion matching network's mean matching accuracy and efficiency are higher than the mainstream Superglue algorithm. Also, SfM achieves better performance regarding the number of sparse point clouds and accuracy. • Propose a keypoint detection and feature-matching algorithm for low-luminance images. • Employ self-supervised methods to improve the robustness of keypoint detection. • Employ the attention mechanism to improve feature matching. • A novel framework for image matching applies to SfM in low-luminance environments. [ABSTRACT FROM AUTHOR]

Establishing high-quality correspondences is a fundamental step for many computer vision tasks. Leveraging the local consistency of correct correspondences (i.e., inliers) has been a prevalent approach to distinguish them from incorrect correspondences (i.e., outliers). However, the random distribution of a significant proportion of outliers complicates the local neighborhood construction of inliers, making it inevitably contain many outliers, which compromises the reliability of the consistency information. In this paper, we propose a Progressive Multi-Stage Adaptive Feature Learning Network (PMA-Net) to solve this problem, which progressively and adaptively learns over multiple stages to remove random outliers embedded in the initial consistency. Specifically, we propose an Adaptive Dynamic Graph Construction (ADGC) module to construct a global topological graph with a high inlier ratio by calculating the affinity between the corresponding neighbors. Additionally, we also design a Feature Mapping Processing (FMP) block and Multi-Stage Prediction (MSP) block to improve the accuracy of subsequent local neighborhood information aggregation, as well as information loss due to channel dimension compression in the prediction phase. Experimental results in camera pose estimation, remote sensing image registration, and homography estimation demonstrate that the proposed PMA-Net performs better than other state-of-the-art methods on various public datasets. Code: https://github.com/XiaojieLi11/PMA-Net [ABSTRACT FROM AUTHOR]

Feature matching is a fundamental problem in many computer vision tasks. This paper proposes a novel effective framework for mismatch removal, named LOcality-guided Global-preserving Optimization (LOGO). To identify inliers from a putative matching set generated by feature descriptor similarity, we introduce a fixed-point progressive approach to optimize a graph-based objective, which represents a two-class assignment problem regarding an affinity matrix containing global structures. We introduce a strategy that a small initial set with a high inlier ratio exploits the topology of the affinity matrix to elicit other inliers based on their reliable geometry, which enhances the robustness to outliers. Geometrically, we provide a locality-guided matching strategy, i.e., using local topology consensus as a criterion to determine the initial set, thus expanding to yield the final feature matching set. In addition, we apply local affine transformations based on reference points to determine the local consensus and similarity scores of nodes and edges, ensuring the validity and generality for various scenarios including complex nonrigid transformations. Extensive experiments demonstrate the effectiveness and robustness of the proposed LOGO, which is competitive with the current state-of-the-art methods. It also exhibits favorable potential for high-level vision tasks, such as essential and fundamental matrix estimation, image registration and loop closure detection. [ABSTRACT FROM AUTHOR]

Reliable feature matching plays an important role in the fields of computer vision and photogrammetry. Due to the complex transformation model caused by photometric and geometric deformations, and the limited discriminative power of local feature descriptors, initial matches with high outlier ratios cannot be addressed very well. This study proposes a reliable outlier-removal algorithm by combining two affine-invariant geometric constraints. First, a very simple geometric constraint, namely, CAI (collinear affine invariance) has been implemented, which is based on the observation that the collinear property of any two points is invariant under affine transformation. Second, after the first-step outlier removal based on the CAI constraint, the SOV (scale-orientation voting) scheme was then adopted to remove remaining outliers and recover the lost inliers, in which the peaks of both scale and orientation voting define the parameters of the geometric transformation model. Finally, match expansion was executed using the Delaunay triangulation of refined matches. By using close-range (rigid and non-rigid images) and UAV (unmanned aerial vehicle) datasets, comprehensive comparison and analysis are conducted in this study. The results demonstrate that the proposed outlier-removal algorithm achieves the best overall performance when compared with RANSAC-like and local geometric constraint-based methods, and it can also be applied to achieve reliable outlier removal in the workflow of SfM-based UAV image orientation. [ABSTRACT FROM AUTHOR]

Pattern recognition from two‐dimensional (2D) images is an indispensable tache in computer vision. However, one legacy hinders its progress: multifarious visual distortions (e.g., partially occluded signs, fisheye respective, affine or 3D projections) caused by spatiotemporal‐varying perturbations (e.g., shrunk, sharpening, overexposure, jitter, and motion) significantly degrade the performance of neural networks in terms of high‐level intelligent behaviors (e.g., target localization and recognition). Leveraging the growing availability of capsule network (CapsNet), we suppress the deformation effect to the final prediction by proposing a CapsNetORB framework to implement distorted target recognition. Two highlights, the customized Siamese CapsNet (S‐CapsNet) and vector‐based oriented fast and rotated brief (VB‐ORB), can cast a mutual positive stimulation: the former encodes capsule feature vectors for the later, whilst the later detects space‐scale invariant interval dimensions (instead of pixels) to bridge association between source standard images (high‐quality training images) and distorted ones (testing images). Thus, the category of one source standard image owning the most correspondences is the final predicted category. We believe that capsule vectors own higher representability and stability compared with conventional pixels/feature maps, which can be well exploited in feature learning while resisting visual distortions. Experimentally, we show that employing our pipeline for distorted target categorization can outperform state‐of‐the‐arts by delivering promising performance on CUB‐200‐2011, Stanford Dogs, Stanford Cars, and our hand‐crafted data set. [ABSTRACT FROM AUTHOR]

Whole pixel registration of non-rigid images with high accuracy and efficiency is a challenging problem in computer vision. To address this issue, we propose a correspondence vector field (CVF) Interpolation approach based on sparse matching of feature seeds. First, we detect and match two types of feature seeds to improve the accuracy of the later dense CVF interpolation. The first type of feature seeds is to guarantee the accuracy of the motion boundary, while the second one is to achieve the uniform distribution of seeds, which is helpful to improve the effect of interpolation. Second, we regionally estimate the dense CVF using the proposed interpolation approach on this basis. At last, we realize the whole-pixel registration of non-rigid images to yield the image alignment. Different from the traditional CVF interpolation approaches based on optical flow field, ours is based on the sparse matching of feature seeds. Thus, it is not limited to the large displacements and tends to achieve the accurate matching of certain key points easily, which is critical to the final interpolation result. Qualitative and quantitative experimental results on several internationally used datasets demonstrate that our approach outperforms the state-of-the-art ones. [ABSTRACT FROM AUTHOR]

Obtaining good feature descriptors for 3D local shape under different conditions, such as noise, clutter, occlusion, and limited overlap, is still a challenging task in computer vision. In this paper, we present a variant of spin-image called dual spin-image (Dual-SI) for 3D local shape description that encodes bi-directional information between an oriented point and its surrounding neighbors within a given radius. By accumulating the parameters that encode the 3D point information into a 2D space bidirectionally, we can generate two different signatures. These signatures are then concatenated to form the Dual-SI feature descriptor. Finally, we propose a multi-scale radius approach to address the problem of occlusion and make use of a weighted kernel to address the noise problem. We tested our Dual-SI feature descriptor on popular datasets addressing 3D object registration, 3D object recognition, and shape retrieval scenarios. We also conduct experiments for 3D point cloud registration to further evaluate the effectiveness of our method. Consensus experimental results show that our Dual-SI achieves outstanding performance on datasets with various nuisances and application contexts. [Display omitted] • A Dual-SI variant of the spin-image descriptor that uses bidirectional encoding. • Dual-SI uses multi-scale radii to address occlusion problems. • Dual-SI uses weighted kernel to reduce the noise effect on scanned data. • Dual-SI achieves balanced performance in different experimental conditions. • Dual-SI offers sufficient consistent feature matches between point clouds. [ABSTRACT FROM AUTHOR]

Video is a rich information source containing both audio and visual information along with motion information embedded in it. Applications such as e-learning, live TV, video on demand, traffic monitoring, etc. need an efficient video retrieval strategy. Content-based video retrieval and superpixel segmentation are two diverse application areas of computer vision. In this work, we are presenting an algorithm for content-based video retrieval with help of Integration of Curvelet transform and Simple Linear Iterative Clustering (ICTSLIC) algorithm. Proposed algorithm consists of two steps: off line processing and online processing. In offline processing, keyframes of the database videos are extracted by employing features: Pearson Correlation Coefficient (PCC) and color moments (CM) and on the extracted keyframes superpixel generation algorithm ICTSLIC is applied. The superpixels generated by applying ICTSLIC on keyframes are used to represent database videos. On other side, in online processing, ICTSLIC superpixel segmentation is applied on query frame and the superpixels generated by segmentation are used to represent query frame. Then videos similar to query frame are retrieved through matching done by calculation of Euclidean distance between superpixels of query frame and database keyframes. Results of the proposed method are irrespective of query frame features such as camera motion, object's pose, orientation and motion due to the incorporation of ICTSLIC superpixels as base feature for matching and retrieval purpose. The proposed method is tested on the dataset comprising of different categories of video clips such as animations, serials, personal interviews, news, movies and songs which is publicly available. For evaluation, the proposed method randomly picks frames from database videos, instead of selecting keyframes as query frames. Experiments were conducted on the developed dataset and the performance is assessed with different parameters Precision, Recall, Jaccard Index, Accuracy and Specificity. The experimental results shown that the proposed method is performing better than the other state-of-art methods. [ABSTRACT FROM AUTHOR]

Estimating depth from RGB images is a long-standing ill-posed problem, which has been explored for decades by the computer vision, graphics, and machine learning communities. Among the existing techniques, stereo matching remains one of the most widely used in the literature due to its strong connection to the human binocular system. Traditionally, stereo-based depth estimation has been addressed through matching hand-crafted features across multiple images. Despite the extensive amount of research, these traditional techniques still suffer in the presence of highly textured areas, large uniform regions, and occlusions. Motivated by their growing success in solving various 2D and 3D vision problems, deep learning for stereo-based depth estimation has attracted a growing interest from the community, with more than 150 papers published in this area between 2014 and 2019. This new generation of methods has demonstrated a significant leap in performance, enabling applications such as autonomous driving and augmented reality. In this paper, we provide a comprehensive survey of this new and continuously growing field of research, summarize the most commonly used pipelines, and discuss their benefits and limitations. In retrospect of what has been achieved so far, we also conjecture what the future may hold for deep learning-based stereo for depth estimation research. [ABSTRACT FROM AUTHOR]

In automated Visual GUI Testing (VGT) for Android devices, the available tools often suffer from low robustness to mobile fragmentation, leading to incorrect results when running the same tests on different devices. To soften these issues, we evaluate two feature matching-based approaches for widget detection in VGT scripts, which use, respectively, the complete full-screen snapshot of the application (Fullscreen) and the cropped images of its widgets (Cropped) as visual locators to match on emulated devices. Our analysis includes validating the portability of different feature-based visual locators over various apps and devices and evaluating their robustness in terms of cross-device portability and correctly executed interactions. We assessed our results through a comparison with two state-of-the-art tools, EyeAutomate and Sikuli. Despite a limited increase in the computational burden, our Fullscreen approach outperformed state-ofthe- art tools in terms of correctly identified locators across a wide range of devices and led to a 30% increase in passing tests. Our work shows that VGT tools’ dependability can be improved by bridging the testing and computer vision communities. This connection enables the design of algorithms targeted to domain-specific needs and thus inherently more usable and robust. [ABSTRACT FROM AUTHOR]

This work is focused on the development of a smart and automatic inspection system for printed labels. This is a challenging problem to solve since the collected labels are typically subjected to a variety of geometric and non-geometric distortions. Even though these distortions do not affect the content of a label, they have a substantial impact on the pixel value of the label image. Second, the faulty area may be extremely small as compared to the overall size of the labelling system. A further necessity is the ability to locate and isolate faults. To overcome this issue, a robust image hashing approach for the detection of erroneous labels has been developed. Image hashing techniques are generally used in image authentication, social event detection and image copy detection. Most of the image hashing methods are computationally extensive and also misjudge the images processed through the geometric transformation. In this paper, we present a novel idea to detect the faults in labels by incorporating image hashing along with the traditional computer vision algorithms to reduce the processing time. It is possible to apply Speeded Up Robust Features (SURF) to acquire alignment parameters so that the scheme is resistant to geometric and other distortions. The statistical mean is employed to generate the hash value. Even though this feature is quite simple, it has been found to be extremely effective in terms of computing complexity and the precision with which faults are detected, as proven by the experimental findings. Experimental results show that the proposed technique achieved an accuracy of 90.12 % . [ABSTRACT FROM AUTHOR]

Seeking reliable correspondences is a fundamental and significant work in computer vision. Recent work has demonstrated that the task can be effectively accomplished by utilizing a deep learning network based on multi-layer perceptrons, which uses the context normalization to deal with the input. However, the context normalization treats each correspondence equally, which will reduce the representation capability of potential inliers. To solve this problem, we propose a novel and effective Local-Global Self-Attention (LAGA) layer based on the self-attention mechanism, to capture contextual information of potential inliers from coarse to fine, and suppress outliers at the same time in processing the input. The global self-attention module is able to capture abundant global contextual information in the whole image, and the local self-attention module is used to obtain rich local contextual information in the local region. After that, to obtain richer contextual information and feature maps with stronger representative capacity, we combine global and local contextual information. The extensive experiments have shown that the networks with our proposed LAGA layer perform better than the original and other comparative networks in outdoor and indoor scenes for outlier removal and camera pose estimation tasks. [ABSTRACT FROM AUTHOR]

The 3D printing process lacks real-time inspection, which is still an open-loop manufacturing process, and the molding accuracy is low. Based on the 3D reconstruction theory of machine vision, in order to meet the applicability requirements of 3D printing process detection, a matching fusion method is proposed. The fast nearest neighbor (FNN) method is used to search matching point pairs. The matching point information of FFT-SIFT algorithm based on fast Fourier transform is superimposed with the matching point information of AKAZE algorithm, and then fused to obtain more dense feature point matching information and rich edge feature information. Combining incremental SFM algorithm with global SFM algorithm, an integrated SFM sparse point cloud reconstruction method is developed. The dense point cloud is reconstructed by PMVs algorithm, the point cloud model is meshed by Delaunay triangulation, and then the accurate 3D reconstruction model is obtained by texture mapping. The experimental results show that compared with the classical SIFT algorithm, the speed of feature extraction is increased by 25.0%, the number of feature matching is increased by 72%, and the relative error of 3D reconstruction results is about 0.014%, which is close to the theoretical error. [ABSTRACT FROM AUTHOR]

In order to get the feature points in the image more quickly and improve the stability of feature matching results, an improved matching strategy is proposed. First, the image is subsampled, matched by a low-resolution image, and the matching pair that matches the failure is quickly filtered out, and then the matched ones that succeed are matched with the corresponding original images, so as to achieve the purpose of acceleration. In addition, the matching result of SIFT and the matching result of SURF are merged, and the fusion result is applied to the 3D reconstruction technology, which effectively improves the accuracy of the 3D point cloud. The algorithm is tested by using self-acquired data set and public data set, and the experimental data is analyzed. The results show that the improved feature matching strategy improves the running speed of 3D reconstruction by 40% and greatly increases the number of 3D points. Therefore, the proposed method can not only reduce the computational burden of feature matching process, but also improve the stability of 3D reconstruction, so it has certain reference value in the research of 3D reconstruction.

The goal of computer vision is to identify objects of interest from images. Copy-move forgery is the process of copying one or more parts of an image and moved into another part of an equivalent image. The detection of copy-move images has many limitations in recognition of objects. In this paper, the proposed work uses Speeded Up Robust Feature (SURF) feature extraction, and the specific object is recognized with the help of the support vector machine. When copy-move forgery was performed, some modifications were done to the image. For instance, turning, scaling, darkening, compression, and noise addition are applied to make effective impersonation forgeries. Here, feature matching process uses the image rotate function, which consists of bicubic and crop operations, and calculates the difference using the blend, scale and joint operation. The results show that forged images are extracted from a given set of test images. The test results exhibit that the proposed technique can get noteworthy and impressive results. [ABSTRACT FROM AUTHOR]

Seeking reliable correspondences has been always the key of achieving image matching task in computer vision. The recent work LFGC aims to achieve this goal by training a binary correspondence classifier, which infers the probabilities of correspondences being inliers and regresses the relative camera pose via learning the input correspondence data. Specially, the LFGC employs multi-layer perceptrons to extract features and context normalization to model underlying contextual relation of correspondences. Nevertheless, the contextual relation obtained by context normalization is easily interfered by outliers, which limits the accuracy and robustness of the correspondence classifier. To address this limitation, based on the observation that the contextual relation in spatial neighborhood is generally well preserved on most scenarios, this paper proposes an improved strategy, i.e. , maximizing local neighborhood consensus to establish robust neighborhood context. Specifically, we construct the neighborhood for correspondences and preserve neighbors' features similar to their medoids, where the neighborhood information is embedded. Subsequently, we combine this neighborhood structure with multi-layer perceptrons to classify the correspondences as inliers or outliers, simultaneously potential inliers are employed to regress the relative camera pose encoded by the essential matrix. Qualitative and quantitative experimental results on benchmark datasets demonstrate that the improved network can effectively promote performance in outlier removal and camera pose estimation comparing with the baseline. [ABSTRACT FROM AUTHOR]

In this work, we consider the multi-image object matching problem in distributed networks of robots. Multi-image feature matching is a keystone of many applications, including Simultaneous Localization and Mapping, homography, object detection, and Structure from Motion. We first review the QuickMatch algorithm for multi-image feature matching. We then present NetMatch, an algorithm for distributing sets of features across computational units (agents) that largely preserves feature match quality and minimizes communication between agents (avoiding, in particular, the need to flood all data to all agents). Finally, we present an experimental application of both QuickMatch and NetMatch on an object matching test with low-quality images. The QuickMatch and NetMatch algorithms are compared with other standard matching algorithms in terms of preservation of match consistency. Our experiments show that QuickMatch and Netmatch can scale to larger numbers of images and features, and match more accurately than standard techniques. [ABSTRACT FROM AUTHOR]

Feature matching is a fundamental problem in computer vision area. In many real applications, one can usually obtain some potential (candidate) matches $\mathcal {C}$ by using some discriminative feature descriptors, such as SIFT descriptor. Then, the feature matching problem can be formulated as the problem of trying to select the correct matches $\mathcal {S}$ from the potential match set $\mathcal {C}$. In this paper, we propose to solve matches selection by developing a novel intra-group sparse matching (IGSM) model. Our IGSM is motivated by a simple observation that the potential match set $\mathcal {C}$ can be divided into several non-overlapping groups $\mathcal {C}_i$ , among which the correct matches $\mathcal {S}$ are uniformly distributed. We thus develop an intra-group selection model to conduct matches selection at the intra-group level to incorporate the one-to-one matching constraint more in matches selection process. Our IGSM model has three main advantages: (1) The selection mechanism is parameter-free; (2) it generates an intra-group sparse solution which better maintains the one-to-one matching constraint in nature; (3) a simple yet effective update algorithm has been derived to solve IGSM model. The optimality and convergence of the algorithm are theoretically guaranteed. Experimental results on several image feature matching datasets show the effectiveness and efficiency of the proposed IGSM matching method. [ABSTRACT FROM AUTHOR]

Local image feature matching lies in the heart of many computer vision applications. Achieving high matching accuracy is challenging when significant geometric difference exists between the source and target images. The traditional matching pipeline addresses the geometric difference by introducing the concept of support region. Around each feature point, the support region defines a neighboring area characterized by estimated attributes like scale, orientation, affine shape, etc. To correctly assign support region is not an easy job, especially when each feature is processed individually. In this article, we propose to estimate the relative affine transformation for every pair of to-be-compared features. This “tailored” measurement of geometric difference is more precise and helps improve the matching accuracy. Our pipeline can be incorporated into most existing 2D local image feature detectors and descriptors. We comprehensively evaluate its performance with various experiments on a diversified selection of benchmark datasets. The results show that the majority of tested detectors/descriptors gain additional matching accuracy with proposed pipeline. [ABSTRACT FROM AUTHOR]

Image feature correspondence selection is pivotal to many computer vision tasks from object recognition to 3D reconstruction. Although many correspondence selection algorithms have been developed in the past decade, there still lacks an in-depth evaluation and comparison in the open literature, which makes it difficult to choose the appropriate algorithm for a specific application. This paper attempts to fill this gap by evaluating eight competing correspondence selection algorithms including both classical methods and current state-of-the-art ones. In addition to preselected correspondences, we have compared different combinations of detector and descriptor on four standard datasets. The diversity of those datasets cover a wide range of uncertainty factors including zoom, rotation, blur, viewpoint change, JPEG compression, light change, different rendering styles and multiple structures. We have measured the quality of competing correspondence selection algorithms in terms of four performance metrics - i.e., precision, recall, F-measure and efficiency. Moreover, we propose to combine the strengths of eight competing methods by combining their correspondence selection results. Extensive experimental results are reported to demonstrate the superiority of several fusion strategies to individual methods, which suggests the possibility of adaptively combining those methods for even better performance. [ABSTRACT FROM AUTHOR]

In this work, we present a novel, theoretical approach to address one of the longstanding problems in computer vision: 2D and 3D affine invariant feature matching. Our proposed Grassmannian Graph (GrassGraph) framework employs a two stage procedure that is capable of robustly recovering correspondences between two unorganized, affinely related feature (point) sets. In the ideal case, the first stage maps the feature sets to an affine invariant Grassmannian representation, where the features are mapped into the same subspace. It turns out that coordinate representations extracted from the Grassmannian differ by an arbitrary orthonormal matrix. In the second stage, by approximating the Laplace-Beltrami operator (LBO) on these coordinates, this extra orthonormal factor is nullified, providing true affine invariant coordinates which we then utilize to recover correspondences via simple mutual nearest neighbor relations. Our validation benchmarks use large number of experimental trials performed on 2D and 3D datasets. Experimental results show that the proposed Grass-Graph method successfully recovers large affine transformations. [ABSTRACT FROM AUTHOR]

The quality of RGB images can be degraded by poor weather or lighting conditions. Thus, to make computer vision techniques work correctly, images need to be enhanced first. This paper proposes an RGB image enhancement method for improving feature matching which is a core step in most computer vision techniques. The proposed method decomposes near-infrared (NIR) image into fine detail, medium detail, and base images by using weighted least squares filters (WLSF) and boosts the medium detail image. Then, the fine and boosted medium detail images are combined, and the combined NIR detail image replaces the luminance detail image of an RGB image. Experiments demonstrates that the proposed method can effectively enhance RGB image; hence more stable image features are extracted. In addition, the method can minimize the loss of the useful visual (or optical) information of the original RGB image that can be used for other vision tasks. [ABSTRACT FROM AUTHOR]

Feature matching aims at generating correspondences across images, which is widely used in many computer vision tasks. Although considerable progress has been made on feature descriptors and fast matching for initial correspondence hypotheses, selecting good ones from them is still challenging and critical to the overall performance. More importantly, existing methods often take a long computational time, limiting their use in real-time applications. This paper attempts to separate true correspondences from false ones at high speed. We term the proposed method (GMS) grid-based motion Statistics, which incorporates the smoothness constraint into a statistic framework for separation and uses a grid-based implementation for fast calculation. GMS is robust to various challenging image changes, involving in viewpoint, scale, and rotation. It is also fast, e.g., take only 1 or 2 ms in a single CPU thread, even when 50K correspondences are processed. This has important implications for real-time applications. What's more, we show that incorporating GMS into the classic feature matching and epipolar geometry estimation pipeline can significantly boost the overall performance. Finally, we integrate GMS into the well-known ORB-SLAM system for monocular initialization, resulting in a significant improvement. [ABSTRACT FROM AUTHOR]

In recent years, facial recognition has been a major problem in the field of computer vision, which has attracted lots of interest in previous years because of its use in different applications by different domains and image analysis. Which is based on the extraction of facial descriptors, it is a very important step in facial recognition. In this article, we compared robust methods (SIFT, PCA-SIFT, ASIFT and SURF) to extract relevant facial information with different facial posture variations (open and unopened mouth, glasses and no glasses, open and closed eyes). The simulation results show that the detector (SURF) is better than others at finding the similarity descriptor and calculation time. Our method is based on the normalization of vector descriptors and combined with the RANSAC algorithm to cancel outliers in order to calculate the Hessian matrix with the objective of reducing the calculation time. To validate our experience, we tested four facial images databases containing several modifications. The results of the simulation show that our method is more efficient than other detectors in terms of speed of recognition and determination of similar points between two images of the same face, one belonging to the base of the text and the other one to the base driven by different modifications. This method, which can be applied on a mobile platform to analyze the content of simple images, for example, to detect driver fatigue, human-machine interaction, human-robot. Using descriptors with properties important for good accuracy and real-time response. [ABSTRACT FROM AUTHOR]

Establishing reliable correspondences between two images is non-trivial for solving various computer vision problems. Many learning-based feature matching methods train end-to-end networks using MLP to infer the probability of each initial correspondence being an inlier. However, these methods utilize continuous PointCN blocks as the main component of their networks, which have limited capacity to obtain contextual information for correspondence learning. In this paper, we propose a simple and effective module called the Multi-Channel Aggregation (MCA) module to capture rich context of correspondences, enabling reliable correspondence identification and accurate camera pose estimation between matching images. Specifically, our MCA module starts from a high-channel PointCN submodule and progressively obtains some high-to-low channel PointCN submodules. Meanwhile, it aggregates mutual information of different submodules to enhance the feature representation capacity. The proposed MCA module is a plug-and-play component that can boost the performance of existing learning-based methods. Extensive experiments across different datasets on the outlier removal and camera pose estimation tasks demonstrate the effectiveness of our MCA module. We have released the source code at https://github.com/guobaoxiao/MCA-Net. [ABSTRACT FROM AUTHOR]

Feature matching aims to identify reliable correspondences between two sets of given initial feature points, which is of considerable importance to photogrammetry and computer vision. In this study, we propose an innovative sparse semantic learning-based network, named SSL-Net, for feature matching. Specifically, SSL-Net includes a novel sparsity constraint (SC) block, which builds a sparse graph for sparse semantic learning. The SC block adopts a region-to-whole learning strategy to measure the confidence of nodes in the sparse graph. It helps the sparse graph preserve the semantic information of positive influence while rejecting unnecessary ones, thereby suppressing the negative influence of incorrect correspondences. In addition, SSL-Net also includes a channel-spatial attention feature gathering block, which gathers features along the spatial direction and channel dimension of correspondences. To mitigate the existence of label ambiguity, we incorporate the accommodation factor into the loss function of SSL-Net for feature matching. As a result, our network outperforms the state-of-the-art method by a considerable margin. Notably, SSL-Net achieves a 9.05% improvement under an error threshold of 5 ° over the state-of-the-art method for the relative pose estimation task on the YFCC100M dataset. Our code will be available at https://github.com/guobaoxiao/SSL-Net. • We present a novel sparse semantic learning network for feature matching. • We introduce a channel-spatial attention feature gathering block to extract relevant information while suppress irrelevant context. • SSL-Net outperforms state-of-the-art methods for outlier removal and relative pose estimation. [ABSTRACT FROM AUTHOR]

Correctly matching feature points in a pair of images is an important preprocessing step for many computer vision applications. In this paper we propose an efficient method for estimating the number of correct matches without explicitly computing them. To this end, we propose to analyze the set of matches using the spatial order of the features, as projected to the $x$ x -axis of the image. The set of features in each image is thus represented by a sequence, and analyzed using the Kendall and Spearman Footrule distance metrics between permutations. This result is interesting in its own right. Moreover, we demonstrate three useful applications of our method: (i) a new halting condition for RANSAC based epipolar geometry estimation methods, (ii) discarding spatially unrelated image pairs in the Structure-from-Motion pipeline, and (iii) computing the probability that a given match is correct based on the rank of the features within the sequences. Our experiments on a large number of synthetic and real data demonstrate the effectiveness of our method. For example, the running time of the image matching stage in the Structure-from-Motion pipeline may be reduced by about 90 percent while preserving about 85 percent of the image pairs with spatial overlap. [ABSTRACT FROM AUTHOR]