Search

Waves are crucial in ocean observation and research. Stereo vision-based wave measurement, offering non-contact, low-cost, and intelligent processing, is an emerging method. However, improving accuracy remains a challenge due to wave complexity. This paper presents a novel approach to measure wave height, period, and direction by combining deep learning-based stereo matching with feature matching techniques. To improve the discontinuity and low accuracy in disparity maps from traditional wave image matching algorithms, this paper proposes the use of a high-precision stereo matching method based on Pyramid Stereo Matching Network (PSM-Net).A 3D reconstruction method integrating Scale-Invariant Feature Transform (SIFT) with stereo matching was also introduced to overcome the limitations of template matching and interleaved spectrum methods, which only provide 2D data and fail to capture the full 3D motion of waves. This approach enables accurate wave direction measurement. Additionally, a six-degree-of-freedom platform was proposed to simulate waves, addressing the high costs and attenuation issues of traditional wave tank simulations. Experimental results show the prototype system achieves a wave height accuracy within 5%, period accuracy within 4%, and direction accuracy of ±2°, proving the method's effectiveness and offering a new approach to stereo vision-based wave measurement. [ABSTRACT FROM AUTHOR]

Existing methods for 3D local feature description often struggle to achieve a good balance between distinctiveness, robustness, and computational efficiency. To address this challenge, a novel 3D local feature descriptor named Histograms of Projected Normal Vector Distribution (HPNVD) is proposed. The HPNVD descriptor consists of two main components. First, a local reference frame (LRF) is constructed based on the covariance matrix and neighborhood projection to achieve invariance to rigid transformations. Then, the local surface normals are projected onto three coordinate planes within the LRF, which allows for effective encoding of the local shape information. The projection planes are further divided into multiple regions, and a histogram is computed for each plane to generate the final HPNVD descriptor. Experimental results demonstrate that the proposed HPNVD descriptor outperforms state-of-the-art methods in terms of descriptiveness and robustness, while maintaining compact storage and computational efficiency. Moreover, the HPNVD-based point cloud registration algorithm shows excellent performance, further validating the effectiveness of the descriptor. [ABSTRACT FROM AUTHOR]

In spacecraft rendezvous and docking, traditional methods that rely on inertial navigation and sensor data face challenges due to sensor inaccuracies, noise, and a lack of multi-approach assurance. Focusing on exploring a new approach as assistance, this study marks the first application of deep learning-based image feature matching in spacecraft docking tasks, introducing the Class-Tuned Invariant Feature Transformer (CtIFT) algorithm. CtIFT incorporates an improved cross-attention mechanism and a custom-designed feature classification module. By using symmetric multi-layer cross-attention, it gradually strengthens inter-feature relationships perception. And, in the feature matcher, it employs feature classification to reduce computational load, thereby achieving high-precision matching. The model is trained on multi-source datasets to enhance its adaptability in complex environments. The method demonstrates outstanding performance across experiments on four spacecraft docking video scenes, with CtIFT being the only feasible solution compared to SIFT and eight state-of-the-art network methods: D2-Net, SuperPoint, SuperGlue, LightGlue, ALIKED, LoFTR, ASpanFormer, and TopicFM+. The number of successfully matched feature points per frame consistently reaches the hundreds, the successful rate remains 100%, and the average processing time is maintained below 0.18 s per frame, an overall performance which far exceeds other methods. The results indicate that this approach achieves strong matching accuracy and robustness in optical docking imaging, supports real-time processing, and provides new technical support for assistance of spacecraft rendezvous and docking tasks. [ABSTRACT FROM AUTHOR]

[Display omitted] Automatic co-registration of multi-epoch Unmanned Aerial Vehicle (UAV) image sets remains challenging due to the radiometric differences in complex dynamic scenes. Specifically, illumination changes and vegetation variations usually lead to insufficient and spatially unevenly distributed common tie points (CTPs), resulting in under-fitting of co-registration near the areas without CTPs. In this paper, we propose a novel Common-Feature-Track-Matching (CFTM) approach for UAV image sets co-registration, to alleviate the shortage of CTPs in complex dynamic scenes. Instead of matching features between multi-epoch images, we first search correspondences between multi-epoch feature tracks (i.e., groups of features corresponding to the same 3D points), which avoids the removal of matches due to unreliable estimation of the relative pose between inter-epoch image pairs. Then, the CTPs are triangulated from the successfully matched track pairs. Since an even distribution of CTPs is crucial for robust co-registration, a block-based strategy is designed, as well as enabling parallel computation. Finally, an iterative optimization algorithm is developed to gradually select the best CTPs to refine the poses of multi-epoch images. We assess the performance of our method on two challenging datasets. The results show that CFTM can automatically acquire adequate and evenly distributed CTPs in complex dynamic scenes, achieving a high co-registration accuracy approximately four times higher than the state-of-the-art in challenging scenario. Our code is available at https://github.com/lixinlong1998/CoSfM. [ABSTRACT FROM AUTHOR]

Introduction: Simultaneous Localization and Mapping (SLAM) is a technology used in intelligent systems such as robots and autonomous vehicles. Visual SLAM has become a more popular type of SLAM due to its acceptable cost and good scalability when applied in robot positioning, navigation and other functions. However, most of the visual SLAM algorithms assume a static environment, so when they are implemented in highly dynamic scenes, problems such as tracking failure and overlapped mapping are prone to occur. Methods: To deal with this issue, we propose ISFM-SLAM, a dynamic visual SLAM built upon the classic ORB-SLAM2, incorporating an improved instance segmentation network and enhanced feature matching. Based on YOLACT, the improved instance segmentation network applies the multi-scale residual network Res2Net as its backbone, and utilizes CIoU_Loss in the bounding box loss function, to enhance the detection accuracy of the segmentation network. To improve the matching rate and calculation efficiency of the internal feature points, we fuse ORB key points with an efficient image descriptor to replace traditional ORB feature matching of ORB-SLAM2. Moreover, the motion consistency detection algorithm based on external variance values is proposed and integrated into ISFM-SLAM, to assist the proposed SLAM systems in culling dynamic feature points more effectively. Results and discussion: Simulation results on the TUM dataset show that the overall pose estimation accuracy of the ISFM-SLAM is 97% better than the ORB-SLAM2, and is superior to other mainstream and state-of-the-art dynamic SLAM systems. Further real-world experiments validate the feasibility of the proposed SLAM system in practical applications. [ABSTRACT FROM AUTHOR]

As technology evolves, the cost of 3D scanners is falling, which makes 3D computer vision for industrial applications increasingly popular. More and more researchers have started to study 3D computer vision. Point cloud feature descriptors are a fundamental task in 3D computer vision, and descriptors that use spatial features tend to perform better than those without them. Point cloud descriptors can generally be divided into local reference frames-based (LRF-based) and local reference frames-free (LRF-free). The former uses LRFs to provide spatial features to the descriptors, while the latter uses point pair features to provide spatial features. However, the performance of those LRF-based descriptors is more affected by local reference frames (LRFs), and the descriptors with spatial information LRF-free tend to be more computationally intensive because of its point pair combination strategy. Therefore, we propose a strategy named Multi-scale Point Pair Combination Strategy (MSPPCS) that reduces the computation of point pair-based feature descriptors by nearly 70 % while ensuring that the performance of the descriptor is almost unaffected. We also propose a new descriptor, Spatial Feature Point Pair Histograms (SFPPH), which has excellent performance and robustness due to the diverse spatial features used. We critically evaluate the performance of our descriptor on the Bologna dataset, Kinect dataset, and UWA dataset. The experimental results show that our descriptor is the most robust and performing point cloud feature descriptor. [ABSTRACT FROM AUTHOR]

Establishing accurate correspondences between aerial and ground images is facing immense challenges because of the drastic viewpoint, illumination, and scale variations resulting from significant differences in viewing angles, shoot timing, and imaging mechanisms. To cope with these issues, we propose an effective aerial-to-ground feature matching method, named Viewpoint-invariant Deformable Feature Transformation (VDFT), which aims to comprehensively enhance the discrimination of local features by utilizing deformable convolutional network (DCN) and seed attention mechanism. Specifically, the proposed VDFT is constructed consisting of three pivotal modules: (1) a learnable deformable feature network is established by using DCN and Depthwise Separable Convolution (DSC) to obtain dynamic receptive fields, addressing local geometric deformations caused by viewpoint variation; (2) an improved joint detection and description strategy is presented through concurrently sharing the multi-level deformable feature representation to enhance the localization accuracy and representation capabilities of feature points; and (3) a seed attention matching module is built by introducing self- and cross- seed attention mechanisms to improve the performance and efficiency for aerial-to-ground feature matching. Finally, we conduct thorough experiments to verify the matching performance of our VDFT on five challenging aerial-to-ground datasets. Extensive experimental evaluations prove that our VDFT is more resistant to perspective distortion and drastic variations in viewpoint, illumination, and scale. It exhibits satisfactory matching performance and outperforms the current state-of-the-art (SOTA) methods in terms of robustness and accuracy. [ABSTRACT FROM AUTHOR]

Feature‐based image matching is a critical technique in photogrammetry and computer vision. Recently, various advanced image matching methods have been proposed. The effectiveness of these methods is significantly challenged in the case of multisource planetary images which often have to deal with unique surface morphologies, observed by different sensors and under different illumination and viewing conditions. This study investigates and evaluates the performances of 13 feature detectors across diverse images from Moon and Mars, captured by different sensor systems under different radiometric and geometric conditions. Also, the performances of 12 feature descriptors are assessed. A ranking for combinations of detectors and descriptors is determined. The results reveal that phase congruency‐based algorithms achieve favourable performance in both feature detection and description. On the other hand methods based on deep learning may obtain better results if training data of high quality were available. Finally, we summarise the capabilities and limitations of multisource remote sensing image matching methods and provide discussions and prospects for future research. [ABSTRACT FROM AUTHOR]

Aiming at the low tracking accuracy and longtime tracking problems of the traditional automatic mobile human body tracking methods, this research proposes an automatic tracking method of the moving human body based on remote sensing spatial information. The proposed automatic tracking method for the moving human body first utilizes remote sensing technology to obtain the spatial information of the moving human body and then constructs a human body target motion model. By segmenting and fusing human body motion images, automatic matching and tracking of human body motion is finally realized. The experimental results showed that the auto-tracking time of the moving human body was only 0.3 s, while the auto-tracking accuracy rate was as high as 98.29%. In summary, the method used in this research has a high human motion tracking recognition effect. In addition, the study still has shortcomings, such as how to maintain high accuracy tracking in low light and bad weather conditions, still need to be studied. [ABSTRACT FROM AUTHOR]

Seeking a reliable inliers selection method to image feature matching is a fundamental and important task in image processing and robotic vision. To improve the precision of feature matching on putative matches with heavy outliers, we propose a new strategy to select the inliers from the rough feature correspondence-set. Firstly, we construct the inliers selection problem as a concise mathematical model with linear time complexity based on the local K-nearest neighbour structure preserving (KNNSP). Then, to improve the robustness of this model, we design a threshold decrease strategy for this model to iteratively solve the optimal inliers set. Extensive experiments on various image pairs demonstrate that the proposed method does not need prior information and can be directly used for rigid or non-rigid image matching problems. In addition, compared with other typical image matching algorithms, our method ensures competitive performance in precision and recall. [ABSTRACT FROM AUTHOR]

In order to improve the intuitiveness and automation of demand‐side resource selection, a demand‐side resource selection method based on load mapping matching is proposed in view of the increasing challenges of supply–demand balance in power networks and the rapid development of power demand‐side management technologies. First, a two‐dimensional load mapping of demand‐side resources is drawn, and the load mapping is processed by Gaussian convolutional difference method. Then, feature points are extracted and located for the target resources and the loads of other resources in the demand‐side resource pool, and similar feature key point pairs of demand‐side resources are obtained. Finally, the demand‐side resources with similar load characteristics to the target resources in the resource pool are screened according to the number of similar feature key point pairs, and the load resources similar to the target resources are finally identified by dividing the resource selection into priority levels. The experimental results show that the method can effectively extract feature key points, clearly and intuitively represent the features of demand‐side resource load mapping, and can match to load resources with similar characteristics, which greatly transforms the demand‐side resource selection mode. © 2024 Institute of Electrical Engineers of Japan and Wiley Periodicals LLC. [ABSTRACT FROM AUTHOR]

The effective management and conservation of forest resources hinge on accurate monitoring. Nonetheless, individual remote-sensing images captured by low-altitude unmanned aerial vehicles (UAVs) fail to encapsulate the entirety of a forest's characteristics. The application of image-stitching technology to high-resolution drone imagery facilitates a prompt evaluation of forest resources, encompassing quantity, quality, and spatial distribution. This study introduces an improved SIFT algorithm designed to tackle the challenges of low matching rates and prolonged registration times encountered with forest images characterized by dense textures. By implementing the SIFT-OCT (SIFT omitting the initial scale space) approach, the algorithm bypasses the initial scale space, thereby reducing the number of ineffective feature points and augmenting processing efficiency. To bolster the SIFT algorithm's resilience against rotation and illumination variations, and to furnish supplementary information for registration even when fewer valid feature points are available, a gradient location and orientation histogram (GLOH) descriptor is integrated. For feature matching, the more computationally efficient Manhattan distance is utilized to filter feature points, which further optimizes efficiency. The fast sample consensus (FSC) algorithm is then applied to remove mismatched point pairs, thus refining registration accuracy. This research also investigates the influence of vegetation coverage and image overlap rates on the algorithm's efficacy, using five sets of Cyclobalanopsis natural forest images. Experimental outcomes reveal that the proposed method significantly reduces registration time by an average of 3.66 times compared to that of SIFT, 1.71 times compared to that of SIFT-OCT, 5.67 times compared to that of PSO-SIFT, and 3.42 times compared to that of KAZE, demonstrating its superior performance. [ABSTRACT FROM AUTHOR]

Forests and trees provide a variety of essential ecosystem services. Maintaining them is becoming increasingly important, as global and regional climate change is already leading to major changes in the structure and composition of forests. To minimize the negative effects of storm damage risk, the tree and stand characteristics on which the storm damage risk depends must be known. Previous work in this field has consisted of tree-pulling tests and targets attached to selected branches. They fail, however, since the mass of such targets is very high compared to the mass of the branches, causing the targets to influence the tree's response significantly, and because they cannot model dynamic wind loads. We, therefore, installed a multi-camera system consisting of nine cameras that are mounted on four masts surrounding a tree. With those cameras acquiring images at a rate of 10 Hz, we use photogrammetry and a semi-automatic feature-matching workflow to deduce a 3D model of the tree crown over time. Together with motion sensors mounted on the tree and tree-pulling tests, we intended to learn more about the wind-induced tree response of all dominant aerial tree parts, including the crown, under real wind conditions, as well as dampening processes in tree motion. [ABSTRACT FROM AUTHOR]

Abstract Feature matching in computer vision is crucial but challenging in weakly textured scenes due to the lack of pattern repetition. We introduce the SwinMatcher feature matching method, aimed at addressing the issues of low matching quantity and poor matching precision in weakly textured scenes. Given the inherently significant local characteristics of image features, we employ a local self-attention mechanism to learn from weakly textured areas, maximally preserving the features of weak textures. To address the issue of incorrect matches in scenes with repetitive patterns, we use a cross-attention and positional encoding mechanism to learn the correct matches of repetitive patterns in two scenes, achieving higher matching precision. We also introduce a matching optimization algorithm that calculates the spatial expected coordinates of local two-dimensional heat maps of correspondences to obtain the final sub-pixel level matches. Experiments indicate that, under identical training conditions, the SwinMatcher outperforms other standard methods in pose estimation, homography estimation, and visual localization. It exhibits strong robustness and superior matching in weakly textured areas, offering a new research direction for feature matching in weakly textured images.

Traditional architectural landscape design often has problems such as poor visual effect, low satisfaction of residents, and unreasonable design in green environmental protection. Therefore, this paper proposes a green landscape space environment optimization design scheme based on virtual generation algorithm. Firstly, according to the virtual generation algorithm, the virtual generation of the building landscape size is optimized, and the optimization targets include the cost of the building and public facilities such as green belt. Secondly, a new algorithm for generating panoramic landscape images in virtual reality based on feature extraction is proposed. The determinant of Hessian matrix of each pixel in the landscape image is calculated to obtain SURF feature point values. The similarity between landscape images is measured by the Euclidean distance between feature points to achieve feature matching. Through orthographic projection, all landscape images to be synthesized are projected onto the cylindrical surface, and the overlapping parts of adjacent landscape images are fused. After obtaining the projected landscape images, the three-dimensional landscape images generated by orthographic projection are seamlessly spliced through feature matching, and the three-dimensional landscape images generated by orthographic projection are split from a certain position and projected on a plane to obtain a visual consistency of the panoramic landscape image. Finally, the experimental results show that there is almost no gap between the overlapping regions of the panoramic landscape images generated by the proposed algorithm. The brightness difference is not large, the "ghost" phenomenon can be eliminated, and the memory and time consumption are less.

Bolt connections are widely used as structural connections in civil engineering, mechanical engineering, and bridge construction. However, problems such as loosening, or breakage can occur with bolts after prolonged use. To address the challenges of detecting bolt loosening, this study reviews existing detection technologies, analyzes their advantages and limitations, and proposes a novel bolt-loosening detection algorithm based on image matching and deep learning. The algorithm comprises the following components: a bolt target detection model based on an improved YOLOv8 algorithm, image correction using perspective transformation, bolt contour detection and image processing, and feature matching to calculate the transformation matrix between images obtained before and after loosening, thereby determining the loosening angle of the bolt. The experiments focused on a rectangular steel plate featuring four M6 standard bolts. The results demonstrate that the bolt target detection model can accurately locate and crop bolt positions and identify loosening angles under various shooting angles, distances, and lighting conditions. At specific shooting angles and appropriate distances, the detection error threshold was less than 2°. Subsequently, experiments conducted in real-world scenarios confirmed the accuracy and feasibility of the proposed algorithm.

Abstract When the coal gangue sorting robot sorts coal gangue, the position of the target coal gangue will change due to belt slippage, deviation, and speed fluctuations of the belt conveyor. This will cause the robotic to fail in grasping or miss grasping. We have developed a solution to this problem: the IMSSP-Net two-stage network gangue image fast matching method. This method will reacquire the target gangue position information and improve the robot’s grasping precision and efficiency. In the first stage, we use SuperPoint to guarantee the scene adaptability and credibility of feature point extraction. We have enhanced Superpoint’s ability to detect feature points further by using the improved Multi-scale Retinex with Color Restoration enhancement algorithm. In the second stage, we introduce SuperGlue for feature matching to improve the robustness of the matching network. We eliminated erroneous feature matching point pairs and improved the accuracy of image matching by adopting the PROSAC algorithm. We conducted image matching comparison experiments under different object distances, scales, rotation angles, and complex conditions. The experimental platform adopts the double-manipulator truss-type coal gangue sorting robot independently developed by the team. The matching precision, recall, and matching time of the method are 98.2%, 98.3%, and 84.6ms, respectively. The method can meet the requirements of efficient and accurate matching between coal gangue recognition images and sorting images.

3D ultrasound imaging is a promising modality for therapy guidance, e.g. in radiation therapy. It is able to provide volumetric soft tissue images in real-time. However, due to low image quality, high noise ratio and high data dimensionality, real-time capable US image processing methods like target tracking are challenging. In this study, a feature-based tracking approach is investigated. The FAST feature detector is used to detect local image features in 3D ultrasound images. Two different feature descriptors are tested and evaluated in terms of target tracking: The BRIEF descriptor as well as a slicedwasserstein autoencoder. On the basis of a feature matching algorithm, tracking experiments are executed and evaluated using eight labeled 3D US sequences. The mean tracking error measured is 2.08±1.50mm and 2.29±1.59mm using the autoencoder and the BRIEF descriptor, respectively. The results indicate that using an autoencoder for feature description improves the tracking performance compared to a binary descriptor. The proposed tracking method could be executed in fast runtimes of 137 ms and 256 ms per image on average making it real-time capable.

Graph Neural Networks (GNNs) have gained popularity in image matching methods, proving useful for various computer vision tasks like Structure from Motion (SfM) and 3D reconstruction. A well-known example is SuperGlue. Lightweight variants, such as LightGlue, have been developed with a focus on stacking fewer GNN layers compared to SuperGlue. This paper proposes the h-GNN, a lightweight image matching model, with improvements in the two processing modules, the GNN and matching modules. After image features are detected and described as keypoint nodes of a base graph, the GNN module, which primarily aims at increasing the h-GNN's depth, creates successive hierarchies of compressed-size graphs from the base graph through a clustering technique termed SC+PCA. SC+PCA combines Principal Component Analysis (PCA) with Spectral Clustering (SC) to enrich nodes with local and global information during graph clustering. A dual non-contrastive clustering loss is used to optimize graph clustering. Additionally, four message-passing mechanisms have been proposed to only update node representations within a graph cluster at the same hierarchical level or to update node representations across graph clusters at different hierarchical levels. The matching module performs iterative pairwise matching on the enriched node representations to obtain a scoring matrix. This matrix comprises scores indicating potential correct matches between the image keypoint nodes. The score matrix is refined with a 'dustbin' to further suppress unmatched features. There is a reprojection loss used to optimize keypoint match positions. The Sinkhorn algorithm generates a final partial assignment from the refined score matrix. Experimental results demonstrate the performance of the proposed h-GNN against competing state-of-the-art (SOTA) GNN-based methods on several image matching tasks under homography, estimation, indoor and outdoor camera pose estimation, and 3D reconstruction on multiple datasets. Experiments also demonstrate improved computational memory and runtime, approximately 38.1% and 26.14% lower than SuperGlue, and an average of about 6.8% and 7.1% lower than LightGlue. Future research will explore the effects of integrating more recent simplicial message-passing mechanisms, which concurrently update both node and edge representations, into our proposed model. [ABSTRACT FROM AUTHOR]

Image matching is an important task in computer vision. The detector-free dense matching method is an important research direction of image matching due to its high accuracy and robustness. The classical detector-free image matching methods utilize convolutional neural networks to extract features and then match them. Due to the lack of scale equivariance in CNNs, this method often exhibits poor matching performance when the images to be matched undergo significant scale variations. However, large-scale variations are very common in practical problems. To solve the above problem, we propose SeLFM, a method that combines scale equivariance and the global modeling capability of transformer. The two main advantages of this method are scale-equivariant CNNs can extract scale-equivariant features, while transformer also brings global modeling capability. Experiments prove that this modification improves the performance of the matcher in matching image pairs with large-scale variations and does not affect the general matching performance of the matcher. The code will be open-sourced at this link: https://github.com/LiaoYun0x0/SeLFM/tree/main [ABSTRACT FROM AUTHOR]

Few-shot counting (FSC) is the task of counting the number of objects in an image that belong to the same category, by using a provided exemplar pattern. By replacing the exemplar, we can effectively count anything, even in cases where we have no prior knowledge of that category's exemplar. However, due to the variations within the same category and the impact of inter-class similarity, it is challenging to achieve accurate intra-class similarity matching using conventional similarity comparison methods. To tackle these issues, we propose a novel few-shot counting method called Multi-stage Exemplar Attention Match Network (MEAMNet), which increases the accuracy of matching, reduces the impact of noise, and enhances similarity feature matching. Specifically, we propose a multi-stage matching strategy to obtain more stable and effective matching results by acquiring similar feature in different feature spaces. In addition, we propose a novel feature matching module called Exemplar Attention Match (EAM). With this module, the intra-class similarity representation in each stage will be enhanced to achieve a better matching of the key feature. Experimental results indicate that our method not only significantly surpasses the state-of-the-art (SOTA) methods in most evaluation metrics on the FSC-147 dataset but also achieves comprehensive superiority on the CARPK dataset. This highlights the outstanding accuracy and stability of our matching performance, as well as its exceptional transferability. We will release the code at https://github.com/hzg0505/MEAMNet. • Propose a novel few-shot counting method MEAMNet to count anything. • Propose a multi-stage matching strategy to obtain reliable matching result. • Propose a novel similarity match module EAM. [ABSTRACT FROM AUTHOR]

Detecting objects in images is crucial for several applications, including surveillance, autonomous navigation, augmented reality, and so on. Although AI-based approaches such as Convolutional Neural Networks (CNNs) have proven highly effective in object detection, in scenarios where the objects being recognized are unknow, it is difficult to generalize an AI model for such tasks. In another trend, feature-based approaches like SIFT, SURF, and ORB offer the capability to search any object but have limitations under complex visual variations. In this work, we introduce a novel edge-based object/scene recognition method. We propose that utilizing feature edges, instead of feature points, offers high performance under complex visual variations. Our primary contribution is a directional pixel voting descriptor based on image segments. Experimental results are promising; compared to previous approaches, ours demonstrates superior performance under complex visual variations and high processing speed. [ABSTRACT FROM AUTHOR]

Graffiti, by their very nature, are ephemeral, sometimes even vanishing before creators finish them. This transience is part of graffiti's allure yet signifies the continuous loss of this often disputed form of cultural heritage. To counteract this, graffiti documentation efforts have steadily increased over the past decade. One of the primary challenges in any documentation endeavour is identifying and recording new creations. Image‐based change detection can greatly help in this process, effectuating more comprehensive documentation, less biased digital safeguarding and improved understanding of graffiti. This paper introduces a novel and largely automated image‐based graffiti change detection method. The methodology uses an incremental structure‐from‐motion approach and synthetic cameras to generate co‐registered graffiti images from different areas. These synthetic images are fed into a hybrid change detection pipeline combining a new pixel‐based change detection method with a feature‐based one. The approach was tested on a large and publicly available reference dataset captured along the Donaukanal (Eng. Danube Canal), one of Vienna's graffiti hotspots. With a precision of 87% and a recall of 77%, the results reveal that the proposed change detection workflow can indicate newly added graffiti in a monitored graffiti‐scape, thus supporting a more comprehensive graffiti documentation. [ABSTRACT FROM AUTHOR]

With the development of image feature matching technology, feature matching algorithms based on deep learning have achieved excellent results, but in scenarios with low texture or extreme perspective changes, the matching accuracy is still difficult to guarantee. In this paper, a superresolution reconstruction method based on a Residual-ESPCN (efficient subpixel convolutional neural network) approach is proposed based on LoFTR (local feature matching with transformers). The superresolution method is used to improve the interpolation method used in ASFF (adaptive spatial feature fusion) to increase the image resolution, enhance the detailed information of the image, and make the extracted features richer. Then, ASFF is introduced into the local feature extraction module of LoFTR, which can alleviate the inconsistency problem of information transmission between different scale features of the feature pyramid and lessen the amount of information lost during transmission from low- to high-resolution levels. Moreover, to improve the adaptability of the algorithm to different scenarios, OTSU is introduced to adaptively calculate the threshold of feature matching. The experimental results show that in different indoor or outdoor scenarios, our proposed algorithm for matching features can effectively improve the adaptability of feature matching and can achieve good results in terms of the area under the curve (AUC), accuracy and recall. [ABSTRACT FROM AUTHOR]

Users are more comfortable trusting their sensitive information to the cloud as its security continues to improve. However, when there are several encrypted files, each with its own set of keywords for indexing, the storage overhead grows exponentially, and search efficiency suffers. Therefore, this work provides a technique for searching encrypted cloud data that makes use of features to match joint keywords (FMJK). Joint keywords are generated by randomly selecting a subset of the data owner's non-duplicated keywords choice among the documents' extracted keywords; together, these keywords form a keyword dictionary. Every combined keyword matches with a document's feature as well as a query keyword, making the former's result considered a dimension of a document's index with the latter's result considered a dimension about the query trapdoor. Its BM25 method is then utilized for arranging the top k results by the inner product between the document index and the trapdoor. [ABSTRACT FROM AUTHOR]

The popularity of Structure from Motion (SfM) techniques has significantly advanced 3D reconstruction in various domains, including construction site mapping. Central to SfM, is the feature extraction and matching process, which identifies and correlates keypoints across images. Previous benchmarks have assessed traditional and learning-based methods for these tasks but have not specifically focused on construction sites, often evaluating isolated components of the SfM pipeline. This study provides a comprehensive evaluation of traditional methods (e.g., SIFT, AKAZE, ORB) and learning-based methods (e.g., D2-Net, DISK, R2D2, SuperPoint, SOSNet) within the SfM pipeline for construction site mapping. It also compares matching techniques, including SuperGlue and LightGlue, against traditional approaches such as nearest neighbor. Our findings demonstrate that deep learning-based methods such as DISK with LightGlue and SuperPoint with various matchers consistently outperform traditional methods like SIFT in both reconstruction quality and computational efficiency. Overall, the deep learning methods exhibited better adaptability to complex construction environments, leveraging modern hardware effectively, highlighting their potential for large-scale and real-time applications in construction site mapping. This benchmark aims to assist researchers in selecting the optimal combination of feature extraction and matching methods for SfM applications at construction sites. [ABSTRACT FROM AUTHOR]

Robust matching, especially the number, precision and distribution of feature point matching, directly affects the effect of 3D reconstruction. However, the existing methods rarely consider these three aspects comprehensively to improve the quality of feature matching, which in turn affects the effect of 3D reconstruction. Therefore, to effectively improve the quality of 3D reconstruction, we propose a circle-based enhanced motion consistency and guided diffusion feature matching algorithm for 3D reconstruction named EMC+GD_C. Firstly, a circle-based neighborhood division method is proposed, which increases the number of initial matching points. Secondly, to improve the precision of feature point matching, on the one hand, we put forward the idea of enhancing motion consistency, reducing the mismatch of high similarity feature points by enhancing the judgment conditions of true and false matching points; on the other hand, we combine the RANSAC optimization method to filter out the outliers and further improve the precision of feature point matching. Finally, a novel guided diffusion idea combining guided matching and motion consistency is proposed, which expands the distribution range of feature point matching and improves the stability of 3D models. Experiments on 8 sets of 908 pairs of images in the public 3D reconstruction datasets demonstrate that our method can achieve better matching performance and show stronger stability in 3D reconstruction. Specifically, EMC+GD_C achieves an average improvement of 24.07% compared to SIFT-based ratio test, 9.18% to GMS and 1.94% to EMC+GD_G in feature matching precision. [ABSTRACT FROM AUTHOR]

Waves are crucial in ocean observation and research. Stereo vision-based wave measurement, offering non-contact, low-cost, and intelligent processing, is an emerging method. However, improving accuracy remains a challenge due to wave complexity. This paper presents a novel approach to measure wave height, period, and direction by combining deep learning-based stereo matching with feature matching techniques. To improve the discontinuity and low accuracy in disparity maps from traditional wave image matching algorithms, this paper proposes the use of a high-precision stereo matching method based on Pyramid Stereo Matching Network (PSM-Net).A 3D reconstruction method integrating Scale-Invariant Feature Transform (SIFT) with stereo matching was also introduced to overcome the limitations of template matching and interleaved spectrum methods, which only provide 2D data and fail to capture the full 3D motion of waves. This approach enables accurate wave direction measurement. Additionally, a six-degree-of-freedom platform was proposed to simulate waves, addressing the high costs and attenuation issues of traditional wave tank simulations. Experimental results show the prototype system achieves a wave height accuracy within 5%, period accuracy within 4%, and direction accuracy of ±2°, proving the method’s effectiveness and offering a new approach to stereo vision-based wave measurement.

ABSTRACTFeature matching is a classic challenge in the computer vision field. In this paper, we propose an innovative graph classification method based on neighborhood motion consistency to eliminate erroneous matches. Specifically, we transform the coordinates of feature matching points into vectors on a unified scale. For a given match, we construct a graph centered around the match and incorporating neighboring matches. Node attributes are designed to represent the similarity between the vector of the central node and those of its neighbors. To facilitate this, we develop a lightweight graph attention neural network dedicated to graph property classification, thereby predicting the accuracy of the match under consideration. To effectively train the model, we employ a random cropping strategy to generate a plethora of diverse graphs for classifier training. We evaluate our method on datasets encompassing translational remote sensing data, rotational and scaled remote sensing imagery produced via random cropping, and nonrigid fisheye datasets. Our algorithm demonstrates superior performance to current state-of-the-art methods.

IntroductionSimultaneous Localization and Mapping (SLAM) is a technology used in intelligent systems such as robots and autonomous vehicles. Visual SLAM has become a more popular type of SLAM due to its acceptable cost and good scalability when applied in robot positioning, navigation and other functions. However, most of the visual SLAM algorithms assume a static environment, so when they are implemented in highly dynamic scenes, problems such as tracking failure and overlapped mapping are prone to occur.MethodsTo deal with this issue, we propose ISFM-SLAM, a dynamic visual SLAM built upon the classic ORB-SLAM2, incorporating an improved instance segmentation network and enhanced feature matching. Based on YOLACT, the improved instance segmentation network applies the multi-scale residual network Res2Net as its backbone, and utilizes CIoU_Loss in the bounding box loss function, to enhance the detection accuracy of the segmentation network. To improve the matching rate and calculation efficiency of the internal feature points, we fuse ORB key points with an efficient image descriptor to replace traditional ORB feature matching of ORB-SLAM2. Moreover, the motion consistency detection algorithm based on external variance values is proposed and integrated into ISFM-SLAM, to assist the proposed SLAM systems in culling dynamic feature points more effectively.Results and discussionSimulation results on the TUM dataset show that the overall pose estimation accuracy of the ISFM-SLAM is 97% better than the ORB-SLAM2, and is superior to other mainstream and state-of-the-art dynamic SLAM systems. Further real-world experiments validate the feasibility of the proposed SLAM system in practical applications.

Inverse synthetic aperture radar (ISAR) image registration enables the analysis of target dynamics by comparing registered images from different viewpoints. However, it faces significant challenges due to various factors, such as the complex scattering characteristics of the target, limited availability of information, and additive noise in ISAR images. This paper proposes a novel ISAR image registration method based on line features. It integrates information from both dominant scatterers and the target’s outer contour to detect lines. According to the consistency principles of multiple lines in rotation and translation, line features from different ISAR images are matched. Simultaneously, the results of the feature matching are utilized to guide the parameter configuration for optimizing the image registration process. Comparative experiments illustrate the advantages of the proposed method in both feature extraction and registration feasibility.

Abstract Robust matching, especially the number, precision and distribution of feature point matching, directly affects the effect of 3D reconstruction. However, the existing methods rarely consider these three aspects comprehensively to improve the quality of feature matching, which in turn affects the effect of 3D reconstruction. Therefore, to effectively improve the quality of 3D reconstruction, we propose a circle-based enhanced motion consistency and guided diffusion feature matching algorithm for 3D reconstruction named EMC+GD_C. Firstly, a circle-based neighborhood division method is proposed, which increases the number of initial matching points. Secondly, to improve the precision of feature point matching, on the one hand, we put forward the idea of enhancing motion consistency, reducing the mismatch of high similarity feature points by enhancing the judgment conditions of true and false matching points; on the other hand, we combine the RANSAC optimization method to filter out the outliers and further improve the precision of feature point matching. Finally, a novel guided diffusion idea combining guided matching and motion consistency is proposed, which expands the distribution range of feature point matching and improves the stability of 3D models. Experiments on 8 sets of 908 pairs of images in the public 3D reconstruction datasets demonstrate that our method can achieve better matching performance and show stronger stability in 3D reconstruction. Specifically, EMC+GD_C achieves an average improvement of 24.07% compared to SIFT-based ratio test, 9.18% to GMS and 1.94% to EMC+GD_G in feature matching precision.

While the current research has achieved satisfactory results for the registration of single mode data, there has always been a significant challenge in the registration of multi-modal images due to the obvious nonlinear radiation differences caused by different imaging mechanisms and imaging time. For example, multi-temporal visible, visible-synthetic aperture radar, visible-near infrared, near infrared-short wave infrared, visible-MAP, etc. To address this problem, we propose a Robust and Accurate Feature Matching Method for Multi-modal Geographic Images Spatial Registration (RAMMR) to fully extract common key points between images, weaken the radiation difference between data, and finally accurately match more inliers to realize multi-modal image registration. Considering the influence of noise and edge information on key point extraction, RAMMR first constructs a new scale space by introducing the Side Window Filter (SWF); Then, we improve Harris algorithm to extract key points based on the SWF scale space; After that, we propose an enhanced log-polar descriptor based on the gradient angles and gradient amplitudes of the scale space, which effectively improves the quality of the descriptor and avoids the mismatch of key points; Based on the standard Euclidean distance, we design a re-match strategy to obtain the initial matching results, and Random Sample Consensus (RANSAC) is used to eliminate outliers. Finally, the affine transformation parameters are calculated based on inliers, and multi-modal image registration is realized. RAMMR is evaluated on different multi-modal datasets and compared with some state-of-art methods. The experimental results show that RAMMR accurately registers multi-modal geographic images and obtains comparative results compared with benchmark methods. Our source datasets are publicly available at https://github.com/RSmfmr/multimodal-dataset.

In recent years, with the rapid development of unmanned aerial vehicle (UAV) technology, multi-view 3D reconstruction has once again become a hot spot in computer vision. Incremental Structure From Motion (SFM) is currently the most prevalent reconstruction pipeline, but it still faces challenges in reconstruction efficiency, accuracy, and feature matching. In this paper, we use deep learning algorithms for feature matching to obtain more accurate matching point pairs. Moreover, we adopted the improved Gauss–Newton (GN) method, which not only avoids numerical divergence but also accelerates the speed of bundle adjustment (BA). Then, the sparse point cloud reconstructed by SFM and the original image are used as the input of the depth estimation network to predict the depth map of each image. Finally, the depth map is fused to complete the reconstruction of dense point clouds. After experimental verification, the reconstructed dense point clouds have rich details and clear textures, and the integrity, overall accuracy, and reconstruction efficiency of the point clouds have been improved. [ABSTRACT FROM AUTHOR]

Three-dimensional (3D) single-object tracking (3D SOT) is a fundamental yet not well-solved problem in 3D vision, where the complexity of feature matching and the sparsity of point clouds pose significant challenges. To handle abrupt changes in appearance features and sparse point clouds, we propose a novel 3D SOT network, dubbed CDTracker. It leverages both cosine similarity and an attention mechanism to enhance the robustness of feature matching. By combining similarity embedding and attention assignment, CDTracker performs template and search area feature matching in a coarse-to-fine manner. Additionally, CDTracker addresses the problem of sparse point clouds, which commonly leads to inaccurate tracking. It incorporates relatively dense sampling based on the concept of point cloud segmentation to retain more target points, leading to improved localization accuracy. Extensive experiments on both the KITTI and Waymo datasets demonstrate clear improvements in CDTracker over its competitors. [ABSTRACT FROM AUTHOR]

In the domain of point cloud registration, the coarse-to-fine feature matching paradigm has received significant attention due to its impressive performance. This paradigm involves a two-step process: first, the extraction of multilevel features, and subsequently, the propagation of correspondences from coarse to fine levels. However, this approach faces two notable limitations. Firstly, the use of the Dual Softmax operation may promote one-to-one correspondences between superpoints, inadvertently excluding valuable correspondences. Secondly, it is crucial to closely examine the overlapping areas between point clouds, as only correspondences within these regions decisively determine the actual transformation. Considering these issues, we propose OAAFormer to enhance correspondence quality. On the one hand, we introduce a soft matching mechanism to facilitate the propagation of potentially valuable correspondences from coarse to fine levels. On the other hand, we integrate an overlapping region detection module to minimize mismatches to the greatest extent possible. Furthermore, we introduce a region-wise attention module with linear complexity during the fine-level matching phase, designed to enhance the discriminative capabilities of the extracted features. Tests on the challenging 3DLoMatch benchmark demonstrate that our approach leads to a substantial increase of about 7% in the inlier ratio, as well as an enhancement of 2%–4% in registration recall. Finally, to accelerate the prediction process, we replace the Conventional Random Sample Consensus (RANSAC) algorithm with the selection of a limited yet representative set of high-confidence correspondences, resulting in a 100 times speedup while still maintaining comparable registration performance. [ABSTRACT FROM AUTHOR]

Computer vision is a catch-all term for a variety of applications. This makes it a good research environment for new ideas and concepts. Feature extraction is considered an essential step in such applications. Several research studies introduced the Scale-Invariant Feature Transform (SIFT) as a benchmark method to extract visual features from objects inside images. This ensures the need for a deep study of SIFT in a variety of settings. Hence, this paper presents an assessment of SIFT from different perspectives that are not explicitly expressed in the literature. In addition, it presents an illustration of the majority of Oriented FAST and Rotated BRIEF (ORB) feature extraction characteristics to facilitate the choice procedure between SIFT and ORB. Several experimental cases are included, each of which evaluates the performance of such methods from distinct and different aspects. At first, the paper presents an assessment of these methods to identify objects inside geometrically–affine-transformed images. This is done by comparing how well their gathered feature descriptors from images perform against one another. Second, this paper presents an evaluation of the invariance of these methods to the changes in illumination. Furthermore, the computational and asymptotic complexity of such methods is investigated to examine its impact on the complexity of any feature-based system. Finally, the efficiency of these methods is verified by assessing their ability to support real-time applications, through the evaluation of their time and space complexities over all investigated test scenarios. [ABSTRACT FROM AUTHOR]

With the rapid development and extensive application of next-generation image processing technologies, the manufacturing industry is increasingly adopting intelligent equipment. To meet the demands of high precision and high efficiency production, there has been a growing focus on researching 3D point cloud processing methods that go beyond traditional approaches. A fundamental and crucial challenge in the field of point cloud processing is establishing a point-to-point correspondence mapping between two point clouds, which relies on leveraging the local feature description information inherent in the point cloud. This paper thoroughly investigates novel local description methods based on point cloud processing. It addresses the issue of inadequate descriptive capability and robustness found in existing local description methods. Specifically, this study explores the encoding of point information in the neighborhood space and multi-view projection mapping, proposing a local point cloud description method based on depth grids. This method leverages a local reference frame and establishes a depth grid after obtaining the local reference frame through neighborhood projection and distance weighting. The contribution of neighboring points to the depth of the grid is calculated to obtain the eigenvalues. To enhance efficiency, the calculation of eigenvalues incorporates normalization and multi-view projection techniques. The proposed method is compared and evaluated against various local description methods to verify its effectiveness and accuracy. [ABSTRACT FROM AUTHOR]

Efficient image stitching plays a vital role in the Non-Destructive Evaluation (NDE) of infrastructures. An essential challenge in the NDE of infrastructures is precisely visualizing defects within large structures. The existing literature predominantly relies on high-resolution close-distance images to detect surface or subsurface defects. While the automatic detection of all defect types represents a significant advancement, understanding the location and continuity of defects is imperative. It is worth noting that some defects may be too small to capture from a considerable distance. Consequently, multiple image sequences are captured and processed using image stitching techniques. Additionally, visible and infrared data fusion strategies prove essential for acquiring comprehensive information to detect defects across vast structures. Hence, there is a need for an effective image stitching method appropriate for infrared and visible images of structures and industrial assets, facilitating enhanced visualization and automated inspection for structural maintenance. This paper proposes an advanced image stitching method appropriate for dual-sensor inspections. The proposed image stitching technique employs self-supervised feature detection to enhance the quality and quantity of feature detection. Subsequently, a graph neural network is employed for robust feature matching. Ultimately, the proposed method results in image stitching that effectively eliminates perspective distortion in both infrared and visible images, a prerequisite for subsequent multi-modal fusion strategies. Our results substantially enhance the visualization capabilities for infrastructure inspection. Comparative analysis with popular state-of-the-art methods confirms the effectiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

Reference-based super resolution (RefSR) aims to recover the lost details in a low-resolution image and generate a high-resolution result, guided by a high-resolution reference image with similar contents or textures. In contrast to the traditional single-image super-resolution, which focuses on the intrinsic properties of the single low-resolution image, the challenge of RefSR lies in matching and aggregating highly-related but misaligned reference images with low-resolution images. Several effective but complex designs have been proposed to address this challenge, which poses difficulties in implementing RefSR in real-world applications. In order to better understand the working mechanism of RefSR and design a more efficient and lightweight architecture, we provide a review about the essential components of the existing deep learning-based RefSR. We decompose and classify the common pipeline into four submodules according to the functionalities. Then, we summarize and describe the implementation details of the commonly-adopted approaches in each submodule. Finally, we discuss the challenges and promising research directions. [ABSTRACT FROM AUTHOR]

Students with complex communication needs have increasingly been using non-dedicated communication systems, such as mobile devices, to support their communication needs. This in turn, has led to an increased used of augmentative and alternative communication apps. The main challenge currently faced is the lack of empirically validated apps and evaluation systems to assess the features of the apps. As a result, this study attempted to determine the reliability of an app evaluation tool that was grounded in the components of the feature matching model. The goal was also to identify if the app evaluation tool could be used to evaluate various types of augmentative and alternative communication apps. Participants evaluated apps across the dimensions of usability, output, and display. Results suggest that expert raters were more reliability than novice raters across the various types of apps. Practical implications and future research directions are discussed. [ABSTRACT FROM AUTHOR]

Semantic information associated Simultaneous Localization and Mapping (SIA-SLAM), a visual SLAM algorithm using semantic information association, is proposed to solve the problems that dynamic objects lead to the decreased accuracy of the localization and feature matching between two frames due to the lack of object semantic information. Firstly, a Solov2 instance segmentation network is used to obtain instance segmentation images, and the feature points are extracted from the RBG images simultaneously. Secondly, the feature points on dynamic objects are removed, and the semantic information of static objects is associated with the remaining feature points. Then, the static feature points are utilized to estimate the camera poses and update the static map point set. Finally, the camera poses are optimized by using closed-loop detection. When tracking the camera poses and inter-frame feature matching during the closed-loop detection, the semantic information of the feature points is checked first, and then the bag-of-words model is used for feature matching. The proposed SIA-SLAM algorithm is tested on the Technische Universität München (TUM) public dataset. As far as the absolute trajectory errors (ATE) are concerned, the Root Mean Square Errors (RMSE) and Standard Deviation (S.D.) improvement values can reach up to 98.15% and 98.18% in high dynamic scene of TUM dataset, respectively. The proposed SIA-SLAM algorithm is superior to other semantic SLAM algorithms which are tested in the specific datasets. Furthermore, the reliability and robustness of the SIA-SLAM algorithm are verified in a real scenario. The SIA-SLAM algorithm effectively improves the accuracy of the camera trajectory estimation and feature matching. [ABSTRACT FROM AUTHOR]