Search

To tackle challenges in road multi‐object detection, such as object occlusion, small object detection, and multi‐scale object detection difficulties, a new YOLOv8n‐RSFM structure is proposed. The key improvement of this structure lies in the introduction of the transformer decoder head, which optimizes the matching between the ground truth and predicted boxes, thereby effectively addressing issues of object overlap and multi‐scale detection. Additionally, a small object detection layer is incorporated to retain crucial information beneficial for detecting small objects, significantly improving the detection accuracy for small targets. To enhance learning capacity and reduce redundant computations, the FasterNet backbone is employed to replace CSPDarknet53, thus accelerating the training process. Finally, the INNER‐MPDIoU loss function is introduced to replace the original algorithm's complete IoU to accelerate the convergence and obtain more accurate regression results. A series of experiments were conducted on different datasets. The experimental results show that the proposed model YOLOv8N‐RSFM outperforms the original model YOLOv8n in small target detection. On the VisDrone, TinyPerson, and VSCrowd datasets, the mean accuracy percentage improved by 7.9%, 12.3%, and 4.5%, respectively. [ABSTRACT FROM AUTHOR]

Insulators, as typical equipment for distribution networks, provide good electrical insulation between live conductors and earth. Timely and accurate detection is essential for insulator detection issues. However, as the complexity of neural networks increases, the detection efficiency is often lower. Therefore, this paper proposes a fast insulator positioning and defect detection method. Firstly, for insulator target localization, the SqueezeNet network is improved using ECA attention mechanism. In addition, to address the issue of low defect detection accuracy, a joint algorithm has been proposed. The integration of convolutional variational autoencoder (CVAE) and generative adversarial network (GAN) solve their own shortcomings due to different image focus angles. The target localization accuracy reaches 94.30%, and the defect detection accuracy reaches 89.60%. It solves the problems of difficulty in locating small targets in a large field of view and inaccurate detection due to a small number of abnormal samples. This method has been tried and tested in practical distribution network systems. [ABSTRACT FROM AUTHOR]

In order to expand the application range of underwater panoramic images, ensure the image matching robustness and effect, and improve the matching accuracy of underwater panoramic image, a matching method of underwater panoramic image based on ORB-GMS is proposed. After noise reduction processing of underwater panoramic images by wavelet thresholding method, the ORB algorithm is used to obtain feature point descriptions of underwater panoramic images after noise reduction by scale space construction, feature point detection and extraction, and descriptor construction. The feature points of underwater panoramic image extracted and described by the ORB algorithm are meshed using the GMS algorithm, and the results of the meshing are used as the neighborhood support estimates. Based on the grid motion, the statistical neighborhood support estimator is used to distinguish the correct matching points from the incorrect ones to complete the underwater panoramic image matching. The experimental results show that the proposed method for underwater panoramic image matching produces more matching points, more number of interior points, larger interior point ratio and less matching time consumption; it can effectively improve the matching accuracy of underwater panoramic images processed by scaling size, changing light and dark, and rotating angle transformation. [ABSTRACT FROM AUTHOR]

As a hot research field at present, computer vision is devoted to the rapid acquisition and application of target information from images or videos by simulating human visual mechanism. In order to improve the accuracy and efficiency of image detection, image saliency region detection technology has received more and more attention in the field of computer vision research; an important research content in the field, the core part of which lies in the research on algorithms related to feature extraction and saliency calculation of targets. This paper analyzes the multi‐feature fusion saliency detection model and visual saliency calculation process, and based on the existing algorithm, by improving the VGG16 network, a fully convolutional network saliency detection algorithm is proposed. The qualitative and quantitative experimental results show that compared with the four mainstream methods of BL, GS, SF, and RFCN, our algorithm not only improves the accuracy of salient object detection, but also effectively solves the problem of target edge blur. Therefore, this study has improved the accuracy and efficiency of saliency detection, which can not only promote the development of computer vision technology, but also provide support for research in the field of image processing. [ABSTRACT FROM AUTHOR]

The article proposes a real‐time apple picking method based on an improved YOLOv5. This method accurately recognizes different apple targets on fruit trees for robots and helps them adjust their position to avoid obstructions during fruit picking. Firstly, the original BottleneckCSP module in the YOLOv5 backbone network is enhanced to extract deeper features from images while maintaining lightweight. Secondly, the ECA module is embedded into the improved backbone network to better extract features of different apple targets. Lastly, the initial anchor frame size of the network is adjusted to avoid recognizing apples in distant planting rows. The results demonstrate that this improved model achieves high accuracy rates and recall rates for recognizing various types of apple picking methods with an average recognition time of 0.025s per image. Compared with other models tested on six types of apple picking methods, including the original YOLOv5 model as well as YOLOv3 and EfficientDet‐D0 algorithms, our improved model shows significant improvements in mAP by 1.95%, 17.6%, and 12.7% respectively. This method provides technical support for robots' picking hands to actively avoid obstructions caused by branches during fruit harvesting, effectively reducing apple loss. [ABSTRACT FROM AUTHOR]

Recently, the number of high-resolution Earth-observing satellite sensors has been increasing owing to the growing needs of intelligence, mapping, and environmental monitoring. An acquired satellite image should be processed for analysis-ready data (ARD) that can be used for many applications. An important step among the processing is georeferencing that assigns geographic coordinates to each image pixel. These days, georeferencing is directly carried out using onboard sensors to produce sensor model information such as rational polynomial coefficients (RPCs). However, postprocessing is required to increase the positional accuracy of RPCs through bias compensation. Recently, bias compensation has been carried out on the basis of an automated process using ground control point (GCP) image chips. Image matching is carried out between the chips and the target satellite image to model the bias over the entire image. However, if the dissimilarity between the chip and the target satellite image increases owing to large differences in acquisition time and seasonal differences, the image matching often fails. Therefore, in this study, we utilized both intensity-based matching and edge-based matching to overcome these issues. We selected normalized cross-correlation (NCC) for intensity-based matching and relative edge cross-correlation (RECC) for edge-based matching. First, GCP chips were projected onto the target satellite images to align the two datasets. Then, both image matching methods were carried out in a pyramid image matching scheme, and their results were merged before RPC bias compensation with outlier removal. The experiments were carried out for two Kompsat-3A strips consisting of 9 and 7 scenes. NCC and RECC showed different matching results per scene, but RECC tended to show better results. NCC + RECC could derive most matching points, but the accuracy was between NCC and RECC. However, NCC + RECC shows potential to suppress a matching outlier. By applying automated bias compensation, 1.1-1.2 pixels of accuracy in root mean square error (RMSE) could be obtained. [ABSTRACT FROM AUTHOR]

Siamese networks excel at comparing two images, serving as an effective class verification technique for a single‐per‐class reference image. However, when multiple reference images are present, Siamese verification necessitates multiple comparisons and aggregation, often unpractical at inference. The Centre‐Loss approach, proposed in this research, solves a class verification task more efficiently, using a single forward‐pass during inference, than sample‐to‐sample approaches. Optimising a Centre‐Loss function learns class centres and minimises intra‐class distances in latent space. The authors compared verification accuracy using Centre‐Loss against aggregated Siamese when other hyperparameters (such as neural network backbone and distance type) are the same. Experiments were performed to contrast the ubiquitous Euclidean against other distance types to discover the optimum Centre‐Loss layer, its size, and Centre‐Loss weight. In optimal architecture, the Centre‐Loss layer is connected to the penultimate layer, calculates Euclidean distance, and its size depends on distance type. The Centre‐Loss method was validated on the Self‐Checkout products and Fruits 360 image datasets. Centre‐Loss comparable accuracy and lesser complexity make it a preferred approach over sample‐to‐sample for the class verification task, when the number of reference image per class is high and inference speed is a factor, such as in self‐checkouts. [ABSTRACT FROM AUTHOR]

Robust and efficient multi-source image matching remains a challenging task due to nonlinear radiometric differences between image features. This paper proposes a pixel-level matching framework for multi-source images to overcome this issue. Firstly, a novel descriptor called channel features of phase congruency (CFPC) is first derived at each control point to create a pixelwise feature representation. The proposed CFPC is not only simple to construct but is also highly efficient and somewhat insensitive to noise and intensity changes. Then, a Fast Sequential Similarity Detection Algorithm (F-SSDA) is proposed to further improve the matching efficiency. Comparative experiments are conducted by matching different types of multi-source images (e.g., Visible–SAR; LiDAR–Visible; visible–infrared). The experimental results demonstrate that the proposed method can achieve pixel-level matching accuracy with high computational efficiency. [ABSTRACT FROM AUTHOR]

The estimation of multiple matching models between wide baseline or large angle images is a quite challenging task in image processing. The existing algorithms can be used to estimate multiple matching models and their inliers between images well, but their results are prone to matching pairs mis-classification issues. In order to accurately estimate the multiple matching models and allocate matching pairs, this paper proposed an image multi-matching model estimation algorithm based on the aggregation of matching points of the same model (AMPSM). Firstly, for improve the proportion of correct matching pairs, it filtered out incorrect matching pairs based on the distribution characteristics of correct matching points in the neighboring region. Furthermore, based on the different matching model degrees to which the matching pairs belong, searched for the suspected intersection matching pairs of multiple models, that was interference matching pairs. Meantime, for reducing the impact of interference matching pairs on the accuracy of matching classification, they were removed. Afterwards, for improve the clustering degree of matching points with the co-model, the position was dynamically moved based on the distance between the points within the same model and the center of gravity of the point set during the sampling process. Finally, classifying clustered matching points by Mean Shift to obtain a multi matching model. And the proposed method was compared with classical framework based algorithms RANSAC, PROSAC, MAGSAC++, GMS, AdaLAM, PEARL, MTC, Sequential RANSAC, and deep learning based algorithms SuperGlue, OANet, CLCNet, CONSAC, etc. Results indicate over 30% increase in the inlier rate, 8.39% reduction in the mis-classification rate of multi model estimation. It is concluded that the new algorithm has significant advantages in incorrect matches filtering and multi-model estimation. [ABSTRACT FROM AUTHOR]

Image matching is an important task in computer vision. The detector-free dense matching method is an important research direction of image matching due to its high accuracy and robustness. The classical detector-free image matching methods utilize convolutional neural networks to extract features and then match them. Due to the lack of scale equivariance in CNNs, this method often exhibits poor matching performance when the images to be matched undergo significant scale variations. However, large-scale variations are very common in practical problems. To solve the above problem, we propose SeLFM, a method that combines scale equivariance and the global modeling capability of transformer. The two main advantages of this method are scale-equivariant CNNs can extract scale-equivariant features, while transformer also brings global modeling capability. Experiments prove that this modification improves the performance of the matcher in matching image pairs with large-scale variations and does not affect the general matching performance of the matcher. The code will be open-sourced at this link: https://github.com/LiaoYun0x0/SeLFM/tree/main [ABSTRACT FROM AUTHOR]

The cross-view matching of local image features is a fundamental task in visual localization and 3D reconstruction. This study proposes FilterGNN, a transformer-based graph neural network (GNN), aiming to improve the matching efficiency and accuracy of visual descriptors. Based on high matching sparseness and coarse-to-fine covisible area detection, FilterGNN utilizes cascaded optimal graph-matching filter modules to dynamically reject outlier matches. Moreover, we successfully adapted linear attention in FilterGNN with post-instance normalization support, which significantly reduces the complexity of complete graph learning from O(N2) to O(N). Experiments show that FilterGNN requires only 6% of the time cost and 33.3% of the memory cost compared with SuperGlue under a large-scale input size and achieves a competitive performance in various tasks, such as pose estimation, visual localization, and sparse 3D reconstruction. [ABSTRACT FROM AUTHOR]

Wave information retrieval from videos captured by a single camera has been increasingly applied in marine observation. However, when the camera observes ocean waves at low grazing angles, the accurate extraction of wave information from videos will be affected by the interference of the fine ripples on the sea surface. To solve this problem, this study develops a method for estimating peak wave periods from videos captured at low grazing angles. The method extracts the motion of the sea surface texture from the video and obtains the peak wave period via the spectral analysis. The calculation results captured from real-world videos are compared with those obtained from X-band radar inversion and tracking buoy movement, with maximum deviations of 8% and 14%, respectively. The analysis of the results shows that the peak wave period of the method has good stability. In addition, this paper uses a pinhole camera model to convert the displacement of the texture from pixel height to actual height and performs moving average filtering on the displacement of the texture, thus conducting a preliminary exploration of the inversion of significant wave height. This study helps to extend the application of sea surface videos. [ABSTRACT FROM AUTHOR]

Abstract To tackle challenges in road multi‐object detection, such as object occlusion, small object detection, and multi‐scale object detection difficulties, a new YOLOv8n‐RSFM structure is proposed. The key improvement of this structure lies in the introduction of the transformer decoder head, which optimizes the matching between the ground truth and predicted boxes, thereby effectively addressing issues of object overlap and multi‐scale detection. Additionally, a small object detection layer is incorporated to retain crucial information beneficial for detecting small objects, significantly improving the detection accuracy for small targets. To enhance learning capacity and reduce redundant computations, the FasterNet backbone is employed to replace CSPDarknet53, thus accelerating the training process. Finally, the INNER‐MPDIoU loss function is introduced to replace the original algorithm's complete IoU to accelerate the convergence and obtain more accurate regression results. A series of experiments were conducted on different datasets. The experimental results show that the proposed model YOLOv8N‐RSFM outperforms the original model YOLOv8n in small target detection. On the VisDrone, TinyPerson, and VSCrowd datasets, the mean accuracy percentage improved by 7.9%, 12.3%, and 4.5%, respectively.

Abstract Insulators, as typical equipment for distribution networks, provide good electrical insulation between live conductors and earth. Timely and accurate detection is essential for insulator detection issues. However, as the complexity of neural networks increases, the detection efficiency is often lower. Therefore, this paper proposes a fast insulator positioning and defect detection method. Firstly, for insulator target localization, the SqueezeNet network is improved using ECA attention mechanism. In addition, to address the issue of low defect detection accuracy, a joint algorithm has been proposed. The integration of convolutional variational autoencoder (CVAE) and generative adversarial network (GAN) solve their own shortcomings due to different image focus angles. The target localization accuracy reaches 94.30%, and the defect detection accuracy reaches 89.60%. It solves the problems of difficulty in locating small targets in a large field of view and inaccurate detection due to a small number of abnormal samples. This method has been tried and tested in practical distribution network systems.

Abstract Siamese networks excel at comparing two images, serving as an effective class verification technique for a single‐per‐class reference image. However, when multiple reference images are present, Siamese verification necessitates multiple comparisons and aggregation, often unpractical at inference. The Centre‐Loss approach, proposed in this research, solves a class verification task more efficiently, using a single forward‐pass during inference, than sample‐to‐sample approaches. Optimising a Centre‐Loss function learns class centres and minimises intra‐class distances in latent space. The authors compared verification accuracy using Centre‐Loss against aggregated Siamese when other hyperparameters (such as neural network backbone and distance type) are the same. Experiments were performed to contrast the ubiquitous Euclidean against other distance types to discover the optimum Centre‐Loss layer, its size, and Centre‐Loss weight. In optimal architecture, the Centre‐Loss layer is connected to the penultimate layer, calculates Euclidean distance, and its size depends on distance type. The Centre‐Loss method was validated on the Self‐Checkout products and Fruits 360 image datasets. Centre‐Loss comparable accuracy and lesser complexity make it a preferred approach over sample‐to‐sample for the class verification task, when the number of reference image per class is high and inference speed is a factor, such as in self‐checkouts.

Abstract The article proposes a real‐time apple picking method based on an improved YOLOv5. This method accurately recognizes different apple targets on fruit trees for robots and helps them adjust their position to avoid obstructions during fruit picking. Firstly, the original BottleneckCSP module in the YOLOv5 backbone network is enhanced to extract deeper features from images while maintaining lightweight. Secondly, the ECA module is embedded into the improved backbone network to better extract features of different apple targets. Lastly, the initial anchor frame size of the network is adjusted to avoid recognizing apples in distant planting rows. The results demonstrate that this improved model achieves high accuracy rates and recall rates for recognizing various types of apple picking methods with an average recognition time of 0.025s per image. Compared with other models tested on six types of apple picking methods, including the original YOLOv5 model as well as YOLOv3 and EfficientDet‐D0 algorithms, our improved model shows significant improvements in mAP by 1.95%, 17.6%, and 12.7% respectively. This method provides technical support for robots' picking hands to actively avoid obstructions caused by branches during fruit harvesting, effectively reducing apple loss.

Abstract As a hot research field at present, computer vision is devoted to the rapid acquisition and application of target information from images or videos by simulating human visual mechanism. In order to improve the accuracy and efficiency of image detection, image saliency region detection technology has received more and more attention in the field of computer vision research; an important research content in the field, the core part of which lies in the research on algorithms related to feature extraction and saliency calculation of targets. This paper analyzes the multi‐feature fusion saliency detection model and visual saliency calculation process, and based on the existing algorithm, by improving the VGG16 network, a fully convolutional network saliency detection algorithm is proposed. The qualitative and quantitative experimental results show that compared with the four mainstream methods of BL, GS, SF, and RFCN, our algorithm not only improves the accuracy of salient object detection, but also effectively solves the problem of target edge blur. Therefore, this study has improved the accuracy and efficiency of saliency detection, which can not only promote the development of computer vision technology, but also provide support for research in the field of image processing.

The present paper aims to conduct an experiment that compares different methods of detecting objects in images. Programs were developed to evaluate the efficiency of SURF, BRISK, MSER, and ORB object detection methods. Four static gray images with sufficiently different histograms were used. The experiment also highlighted the need for image preprocessing to improve feature extraction and detection. Thus, a programmed method for adjusting pixel groups was developed. This method proved useful when one of the listed algorithms failed to detect the object in the original image, but succeeded after adjustment. The effectiveness of detection methods and the evaluation of their performance depend on the application, image preparation, algorithms used, and their implementation. Results of the detection methods were presented numerically (similarities, gradients, distances, etc.) and graphically.

Abstract Synthetic images facilitate stereo matching. However, synthetic images may suffer from image distortion, domain bias, and stereo mismatch, which would significantly restrict the widespread use of stereo matching models in the real world. The first goal in this paper is to synthesize real‐looking images for minimizing the domain bias between the synthesized and real images. For this purpose, sharpened disparity maps are produced from a mono real image. Then, stereo image pairs are synthesized using these imperfect disparity maps and the single real image in the proposed pipeline. Although the synthesized images are as realistic as possible, the domain styles of the synthesized images are always very different from the real images. Thus, the second goal is to enhance the domain generalization ability of the stereo matching network. For that, the feature extraction layer is replaced with a teacher–student model. Then, a constraint of binocular contrast features is imposed on the output of the model. When tested on the KITTI, ETH3D, and Middlebury datasets, the accuracy of the method outperforms traditional methods by at least 30%. Experiments demonstrate that the approaches are general and can be conveniently embedded into existing stereo networks.

Synthetic images facilitate stereo matching. However, synthetic images may suffer from image distortion, domain bias, and stereo mismatch, which would significantly restrict the widespread use of stereo matching models in the real world. The first goal in this paper is to synthesize real‐looking images for minimizing the domain bias between the synthesized and real images. For this purpose, sharpened disparity maps are produced from a mono real image. Then, stereo image pairs are synthesized using these imperfect disparity maps and the single real image in the proposed pipeline. Although the synthesized images are as realistic as possible, the domain styles of the synthesized images are always very different from the real images. Thus, the second goal is to enhance the domain generalization ability of the stereo matching network. For that, the feature extraction layer is replaced with a teacher–student model. Then, a constraint of binocular contrast features is imposed on the output of the model. When tested on the KITTI, ETH3D, and Middlebury datasets, the accuracy of the method outperforms traditional methods by at least 30%. Experiments demonstrate that the approaches are general and can be conveniently embedded into existing stereo networks. [ABSTRACT FROM AUTHOR]

The accuracy and reliability of unmanned aerial vehicle (UAV) visual positioning systems are dependent on the performance of multi-source image matching algorithms. Despite many advancements, targeted performance evaluation frameworks and datasets for UAV positioning are still lacking. Moreover, existing consistency verification methods such as Random Sample Consensus (RANSAC) often fail to entirely eliminate mismatches, affecting the precision and stability of the matching process. The contributions of this research include the following: (1) the development of a benchmarking framework accompanied by a large evaluation dataset for assessing the efficacy of multi-source image matching algorithms; (2) the results of this benchmarking framework indicate that combinations of multiple algorithms significantly enhance the Match Success Rate (MSR); (3) the introduction of a novel Geographic Geometric Consistency (GGC) method that effectively identifies mismatches within RANSAC results and accommodates rotational and scale variations; and (4) the implementation of a distance threshold iteration (DTI) method that, according to experimental results, achieves an 87.29% MSR with a Root Mean Square Error (RMSE) of 1.11 m (2.22 pixels) while maintaining runtime at only 1.52 times that of a single execution, thus optimizing the trade-off between MSR, accuracy, and efficiency. Furthermore, when compared with existing studies on UAV positioning, the multi-source image matching algorithms demonstrated a sub-meter positioning error, significantly outperforming the comparative method. These advancements are poised to enhance the application of advanced multi-source image matching technologies in UAV visual positioning. [ABSTRACT FROM AUTHOR]

Airborne aircrafts are dependent on the Global Navigation Satellite System (GNSS), which is susceptible to interference due to the satellite base-station and cooperative communication. Synthetic aperture radar altimeter (SARAL) provides the ability to measure the topographic terrain for matching with Digital Elevation Model (DEM) to achieve positioning without relying on GNSS. However, due to the near-vertical coupling in the delay-Doppler map (DDM), the similarity of DDMs of adjacent apertures is high, and the probability of successful matching is low. To this end, a novel neural network of terrain matching and aircraft positioning is proposed based on the airborne SARAL imagery. The model-driven terrain matching and aircraft positioning network (TMP-Net) is capable of realizing aircraft positioning by utilizing the real-time DDMs to match with the DEM-based DDM references, which are generated by a point-by-point coupling mechanism between the airborne routine and ground terrain DEM. Specifically, the training dataset is established by a numerical simulation method based on a semi-analytical model. Therefore, DEM-based DDM references can be generated by forward deduction when only regional DEM can be obtained. In addition to the model-based DDM generation, feature extraction, and similarity measurement, an aircraft positioning module is added. Three different positioning methods are designed to achieve the aircraft positioning, where three-point weighting exhibits the best performance in terms of positioning accuracy. Due to the fact that both the weighted triplet loss and softmax loss are employed in a cooperative manner, the matching accuracy can be improved and the positioning error can be reduced. Finally, both simulated and measured airborne datasets are used to validate the effectiveness of the proposed algorithm. Quantitative and qualitative evaluations show the superiority. [ABSTRACT FROM AUTHOR]

Featured Application: Simultaneous Localization and Mapping of autonomous cleaning and inspection robots operating in the photovoltaic power station scene. Due to various typical unstructured factors in the environment of photovoltaic power stations, such as high feature similarity, weak textures, and simple structures, the motion model of the ORB-SLAM2 algorithm performs poorly, leading to a decline in tracking accuracy. To address this issue, we propose PE-SLAM, which improves the ORB-SLAM2 algorithm's motion model by incorporating the particle swarm optimization algorithm combined with epipolar constraint to eliminate mismatches. First, a new mutation strategy is proposed to introduce perturbations to the pbest (personal best value) during the late convergence stage of the PSO algorithm, thereby preventing the PSO algorithm from falling into local optima. Then, the improved PSO algorithm is used to solve the fundamental matrix between two images based on the feature matching relationships obtained from the motion model. Finally, the epipolar constraint is applied using the computed fundamental matrix to eliminate incorrect matches produced by the motion model, thereby enhancing the tracking accuracy and robustness of the ORB-SLAM2 algorithm in unstructured photovoltaic power station scenarios. In feature matching experiments, compared to the ORB algorithm and the ORB+HAMMING algorithm, the ORB+PE-match algorithm achieved an average accuracy improvement of 19.5%, 14.0%, and 6.0% in unstructured environments, respectively, with better recall rates. In the trajectory experiments of the TUM dataset, PE-SLAM reduced the average absolute trajectory error compared to ORB-SLAM2 by 29.1% and the average relative pose error by 27.0%. In the photovoltaic power station scene mapping experiment, the dense point cloud map constructed has less overlap and is complete, reflecting that PE-SLAM has basically overcome the unstructured factors of the photovoltaic power station scene and is suitable for applications in this scene. [ABSTRACT FROM AUTHOR]

In response to the significant positioning errors that arise in visual localization algorithms for unmanned aerial vehicles (UAVs) when relying on drone image matching in areas devoid of satellite signals, we propose a deep learning-based algorithm named PnP-UGCSuperGlue. This algorithm employs a convolutional neural network (CNN) that is enhanced with a graph encoding module. The resulting enriched features contain vital information that refines the feature map and improves the overall accuracy of the visual localization process. The PnP-UGCSuperGlue framework initiates with the semantic feature extraction from both the real-time drone image and the geo-referenced image. This extraction process is facilitated by a CNN-based feature extractor. In the subsequent phase, a graph encoding module is integrated to aggregate the extracted features. This integration significantly enhances the quality of the generated feature keypoints and descriptors. Following this, a graph matching network is applied to leverage the generated descriptors, thereby facilitating a more precise feature point matching and filtering process. Ultimately, the perspective-n-point (PnP) method is utilized to calculate the rotation matrix and translation vector. This calculation is based on the results of the feature matching phase, as well as the camera intrinsic parameters and distortion coefficients. The proposed algorithm's efficacy is validated through experimental evaluation, which demonstrates a mean absolute error of 0.0005 during the drone's hovering state and 0.0083 during movement. These values indicate a significant reduction of 0.0010 and 0.0028, respectively, compared to the USuperGlue network. [ABSTRACT FROM AUTHOR]

This study addresses the issues of the limited or unavailable unmanned aerial vehicle (UAV) image matching labels, differences in imagery between UAV and ground cameras, and the limited UAV airborne computer memory in UAV visual navigation. To tackle these challenges, a novel unsupervised UAV image matching network called USuperGlue is developed, which leverages local self-attention. The aim is to enhance the precision and recall of the matching process. To achieve this, a local self-attention module is designed to specialize each feature point at the end of the encoding, thereby further improving the matching precision. Subsequently, the encoded features are decoded to extract scores, keypoints, and feature descriptors. During the feature matching stage, an unsupervised matching network is constructed using SuperGlue, a graph neural network that incorporates multi-head attention, to match the feature descriptors of image pairs. The experiment conducted training of the unsupervised network using the Drone dataset and COCO dataset. The simulation results demonstrate that the trained USuperGlue model achieves superior precision and recall, making it a highly effective solution for UAV image matching. [ABSTRACT FROM AUTHOR]

Accurate frost observations are crucial for developing and validating frost prediction models. In 2022, the multi-sensor-based automatic frost observation system (MFOS), including an RGB camera, a thermal infrared camera, a leaf wetness sensor (LWS), LED lighting, and three glass plates, was developed to replace the naked-eye observation of frost. The MFOS, herein installed and operated in an apple orchard, provides temporally high-resolution frost observations that show the onset, end, duration, persistence, and discontinuity of frost more clearly than conventional naked-eye observations. This study introduces recent additions to the MFOS and presents the results of its application to frost weather analysis and forecast evaluation in an orchard in South Korea. The NCAM's Weather Research and Forecasting (WRF) model was employed as a weather forecast model. The main findings of this study are as follows: (1) The newly added image-based object detection capabilities of the MFOS helped with the extraction and quantitative comparison of surface temperature data for apples, leaves, and the LWS. (2) The resolution matching of the RGB and thermal infrared images was made successful by resizing the images, matching them according to horizontal movement, and conducting apple-centered averaging. (3) When applied to evaluate the frost-point predictions of the numerical weather model at one-hour intervals, the results showed that the MFOS could be used as a much more objective tool to verify the accuracy and characteristics of frost predictions compared to the naked-eye view. (4) Higher-resolution and realistic land-cover and vegetation representation are necessary to improve frost forecasts using numerical grid models based on land–atmosphere physics. [ABSTRACT FROM AUTHOR]

Local feature detection and description play a crucial role in various computer vision tasks, including image matching. Variations in illumination conditions significantly affect the accuracy of these applications. However, existing methods inadequately address this issue. In this paper, a novel algorithm based on illumination auxiliary learning module (IALM) is introduced. Firstly, a new local feature extractor named illumination auxiliary Superpoint (IA-Superpoint) is established, based on the integration of IALM and Superpoint. Secondly, illumination-aware auxiliary training focuses on capturing the effects of illumination variations during feature extraction through tailored loss functions and jointly learning mechanisms. Lastly, in order to evaluate the illumination robustness of local features, a metric is proposed by simulating various illumination disturbances. Experiments on HPatches and RDNIM datasets demonstrate that the performance of local feature extraction is greatly improved by our method. Compared to the baseline method, the proposed method exhibits improvements in both mean matching accuracy and homography estimation. [ABSTRACT FROM AUTHOR]

The matching of remote sensing images is a critical and necessary procedure that directly impacts the correctness and accuracy of underwater topography, change detection, digital elevation model (DEM) generation, and object detection. The texture of images becomes weaker with increasing water depth, and this results in matching-extraction failure. To address this issue, a novel method, homography-based motion statistics with an epipolar constraint (HMSEC), is proposed to improve the number, reliability, and robustness of matching points for weak-textured seafloor images. In the matching process of HMSEC, a large number of reliable matching points can be identified from the preliminary matching points based on the motion smoothness assumption and motion statistics. Homography and epipolar geometry are also used to estimate the scale and rotation influences of each matching point in image pairs. The results show that the matching-point numbers for the seafloor and land regions can be significantly improved. In this study, we evaluated this method for the areas of Zhaoshu Island, Ganquan Island, and Lingyang Reef and compared the results to those of the grid-based motion statistics (GMS) method. The increment of matching points reached 2672, 2767, and 1346, respectively. In addition, the seafloor matching points had a wider distribution and reached greater water depths of −11.66, −14.06, and −9.61 m. These results indicate that the proposed method could significantly improve the number and reliability of matching points for seafloor images. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Zhengzhou University (Natural Science Edition) is the property of Journal of Zhengzhou University (Natural Science Edition) Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Local features of an image provide a robust way of image matching if they are invariant to large variations in scale, viewpoint, illumination, rotation, and affine transformations. In this paper, we propose a novel feature descriptor based on circular and elliptical local sampling of image pixels to attain fast and robust results under varying imaging conditions. The proposed descriptor is tested on a standard benchmark dataset comprising of images with varying imaging conditions and compression quality. Results show that the proposed method generates sufficient or more number of stable and correct matches between an image pair (original image and distorted image) as compared to SIFT with a speedup of 1.6 on average basis. The paper also discusses the reason of choosing SIFT descriptor for comparison and its efficacy in different scenarios. The paper also reasons the robustness of hand crafted feature descriptors and why they hold an upper hand among many other deep learning methods. [ABSTRACT FROM AUTHOR]

The present paper aims to conduct an experiment that compares different methods of detecting objects in images. Programs were developed to evaluate the efficiency of SURF, BRISK, MSER, and ORB object detection methods. Four static gray images with sufficiently different histograms were used. The experiment also highlighted the need for image preprocessing to improve feature extraction and detection. Thus, a programmed method for adjusting pixel groups was developed. This method proved useful when one of the listed algorithms failed to detect the object in the original image, but succeeded after adjustment. The effectiveness of detection methods and the evaluation of their performance depend on the application, image preparation, algorithms used, and their implementation. Results of the detection methods were presented numerically (similarities, gradients, distances, etc.) and graphically. [ABSTRACT FROM AUTHOR]

During the process of unmanned aerial vehicle (UAV) flight, images are typically captured along pre-planned routes, resulting in large datasets covering various types of terrain. These images exhibit different scale and rotation differences due to factors such as geographic location and terrain variations. To address the issue of low image matching accuracy and poor photogrammetric results, this paper compared multiple deep learning methods with the traditional SIFT image matching method, investigating the feasibility of deep learning matching methods in railway surveying scenarios. By integrating project data from railway surveys, a comparative analysis was conducted on the number of matching points, average reprojection error, average length of point trajectories, processing time, and the visual effects of matching and sparse reconstruction results between traditional and deep learning matching methods. The research findings indicate that, deep learning-based image matching methods exhibit advantages over traditional methods in terms of a higher number of key points, higher correct matching rates, and higher accuracy in photogrammetric positioning. Therefore, deep learning image matching methods show great potential to replace traditional SIFT matching methods in railway surveying scenarios, potentially overcoming challenges such as complex terrain coverage and scale rotation differences. [ABSTRACT FROM AUTHOR]

With the increasing use of unmanned aerial vehicles (UAVs) in various fields, achieving the precise localization of UAV images is crucial for enhancing their utility. Photogrammetry-based techniques, particularly bundle adjustment, serve as foundational methods for accurately determining the spatial coordinates of UAV images. The effectiveness of bundle adjustment is significantly influenced by the selection of input data, particularly the composition of overlapping image sets. The selection process of overlapping images significantly impacts both the accuracy of spatial coordinate determination and the computational efficiency of UAV image localization. Therefore, a strategic approach to this selection is crucial for optimizing the performance of bundle adjustment in UAV image processing. In this context, we propose an efficient methodology for determining overlapping image sets. The proposed method selects overlapping images based on image similarity, leveraging the complementary strengths of the bag of visual words and number of matches techniques. Essentially, our method achieves both high accuracy and high speed by utilizing a Bag of Visual Words for candidate selection and the number of matches for additional similarity assessment for overlapping image-set determination. We compared the performance of our proposed methodology with the conventional number of matches and bag-of-visual word-based methods for overlapping image-set determination. In the comparative evaluation, the proposed method demonstrated an average precision of 96%, comparable to that of the number of matches-based approach, while surpassing the 62% precision achieved by both bag-of-visual-word methods. Moreover, the processing time decreased by approximately 0.11 times compared with the number of matches-based methods, demonstrating relatively high efficiency. Furthermore, in the bundle adjustment results using image sets, the proposed method, along with the number of matches-based methods, showed reprojection error values of less than 1, indicating relatively high accuracy and contributing to the improvement in accuracy in estimating image positions. [ABSTRACT FROM AUTHOR]

Matching optical and synthetic aperture radar (SAR) images is often challenged by intricate geometric distortion and nonlinear radiation differences, leading to insufficient and unevenly distributed corresponding points. To tackle this issue, we propose a lightweight deep convolutional network with inverted residuals for optical and SAR image matching. Initially, a fully convolutional neural network (FCNN) is designed to extract high-level and semantic features, robustly capturing universal characteristics between optical and SAR images, adept at handling geometric distortion and nonlinear radiation changes. Notably, we integrate a lightweight architecture with inverted residuals into FCNN to adeptly extract local and global contextual information, facilitating feature reuse and minimizing the loss of crucial features. Additionally, a vector-refined module is deployed to refine dense features, filtering out redundant information. Subsequently, a coarse-to-fine strategy is employed to further eliminate gross errors or incorrect matches. Finally, we evaluate the performance of the proposed network in optical and SAR image matching against manually-designed methods and state-of-the-art deep learning techniques. Experimental results demonstrate that our network significantly surpasses existing methods in terms of the number of correct matches and matching accuracy. Specifically, our proposed network achieves at least a 2.8 times increase in correct matches and an 18% improvement in matching accuracy compared to existing methods. [ABSTRACT FROM AUTHOR]

Feature point matching is a fundamental task in computer vision such as vision simultaneous localization and mapping (VSLAM) and structure from motion (SFM). Due to the similarity or interference of features, mismatches are often unavoidable. Therefore, how to eliminate mismatches is important for robust matching. Smoothness constraint is widely used to remove mismatch, but it cannot effectively deal with the issue in the rapidly changing scene. In this paper, a novel LCS-SSM (Local Cell Statistics and Structural Similarity Measurement) mismatch removal method is proposed. LCS-SSM integrates the motion consistency and structural similarity of a local image block as the statistical likelihood of matched key points. Then, the Random Sampling Consensus (RANSAC) algorithm is employed to preserve the isolated matches that do not satisfy the statistical likelihood. Experimental and comparative results on the public dataset show that the proposed LCS-SSM can effectively and reliably differentiate true and false matches compared with state-of-the-art methods, and can be used for robust matching in scenes with fast motion, blurs, and clustered noise. [ABSTRACT FROM AUTHOR]

To investigate the impact of sagging correction calibration errors in radiotherapy software on image matching. Three software applications were used, with and without a polymethyl methacrylate rod supporting the ball bearings (BB). The calibration error for sagging correction across nine flex maps (FMs) was determined by shifting the BB positions along the Left–Right (LR), Gun–Target (GT), and Up–Down (UD) directions from the reference point. Lucy and pelvic phantom cone-beam computed tomography (CBCT) images underwent auto-matching after modifying each FM. Image deformation was assessed in orthogonal CBCT planes, and the correlations among BB shift magnitude, deformation vector value, and differences in auto-matching were analyzed. The average difference in analysis results among the three softwares for the Winston–Lutz test was within 0.1 mm. The determination coefficients (R2) between the BB shift amount and Lucy phantom matching error in each FM were 0.99, 0.99, and 1.00 in the LR-, GT-, and UD-directions, respectively. The pelvis phantom demonstrated no cross-correlation in the GT direction during auto-matching error evaluation using each FM. The correlation coefficient (r) between the BB shift and the deformation vector value was 0.95 on average for all image planes. Slight differences were observed among software in the evaluation of the Winston–Lutz test. The sagging correction calibration error in the radiotherapy imaging system was caused by an auto-matching error of the phantom and deformation of CBCT images. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Remote Sensing is the property of Editorial Office of Journal of Remote Sensing & Science Publishing Co. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Bumblebees (Bombus terrestris) have been shown to engage in string-pulling behavior to access rewards. The objective of this study was to elucidate whether bumblebees display means-end comprehension in a string-pulling task. We presented bumblebees with two options: one where a string was connected to an artificial flower containing a reward and the other presenting an interrupted string. Bumblebees displayed a consistent preference for pulling connected strings over interrupted ones after training with a stepwise pulling technique. When exposed to novel string colors, bees continued to exhibit a bias towards pulling the connected string. This suggests that bumblebees engage in featural generalization of the visual display of the string connected to the flower in this task. If the view of the string connected to the flower was restricted during the training phase, the proportion of bumblebees choosing the connected strings significantly decreased. Similarly, when the bumblebees were confronted with coiled connected strings during the testing phase, they failed to identify and reject the interrupted strings. This finding underscores the significance of visual consistency in enabling the bumblebees to perform the task successfully. Our results suggest that bumblebees’ ability to distinguish between continuous strings and interrupted strings relies on a combination of image matching and associative learning, rather than means-end understanding. These insights contribute to a deeper understanding of the cognitive processes employed by bumblebees when tackling complex spatial tasks.

With the advancement of technology in recent years, the application of artificial intelligence in real life has become more extensive. Graphic recognition is a hot spot in the current research of related technologies. It involves machines extracting key information from pictures and combining it with natural language processing for in-depth understanding. Existing methods still have obvious deficiencies in fine-grained recognition and deep understanding of contextual context. Addressing these issues to achieve high-quality image-text recognition is crucial for various application scenarios, such as accessibility technologies, content creation, and virtual assistants. To tackle this challenge, a novel approach is proposed that combines the Mask R-CNN, DCGAN, and ALBERT models. Specifically, the Mask R-CNN specializes in high-precision image recognition and segmentation, the DCGAN captures and generates nuanced features from images, and the ALBERT model is responsible for deep natural language processing and semantic understanding of this visual information. Experimental results clearly validate the superiority of this method. Compared to traditional image-text recognition techniques, the recognition accuracy is improved from 85.3% to 92.5%, and performance in contextual and situational understanding is enhanced. The advancement of this technology has far-reaching implications for research in machine vision and natural language processing and open new possibilities for practical applications.