Search

The recent advance in Neural Radiance Fields (NeRF), which utilizes Multilayer Perceptrons (MLP) for implicit scene representation, enables the synthesis of realistic views from new perspectives. However, degradation in the quantity and quality of input images can lead to failure in scene reconstruction and difficulty in synthesizing high-quality views. To address these limitations, this paper presents a NeRF-based framework (MBS-NeRF), which can reconstruct sharp NeRF from a limited number of motion-blurred input images for high-quality view synthesis. The framework integrates depth information as a constraint to counter the lack of sufficient view information and introduces a Motion Blur Simulation Module (MBSM) to simulate the physical formation process of motion blur. We further introduce camera trajectory optimization during the exposure process to robust the incorrect camera position. MBS-NeRF is thoroughly trained considering photometric consistency and depth supervision. Comprehensive experiments on synthetic and real datasets validate the effectiveness of the model in reconstructing sharp NeRF and achieving high-quality view synthesis from sparse, motion-blurred inputs. [ABSTRACT FROM AUTHOR]

Simultaneous Localization and Mapping (SLAM) technology has garnered significant interest in the robotic vision community over the past few decades. The rapid development of SLAM technology has resulted in its widespread application across various fields, including autonomous driving, robot navigation, and virtual reality. Although SLAM, especially Visual–Inertial SLAM (VI-SLAM), has made substantial progress, most classic algorithms in this field are designed based on the assumption that the observed scene is static. In complex real-world environments, the presence of dynamic objects such as pedestrians and vehicles can seriously affect the robustness and accuracy of such systems. Event cameras, which use recently introduced motion-sensitive biomimetic sensors, efficiently capture scene changes (referred to as "events") with high temporal resolution, offering new opportunities to enhance VI-SLAM performance in dynamic environments. Integrating this kind of innovative sensor, we propose the first event-enhanced Visual–Inertial SLAM framework specifically designed for dynamic environments, termed E 2 -VINS. Specifically, the system uses visual–inertial alignment strategy to estimate IMU biases and correct IMU measurements. The calibrated IMU measurements are used to assist in motion compensation, achieving spatiotemporal alignment of events. The event-based dynamicity metrics, which measure the dynamicity of each pixel, are then generated on these aligned events. Based on these metrics, the visual residual terms of different pixels are adaptively assigned weights, namely, dynamicity weights. Subsequently, E 2 -VINS jointly and alternately optimizes the system state (camera poses and map points) and dynamicity weights, effectively filtering out dynamic features through a soft-threshold mechanism. Our scheme enhances the robustness of classic VI-SLAM against dynamic features, which significantly enhances VI-SLAM performance in dynamic environments, resulting in an average improvement of 1.884% in the mean position error compared to state-of-the-art methods. The superior performance of E 2 -VINS is validated through both qualitative and quantitative experimental results. To ensure that our results are fully reproducible, all the relevant data and codes have been released. [ABSTRACT FROM AUTHOR]

This paper introduces an efficient localization algorithm for robotic systems, utilizing deep learning to identify and exploit natural fiduciary patterns within the environment. Diverging from conventional localization techniques that depend on artificial markers, this method capitalizes on the inherent environmental features to enhance both accuracy and computational efficiency. By integrating advanced deep learning frameworks with natural scene analysis, the proposed algorithm facilitates robust, real-time localization in dynamic and unstructured settings. The resulting approach offers significant improvements in adaptability, precision, and operational efficiency, representing a substantial contribution to the field of autonomous robotics. We are aiming at analyzing an automotive manufacturing scenario to achieve robotic localization related to a moving target. To work with a simpler and more accessible scenario we have chosen a demonstrative context consisting of a laboratory wall containing some elements. This paper will focus on the first part of the case study, with a continuation planned for future work. It will demonstrate a scenario in which a camera is mounted on a robot, capturing images of the underside of a car (which we assume to be represented by a gray painted surface with specific elements to be described in Materials and Methods). These images are processed by a convolutional neural network (CNN), designed to detect the most distinctive features of the environment. The extracted information is crucial, as the identified characteristic areas will serve as reference points for the real-time localization of the industrial robot. In this work, we have demonstrated the potential of leveraging natural fiduciary patterns for efficient and accurate robot localization. By utilizing deep learning, specifically convolutional neural networks. The experimental results suggest that this approach is not only feasible but also scalable across a wide range of applications, including industrial automation autonomous vehicles, and aerospace navigation. As robots increasingly operate in environments where computational efficiency and adaptability are paramount, our methodology offers a viable solution to enhance localization without compromising accuracy or speed. The proposal of an algorithm that enables the application of the proposed method for natural fiduciary patterns based on neural networks to more complex scenarios is highlighted, along with the efficiency of the method for robot localization compared to others. [ABSTRACT FROM AUTHOR]

Event-based cameras are a new type of vision sensor in which pixels operate independently and respond asynchronously to changes in brightness with microsecond resolution, instead of providing standard intensity frames. Compared with traditional cameras, event-based cameras have low latency, no motion blur, and high dynamic range (HDR), which provide possibilities for robots to deal with some challenging scenes. We propose a visual–inertial odometry for stereo event-based cameras based on Error-State Kalman Filter (ESKF). The vision module updates the pose by relying on the edge alignment of a semi-dense 3D map to a 2D image, while the IMU module updates the pose using median integration. We evaluate our method on public datasets with general 6-DoF motion (three-axis translation and three-axis rotation) and compare the results against the ground truth. We compared our results with those from other methods, demonstrating the effectiveness of our approach. [ABSTRACT FROM AUTHOR]

The maturity of simultaneous localization and mapping (SLAM) methods has now reached a significant level that motivates in-depth and problem-specific reviews. The focus of this study is to investigate the evolution of vision-based, LiDAR-based, and a combination of these methods and evaluate their performance in enclosed and GPS-denied (EGD) conditions for infrastructure inspection. This paper categorizes and analyzes the SLAM methods in detail, considering the sensor fusion type and chronological order. The paper analyzes the performance of eleven open-source SLAM solutions, containing two visual (VINS-Mono, ORB-SLAM 2), eight LiDAR-based (LIO-SAM, Fast-LIO 2, SC-Fast-LIO 2, LeGO-LOAM, SC-LeGO-LOAM A-LOAM, LINS, F-LOAM) and one combination of the LiDAR and vision-based method (LVI-SAM). The benchmarking section analyzes accuracy and computational resource consumption using our collected dataset and a test dataset. According to the results, LiDAR-based methods performed well under EGD conditions. Contrary to common presumptions, some vision-based methods demonstrate acceptable performance in EGD environments. Additionally, combining vision-based techniques with LiDAR-based methods demonstrates superior performance compared to either vision-based or LiDAR-based methods individually. [ABSTRACT FROM AUTHOR]

To improve the accuracy of camera pose estimation for RGBD-based 3D reconstruction in dynamic scenes, a method based on instance segmentation and optical flow is proposed. Firstly, instance segmentation is used to detect objects, and a semantic map is constructed by removing non-rigid objects. Secondly, motion residual is calculated by optical flow and camera flow to detect dynamic rigid objects, and nonlinear optimization is used to track the dynamic rigid objects extracted from the semantic map. Thirdly, after removing features of non-rigid objects and dynamic rigid ones in each frame, the remaining features are used to optimize the camera pose. Finally, a TSDF model is used to reconstruct the static background, and point clouds are used to reconstruct dynamic rigid objects. Experiments on TUM and Bonn datasets show that the method produces better camera poses than the current state-of-the-art methods in most dynamic scenes. Ablation experiments on Bonn dataset show that retaining features of static rigid objects significantly improves camera pose estimation precision. The annotated datasets and the source code are available at https://github.com/CodingMaplee/IOFusion/tree/main. [ABSTRACT FROM AUTHOR]

Displacement is a critical indicator that reflects the degeneration of overall stiffness for bridges. However, for most bridges, contact displacement measurement is difficult to achieve due to the lack of reference points and technical limitations. In this paper, a unique dual-camera visual sensor system for the whole field displacement measurement of bridge undersurfaces was developed. The visual sensor system relies on feature point algorithm and several algorithms are involved to improve the accuracy and efficiency. The dual-camera equipment can independently eliminate the impact of the camera vibration, thereby solving the problem of camera vibration caused by ground vibrations and wind loads at outdoor inspection sites when measuring bridge displacement. In a laboratory testing environment, a steel plate simulated a bridge and different experiments were conducted to test the visual system in situations with and without vibration disturbances to the camera. The results verify the accuracy of the developed visual sensor, especially the effectiveness of the vibration reduction method. [ABSTRACT FROM AUTHOR]

Scarce feature points are a critical limitation affecting the accuracy and stability of incremental structure from motion (SfM) in small-scale scenes. In this paper, we propose an incremental SfM method for small-scale scenes, combined with an auxiliary calibration plate. This approach increases the number of feature points in sparse regions, and we randomly generate feature points within those areas. At the same time, we obtain a coarse matching set of feature points using pairwise polar geometric constraints. The positional results from the geometric constraints of the calibration plate are then used to filter out high-precision matching points, thereby improving the accuracy of the three-dimensional reconstruction. Experimental results demonstrate that the proposed method achieves superior reconstruction completeness and accuracy. In three real-world experiments, the average re-projection errors were 0.5245, 0.4151, and 0.4996 pixels, outperforming competing methods. This approach ensures robust pose estimation and facilitates precise 3D reconstructions. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

In this paper, an exhaustive review and comprehensive analysis of recent and former deep learning methods in 3D semantic segmentation (3DSS) is presented. In the related literature, the taxonomy scheme used for the classification of 3DSS deep learning methods is ambiguous. Based on the taxonomy schemes of nine existing review papers, a new taxonomy scheme for 3DSS deep learning methods is proposed, aiming to standardize it and improve the comparability and clarity across related studies. Furthermore, an extensive overview of the available 3DSS indoor and outdoor datasets is provided along with their links. The core part of this review is the detailed presentation of recent and former 3DSS deep learning methods and their classification using the proposed taxonomy scheme along with their GitHub repositories. Additionally, a brief but informative analysis of the evaluation metrics and loss functions used in 3DSS is included. Finally, a fruitful discussion of the examined 3DSS methods and datasets is presented to foster new research directions and applications in the field of 3DSS. In addition to this review, a GitHub repository is provided, including an initial classification of over 400 3DSS methods, using the proposed taxonomy scheme. [ABSTRACT FROM AUTHOR]

Feature correspondences identification between consecutive frames is a critical prerequisite in the monocular Visual-Inertial Navigation System (VINS). In this paper, we propose a novel self-adaptation feature point correspondences identification algorithm in terms of IMU-aided information fusion at the level of feature tracking for nonlinear optimization framework-based VINS. This method starts with an IMU pre-integration predictor to predict the pose of each new coming frame. In weak texture scenes and motion blur situations, in order to increase the number of feature correspondences and improve the track lengths of feature points, we introduce a novel predicting-matching based feature point tracking strategy to build new matches. On the other hand, the predicted pose is incorporated into the outliers rejection step to deal with mismatch caused by dynamic objects. Finally, the proposed self-adaptation feature correspondences identification algorithm is implemented based on VINS-Fusion and validated through public datasets. The experimental results show that it effectively improves the accuracy and tracking length of feature matching, and demonstrates better performance in terms of camera pose estimation as compared to state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Existing end-to-end networks for steering angle prediction usually use images generated by standard cameras as input. However, standard cameras are susceptible to poor lighting conditions and motion blur, which is not conducive to training an accurate and robust end-to-end network. In contrast, biological vision-inspired event cameras overcome the aforementioned shortcomings with their unique working principle and offer significant advantages such as high temporal resolution, high dynamic range and low power consumption. Nevertheless, event cameras generate a lot of noise and are unable to provide texture information on static region. Therefore, these two types of cameras are complementary to each other to some extent. To explore the benefits of fusing information from these two types of cameras in autonomous driving tasks, we propose GFENet, an attention-based two-stream encoder-decoder architecture for steering angle prediction by combining events and images. Firstly, asynchronous and sparse events are converted into synchronous and dense event frames. Then, event frames and corresponding image frames are fed into two symmetric encoders to extract features. Next, We introduce a Group-Wise Feature-Enhanced (GEF) module that can refine features and suppress noise to guide the fusion of two modalities features at different levels. Finally, The final fused features are passed through a simple decoder to predict the steering angle. Experiments results on the DDD20 and EventScape datasets shows that our GFEFNet outperforms the state-of-the-art image-event fusion method. [ABSTRACT FROM AUTHOR]

One of the predominant technologies for multi-source navigation in vehicles involves the fusion of GNSS/IMU/ODO through a factor graph. To address issues such as the asynchronous sampling frequencies between the IMU and ODO, as well as diminished accuracy during GNSS signal loss, we propose a GNSS/IMU/ODO integrated navigation method based on an adaptive sliding window factor graph. The measurements from the ODO are utilized as observation factors to mitigate prediction interpolation errors associated with traditional ODO pre-integration methods. Additionally, online estimation and compensation for both installation angle deviations and scale factors of the ODO further enhance its ability to constrain pose errors during GNSS signal loss. A multi-state marginalization algorithm is proposed and then utilized to adaptively adjust the sliding window size based on the quality of GNSS observations, enhancing pose optimization accuracy in multi-source fusion while prioritizing computational efficiency. Tests conducted in typical urban environments and mountainous regions demonstrate that our proposed method significantly enhances fusion navigation accuracy under complex GNSS conditions. In a complex city environment, our method achieves a 55.3% and 29.8% improvement in position and velocity accuracy and enhancements of 32.0% and 61.6% in pitch and heading angle accuracy, respectively. These results match the precision of long sliding windows, with a 75.8% gain in computational efficiency. In mountainous regions, our method enhances the position accuracy in the three dimensions by factors of 89.5%, 83.7%, and 43.4%, the velocity accuracy in the three dimensions by factors of 65.4%, 32.6%, and 53.1%, and reduces the attitude errors in roll, pitch, and yaw by 70.5%, 60.8%, and 26.0%, respectively, demonstrating strong engineering applicability through an optimal balance of precision and efficiency. [ABSTRACT FROM AUTHOR]

Indoor navigation is becoming increasingly essential for multiple applications. It is complex and challenging due to dynamic scenes, limited space, and, more importantly, the unavailability of global navigation satellite system (GNSS) signals. Recently, new sensors have emerged, namely event cameras, which show great potential for indoor navigation due to their high dynamic range and low latency. In this study, an event-based visual–inertial odometry approach is proposed, emphasizing adaptive event accumulation and selective keyframe updates to reduce computational overhead. The proposed approach fuses events, standard frames, and inertial measurements for precise indoor navigation. Features are detected and tracked on the standard images. The events are accumulated into frames and used to track the features between the standard frames. Subsequently, the IMU measurements and the feature tracks are fused to continuously estimate the sensor states. The proposed approach is evaluated using both simulated and real-world datasets. Compared with the state-of-the-art U-SLAM algorithm, our approach achieves a substantial reduction in the mean positional error and RMSE in simulated environments, showing up to 50% and 47% reductions along the x- and y-axes, respectively. The approach achieves 5–10 ms latency per event batch and 10–20 ms for frame updates, demonstrating real-time performance on resource-constrained platforms. These results underscore the potential of our approach as a robust solution for real-world UAV indoor navigation scenarios. [ABSTRACT FROM AUTHOR]

Copyright of Application Research of Computers / Jisuanji Yingyong Yanjiu is the property of Application Research of Computers Edition and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Forest mapping provides critical observational data needed to understand the dynamics of forest environments. Notably, tree diameter at breast height (DBH) is a metric used to estimate forest biomass and carbon dioxide ( CO 2 ) sequestration. Manual methods of forest mapping are labor intensive and time consuming, a bottleneck for large-scale mapping efforts. Automated mapping relies on acquiring dense forest reconstructions, typically in the form of point clouds. Terrestrial laser scanning (TLS) and mobile laser scanning (MLS) generate point clouds using expensive LiDAR sensing and have been used successfully to estimate tree diameter. Neural radiance fields (NeRFs) are an emergent technology enabling photorealistic, vision-based reconstruction by training a neural network on a sparse set of input views. In this paper, we present a comparison of MLS and NeRF forest reconstructions for the purpose of trunk diameter estimation in a mixed-evergreen Redwood forest. In addition, we propose an improved DBH-estimation method using convex-hull modeling. Using this approach, we achieved 1.68 cm RMSE (2.81%), which consistently outperformed standard cylinder modeling approaches. [ABSTRACT FROM AUTHOR]

To enhance the intelligence of robots, constructing accurate object-level instance maps is essential. However, the diversity and clutter of objects in indoor scenes present significant challenges for instance map construction. To tackle this issue, we propose a method for constructing object-level instance maps based on RGBD images. First, we utilize the advanced visual odometer ORB-SLAM3 to estimate the poses of image frames and extract keyframes. Next, we perform semantic and geometric segmentation on the color and depth images of these keyframes, respectively, using semantic segmentation to optimize the geometric segmentation results and address inaccuracies in the target segmentation caused by small depth variations. The segmented depth images are then projected into point cloud segments, which are assigned corresponding semantic information. We integrate these point cloud segments into a global voxel map, updating each voxel's class using color, distance constraints, and Bayesian methods to create an object-level instance map. Finally, we construct an ellipsoids scene from this map to test the robot's localization capabilities in indoor environments using semantic information. Our experiments demonstrate that this method accurately and robustly constructs the environment, facilitating precise object-level scene segmentation. Furthermore, compared to manually labeled ellipsoidal maps, generating ellipsoidal maps from extracted objects enables accurate global localization. [ABSTRACT FROM AUTHOR]

Semantic segmentation tasks encompass various applications, such as autonomous driving, medical imaging, and robotics. Achieving accurate semantic information retrieval under conditions of high dynamic range and rapid scene changes remains a significant challenge for image-based algorithms. This challenge is primarily attributable to the limitations of conventional image sensors, which can experience motion blur or exposure artifacts. In contrast, event-based vision sensors, which asynchronously report changes in pixel intensity, offer a compelling solution by acquiring visual information at the same rate as the scene dynamics, thereby mitigating these limitations. However, we encounter a significant challenge in event-based semantic segmentation tasks: the need to expend time on converting event data into frame images to align with existing image-based semantic segmentation techniques. This approach squanders the inherently high temporal resolution of event data, compromising the accuracy and real-time performance of semantic segmentation tasks. To address these issues, this work explores a sparse semantic segmentation approach that directly addresses event data. We propose a network named EventSegNet that improves the ability to extract geometric features from event data by combining geometric feature enhancement operations and attention mechanisms. Based on this, we propose a large-scale event-based semantic segmentation dataset that provides labels for each event. Our approach achieved a new F1 score of 84.2% on the dataset. In addition, a lightweight and edge-oriented AI inference deployment technique was implemented for the network model. Compared to the baseline model, the optimized network model reduces the F1 score by 1.1% but is more than twice as fast computationally, enabling real-time inference on the NVIDIA AGX Xavier. [ABSTRACT FROM AUTHOR]

Recently, several initiatives regarding maritime autonomous surface ships (MASSs) have been implemented worldwide. One of the fundamental technologies for attaining MASSs is the recognition and localization of surrounding ships. Traditional navigational instruments are inadequate for recognizing objects, and the authors investigated the potential of stereo vision. Conventional stereo camera systems are not suitable for localizing very distant objects. One proposed solution is to use an additional camera, thus using three-camera measurements of objects at long distances to reduce positional measurement errors, incorporating time-series averaging and keypoint-based techniques. This study evaluated experimentally the accuracy of measurements using three ship-mounted cameras. The accuracy and precision of stereo measurements depend on the distance between the camera positions, referred to as the baseline length. Conventional stereo cameras are typically used to measure objects at distances of up to 200 times the baseline length. This study indicates that, using trinocular stereo vision, a target ship at distances up to 2500 m, which is 500 times the baseline length, can be measured with an accuracy of approximately 5% of the RMSE. [ABSTRACT FROM AUTHOR]

Purpose: Wheeled robots have been widely used in People's Daily life. Accurate positioning is the premise of autonomous navigation. In this paper, an optimization-based visual-inertial-wheel odometer tightly coupled system is proposed, which solves the problem of failure of visual inertia initialization due to unobservable scale.The aim of this paper is to achieve robust localization of visually challenging scenes. Design/methodology/approach: During system initialization, the wheel odometer measurement and visual-inertial odometry (VIO) fusion are initialized using maximum a posteriori (MAP). Aiming at the visual challenge scene, a fusion method of wheel odometer and inertial measurement unit (IMU) measurement is proposed, which can still be robust initialization in the scene without visual features. To solve the problem of low track accuracy caused by cumulative errors of VIO, the local and global positioning accuracy is improved by integrating wheel odometer data. The system is validated on a public data set. Findings: The results show that our system performs well in visual challenge scenarios, can achieve robust initialization with high efficiency and improves the state estimation accuracy of wheeled robots. Originality/value: To realize robust initialization of wheeled robot, wheel odometer measurement and vision-inertia fusion are initialized using MAP. Aiming at the visual challenge scene, a fusion method of wheel odometer and IMU measurement is proposed. To improve the accuracy of state estimation of wheeled robot, wheel encoder measurement and plane constraint information are added to local and global BA, so as to achieve refined scale estimation. [ABSTRACT FROM AUTHOR]

Microscopic and nanoscopic motors, often referred to as micro-/nanomotors, are autonomous devices capable of converting chemical energy from their surroundings into mechanical motion or forces necessary for propulsion. These devices draw inspiration from natural biomolecular motor proteins, and in recent years, synthetic micro-/nanomotors have attracted significant attention. Among these, catalytic micro-/nanomotors have emerged as a prominent area of research. Despite considerable progress in their design and functionality, several obstacles remain, especially regarding the development of biocompatible materials and fuels, the integration of intelligent control systems, and the translation of these motors into practical applications. Thus, a comprehensive understanding of the current advancements in catalytic micro-/nanomotors is critical. This review aims to provide an in-depth overview of their fabrication techniques, propulsion mechanisms, key influencing factors, control methodologies, and potential applications. Furthermore, we examine their physical and hydrodynamic properties in fluidic environments to optimize propulsion efficiency. Lastly, we evaluate their biosafety and biocompatibility to facilitate their use in biological systems. The review also addresses key challenges and proposes potential solutions to advance their practical deployment. [ABSTRACT FROM AUTHOR]

MINFLUX nanoscopy relies on the localization of single fluorophores with expected ~ 2 nm precision in 3D mapping, roughly one order of magnitude better than standard stimulated emission depletion microscopy or stochastic optical reconstruction microscopy. This "brilliant" technique takes advantage of specialized localization principles and algorithms that require only dim fluorescence signals with a minimum flux of photons; hence the name follows. With this level of performance, MINFLUX imaging and tracking should allow for the routine study of biological processes down to the molecular scale, revealing previously unresolved details in cell structures, such as the organization of calcium channels in muscle cells or the clustering of receptors in synapses. Whereas the high localization precision is definitely a strength of the MINFLUX technique, limitations and challenges also exist, especially in the labeling procedures aiming at appropriate density and on/off switching kinetics. This primer presents some significant results achieved with MINFLUX so far and highlights specific operational procedures crucial for this technique., (© 2024 The Author(s). Microscopy Research and Technique published by Wiley Periodicals LLC.)

The integration of Machine Learning (ML) with super-resolution microscopy represents a transformative advancement in biomedical research. Recent advances in ML, particularly deep learning (DL), have significantly enhanced image processing tasks, such as denoising and reconstruction. This review explores the growing potential of automation in super-resolution microscopy, focusing on how DL can enable autonomous imaging tasks. Overcoming the challenges of automation, particularly in adapting to dynamic biological processes and minimizing manual intervention, is crucial for the future of microscopy. Whilst still in its infancy, automation in super-resolution can revolutionize drug discovery and disease phenotyping leading to similar breakthroughs as have been recognized in this year's Nobel Prizes for Physics and Chemistry., (© 2025 The Author(s). Small Methods published by Wiley‐VCH GmbH.)