Search

Localization and perception play an important role as the basis of autonomous Unmanned Aerial Vehicle (UAV) applications, providing the internal state of movements and the external understanding of environments. Simultaneous Localization And Mapping (SLAM), one of the critical techniques for localization and perception, is facing technical upgrading, due to the development of embedded hardware, multi-sensor technology, and artificial intelligence. This survey aims at the development of visual SLAM and the basis of UAV applications. The solutions to critical problems for visual SLAM are shown by reviewing state-of-the-art and newly presented algorithms, providing the research progression and direction in three essential aspects: real-time performance, texture-less environments, and dynamic environments. Visual–inertial fusion and learning-based enhancement are discussed for UAV localization and perception to illustrate their role in UAV applications. Subsequently, the trend of UAV localization and perception is shown. The algorithm components, camera configuration, and data processing methods are also introduced to give comprehensive preliminaries. In this paper, we provide coverage of visual SLAM and its related technologies over the past decade, with a specific focus on their applications in autonomous UAV applications. We summarize the current research, reveal potential problems, and outline future trends from academic and engineering perspectives. [ABSTRACT FROM AUTHOR]

Photometric calibration is essential to many computer vision applications. One of its key benefits is enhancing the performance of Visual SLAM, especially when it depends on a direct method for tracking, such as the standard KLT algorithm. Additionally, it proves valuable in extracting sensor irradiance values from measured intensities, serving as a pre-processing step for a number of vision algorithms such as shape-from-shading. Current photometric calibration systems rely on a joint optimization problem and encounter an ambiguity in the estimates, which can only be resolved using ground truth information. We propose a novel method that solves for photometric parameters using a sequential estimation approach. To enhance the decoupling of CRF and Vignette estimation, we strategically utilize keyframes with high exposure ratios and small displacements for the former, and keyframes with relatively large displacements for the latter. Our proposed method achieves high accuracy in estimating all parameters; furthermore, the formulations are linear and convex, which makes the solution fast and suitable for online applications. Experiments on a Visual Odometry system validate the proposed method and demonstrate its advantages. • Overcoming the exponential ambiguity found in previous approaches. • Eliminating the need for CRF ground truth data point. • Yielding a quick and computationally efficient estimation of the parameters. • Accuracies that are at par with (or better than) state-of-the-art in estimating photometric parameters. [ABSTRACT FROM AUTHOR]

Tracking failure in state-of-the-art visual SLAM has been reported to be frequent and hampers real-world deployment of SLAM solutions. Very recently, efforts have been made to avoid tracking failure using various methods (e.g. using deep reinforcement learning to plan a path where the risk of tracking failure is minimal). The results of those approaches are encouraging but are far from producing a failure-free visual system. Failure is inevitable in vision-based systems. Therefore, developing recovery (post-failure) solutions might be a better approach in developing more reliable systems. To this end, we propose a novel, easily trainable, deep recovery maneuver algorithm. Instead of predicting tracking failures, our algorithm predicts the back stepping move, post failure, which helps regain tracking. With our proposed simple approach, state-of-the-art SLAM manages to complete 13 out of 50 navigation episodes (13x improvement over the baseline). Furthermore, even for cases where our algorithm fails to complete the route, it traverses 2x longer distances as compared to baseline methods. [ABSTRACT FROM AUTHOR]

In the autonomous navigation of mobile robots, precise positioning is crucial. In forest environments with weak satellite signals or in sites disturbed by complex environments, satellite positioning accuracy has difficulty in meeting the requirements of autonomous navigation positioning accuracy for robots. This article proposes a vision SLAM/UWB tightly coupled localization method and designs a UWB non-line-of-sight error identification method using the displacement increment of the visual odometer. It utilizes the displacement increment of visual output and UWB ranging information as measurement values and applies the extended Kalman filtering algorithm for data fusion. This study utilized the constructed experimental platform to collect images and ultra-wideband ranging data in outdoor environments and experimentally validated the combined positioning method. The experimental results show that the algorithm outperforms individual UWB or loosely coupled combination positioning methods in terms of positioning accuracy. It effectively eliminates non-line-of-sight errors in UWB, improving the accuracy and stability of the combined positioning system. [ABSTRACT FROM AUTHOR]

In challenging environments, feature-based visual SLAM encounters frequent failures in frame tracking, introducing unknown poses to robotic applications. This paper introduces an immediate approach for recovering untracked camera poses. Through the retrieval of key information from elapsed untracked frames, lost poses are efficiently restored with a short time consumption. Taking account of reconstructed poses and map points during local optimizing, a denser local map is constructed around ambiguous frames to enhance the further SLAM procedure. The proposed method is implemented in a SLAM system, and monocular experiments are conducted on datasets. The experimental results demonstrate that our method can reconstruct the untracked frames in nearly real time, effectively complementing missing segments of the trajectory. Concurrently, the accuracy and robustness for subsequent tracking are improved through the integration of recovered poses and map points. [ABSTRACT FROM AUTHOR]

This work presents a novel RGB-D dynamic Simultaneous Localisation and Mapping (SLAM) method that improves the precision, stability, and efficiency of localisation while relying on lightweight deep learning in a dynamic environment compared to the traditional static feature-based visual SLAM algorithm. Based on ORB-SLAM3, the GCNv2-tiny network instead of the ORB method, improves the reliability of feature extraction and matching and the accuracy of position estimation; then, the semantic segmentation thread employs the lightweight YOLOv5s object detection algorithm based on the GSConv network combined with a depth image to determine potentially dynamic regions of the image. Finally, to guarantee that the static feature points are used for position estimation, dynamic probability is employed to determine the true dynamic feature points based on the optical flow, semantic labels, and the state in last frame. We have performed experiments on the TUM datasets to verify the feasibility of the algorithm. Compared with the classical dynamic visual SLAM algorithm, the experimental results demonstrate that the absolute trajectory error is greatly reduced in dynamic environments, and that the computing efficiency is improved by 31.54% compared with the real-time dynamic visual SLAM algorithm with close accuracy, demonstrating the superiority of DLD-SLAM in accuracy, stability, and efficiency. [ABSTRACT FROM AUTHOR]

Intelligent vehicles are constrained by road, resulting in a disparity between the assumed six degrees of freedom (DoF) motion within the Visual Simultaneous Localization and Mapping (SLAM) system and the approximate planar motion of vehicles in local areas, inevitably causing additional pose estimation errors. To address this problem, a stereo Visual SLAM system with road constraints based on graph optimization is proposed, called RC-SLAM. Addressing the challenge of representing roads parametrically, a novel method is proposed to approximate local roads as discrete planes and extract parameters of local road planes (LRPs) using homography. Unlike conventional methods, constraints between the vehicle and LRPs are established, effectively mitigating errors arising from assumed six DoF motion in the system. Furthermore, to avoid the impact of depth uncertainty in road features, epipolar constraints are employed to estimate rotation by minimizing the distance between road feature points and epipolar lines, robust rotation estimation is achieved despite depth uncertainties. Notably, a distinctive nonlinear optimization model based on graph optimization is presented, jointly optimizing the poses of vehicle trajectories, LPRs, and map points. The experiments on two datasets demonstrate that the proposed system achieved more accurate estimations of vehicle trajectories by introducing constraints between the vehicle and LRPs. The experiments on a real-world dataset further validate the effectiveness of the proposed system. [ABSTRACT FROM AUTHOR]

At work, three tasks were presented: road lane detection and trajectory estimation, environment mapping, and the application of a neural network. All these tasks are based on the results of the lane detection method. The presented lane detection method stands out due to the execution of an interpolation transformation for all previously detected edge points. This transformation transfers these points to a "bird's-eye" coordinate system and distributes them on a grid. Road lanes are identified by a lane feature filter based on the analysis of the distances between unique points. This allows lane views to be obtained in a coordinate system while preserving the distance condition. The road environment map is constructed from the obtained images using a probabilistic algorithm called Distributed Particle-SLAM (DP-SLAM). Based on the map result, a method for representing characteristic points describing the path of road lanes in each incoming camera image has been developed. These points are then used for training the neural network. The neural network solves a regression task for the coordinates of the points on the road lanes, enabling the identification of coefficients for parabolic fitting. Validation has been performed. [ABSTRACT FROM AUTHOR]

Traditional Simultaneous Localization and Mapping application in dynamic situations is constrained by static assumptions. However, the majority of well-known dynamic SLAM systems use deep learning to identify dynamic objects, which creates the issue of trade-offs between accuracy and real-time. To tackle this issue, this work suggests a unique dynamic semantics method(DYS-SLAM) for semantic simultaneous localization and mapping that strikes a trade-off between high accuracy and high real-time performance. We propose M-LK, an enhanced Lucas-Kanade(LK) optical flow method. This technique keeps the continuous motion and greyscale consistency assumptions from the original method while switching out the spatial consistency assumption for a motion consistency assumption, reducing sensitivity to image gradients to identify dynamic feature points and regions efficiently. In order to increase segmentation accuracy while maintaining real-time performance, we develop a segmentation refinement scheme that projects 3D point cloud segmentation results into 2D object detection zones. A dense semantic octree graph is built in the interim to expedite the high-level process. Compared to the four equivalent dynamic SLAM approaches, experiments on the publicly available TUM RGB-D dataset demonstrate that the DYS-SLAM method offers competitive localization accuracy and satisfactory real-time performance in both high and low-dynamic scenarios. [ABSTRACT FROM AUTHOR]

A semantic SLAM algorithm based on dynamic object tracking is proposed to address the issue of decreased localization accuracy in traditional visual SLAM methods due to feature matching errors in dynamic scenes. The algorithm, built upon the classic visual SLAM framework, extracted dynamic objects for inter-frame tracking and utilized their pose information to assist the camera’s own localization. Firstly, YOLACT, RAFT, and SC-Depth networks were employed in the data preprocessing stage to extract semantic masks, optical flow vectors, and pixel depths from the images. Subsequently, the visual front-end module utilized the extracted information to compute probability maps,employing semantic segmentation masks, motion consistency checks, and occlusion point verification algorithms.These probability maps aided in effectively distinguishing between dynamic and static features in the scene. Then, the bundle adjustment module in the back-end integrated multiple feature constraints derived from object motion to enhance the algorithm’s pose estimation performance in dynamic scenes. Finally, comprehensive comparisons and validations were conducted on the dynamic scenes of the KITTI and OMD datasets. The experimental results demonstrate that the proposed algorithm accurately tracks dynamic objects and exhibits robust and accurate localization performance in both indoor and outdoor dynamic scenes. [ABSTRACT FROM AUTHOR]

Visual simultaneous localization and mapping is a widely used technology for mobile robots to carry out precise positioning in the environment of GNSS technology failure. However, as the robot moves around indoors, its position accuracy will gradually decrease over time due to common and unavoidable environmental factors. In this paper, we propose an improved method called RTABMAP-VIWO, which is based on RTABMAP. The basic idea is to use an Extended Kalman Filter (EKF) framework for fusion attitude estimates from the wheel odometry and IMU, and provide new prediction values. This helps to reduce the local cumulative error of RTABMAP and make it more accurate. We compare and evaluate three kinds of SLAM methods using both public datasets and real indoor scenes. In the dataset experiments, our proposed method reduces the Root-Mean-Square Error (RMSE) coefficient by 48.1% compared to the RTABMAP, and the coefficient is also reduced by at least 29.4% in the real environment experiments. The results demonstrate that the improved method is feasible. By incorporating the IMU into the RTABMAP method, the trajectory and posture errors of the mobile robot are significantly improved. [ABSTRACT FROM AUTHOR]

Simultaneous localization and mapping (SLAM) plays a crucial role in the field of intelligent mobile robots. However, the traditional Visual SLAM (VSLAM) framework is based on strong assumptions about static environments, which are not applicable to dynamic real-world environments. The correctness of re-localization and recall of loop closure detection are both lower when the mobile robot loses frames in a dynamic environment. Thus, in this paper, the re-localization and loop closure detection method with a semantic topology graph based on ORB-SLAM2 is proposed. First, we use YOLOv5 for object detection and label the recognized dynamic and static objects. Secondly, the topology graph is constructed using the position information of static objects in space. Then, we propose a weight expression for the topology graph to calculate the similarity of topology in different keyframes. Finally, the re-localization and loop closure detection are determined based on the value of topology similarity. Experiments on public datasets show that the semantic topology graph is effective in improving the correct rate of re-localization and the accuracy of loop closure detection in a dynamic environment. [ABSTRACT FROM AUTHOR]

The traditional Simultaneous Localization and Mapping (SLAM) systems are based on the strong static assumption, and their performance will degrade significantly due to the presence of dynamic objects located in dynamic environments. To decrease the effects of the dynamic objects, based on the ORB-SLAM2 system, a novel dynamic semantic SLAM system called MOR-SLAM is presented using a mask repair method, which can accurately detect dynamic objects and realize high-precision positioning and tracking of the system in dynamic indoor environments. First, an instance segmentation module is added to the front end of ORB-SLAM2 to distinguish dynamic and static objects in the environment and obtain a preliminary mask. Next, to overcome the under-segmentation problem in instance segmentation, a new mask inpainting model is proposed to ensure that the integrity of object masks, which repairs large objects and small objects in the image with the depth value fusion method and morphological method, respectively. Then, a reliable basic matrix can be obtained based on the above-repaired mask. Finally, the potential dynamic feature points in the environment are detected and removed through the reliable basic matrix, and the remaining static feature points are input into the tracking module of the system to realize the high-precision positioning and tracking in dynamic environments. The experiments on the public TUM dataset show that, compared with ORB-SLAM2, the MOR-SLAM improves the absolute trajectory accuracy by 95.55%. In addition, compared with DynaSLAM and DS-SLAM on the high-dynamic sequences (fr3/w/rpy and fr3/w/static), the MOR-SLAM improves the absolute trajectory accuracy by 15.20% and 59.71%, respectively. [ABSTRACT FROM AUTHOR]

With the rapid development of autonomous driving and robotics applications in recent years, visual Simultaneous Localization and Mapping (SLAM) has become a hot research topic. The majority of visual SLAM systems relies on the assumption of scene rigidity, which may not always hold true in real applications. In dynamic environments, SLAM systems, without accounting for dynamic objects, will easily fail to estimate the camera pose. Some existing methods attempt to address this issue by simply excluding the dynamic features lying in moving objects. But this may lead to a shortage of features for tracking. To tackle this problem, we propose OTE-SLAM, an object tracking enhanced visual SLAM system, which not only tracks the camera motion, but also tracks the movement of dynamic objects. Furthermore, we perform joint optimization of both the camera pose and object 3D position, enabling a mutual benefit between visual SLAM and object tracking. The results of experiences demonstrate that the proposed approach improves the accuracy of the SLAM system in challenging dynamic environments. The improvements include a maximum reduction in both absolute trajectory error and relative trajectory error by 22% and 33%, respectively. [ABSTRACT FROM AUTHOR]

Tracking head movement in outdoor activities is more challenging than in controlled indoor lab environments. Large-magnitude head scanning is common under natural conditions. Compensatory gaze (head and eye) scanning while walking may be critical for people with visual field loss. We compared the accuracy of two outdoor head tracking methods: differential inertial measurement units (IMU) and simultaneous localization and mapping (SLAM). At a fixed location experiment, a gaze aiming test showed that SLAM outperforms IMU in terms of error (IMU: 9.6°, SLAM: 4.47°). In an urban street walking experiment conducted with five patients with hemifield loss, the IMU drift, quantified by root-mean-square deviation, was as high as 68.1°, while the drift of SLAM was only 5.3°. However, the SLAM method suffered from data loss due to tracking failure (~10% overall, and ~ 18% when crossing streets). Our results show that the SLAM and IMU methods have complementary properties. Because of no data gaps, the differential IMU method may be desirable as compared to SLAM in settings where the signal drift can be removed in post-processing and small gaze estimation errors can be tolerated. [ABSTRACT FROM AUTHOR]

In scenarios of indoor localization of mobile robots, Global Positioning System (GPS) signals are prone to loss due to interference from urban building environments and cannot meet the needs of robot localization. On the other hand, traditional indoor localization methods based on wireless signals such as Bluetooth and WiFi often require the deployment of multiple devices in advance, and these methods can only obtain distance information and are unable to obtain the attitude of the positioning target in space. This paper proposes a method for the indoor localization of mobile robots based on a depth camera. Firstly, we extracted ORB feature points from images captured by a depth camera and performed homogenization processing. Then, we performed feature matching between adjacent two frames of images, and the mismatched points are eliminated to improve the accuracy of feature matching. Finally, we used the Iterative Closest Point (ICP) algorithm to estimate the camera's pose, thus achieving the localization of mobile robots in indoor environments. In addition, an experimental evaluation was conducted on the TUM dataset of the Technical University of Munich to validate the feasibility of the proposed depth-camera-based indoor localization system for mobile robots. The experimental results show that the average localization accuracy of our algorithm on three datasets is 0.027 m, which can meet the needs of indoor localization for mobile robots. [ABSTRACT FROM AUTHOR]

Loop closure detection can eliminate the accumulated error of visual SLAM, which is of great significance to the SLAM system. Among them, the widely used visual bag-of-word model algorithm has the problem of visual word identity and ambiguity, which affects the loop closure detection effect. To improve these problems and loop back detection effect, this paper proposed a loop closure detection algorithm based on soft-assigned SIFT features. The algorithm allocated the SIFT feature points extracted from the image to several words with the closest Euclidean distance, according to the distance sorting and weighting, and removed the feature points that were far away from the words to generate more discriminating descriptors. In addition, when filtering candidates, the algorithm added the proportion constraint of the same word feature points and the stability constraint of word shift to filter out a few candidates. In the experimental results, the algorithm compared with the traditional visual bag-of-words model and several loop closure detection algorithms in recent years, the recall rate at 100% accuracy rate improved under three data sets, and the average image query time was about 40 ms. The results show that the algorithm can improve the loop detection effect and ensure the real-time performance. [ABSTRACT FROM AUTHOR]

At present, SLAM is widely used in all kinds of dynamic scenes. It is difficult to distinguish dynamic targets in scenes using traditional visual SLAM. In the matching process, dynamic points are incorrectly added to the pose calculation with the camera, resulting in low precision and poor robustness in the pose estimation. This paper proposes a new dynamic scene visual SLAM algorithm based on adaptive threshold homogenized feature extraction and YOLOv5 object detection, named AHY-SLAM. This new method adds three new modules based on ORB-SLAM2: a keyframe selection module, a threshold calculation module, and an object detection module. The optical flow method is used to screen keyframes for each frame input in AHY-SLAM. An adaptive threshold is used to extract feature points for keyframes, and dynamic points are eliminated with YOLOv5. Compared with ORB-SLAM2, AHY-SLAM has significantly improved pose estimation accuracy over multiple dynamic scene sequences in the TUM open dataset, and the absolute pose estimation accuracy can be increased by up to 97%. Compared with other dynamic scene SLAM algorithms, the speed of AHY-SLAM is also significantly improved under a guarantee of acceptable accuracy. [ABSTRACT FROM AUTHOR]

At present, visual simultaneous localization and mapping is a hot topic in the field of unmanned systems, which is popular among academic workers because of its advantages of accurate localization, low cost, large amount of information, and wide range of applications, but it still has some problems, including the camera’s vulnerability to the number of feature points and the noise impact of the inertial measurement unit during uniform linear motion. In response to the above problem this paper carries out the research on multi-sensor fusion localization algorithm, the main work is as follows: Based on ORB-SLAM3, a visual-inertial-laser SLAM algorithm is designed. The relative motion of laser location between image frames is obtained from the data of 2D Lidar and laser height sensor. The relative motion of inertial measurement unit between image frames is obtained from inertial measurement unit preintegration. Based on the method of factor graph optimization, the pose of image frame is optimized by reprojection of map point, relative motion increment of inertial measurement unit, and relative motion increment of laser location. The algorithm improves the localization accuracy by about 24.4% over the ORB-SLAM3 visual mode and about 22.6% over the ORB-SLAM3 visual-inertial mode on the data of the UAV physical platform. [ABSTRACT FROM AUTHOR]

Visual simultaneous localization and mapping (SLAM), based on point features, achieves high localization accuracy and map construction. They primarily perform simultaneous localization and mapping based on static features. Despite their efficiency and high precision, they are prone to instability and even failure in complex environments. In a dynamic environment, it is easy to keep track of failures and even failures in work. The dynamic object elimination method, based on semantic segmentation, often recognizes dynamic objects and static objects without distinction. If there are many semantic segmentation objects or the distribution of segmentation objects is uneven in the camera view, this may result in feature offset and deficiency for map matching and motion tracking, which will lead to problems, such as reduced system accuracy, tracking failure, and track loss. To address these issues, we propose a novel point-line SLAM system based on dynamic environments. The method we propose obtains the prior dynamic region features by detecting and segmenting the dynamic region. It realizes the separation of dynamic and static objects by proposing a geometric constraint method for matching line segments, combined with the epipolar constraint method of feature points. Additionally, a dynamic feature tracking method based on Bayesian theory is proposed to eliminate the dynamic noise of points and lines and improve the robustness and accuracy of the SLAM system. We have performed extensive experiments on the KITTI and HPatches datasets to verify these claims. The experimental results show that our proposed method has excellent performance in dynamic and complex scenes. [ABSTRACT FROM AUTHOR]

Dynamic environments are challenging for visual Simultaneous Localization and Mapping, as dynamic elements can disrupt the camera pose estimation and thus reduce the reconstructed map accuracy. To solve this problem, this study proposes an approach for eliminating dynamic elements and reconstructing static background in indoor dynamic environments. To check out dynamic elements, the geometric residual is exploited, and the static background is obtained after removing the dynamic elements and repairing images. The camera pose is estimated based on the static background. Keyframes are then selected using randomized ferns, and loop closure detection and relocalization are performed according to the keyframes set. Finally, the 3D scene is reconstructed. The proposed method is tested on the TUM and BONN datasets, and the map reconstruction accuracy is experimentally demonstrated. [ABSTRACT FROM AUTHOR]

Recent developments in robotics have heightened the need for visual SLAM. Dynamic objects are a major problem in visual SLAM which reduces the accuracy of localization due to the wrong epipolar geometry. This study set out to find a new method to address the low accuracy of visual SLAM in outdoor dynamic environments. We propose an adaptive feature point selection system for outdoor dynamic environments. Initially, we utilize YOLOv5s with the attention mechanism to obtain a priori dynamic objects in the scene. Then, feature points are selected using an adaptive feature point selector based on the number of a priori dynamic objects and the percentage of a priori dynamic objects occupied in the frame. Finally, dynamic regions are determined using a geometric method based on Lucas-Kanade optical flow and the RANSAC algorithm. We evaluate the accuracy of our system using the KITTI dataset, comparing it to various dynamic feature point selection strategies and DynaSLAM. Experiments show that our proposed system demonstrates a reduction in both absolute trajectory error and relative trajectory error, with a maximum reduction of 39% and 30%, respectively, compared to other systems. [ABSTRACT FROM AUTHOR]

Many visual SLAM systems are generally solved using natural landmarks or optical flow. However, due to textureless areas, illumination change or motion blur, they often acquire poor camera poses or even fail to track. Additionally, they cannot obtain camera poses with a metric scale in the monocular case. In some cases (such as when calibrating the extrinsic parameters of camera-IMU), we prefer to sacrifice the flexibility of such methods to improve accuracy and robustness by using artificial landmarks. This paper proposes enhancements to the traditional SPM-SLAM, which is a system that aims to build a map of markers and simultaneously localize the camera pose. By placing the markers in the surrounding environment, the system can run stably and obtain accurate camera poses. To improve robustness and accuracy in the case of rotational movements, we improve the initialization, keyframes insertion and relocalization. Additionally, we propose a novel method to estimate marker poses from a set of images to solve the problem of planar-marker pose ambiguity. Compared with the state-of-art, the experiments show that our system achieves better accuracy in most public sequences and is more robust than SPM-SLAM under rotational movements. Finally, the open-source code is publicly available and can be found at GitHub. [ABSTRACT FROM AUTHOR]

Most deep learning-based depth estimation models that learn scene structure self-supervised from monocular video base their estimation on visual cues such as vanishing points. In the established depth estimation benchmarks depicting, for example, street navigation or indoor offices, these cues can be found consistently, which enables neural networks to predict depth maps from single images. In this work, we are addressing the challenge of depth estimation from a real-world bird’s-eye perspective in an industry environment which contains, conditioned by its special geometry, a minimal amount of visual cues and, hence, requires incorporation of the temporal domain for structure from motion estimation. To enable the system to incorporate structure from motion from pixel translation when facing context-sparse, i.e., visual cue sparse, scenery, we propose a novel architecture built upon the structure from motion learner, which uses temporal pairs of jointly unrotated and stacked images for depth prediction. In order to increase the overall performance and to avoid blurred depth edges that lie in between the edges of the two input images, we integrate a geometric consistency loss into our pipeline. We assess the model’s ability to learn structure from motion by introducing a novel industry dataset whose perspective, orthogonal to the floor, contains only minimal visual cues. Through the evaluation with ground truth depth, we show that our proposed method outperforms the state of the art in difficult context-sparse environments. [ABSTRACT FROM AUTHOR]

• Various data from smartphone sensors are utilized and integrated with GCPs. • Scale update and refreshed-SLAM processes constrain the accumulated drift of V-SLAM. • Performance of schemes are tested in different GNSS-challenged environments. Contemporary smartphones contain embedded inertial measurement unit (IMU), global navigation satellite system (GNSS), camera, and other sensors that are capable of providing user position, velocity, and attitude. However, the actual navigation performance capabilities of smartphones are difficult to use because of the low-cost and disparate sensors employed, differing software technologies adopted by manufacturers, and considerable influence of environmental conditions. In this study, we proposed multifusion schemes that integrated sensor data from smartphone IMU, GNSS chipsets, cameras, and ground control points (GCPs), using an extended Kalman filter to enhance the system navigation performance. Different processes of scale recovery and refreshed-simultaneous localization and mapping (SLAM) based on GNSS and GCPs corresponding to outdoor and indoor environments were proposed to increase the accuracy and robustness of the integrated system. To verify the performance of the integrated system, field test data were collected in an urban area of Tainan City, Taiwan. The experimental results indicated improvements of 87.09% and 36.27% for the refreshed-SLAM and its integration system, respectively, compared with visual-SLAM one-scale recovery and conventional integrated schemes. The proposed integrated system that uses smartphone sensor data increased navigation accuracy in GNSS-challenged environments. [ABSTRACT FROM AUTHOR]

After decades of development, LIDAR and visual SLAM technology has relatively matured and been widely used in the military and civil fields. SLAM technology enables the mobile robot to have the abilities of autonomous positioning and mapping, which allows the robot to move in indoor and outdoor scenes where GPS signals are scarce. However, SLAM technology relying only on a single sensor has its limitations. For example, LIDAR SLAM is not suitable for scenes with highly dynamic or sparse features, and visual SLAM has poor robustness in low-texture or dark scenes. However, through the fusion of the two technologies, they have great potential to learn from each other. Therefore, this paper predicts that SLAM technology combining LIDAR and visual sensors, as well as various other sensors, will be the mainstream direction in the future. This paper reviews the development history of SLAM technology, deeply analyzes the hardware information of LIDAR and cameras, and presents some classical open source algorithms and datasets. According to the algorithm adopted by the fusion sensor, the traditional multi-sensor fusion methods based on uncertainty, features, and novel deep learning are introduced in detail. The excellent performance of the multi-sensor fusion method in complex scenes is summarized, and the future development of multi-sensor fusion method is prospected. [ABSTRACT FROM AUTHOR]

In recent years, Simultaneous Localization and Mapping (SLAM) systems have shown significant performance, accuracy, and efficiency gain. In this regard, Visual Simultaneous Localization and Mapping (VSLAM) methods refer to the SLAM approaches that employ cameras for pose estimation and map reconstruction and are preferred over Light Detection And Ranging (LiDAR)-based methods due to their lighter weight, lower acquisition costs, and richer environment representation. Hence, several VSLAM approaches have evolved using different camera types (e.g., monocular or stereo), and have been tested on various datasets (e.g., Technische Universität München (TUM) RGB-D or European Robotics Challenge (EuRoC)) and in different conditions (i.e., indoors and outdoors), and employ multiple methodologies to have a better understanding of their surroundings. The mentioned variations have made this topic popular for researchers and have resulted in various methods. In this regard, the primary intent of this paper is to assimilate the wide range of works in VSLAM and present their recent advances, along with discussing the existing challenges and trends. This survey is worthwhile to give a big picture of the current focuses in robotics and VSLAM fields based on the concentrated resolutions and objectives of the state-of-the-art. This paper provides an in-depth literature survey of fifty impactful articles published in the VSLAMs domain. The mentioned manuscripts have been classified by different characteristics, including the novelty domain, objectives, employed algorithms, and semantic level. The paper also discusses the current trends and contemporary directions of VSLAM techniques that may help researchers investigate them. [ABSTRACT FROM AUTHOR]

The recent advancements in Information and Communication Technology (ICT) as well as increasing demand for vehicular safety has led to significant progressions in Autonomous Vehicle (AV) technology. Perception and Localisation are major operations that determine the success of AV development and usage. Therefore, significant research has been carried out to provide AVs with the capabilities to not only sense and understand their surroundings efficiently, but also provide detailed information of the environment in the form of 3D maps. Visual Simultaneous Localisation and Mapping (V-SLAM) has been utilised to enable a vehicle understand its surroundings, map the environment, and identify its position within the area. This paper presents a detailed review of V-SLAM techniques implemented for AV perception and localisation. An overview of SLAM techniques is presented. In addition, an in-depth review is conducted to highlight various V-SLAM schemes, their strengths, and limitations. Challenges associated with V-SLAM deployment and future research directions are also provided in this paper. [ABSTRACT FROM AUTHOR]

Simultaneous localization and mapping (SLAM) technology can be used to locate and build maps in unknown environments, but the constructed maps often suffer from poor readability and interactivity, and the primary and secondary information in the map cannot be accurately grasped. For intelligent robots to interact in meaningful ways with their environment, they must understand both the geometric and semantic properties of the scene surrounding them. Our proposed method can not only reduce the absolute positional errors (APE) and improve the positioning performance of the system but also construct the object-oriented dense semantic point cloud map and output point cloud model of each object to reconstruct each object in the indoor scene. In fact, eight categories of objects are used for detection and semantic mapping using coco weights in our experiments, and most objects in the actual scene can be reconstructed in theory. Experiments show that the number of points in the point cloud is significantly reduced. The average positioning error of the eight categories of objects in Technical University of Munich (TUM) datasets is very small. The absolute positional error of the camera is also reduced with the introduction of semantic constraints, and the positioning performance of the system is improved. At the same time, our algorithm can segment the point cloud model of objects in the environment with high accuracy. [ABSTRACT FROM AUTHOR]

In dynamic indoor environments and for a Visual Simultaneous Localization and Mapping (vSLAM) system to operate, moving objects should be considered because they could affect the system's visual odometer stability and its position estimation accuracy. vSLAM can use feature points or a sequence of images, as it is the only source of input that can perform localization while simultaneously creating a map of the environment. A vSLAM system based on ORB-SLAM3 and on YOLOR was proposed in this paper. The newly proposed system in combination with an object detection model (YOLOX) applied on extracted feature points is capable of achieving 2–4% better accuracy compared to VPS-SLAM and DS-SLAM. Static feature points such as signs and benches were used to calculate the camera position, and dynamic moving objects were eliminated by using the tracking thread. A specific custom personal dataset that includes indoor and outdoor RGB-D pictures of train stations, including dynamic objects and high density of people, ground truth data, sequence data, and video recordings of the train stations and X, Y, Z data was used to validate and evaluate the proposed method. The results show that ORB-SLAM3 with YOLOR as object detection achieves 89.54% of accuracy in dynamic indoor environments compared to previous systems such as VPS-SLAM. [ABSTRACT FROM AUTHOR]

Simultaneous Localisation and Mapping (SLAM) is one of the fundamental problems in autonomous mobile robots where a robot needs to reconstruct a previously unseen environment while simultaneously localising itself with respect to the map. In particular, Visual-SLAM uses various sensors from the mobile robot for collecting and sensing a representation of the map. Traditionally, geometric model-based techniques were used to tackle the SLAM problem, which tends to be error-prone under challenging environments. Recent advancements in computer vision, such as deep learning techniques, have provided a data-driven approach to tackle the Visual-SLAM problem. This review summarises recent advancements in the Visual-SLAM domain using various learning-based methods. We begin by providing a concise overview of the geometric model-based approaches, followed by technical reviews on the current paradigms in SLAM. Then, we present the various learning-based approaches to collecting sensory inputs from mobile robots and performing scene understanding. The current paradigms in deep-learning-based semantic understanding are discussed and placed under the context of Visual-SLAM. Finally, we discuss challenges and further opportunities in the direction of learning-based approaches in Visual-SLAM. [ABSTRACT FROM AUTHOR]

In complex environments such as those with low textures or obvious brightness changes, point features extracted from a traditional FAST algorithm cannot perform well in pose estimation. Simultaneously, the number of point features extracted from FAST is too large, which increases the complexity of the build map. To solve these problems, we propose an L-FAST algorithm based on FAST, in order to reduce the number of extracted points and increase their quality. L-FAST pays more attention to the intersection of line elements in the image, which can be extracted directly from the related edge image. Hence, we improved the Canny edge extraction algorithm, including denoising, gradient calculation and adaptive threshold. These improvements aimed to enhance the sharpness of image edges and effectively extract the edges of strong light or dark areas in the images as brightness changed. Experiments on digital standard images showed that our improved Canny algorithm was smoother and more continuous for the edges extracted from images with brightness changes. Experiments on KITTI datasets showed that L-FAST extracted fewer point features and increased the robustness of SLAM. [ABSTRACT FROM AUTHOR]

3D reconstruction of crops is important for researching their biological properties, canopy light distribution and robotic harvesting. However, the complex field environment makes the real-time 3D reconstruction of crops difficult. Due to the low-textured in the field, it is difficult to obtain effective features to construct accurate and real-time 3D maps of the field from existing single-feature SLAM methods. In this paper, we propose a novel RGB-D SLAM based on point-line feature fusion for the real-time field 3D scene reconstruction. By optimizing the point-line features joint poses, we first build a 3D scene map of the field based on the point-line feature structure. Then, a joint point cloud filtering method is designed based on the keyframes optimization of the point-line feature. Finally, we obtain the consistently high-quality dense map in the global respect. The overall performance in terms of pose estimation and reconstruction is evaluated on public benchmarks and shows improved performance compared to state-of-the-art methods. Qualitative experiments on the field scenes show that our method enables real-time 3D reconstruction of crops with high robustness. • A point-line feature fusion optimization method is proposed to improve the accuracy of pose estimation. • A keyframe optimization strategy and a dense map optimization scheme are proposed to achieve a globally consistent dense map. • A real-time RGB-D SLAM method is presented for field 3D scenes reconstruction based on point-line feature fusion. [Display omitted] [ABSTRACT FROM AUTHOR]

Key-frame selection methods were developed in the past years to reduce the complexity of frame processing in visual odometry (VO) and visual simultaneous localization and mapping (VSLAM) algorithms. Key-frames help increasing algorithm's performances by sparsifying frames while maintaining its accuracy and robustness. Unlike current selection methods that rely on many heuristic thresholds to decide which key-frame should be selected, this paper proposes a photogrammetric-based key-frame selection method built upon ORB-SLAM3. The proposed algorithm, named Photogrammetric Key-frame Selection (PKS), replaces static heuristic thresholds with photogrammetric principles, ensuring algorithm's robustness and better point cloud quality. A key-frame is chosen based on adaptive thresholds and the Equilibrium Of Center Of Gravity (ECOG) criteria as well as Inertial Measurement Unit (IMU) observations. To evaluate the proposed PKS method, the European Robotics Challenge (EuRoC) and an in-house datasets are used. Quantitative and qualitative evaluations are made by comparing trajectories, point clouds quality and completeness and Absolute Trajectory Error (ATE) in mono-inertial and stereo-inertial modes. Moreover, for the generated dense point clouds, extensive evaluations, including plane-fitting error, model deformation, model alignment error, and model density and quality, are performed. The results show that the proposed algorithm improves ORB-SLAM3 positioning accuracy by 18% in stereo-inertial mode and 20% in mono-inertial mode without the use of heuristic thresholds, as well as producing a more complete and accurate point cloud up to 50%. The open-source code of the presented method is available at https://github.com/arashazimi0032/PKS. [ABSTRACT FROM AUTHOR]

Exploration of marine habitats is one of the key pillars of underwater science, which often involves collecting images at close range. As acquiring imagery close to the seabed involves multiple hazards, the safety of underwater vehicles, such as remotely operated vehicles (ROVs) and autonomous underwater vehicles (AUVs), is often compromised. Common applications for obstacle avoidance in underwater environments are often conducted with acoustic sensors, which cannot be used reliably at very short distances, thus requiring a high level of attention from the operator to avoid damaging the robot. Therefore, developing capabilities such as advanced assisted mapping, spatial awareness and safety, and user immersion in confined environments is an important research area for human-operated underwater robotics. In this paper, we present a novel approach that provides an ROV with capabilities for navigation in complex environments. By leveraging the ability of omnidirectional multi-camera systems to provide a comprehensive view of the environment, we create a 360° real-time point cloud of nearby objects or structures within a visual SLAM framework. We also develop a strategy to assess the risk of obstacles in the vicinity. We show that the system can use the risk information to generate warnings that the robot can use to perform evasive maneuvers when approaching dangerous obstacles in real-world scenarios. This system is a first step towards a comprehensive pilot assistance system that will enable inexperienced pilots to operate vehicles in complex and cluttered environments. [ABSTRACT FROM AUTHOR]

With the development of artificial intelligence technology, visual simultaneous localization and mapping (SLAM) has become a cheap and efficient localization method for underwater robots. However, there are many problems in underwater visual SLAM, such as more serious underwater imaging distortion, more underwater noise, and unclear details. In this paper, we study these two problems and chooses the ORB-SLAM2 algorithm as the method to obtain the motion trajectory of the underwater robot. The causes of radial distortion and tangential distortion of underwater cameras are analyzed, a distortion correction model is constructed, and five distortion correction coefficients are obtained through pool experiments. Comparing the performances of contrast-limited adaptive histogram equalization (CLAHE), median filtering (MF), and dark channel prior (DCP) image enhancement methods in underwater SLAM, it is found that the DCP method has the best image effect evaluation, the largest number of oriented fast and rotated brief (ORB) feature matching, and the highest localization trajectory accuracy. The results show that the ORB-SLAM2 algorithm can effectively locate the underwater robot, and the correct distortion correction coefficient and DCP improve the stability and accuracy of the ORB-SLAM2 algorithm. [ABSTRACT FROM AUTHOR]

The visual SLAM (vSLAM) is a research topic that has been developing rapidly in recent years, especially with the renewed interest in machine learning and, more particularly, deep-learning-based approaches. Nowadays, main research is carried out to improve accuracy and robustness in complex and dynamic environments. This scorching topic has reached a significant level of maturity. This paper presents a relatively detailed and easily understood survey of vSLAM within deep learning. This study attempts to meet this challenge by better organizing the literature, explaining the basic concepts and tools, and presenting the current trends. The contributions of this study can be summarized in three essential steps. The first one is to provide the state-of-the-art in an incremental way following the classical processes of vSLAM-based systems. The second is to give our short- and medium-term view of the development of this very active and evolving field. Finally, we share our opinions on this subject and its interactions with new trends and, more particularly, the deep learning paradigm. We believe that this contribution will be an overview and, more importantly, a critical and detailed vision that serves as a roadmap in the field of vSLAMs both in terms of models and concepts and in terms of associated technologies. [ABSTRACT FROM AUTHOR]

Indirect methods for visual SLAM are gaining popularity due to their robustness to environmental variations. ORB-SLAM2 (Mur-Artal and Tardós, 2017) is a benchmark method in this domain, however, it consumes significant time for computing descriptors that never get reused unless a frame is selected as a keyframe. To overcome these problems, we present FastORB-SLAM which is light-weight and efficient as it tracks keypoints between adjacent frames without computing descriptors. To achieve this, a two stage descriptor-independent keypoint matching method is proposed based on sparse optical flow. In the first stage, we predict initial keypoint correspondences via a simple but effective motion model and then robustly establish the correspondences via pyramid-based sparse optical flow tracking. In the second stage, we leverage the constraints of the motion smoothness and epipolar geometry to refine the correspondences. In particular, our method computes descriptors only for keyframes. We test FastORB-SLAM on TUM and ICL-NUIM RGB-D datasets and compare its accuracy and efficiency to nine existing RGB-D SLAM methods. Qualitative and quantitative results show that our method achieves state-of-the-art accuracy and is about twice as fast as the ORB-SLAM2. [ABSTRACT FROM AUTHOR]

Some underwater applications involve deploying multiple underwater Remotely Operated Vehicles in a common area. Such applications require the localization of these vehicles, not only with respect to each other but also with respect to a previously unknown environment. To this end, this work presents MAM 3 SLAM, a new fully centralized multi-agent and multi-map monocular Visual Simultaneous Localization And Mapping (VSLAM) framework. Multi-agent evaluation metrics are introduced to provide an extensive evaluation of MAM 3 SLAM compared to the state-of-the-art multi-agent VSLAM on four two-agent scenarios: one standard airborne dataset and three new underwater datasets recorded in a pool and the sea. The results show that MAM 3 SLAM is robust to underwater visual conditions and tracking failures, outperforms the other evaluated methods in estimating the individual and relative poses of the agents and in collaborative mapping accuracy. MAM3SLAM successfully estimates the individual and relative localization of the agents with an error lower than 5 cm on three out of the four test sequences, and is twice as accurate as competing multi-agent works in challenging visual conditions with frequent visual dropouts, poor textures, low framerate and fast motion. MAM 3 SLAM's source code is made available, as are the underwater datasets. • A new, centralized multi-agent VSLAM solution for ROVs localization is introduced. • Individual, relative and co-mapping-related evaluation metrics are introduced. • Three new underwater two-agent datasets, in pools and in the sea, are released. • Evaluations show the robustness of our approach compared to competing visual works. [ABSTRACT FROM AUTHOR]

In recent years, research on simultaneous localization and mapping (SLAM) using neural implicit representation has shown promising outcomes due to its smooth mapping and low memory consumption, particularly suitable for structured environments with limited boundaries. However, there is currently no implicit SLAM that can effectively utilize prior structural constraints to accurately build 3D maps. In this study, we propose an RGB-D dense tracking and mapping approach, Structerf-SLAM, that combines visual odometry with neural implicit representation. Our scene representation consists of dual-layer feature grids and pre-trained decoders that decode the interpolated features into RGB and depth values. Moreover, structured planar constraints are integrated. In the tracking stage, utilizing the three-dimensional plane features under the Manhattan assumption achieves more stable and rapid data association, consequently resolving the tracking misalignment issue in textureless regions (e.g., floor, wall, etc.). In the mapping stage, by enforcing planar consistency, the depth predicted by the neural radiation field is well-fitted by a plane, resulting in smoother and more realistic map reconstruction. Experiments on synthetic and real scene datasets demonstrate competitive results of Structerf-SLAM in both mapping and tracking quality. [Display omitted] • The first neural implicit representation SLAM tailored for structured scenes. • Combining VO with implicit SLAM, superior robustness is maintained under sparse data. • Plane features are used to achieve stable data association in the tracking thread. • A dual-layer neural radiance field with planar consistency achieves SOTA reconstruction. [ABSTRACT FROM AUTHOR]

Traditional visual SLAM algorithms run robustly under the assumption of a static environment, but always fail in dynamic scenes, since moving objects will impair camera pose tracking. Given this, this paper presents an efficient semantic dynamic SLAM (ESD-SLAM), which is suitable for dynamic scenarios. Based on the ORB-SLAM2 framework, the ESD-SLAM we proposed employs lightweight semantic segmentation network FcHarDNet to extract semantic information, and uses the region growing algorithm to optimize the semantic segmentation boundary. Then dynamic objects are removed by combining semantic information with multi-view geometry, and it further improves the localization accuracy. Combining semantic information and depth information, a dense point cloud map of static scene is constructed to serve the planning task of mobile robot. We conduct the experiments on the public TUM RGB-D dataset and in the real-world environment. Experimental results show that the proposed algorithm can improve the performance of the ORB-SLAM2 system in dynamic scenes, and significantly improve the real-time performance compared with other same type dynamic SLAM algorithms. [ABSTRACT FROM AUTHOR]

In order to improve the accuracy of visual SLAM algorithms in a dynamic scene, instance segmentation is widely used to eliminate dynamic feature points. However, the existing segmentation technology has low accuracy, especially for the contour of the object, and the amount of calculation of instance segmentation is large, limiting the speed of visual SLAM based on instance segmentation. Therefore, this paper proposes a contour optimization hybrid dilated convolutional neural network (CO-HDC) algorithm, which can perform a lightweight calculation on the basis of improving the accuracy of contour segmentation. Firstly, a hybrid dilated convolutional neural network (HDC) is used to increase the receptive field, which is defined as the size of the region in the input that produces the feature. Secondly, the contour quality evaluation (CQE) algorithm is proposed to enhance the contour, retaining the highest quality contour and solving the problem of distinguishing dynamic feature points from static feature points at the contour. Finally, in order to match the mapping speed of visual SLAM, the Beetle Antennae Search Douglas–Peucker (BAS-DP) algorithm is proposed to lighten the contour extraction. The experimental results have demonstrated that the proposed visual SLAM based on the CO-HDC algorithm performs well in the field of pose estimation and map construction on the TUM dataset. Compared with ORB-SLAM2, the Root Mean Squared Error (Rmse) of the proposed method in absolute trajectory error is about 30 times smaller and is only 0.02 m. [ABSTRACT FROM AUTHOR]

This paper presents a new method of removing mismatches of redundant points based on oriented fast and rotated brief (ORB) in vision simultaneous localization and mapping (SLAM) systems. On the one hand, the grid-based motion statistics (GMS) algorithm reduces the processing time of key frames with more feature points and greatly increases the robustness of the original algorithm in a complex environment. On the other hand, aiming at the situation that the GMS algorithm is prone to false matching when there are few symmetry feature point pairs, the random sample consensus (RANSAC) algorithm is used to optimize and correct it. Experiments show that the method we propose has an average error correction rate of 28.81% for individual GMS while the time consumed at the same accuracy threshold is reduced by 72.18% on average. At the same time, we compared it to locality preserving matching (LPM) and progressive sample consensus (PROSAC), and it performed the best. Finally, we integrated GMS-RANSAC into the ORB-SLAM2 system for monocular initialization, which results in a significant improvement. [ABSTRACT FROM AUTHOR]

Simultaneous localization and mapping (SLAM), as one of the core prerequisite technologies for intelligent mobile robots, has attracted much attention in recent years. However, the traditional SLAM systems rely on the static environment assumption, which becomes unstable for the dynamic environment and further limits the real-world practical applications. To deal with the problem, this paper presents a dynamic-environment-robust visual SLAM system named YOLO-SLAM. In YOLO-SLAM, a lightweight object detection network named Darknet19-YOLOv3 is designed, which adopts a low-latency backbone to accelerate and generate essential semantic information for the SLAM system. Then, a new geometric constraint method is proposed to filter dynamic features in the detecting areas, where dynamic features can be distinguished by utilizing the depth difference with Random Sample Consensus (RANSAC). YOLO-SLAM composes the object detection approach and the geometric constraint method in a tightly coupled manner, which is able to effectively reduce the impact of dynamic objects. Experiments are conducted on the challenging dynamic sequences of TUM dataset and Bonn dataset to evaluate the performance of YOLO-SLAM. The results demonstrate that the RMSE index of absolute trajectory error can be significantly reduced to 98.13% compared with ORB-SLAM2 and 51.28% compared with DS-SLAM, indicating that YOLO-SLAM is able to effectively improve stability and accuracy in the highly dynamic environment. [ABSTRACT FROM AUTHOR]

To solve the problems of difficult initialization and low positioning accuracy of SLAM algorithm caused by sparse feature points and fast camera movement in practical application scenes, this paper proposed a visual-inertial SLAM algorithm. It used EDLines feature extraction and matching algorithm to introduce rich line features to improve the robustness of feature tracking. Using the complementary advantages of the inertial sensor and the visual sensor, it carried out the optimization in three steps in the initial stage through the visual-inertial initialization by steps method, so as to improve the accuracy and speed of the initialization. Experiments show that the line feature extraction and matching algorithm used in this algorithm provides faster matching speed and lower mismatching rate than the traditional LSD algorithm. Moreover, the proposed algorithm has higher accuracy and stronger robustness in the estimation of camera pose in the visual-inertial data set. [ABSTRACT FROM AUTHOR]

Intelligent agents need to understand the surrounding environment to provide meaningful services to or interact intelligently with humans. The agents should perceive geometric features as well as semantic entities inherent in the environment. Contemporary methods in general provide one type of information regarding the environment at a time, making it difficult to conduct high-level tasks. Moreover, running two types of methods and associating two resultant information requires a lot of computation and complicates the software architecture. To overcome these limitations, we propose a neural architecture that simultaneously performs both geometric and semantic tasks in a single thread: simultaneous visual odometry, object detection, and instance segmentation (SimVODIS). SimVODIS is built on top of Mask-RCNN which is trained in a supervised manner. Training the pose and depth branches of SimVODIS requires unlabeled video sequences and the photometric consistency between input image frames generates self-supervision signals. The performance of SimVODIS outperforms or matches the state-of-the-art performance in pose estimation, depth map prediction, object detection, and instance segmentation tasks while completing all the tasks in a single thread. We expect SimVODIS would enhance the autonomy of intelligent agents and let the agents provide effective services to humans. [ABSTRACT FROM AUTHOR]

Visual Simultaneous Localization and Mapping (VSLAM) technology has become a key technology in autonomous driving and robot navigation. Relying on camera sensors, VSLAM can provide a richer and more precise perception means, and its advancement has accelerated in recent years. However, current review studies are often limited to in-depth analysis of a specific module and lack a comprehensive review of the entire VSLAM framework. The VSLAM system consists of five core components: (1) The camera sensor module is responsible for capturing visual information about the surrounding environment. (2) The front-end module uses image data to roughly estimate the camera's position and orientation. (3) The back-end module optimizes and processes the pose information estimated by the front-end. (4) The loop detection module is used to correct accumulated errors in the system. (5) The mapping module is responsible for generating environmental maps. This review provides a systematic and comprehensive analysis of the SLAM framework by taking the core components of VSLAM as the entry point. Deep learning brings new development opportunities for VSLAM, but it still needs to solve the problems of data dependence, cost and real-time in practical application. We deeply explore the challenges of combining VSLAM with deep learning and feasible solutions. This review provides a valuable reference for the development of VSLAM. This will help push VSLAM technology to become smarter and more efficient. Thus, it can better meet the needs of future intelligent autonomous systems in multiple fields. • We comprehensively cover the core components of VSLAM technology, including data acquisition, front-end, back-end, loop closure, and mapping modules, to provide a systematic perspective. In contrast to previous studies that focused primarily on individual modules, this comprehensive approach aims to assist readers in gaining a more holistic understanding of the overall framework of VSLAM technology and the interplay among its key components. • Deep learning significantly enhances feature extraction and matching accuracy in VSLAM, supporting localization and mapping tasks robustly. However, the practical application of deep learning in this context also encounters various challenges, such as the reliance on high-quality annotated data, the demand for significant computational resources, and the necessity for real-time processing. This analysis of the application and challenges of deep learning in practice provides readers with a clear understanding of its value and limitations within VSLAM technology. • We propose specific solutions to address the challenges in practical deep learning applications of VSLAM. For example, synthetic data can make up for the lack of actual annotated data. The lightweight model architecture reduces the computational expenses, while hardware structure optimization improves efficiency. Techniques such as model pruning and inference engine optimization speed up deep learning models during inference. These measures together effectively solve the challenge and provide a solid theoretical foundation and practical guidance for advancing VSLAM technology. • To be more specific, research on Visual Simultaneous Localization and Mapping (VSLAM) can be distinctly categorized into two major groups. On one hand, works like [1][2][3] have delved into specific modules of VSLAM or the underlying assumptions based on particular application scenarios, primarily focusing on a single perspective. On the other hand, studies such as [4][5] may have emphasized the importance of combining deep learning with VSLAM, especially in the context of traditional VSLAM and semantic VSLAM integrated with deep learning. Nevertheless, these investigations exhibit certain limitations. Firstly, they fail to provide a comprehensive and systematic exposition of the overall development of VSLAM, thus lacking a panoramic view. Secondly, existing literature has not conducted an in-depth analysis of the challenges faced by the integration of deep learning, and solutions are seldom provided. [1] Eyvazpour R, Shoaran M, Karimian G. Hardware implementation of SLAM algorithms: a survey on implementation approaches and platforms[J]. Artificial Intelligence Review, 2023, 56(7): 6187-6239. [2] Cheng J, Zhang L, Chen Q, et al. A review of visual SLAM methods for autonomous driving vehicles[J]. Engineering Applications of Artificial Intelligence, 2022, 114: 104992. [3] Kazerouni I A, Fitzgerald L, Dooly G, et al. A survey of state-of-the-art on visual SLAM[J]. Expert Systems with Applications, 2022, 205: 117734. [4] Chen W, Shang G, Ji A, et al. An overview on visual slam: From tradition to semantic[J]. Remote Sensing, 2022, 14(13): 3010. [5] Tang Y, Zhao C, Wang J, et al. Perception and navigation in autonomous systems in the era of learning: A survey[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022. [ABSTRACT FROM AUTHOR]

Visual navigation in unknown environments poses significant challenges due to the presence of many obstacles and low-texture scenes. These factors may cause frequent collisions and tracking failure of feature-based visual Simultaneous Localization and Mapping (vSLAM). To avoid these issues, this paper proposes a spatial memory-augmented visual navigation system that combines a vSLAM module, a conventional global planner module, and a Hierarchical Reinforcement Learning (HRL)-based local planner module. Firstly, a real-time vSLAM named Salient-SLAM is proposed to augment the performance of visual navigation. Salient-SLAM creates a navigation mapping thread by combining a saliency prediction model to build a navigation map that categorizes environmental regions as occupied, explored, or noticeable. Spatial memory that contains spatial abstraction and saliency information of the environment can be further formed by encoding navigation maps, which helps the agent determine an optimal path towards its destination. An open-sourced saliency dataset is proposed to train the saliency prediction model by mimicking the visual attention mechanism. Secondly, a HRL method is proposed to automatically decompose local planning into a high-level policy selector and several low-level policies, where the latter produces actions to interact with the environment. We maximize entropy and minimize option correlation in learning low-level policies, aiming at acquiring diverse and independent behaviors. The simulation results show that the proposed HRL method outperforms competitive baselines by 6.29–10.85 % on Success Rate (SR) and 3.87–11.1 % on Success weighted by Path Length (SPL) metrics. By incorporating the spatial memory, SR, and SPL metrics can be augmented by an average of 9.85 % and 10.89 %, respectively. [ABSTRACT FROM AUTHOR]

The goal of visual SLAM technology is to build a more complete map and estimate a more accurate camera pose. To construct a more detailed and complete 3D map, a semi-dense 3D reconstruction ORB-SLAM algorithm with improved ORB feature matching is proposed to realize the construction of a sparse 3D point cloud map of the environment. On the basis of the ORB-SLAM algorithm, a semi-dense mapping thread is added to establish a semi-dense 3D point cloud map. Then, the scale invariance of SURF feature matching algorithm is used to improve ORB feature matching. Simulation experiments in the TUM RGBD datasets reveal that the 3D map established by the improved ORB-SLAM algorithm can more intuitively exhibit the contours of objects in the environment when compared with the ORB-SLAM algorithm, and the feature matching accuracy is increased. Finally, the consistency of the algorithm is verified by experiments in two datasets. [ABSTRACT FROM AUTHOR]