Search

The traditional simultaneous localization and mapping (SLAM) system uses static points of the environment as features for real-time localization and mapping. When there are few available point features, the system is difficult to implement. A feasible solution is to introduce line features. In complex scenarios containing rich line segments, the description of line segments is not strongly differentiated, which can lead to incorrect association of line segment data, thus introducing errors into the system and aggravating the cumulative error of the system. To address this problem, a point-line stereo visual SLAM system incorporating semantic invariants is proposed in this paper. This system improves the accuracy of line feature matching by fusing line features with image semantic invariant information. When defining the error function, the semantic invariant is fused with the reprojection error function, and the semantic constraint is applied to reduce the cumulative error of the poses in the long-term tracking process. Experiments on the Office sequence of the TartanAir dataset and the KITTI dataset show that this system improves the matching accuracy of line features and suppresses the cumulative error of the SLAM system to some extent, and the mean relative pose error (RPE) is 1.38 and 0.0593 m, respectively.

The accurate and reliable high-altitude orientation estimation is of great significance for unmanned aerial vehicles (UAVs) localization, and further assists them to conduct some fundamental functions, such as aerial mapping, environmental monitoring, and risk management. However, the traditional orientation estimation is susceptible to electromagnetic interference, high maneuverability, and substantial scale variations. Hence, this paper aims to present a new visual compass algorithm to estimate the orientation of a UAV employing the appearance and geometry structure of the point and line features in the remote sensing images. In this study, a coarse-to-fine feature tracking method is used to locate the matched keypoints precisely. An LK-ZNCC algorithm is proposed to match line segments in real-time. A hierarchical fusion method for point and line features is designed to expand the scope of the usage of this system. Many comparative experiments between this algorithm and others are conducted on a UAV. Experimental results show that the proposed visual compass algorithm is a reliable, precise, and versatile system applicable to other UAV navigation systems, especially when they do not work in particular situations.

Mobile platform visual image sequence inevitably has large areas with various types of weak textures, which affect the acquisition of accurate pose in the subsequent platform moving process. The visual–inertial odometry (VIO) with point features and line features as visual information shows a good performance in weak texture environments, which can solve these problems to a certain extent. However, the extraction and matching of line features are time consuming, and reasonable weights between the point and line features are hard to estimate, which makes it difficult to accurately track the pose of the platform in real time. In order to overcome the deficiency, an improved effective point–line visual–inertial odometry system is proposed in this paper, which makes use of geometric information of line features and combines with pixel correlation coefficient to match the line features. Furthermore, this system uses the Helmert variance component estimation method to adjust weights between point features and line features. Comprehensive experimental results on the two datasets of EuRoc MAV and PennCOSYVIO demonstrate that the point–line visual–inertial odometry system developed in this paper achieved significant improvements in both localization accuracy and efficiency compared with several state-of-the-art VIO systems.

To address the problem of estimating camera trajectory and to build a structural three-dimensional (3D) map based on inertial measurements and visual observations, this paper proposes point–line visual–inertial odometry (PL-VIO), a tightly-coupled monocular visual–inertial odometry system exploiting both point and line features. Compared with point features, lines provide significantly more geometrical structure information on the environment. To obtain both computation simplicity and representational compactness of a 3D spatial line, Plücker coordinates and orthonormal representation for the line are employed. To tightly and efficiently fuse the information from inertial measurement units (IMUs) and visual sensors, we optimize the states by minimizing a cost function which combines the pre-integrated IMU error term together with the point and line re-projection error terms in a sliding window optimization framework. The experiments evaluated on public datasets demonstrate that the PL-VIO method that combines point and line features outperforms several state-of-the-art VIO systems which use point features only.

This paper presents a novel approach for estimating the ego-motion of a vehicle in dynamic and unknown environments using tightly-coupled inertial and visual sensors. To improve the accuracy and robustness, we exploit the combination of point and line features to aid navigation. The mathematical framework is based on trifocal geometry among image triplets, which is simple and unified for point and line features. For the fusion algorithm design, we employ the Extended Kalman Filter (EKF) for error state prediction and covariance propagation, and the Sigma Point Kalman Filter (SPKF) for robust measurement updating in the presence of high nonlinearities. The outdoor and indoor experiments show that the combination of point and line features improves the estimation accuracy and robustness compared to the algorithm using point features alone.

Scan matching, an approach to recover the relative position and orientation of two laser scans, is a very important technique for indoor positioning and indoor modeling. The iterative closest point (ICP) algorithm and its variants are the most well-known techniques for such a problem. However, ICP algorithms rely highly on the initial guess of the relative transformation, which will reduce its power for practical applications. In this paper, an initial-free 2D laser scan matching method based on point and line features is proposed. We carefully design a framework for the detection of point and line feature correspondences. First, distinct feature points are detected based on an extended 1D SIFT, and line features are extracted via a modified Split-and-Merge algorithm. In this stage, we also give an effective strategy for discarding unreliable features. The point and line features are then described by a distance histogram; the pairs achieving best matching scores are accepted as potential correct correspondences. The histogram cluster technique is adapted to filter outliers and provide an accurate initial value of the rigid transformation. We also proposed a new relative pose estimation method that is robust to outliers. We use the lq-norm (0 < q < 1) metric in this approach, in contrast to classic optimization methods whose cost function is based on the l2-norm of residuals. Extensive experiments on real data demonstrate that the proposed method is almost as accurate as ICPs and is initial free. We also show that our scan matching method can be integrated into a simultaneous localization and mapping (SLAM) system for indoor mapping.

This thesis is concerned with robot vision. More precisely, we are interested in the integration of panoramic camera in visual servoing. Visual servoing consists in controlling a robotic system by using information extracted from images acquired by one or more cameras. Classical approaches are based on the regulation to zeros of an error function computed from the current and desired measurements. Using such approaches, it is not obvious to insure that the object of interest remains in the camera field of view. To resolve this dificiency, we propose to couple omnidirectional cameras to visual servoing. In order to design suitable control schemes, appropriate tools for geometrical and kinematical modelling of features extracted from the panoramic images have been developped. Dealing with geometrical modelling, relationships between imaged points and/or lines in two views have been exploited to estimate a generic homography matrix valid for the entire class of central cameras. The homography matrix can then be decomposed to achieve partial 3D reconstruction. Two class of visual servoing schemes with omnidirectional cameras have been explored, namely 2D visual servoing and hybrid visual servoing. In particular, a new hybrid approach allow to fully decouple rotational motions from translational motions. The proposed control schemes (2D and hybrid) had been validated throught extensive simulations as well as experimental results obtained with a mobile robot and a 6 -DOF robotic arm in eye-in-hand configuration.; Ce travail de thèse s'inscrit dans le domaine de la vision pour la robotique. Plus précisément, nous nous sommes intéressés à l'exploitation du champ de vu des caméras panoramiques dans le cadre de l'asservissement visuel. En effet, l'asservissement visuel consiste à intégrer des informations issues d'une ou de plusieurs caméras, dans la boucle de commande des systèmes robotiques. Les approches classiques d'asservissement visuel sont basées sur la régulation à zéro du signal d'erreur entre les mesures visuelles courantes et désirées. Lorsque la configuration souhaitée est très éloignée de la configuration initiale, les informations visuelles peuvent sortir du champ de vu de la caméra. L'utilisation d'une caméra panoramique permet naturellement de réduire les sorties de champ de vu. Nous proposons, à cette fin, des outils adéquats pour la modélisation géométrique et cinématique des informations extraites de l'image panoramique. Nous présentons dans un premier temps les méthodologies pour obtenir une transformation homographique à partir des images panoramiques d'un ensemble d'informations visuelles de type point et droite. Nous montrons également qu'il est possible d'utiliser les droites polaires associées aux droites projetées (des coniques) pour procéder à une simple estimation du point principal et de là à une reconstruction projective. Ensuite, nous présentons des schémas de commande qui utilisent des informations visuelles de natures 2D (asservissement visuel 2D) et mixte (asservissement visuel hybride). Les nouvelles approches hybrides que nous proposons permettent de découpler complètement les vitesses de translation de celles de rotation. Les schémas de commande proposés (2D et hybrides) ont été validés en simulation et sur un robot cartésien à 6 degrés de liberté. Nous présentons aussi une approche de commande de robot mobile non holonome pour le suivi de droite à partir des mesures extraites des images panoramiques.