Search

Zusammenfassung In dieser Arbeit wird eine effiziente Methode zur Schätzung der Ego-Fahrstreifen vorgestellt. Das System ist ausgelegt auf die Anwendung für Mikromobilitätssysteme wie E-Fahrräder, E-Scooter oder E-Dreiräder und basiert auf Sequenzen von RGB-Bildern, die aus der Fahrerperspektive aufgenommen werden. Um die Grenzen verschiedener Fahrbahntypen, insbesondere Fußwegen, Radwegen und Straßen, robust zu detektieren, wird ein gradientenbasierter Kantendetektor mit einem Farbsegmentierungansatz kombiniert. Sowohl geometrische Gegebenheiten als auch zeitliche Inferenz werden genutzt, um die Schätzung zu optimieren, insbesondere in schwierigen Fällen wie Kurven. Dabei ist insbesondere die dynamische Schätzung des Fluchtpunkts mithilfe des Optischen Flusses zwischen zwei aufeinanderfolgenden Bildern von Bedeutung. Tests mit zwei Aufnahmesetups und drei verschiedenen Aufrahmeraten zeigen, dass der Ego-Fahrstreifen in den meisten Fällen mit hoher Genauigkeit detektiert und in Kurven passend approximiert wird. Dies wird von der quantitative Evaluation anhand von über 2000 Testbildern bestätigt, die eine IoU von 75,28 % ergibt. Eine Performanz von 12 fps auf einem Raspberry Pi 3 zeigt zudem, dass die Methode wenig rechenintensiv ist, was als Vorraussetzung für die Anwendung in der Mikromobilität gilt.

This paper presents an incremental stereo algorithm designed to calculate a real-time disparity image. The algorithm is designed for stereo video sequences and uses previous information to reduce computation time and improve disparity image quality. It is based on the semi-global matching stereo algorithm but modified to reuse previous calculation information. Storing and reusing this information not only reduces computation time but improves accuracy in a cost filtering scheme. Some tests are presented to compare the computation time and results of the algorithm, which show that it can achieve better results in terms of quality and time than standard algorithms for some scenarios.

Visual tracking of multiple objects is an essential component for a perception system in autonomous driving vehicles. One of the favorable approaches is the tracking-by-detection paradigm, which links current detection hypotheses to previously estimated object trajectories (also known as tracks) by searching appearance or motion similarities between them. As this search operation is usually based on a very limited spatial or temporal locality, the association can fail in cases of motion noise or long-term occlusion. In this paper, we propose a novel tracking method that solves this problem by putting together information from both enlarged structural and temporal domain. For efficiency without loss of optimality, this approach is decomposed in to three stages, with each dealing with only one constrained association task, and thus, it follows the alternating optimization fashion. In our approach, detections are first assembled into small tracklets based on meta-measurements of object affinity. The association task for tracklets-to-tracks is solved by structural information based on a motion pattern between them. Here, we propose new rules to decouple the processing time from the tracklet length. Furthermore, constraints from temporal domain are introduced to recover objects, which are long-time disappearing due to failed detection or long-term occlusion. By putting together the heterogeneous domain information, our approach exhibits an improved state-of-the-art performance on standard benchmarks. With relatively little processing time, an online and real-time tracking is also permitted in our approach.

Cuboid objects are widely used in logistics and manufacturing, since they are ideal for automatic manipulation. For that, a reliable detection, identification and tracking algorithm of cuboid objects is a key requirement. RGBD cameras are suited for these tasks, however they suffer from limitations such as occlusions from clutter or a small field of view. A multi RGBD camera setup would remedy many of these limitations. In this work, we present a cuboid detection, identification and tracking system using a multi RGBD camera setup. First, we calibrate our multi camera setup using a robot arm and a colored spherical calibration target. By using spatial and color information, a preprocessing pipeline segments then the unified point cloud into locally connected planar segments. Afterwards, custom spatial criteria allows for assigning the resulting segments to cuboid groups. We present a new modified bounding box regression method to estimate cuboid boxes and a novel intersection over union based approach to track cuboids across frames. By applying our approach, the automatic manipulation and transportation in manufacturing industry will become more efficient, which helps to save more time and cost. Our system is tested and analyzed with a recorded dataset, which contents variant constellations. The reliability and accuracy are demonstrated.

Estimating motion and shape of surrounding objects reliably and accurately is a fundamental challenge in the study of interactions between cooperative traffic participants. This paper proposes a new approach that utilizes free space information obtained from a LiDAR sensor and object local grid maps in order to simultaneously estimate the shape and movement state of objects with arbitrary shape. We evaluated our approach in several simulated scenarios and found that the movement and shape estimation results are very close to the ground truth. Finally, we did a qualitative evaluation on real data extracted from the KITTI benchmark.

Despite the fact that in recent years, vision-based tracking approaches have made significant progress, the task of tracking vehicles at night still remains challenging. Visual information is strongly deteriorated or at least degraded due to poor illumination conditions. This reduces the perceptive ability of vision systems significantly and can even lead to target loss, resulting in false estimation and/or false prediction of object behavior. In this paper, we propose a novel online-learning method to track vehicles at night. Our method is based on the kernelized correlation filter and assembles different feature channels to kernelized experts. By estimating their reliabilities, we force the appearance model to focus on the most discriminative visual features to accomplish the classification. In addition, a temporal optimization step in conjunction with a memory model is used to remove outliers and keep the most reliable samples to train the tracker models. Experiments over various daytime and weather conditions show that our approach outperforms existing trackers at night and in case of bad weather while offering state-of-the-art performance in more favorable situations. As our tracker has only little computational cost, it is appropriate for use cases with real-time requirements like in automotive or industrial applications.

The number of visually impaired people is constantly increasing. Mobility aids such as white canes are designed to help these people navigate in their everyday life. However, these systems have limitations in certain environments or activities. To meet this challenge, we propose an assistance system to support the navigation of visually impaired people in low structured environments. The system captures information from its environment with a stereo camera. The disparity and color images are processed by an embedded computer which calculates a safe walking direction and accordingly provides vibrotactile feedback to the visually impaired person. This paper presents an approach to segment the traversable path in the captured color image of low structured environments using artificial neural networks. Therefore, a dataset is generated and several encoder-decoder architectures are evaluated and optimized to achieve a sufficient accuracy and real-time framerate of the binary segmentation on a mobile computer. The optimized architecture manages to segment the path with a test Intersection over Union value of 0.9378 and a segmentation time of 96 ms per image in various low structured environments.

In this work, we present a visual pose regression network: ViPNet. It is robust and real-time capable on mobile platforms such as self-driving vehicles. We train a convolutional neural network to estimate the six degrees of freedom camera pose from a single monocular image in an end-to-end manner. In order to estimate camera poses with uncertainty, we use a Bayesian version of the ResNet-50 as our basic network. SEBlocks are applied in residual units to increase our model’s sensitivity to informative features. Our ViPNet is trained using a geometric loss function with trainable parameters, which can simplify the fine-tuning process significantly. We evaluate our ViPNet on the Cambridge Landmarks dataset and also on our Karl-Wilhelm-Plaza dataset, which is recorded with an experimental vehicle. As evaluation results, our ViPNet outperforms other end-to-end monocular camera pose estimation methods. Our ViPNet requires only 9-15ms to predict one camera pose, which allows us to run it with a very high frequency.

In this work, we present an uniform B-spline based continuous fusion approach, which fuses the motion data from an inertial measurement unit and the pose data from a visual localization system accurately, efficiently and continu-ously. Currently, in the domain of robotics and autonomous driving, most of the ego motion fusion approaches are filter based or pose graph based. By using the filter based approaches like the Kalman Filter or the Particle Filter, usually, many parameters should be set carefully, which is a big overhead. Besides that, the filter based approaches can only fuse data in a time forwards direction, which is a big disadvantage in processing async data. Since the pose graph based approaches only fuse the pose data, the inertial measurement unit data should be integrated to estimate the corresponding pose data firstly, which can however bring accumulated error into the fusion system. Additionally, the filter based approaches and the pose graph based approaches only provide discrete fusion results, which may decrease the accuracy of the data processing steps afterwards. Since the fusion approach is generally needed for robots and automated driving vehicles, it is a major goal to make it more accurate, robust, efficient and continuous. Therefore, in this work, we address this problem and apply the axis-angle rotation representation method, the Rodrigues’ formula and the uniform B-spline implementation to solve the ego motion fusion problem continuously. Evaluation results performed on the real world data show that our approach provides accurate, robust and continuous fusion results.

There is an ever increasing demand for semantically annotated images to train and evaluate image classifiers. Currently this data is obtained by manually labeling each individual image thus making the process labor intensive and costly. We therefore present an approach that generates semantically annotated images for classes road and curb in a fully automatic way. The advantage of our method is that it relies on cues from range sensors only thus making its output suitable for training and evaluating image classifiers. We use a normal based curb detector and extract the road by running an active contour on these detections. Since our algorithm does not rely on parametric models it is possible to detect a wide range of road geometries in different environments. Sequences of up to a minute can be accurately labeled without any user interference in less than a minute.

The Vision Meets Drone Multiple Object Tracking (MOT) Challenge 2019 is the second annual activity focusing on evaluating multi-object tracking algorithms on drones, held in conjunction with the 17-th International Conference on Computer Vision (ICCV 2019). Results of 12 submitted MOT algorithms on the collected drone-based dataset are presented. Meanwhile, we also report the results of 6 state-of-the-art MOT algorithms, and provide a comprehensive analysis and discussion of the results. The results of all submissions are publicly available at the website: http://www.aiskyeye.com/. The challenge results show that MOT on drones is far from being solved. We believe the challenge can largely boost the research and development in MOT on drone platforms.

Vision-based object tracking approach has been drawing increased attention from both the academia and the industry in recent years. One of its most successful applications is the monitoring system, which can be installed on infrastructures or on mobile platforms (e.g., vehicles) to track pedestrians or bicyclists in a pre-defined region and further to prevent probable accidents. Despite tremendous progress achieved, the task of visual tracking is still challenging, especially in dealing with severe occlusions, where the tracker may fail due to the abrupt change of object appearance. Such case is common not only in traffic scenarios but also in other tracking tasks. Aiming to tackle this problem, in this paper, we propose a new tracking approach by adopting part based trackers, which are built in the form of correlation filter. In this approach, the occluded object parts are identified by leveraging the knowledge derived from image features and filter responses. With the help of a masking process, visible object areas are acquired in a pixel-wise precision and utilized to build part filters. As both the number and size of part filters are adapted to the current object appearance, the influence of occlusions can be significantly suppressed. Experimental results on traffic sequences demonstrate that our tracker performs robust against occlusion, especially in cases, where long term and severe occlusions appear. A further experiment on the standard benchmark proves that our approach outperforms state-of-the-art methods in tracking various object classes under varied circumstances. Furthermore, the proposed tracker is sophisticatedly designed and is feasible for real time applications.

Highly accurate localization with very limited amount of memory and computational power is one of the big challenges for next generation series cars. We propose localization based on geometric primitives which are compact in representation and further valuable for other tasks like planning and behavior generation. The primitives lack distinctive signature which makes association between detections and map elements highly ambiguous. We resolve ambiguities early in the pipeline by online building up a local map which is key to runtime efficiency. Further, we introduce a new framework to fuse association and odometry measurements based on robust pose graph optimization.We evaluate our localization framework on over 30 min of data recorded in urban scenarios. Our map is memory efficient with less than 8 kB/km and we achieve high localization accuracy with a mean position error of less than 10 cm and a mean yaw angle error of less than 0. 25° at a localization update rate of 50Hz.

We propose a deep convolutional object detector for automated driving applications that also estimates classification, pose and shape uncertainty of each detected object. The input consists of a multi-layer grid map which is well-suited for sensor fusion, free-space estimation and machine learning. Based on the estimated pose and shape uncertainty we approximate object hulls with bounded collision probability which we find helpful for subsequent trajectory planning tasks. We train our models based on the KITTI object detection data set. In a quantitative and qualitative evaluation some models show a similar performance and superior robustness compared to previously developed object detectors. However, our evaluation also points to undesired data set properties which should be addressed when training data-driven models or creating new data sets., 8 pages, 8 figures, 2 tables

Single-object tracking, also known as visual tracking, on the drone platform attracts much attention recently with various applications in computer vision, such as filming and surveillance. However, the lack of commonly accepted annotated datasets and standard evaluation platform prevent the developments of algorithms. To address this issue, the Vision Meets Drone Single-Object Tracking (VisDrone-SOT2018) Challenge workshop was organized in conjunction with the 15th European Conference on Computer Vision (ECCV 2018) to track and advance the technologies in such field. Specifically, we collect a dataset, including 132 video sequences divided into three non-overlapping sets, i.e., training (86 sequences with 69, 941 frames), validation (11 sequences with 7,046 frames), and testing (35 sequences with 29, 367 frames) sets. We provide fully annotated bounding boxes of the targets as well as several useful attributes, e.g., occlusion, background clutter, and camera motion. The tracking targets in these sequences include pedestrians, cars, buses, and animals. The dataset is extremely challenging due to various factors, such as occlusion, large scale, pose variation, and fast motion. We present the evaluation protocol of the VisDrone-SOT2018 challenge and the results of a comparison of 22 trackers on the benchmark dataset, which are publicly available on the challenge website: http://www.aiskyeye.com/. We hope this challenge largely boosts the research and development in single object tracking on drone platforms Refereed/Peer-reviewed

Video object detection has drawn great attention recently. The Vision Meets Drone Object Detection in Video Challenge 2019 (VisDrone-VID2019) is held to advance the state-of-the-art in video object detection for videos captured by drones. Specifically, there are 13 teams participating the challenge. We also report the results of 6 state-of-the-art detectors on the collected dataset. A short description is provided in the appendix for each participating detector. We present the analysis and discussion of the challenge results. Both the dataset and the challenge results are publicly available at the challenge website: http://www.aiskyeye.com/.

In this work we present a novel approach for multi-sensor calibration that significantly outperforms current state-of-the-art. We introduce a new spherical calibration target which has major benefits over existing targets. Those are subresolution detection accuracy in both camera and depth sensor, view invariance and applicability to a wider range of sensor setups than current approaches. With our method a single person achieves high quality calibration in less than a minute. No preparations for setting up the environment for calibration is needed. Our method is fast, easy to use and fully automatic. We evaluate our method in simulation and show high accuracy with an error of less than 3mm in translation and 0.1 0 in rotation on real data.

This article investigates in which way a lidar sensor can be used in a train-borne localization system. The idea is to sense infrastructure elements like rails and turnouts with the lidar sensor and to recognize those objects with a template-matching approach. A requirement analysis for the lidar sensor is presented and a market review based on these requirements is performed. Furthermore, an approach for template matching on lidar scans to recognize infrastructure objects is introduced and its empirical performance is demonstrated based on measurements taken in a light rail environment. The overall goal of the integration of lidar sensors is to fill the sensory gap of existing train localization approaches, which are able to determine the exact, track-selective train position only if highly accurate position measurements from satellite navigation systems are available, which is often not the case. By integrating a lidar sensor, the localization system becomes more diverse, more robust, and can tolerate missing or faulty measurements from the satellite navigation system.

Higher level functionality in autonomous driving depends strongly on a precise motion estimate of the vehicle. Powerful algorithms have been developed. However, their great majority focuses on either binocular imagery or pure LIDAR measurements. The promising combination of camera and LIDAR for visual localization has mostly been unattended. In this work we fill this gap, by proposing a depth extraction algorithm from LIDAR measurements for camera feature tracks and estimating motion by robustified keyframe based Bundle Adjustment. Semantic labeling is used for outlier rejection and weighting of vegetation landmarks. The capability of this sensor combination is demonstrated on the competitive KITTI dataset, achieving a placement among the top 15. The code is released to the community., Accepted at IROS 2018

Trajectory and intention prediction of traffic participants is an important task in automated driving and crucial for safe interaction with the environment. In this paper, we present a new approach to vehicle trajectory prediction based on automatically generated maps containing statistical information about the behavior of traffic participants in a given area. These maps are generated based on trajectory observations using image processing and map matching techniques and contain all typical vehicle movements and probabilities in the considered area. Our prediction approach matches an observed trajectory to a behavior contained in the map and uses this information to generate a prediction. We evaluated our approach on a dataset containing over 14000 trajectories and found that it produces significantly more precise mid-term predictions compared to motion model-based prediction approaches., 6 Pages, 9 figures, submitted to IV 2018

Vision systems become more and more popular to be applied in monitoring tasks such as controlling traffic flows or for security issues. The analysis of target behavior is always based on its observed trajectory, which can be acquired by tracking approaches. Although the fashion of tracking-by-detection is favored by the research community, it still faces challenges like unexpected occlusion caused by background objects or other tracked targets, which can interfere the matching operation and result in tracking errors. In this paper, we propose a novel approach by aggregating prediction events within target groups and integrating a graph-modeling based stitching procedure to handle the above mentioned problems. The evaluation results on the UA-DETRAC benchmark demonstrated the state-of-the-art performance of our tracking approach.

Knowledge about the location of a vehicle is indispensable for autonomous driving. In order to apply global localisation methods, a pose prior must be known which can be obtained from visual odometry. The quality and robustness of that prior determine the success of localisation. Momo is a monocular frame-to-frame motion estimation methodology providing a high quality visual odometry for that purpose. By taking into account the motion model of the vehicle, reliability and accuracy of the pose prior are significantly improved. We show that especially in low-structure environments Momo outperforms the state of the art. Moreover, the method is designed so that multiple cameras with or without overlap can be integrated. The evaluation on the KITTI-dataset and on a proper multi-camera dataset shows that even with only 100--300 feature matches the prior is estimated with high accuracy and in real-time.

The visual based tracking has been an active area among the research community in recent years. Its application can be broadly found in intelligent transportation systems like advanced driver assistance systems (ADAS) or autonomous vehicles. One objective is to monitor the trajectory and behavior of the vulnerable road users (VRUs), e.g., pedestrians and cyclists, to prevent the probable collisions. However, the tracking of VRUs is still challenging, especially in cases, where severe occlusion occurs and the tracker fails due to the abrupt change of object appearance. To tackle this problem we propose a new tracking approach leveraging the part based trackers, which are built based on correlation filters. In this method both the number and the size of part filters are adapted to the current appearance of the object, to eliminate the influence by occluded parts. Experimental results show that our tracker performs robust with respect to occlusions, especially in cases, where long term and severe occlusions appear. Due to a sophisticated design, a real time processing performance can also be achieved by our approach.

Stereo cameras are among the most promising sensors for automated driving. For their deployment, however, calibration should be automated and possible in-situ. We propose a restructuring of bundle adjustment into an incremental online calibration system. It allows us to estimate all observable camera parameters on the fly. Both simulations and experiments with real world cameras show its capability to calibrate stereo rigs in real time while driving. With this method, cameras can be employed with almost no calibration overhead. Only the non-observable parameter of scale has to be defined in advance.

Intelligent vehicles heavily rely on robust and accurate self-localization. Global navigation satellite systems (GNSS) are not reliable in urban environments due to multipath and shadowing effects. Vision-based localization offers a promising alternative. We present a high-precision six degrees of freedom self-localization method using multiple cameras covering the surrounding environment. First, a point feature map is created using images from a previous pass of the area to map. Thereafter, the map is used for high-precision localization in real-time. While localization, a rough prior estimate of the current pose is used to shrink the search space for feature matching by projecting mapped landmarks into current images. Then, stored observations of the projected landmarks are matched to actual observations and the egopose is estimated by back-projection error minimization. Thereby, our map structure provides mapped landmarks efficiently towards localization with multiple cameras. In real-world experiments we show that our approach provides reliable localization results while passing the mapped area in arbitrary orientation.

We present an improved model for MRF-based depth upsampling, guided by image- as well as 3D surface normal features. By exploiting the underlying camera model we define a novel regularization term that implicitly evaluates the planarity of arbitrary oriented surfaces. Our method improves upsampling quality in scenes composed of predominantly planar surfaces, such as urban areas. We use a synthetic dataset to demonstrate that our approach outperforms recent methods that implement distance-based regularization terms. Finally, we validate our approach for mapping applications on our experimental vehicle., Comment: 6 pages, 6 figures

The rapid advances of transportation infrastructure have led to a dramatic increase in the demand for smart systems capable of monitoring traffic and street safety. Fundamental to these applications are a community-based evaluation platform and benchmark for object detection and multi-object tracking. To this end, we organize the AVSS2017 Challenge on Advanced Traffic Monitoring, in conjunction with the International Workshop on Traffic and Street Surveillance for Safety and Security (IWT4S), to evaluate the state-of-the-art object detection and multi-object tracking algorithms in the relevance of traffic surveillance. Submitted algorithms are evaluated using the large-scale UA-DETRAC benchmark and evaluation protocol. The benchmark, the evaluation toolkit and the algorithm performance are publicly available from the website http://detrac-db.rit.albany.edu.

Human detection is an important task for many autonomous robots as well as automated driving systems. The Regionlets detector was one of the best-performing approaches for pedestrian detection on the KITTI dataset during 2015. We analysed the Regionlets detector and its performance. This paper discusses the improvements in accuracy that were achieved by the different ideas of the Regionlets detector. It also analyses what the boosting algorithm learns and how this relates to the expectations. We found that the random generation of regionlet configurations can be replaced by a regular grid of regionlets. Doing so reduces the dimensionality of the feature space drastically but does not decrease detection performance. This translates into a decrease in memory consumption and computing time during training.

In modern automated systems the usage of sensors with orthogonal measurement principles is indispensable for assuring a redundant system. The combination of laser scanners and cameras is particularly prevalent. To profit from the potential of fusing information from these sensors, the basic step is sensor registration. State of the art cross-calibration methods for laser scanners and cameras establish geometric correspondences for example by detecting checker-boards in size and form. However these approaches suffer from ambivalent features and are only usable if the laser scanner has a high scanning density. Therefore, we present a reconstruction-free and vision-based extrinsic calibration algorithm with distinct features, which was developed with special focus on low-density laser scanners and high measurement noise. Moreover, we contribute an evaluation methodology demonstrating the capability of our approach. Along with this paper, we publish the associated code.

Similar to autonomous vehicles, future train applications require an accurate on-board self-localization for railway vehicles. Therefore, a reliable and real-time capable environment perception is required. In particular, the knowledge of the track taken at a turnout overcomes ambiguities in self-localization. As the most important groundwork for this, the paper introduces a new approach for the detection of rails and tracks solely from 2d lidar measurements. The technique is based on a new feature point method for lidar data, a template matching approach, and a spatial clustering technique to extract rails and tracks from the detected rail elements. The new approach is evaluated on six different datasets taken outdoors at a demanding test ground. It provides reliable and accurate detection results with centimeter accuracy, a recall of about 90 %, and a precision of about 95 %. The approach is able to detect rails even in complex real-world topologies such as at turnouts and even on tracks with more than two rails.

We present a wearable assistance system for visually impaired persons that perceives the environment with a stereo camera and communicates obstacles and other objects to the user. We develop our idea of combining perception on an increased level of scene understanding with acoustic feedback to obtain an intuitive mobility aid. We describe our core techniques of scene modelling, object tracking, and acoustic feedback and show in an experimental study how our system can help improving the mobility and safety of visually impaired users.

We present a wearable assistance system for visually impaired persons that perceives the environment with a stereo camera system and communicates obstacles and other objects to the user in form of intuitive acoustic feedback. The system is designed to complement traditional assistance aids. We describe the core techniques of scene understanding, head tracking, and sonification and show in an experimental study that it enables users to walk in unknown urban terrain and to avoid obstacles safely.

This paper presents a novel technique of stereo vision based on the combination of an omnidirectional camera and a perspective camera. The technique combines the 360° field of view of the omnidirectional camera with the long field of view of a perspective camera. We describe the setup of such a camera system and how it can be used to achieve 3D-position estimates. Furthermore, we develop a maximum likelihood approach and a Bayesian approach that are able to fuse monocular and binocular observations of the same object to estimate its position and movement and show how this technique can be applied successfully in the RoboCup MiddleSizeLeague.

In this paper, we present a novel probabilistic generative model for multi-object traffic scene understanding from movable platforms which reasons jointly about the 3D scene layout as well as the location and orientation of objects in the scene. In particular, the scene topology, geometry, and traffic activities are inferred from short video sequences. Inspired by the impressive driving capabilities of humans, our model does not rely on GPS, lidar, or map knowledge. Instead, it takes advantage of a diverse set of visual cues in the form of vehicle tracklets, vanishing points, semantic scene labels, scene flow, and occupancy grids. For each of these cues, we propose likelihood functions that are integrated into a probabilistic generative model. We learn all model parameters from training data using contrastive divergence. Experiments conducted on videos of 113 representative intersections show that our approach successfully infers the correct layout in a variety of very challenging scenarios. To evaluate the importance of each feature cue, experiments using different feature combinations are conducted. Furthermore, we show how by employing context derived from the proposed method we are able to improve over the state-of-the-art in terms of object detection and object orientation estimation in challenging and cluttered urban environments.

Environment perception is an important task in computer vision for many applications in robotics. Especially for robots navigating through different levels of a building, stair detection constitutes an important perception task. In this paper, we propose a stair detection algorithm using range data. Firstly, we introduce a parameter, which describes local surface orientations w.r.t. a global reference. Secondly, a matched filter is used to detect relevant edges in the orientation data. Afterwards, line segments are determined using these edge data which are further used to estimate stairs. The proposed method is invariant against rotations of the sensor. We show that the system can handle typical outdoor stair types and outperforms the accuracy of state-of-the-art stair detection methods. Moreover, the method is used in real time to assist visually impaired people who wear the camera system on a helmet.

In this paper we present a vision-based detection system for cyclists. We build cascaded detectors with different classifiers and shared features to detect cyclists from multiple viewpoints. To improve the performance, we reveal the dependence between the size and the position of an object in the image by a regression method. We also explore the applications of this geometric constraint with different camera setups. Based on experiments we demonstrate that our detector is suitable for real time applications.

While monocular visual odometry has been widely investigated, one of its key issues restrains its broad appliance: the scale drift. To tackle it, we leverage scene inherent information about the ground plane to estimate the scale for usage on Advanced Driver Assistance Systems. The algorithm is conceived so that it is independent of the unscaled ego-motion estimation, augmenting its adaptability to other frameworks. A ground plane estimation using Structure From Motion techniques is complemented by a vanishing point estimation to render our algorithm robust in urban scenarios. The method is evaluated on the KITTI dataset, outperforming state of the art algorithms in areas where urban scenery is dominant.

Estimating the ground plane is often one of the first steps in geometric reasoning processes as it offers easily accessible context knowledge. Especially unconstrained platforms that capture video from egocentric viewpoints can benefit from such knowledge in various ways. A key requirement here is keeping orientation, which can be greatly achieved by keeping track of the ground. We present an approach to keep track of the ground plane in cluttered inner-urban environments using stereo vision in real-time. We fuse a planar model fit in low-resolution disparity data with the direction of the vertical vanishing point. Our experiments show how this effectively decreases the error of plane attitude estimation compared to classic least-squares fitting and allows to track the plane with camera configurations in which the ground is not visible. We evaluate the approach using ground-truth from an inertial measurement unit and demonstrate long-term stability on a dataset of challenging inner city scenes.

Novel safety systems are needed to meet the growing demand of railway operation. In this paper we introduce general techniques for the detection of tracks and their components in 3D laser scanning data. These techniques make use of feature based methods, such as support vector machines, as well as model based methods, such as template matching. The focus of this work are robust and precise detectors for infrastructure elements, such as rails, tracks, closure rails, and frogs. These parts can be used for both, track maintenance and train-borne localization. The approach is evaluated experimentally on 3D laser scanning data and compared with a reference system. Furthermore, the approach is generic such that it can be used for data of any suitable laser scanning system.

We investigate the problem of estimating the local geometric scene structure of urban street canyons captured from an egocentric viewpoint with a small-baseline stereo camera setup. We model the facades of buildings as planar surfaces and estimate their parameters based on a dense disparity map as only input. After demonstrating the importance of considering the stereo reconstruction uncertainties, we present two approaches to solve this model-fitting problem. The first approach is based on robust planar segmentation using random sampling, the second approach transforms the disparity into an elevation map from which the main building orientations can be obtained. We evaluate both approaches on a set of challenging inner city scenes and show how visual odometry can be incorporated to keep track of the estimated geometry in real-time.

3D scene understanding is key for the success of applications such as autonomous driving and robot navigation. However, existing approaches either produce a mild level of understanding, e.g., segmentation, object detection, or are not accurate enough for these applications, e.g., 3D pop-ups. In this paper we propose a principled generative model of 3D urban scenes that takes into account dependencies between static and dynamic features. We derive a reversible jump MCMC scheme that is able to infer the geometric (e.g., street orientation) and topological (e.g., number of intersecting streets) properties of the scene layout, as well as the semantic activities occurring in the scene, e.g., traffic situations at an intersection. Furthermore, we show that this global level of understanding provides the context necessary to disambiguate current state-of-the-art detectors. We demonstrate the effectiveness of our approach on a dataset composed of short stereo video sequences of 113 different scenes captured by a car driving around a mid-size city.

We propose an algorithm to estimate the ego-motion of an omnidirectional robot based on a sequence of position estimates. Thereto, we derive a motion model for omnidirectional robots and an estimation procedure to fit the model to observed positions. Additionally, we show how we can benefit from the velocity estimates deriving an algorithm that recognizes situations in which a robot is blocked by an obstacle.

In this paper we discuss the topic of modeling a moving object in a robot application considering as example the ball movement in a robot soccer environment. The paper focuses on the question of how to estimate the ball velocity reliably even if the ball collides with an obstacle or the observing robot is moving itself. We propose a new estimation algorithm based on a direct estimate using ridge regression and show its performance in reality.