Search

Estimating human motion from video is an active research area due to its many potential applications. Most state-of-the-art methods predict human shape and posture estimates for individual images and do not leverage the temporal information available in video. Many "in the wild" sequences of human motion are captured by a moving camera, which adds the complication of conflated camera and human motion to the estimation. We therefore present BodySLAM, a monocular SLAM system that jointly estimates the position, shape, and posture of human bodies, as well as the camera trajectory. We also introduce a novel human motion model to constrain sequential body postures and observe the scale of the scene. Through a series of experiments on video sequences of human motion captured by a moving monocular camera, we demonstrate that BodySLAM improves estimates of all human body parameters and camera poses when compared to estimating these separately.

In robotic applications, a key requirement for safe and efficient motion planning is the ability to map obstacle-free space in unknown, cluttered 3D environments. However, commodity-grade RGB-D cameras commonly used for sensing fail to register valid depth values on shiny, glossy, bright, or distant surfaces, leading to missing data in the map. To address this issue, we propose a framework leveraging probabilistic depth completion as an additional input for spatial mapping. We introduce a deep learning architecture providing uncertainty estimates for the depth completion of RGB-D images. Our pipeline exploits the inferred missing depth values and depth uncertainty to complement raw depth images and improve the speed and quality of free space mapping. Evaluations on synthetic data show that our approach maps significantly more correct free space with relatively low error when compared against using raw data alone in different indoor environments; thereby producing more complete maps that can be directly used for robotic navigation tasks. The performance of our framework is validated using real-world data.

With the aim of bridging the gap between high quality reconstruction and mobile robot motion planning, we propose an efficient system that leverages the concept of adaptive-resolution volumetric mapping, which naturally integrates with the hierarchical decomposition of space in an octree data structure. Instead of a Truncated Signed Distance Function (TSDF), we adopt mapping of occupancy probabilities in log-odds representation, which allows to represent both surfaces, as well as the entire free, i.e. observed space, as opposed to unobserved space. We introduce a method for choosing resolution -- on the fly -- in real-time by means of a multi-scale max-min pooling of the input depth image. The notion of explicit free space mapping paired with the spatial hierarchy in the data structure, as well as map resolution, allows for collision queries, as needed for robot motion planning, at unprecedented speed. We quantitatively evaluate mapping accuracy, memory, runtime performance, and planning performance showing improvements over the state of the art, particularly in cases requiring high resolution maps.

Dense image alignment from RGB-D images remains a critical issue for real-world applications, especially under challenging lighting conditions and in a wide baseline setting. In this paper, we propose a new framework to learn a pixel-wise deep feature map and a deep feature-metric uncertainty map predicted by a Convolutional Neural Network (CNN), which together formulate a deep probabilistic feature-metric residual of the two-view constraint that can be minimised using Gauss-Newton in a coarse-to-fine optimisation framework. Furthermore, our network predicts a deep initial pose for faster and more reliable convergence. The optimisation steps are differentiable and unrolled to train in an end-to-end fashion. Due to its probabilistic essence, our approach can easily couple with other residuals, where we show a combination with ICP. Experimental results demonstrate state-of-the-art performance on the TUM RGB-D dataset and 3D rigid object tracking dataset. We further demonstrate our method's robustness and convergence qualitatively.

In this work we introduce a new bounding-box free network (BBFNet) for panoptic segmentation. Panoptic segmentation is an ideal problem for a bounding-box free approach as it already requires per-pixel semantic class labels. We use this observation to exploit class boundaries from an off-the-shelf semantic segmentation network and refine them to predict instance labels. Towards this goal BBFNet predicts coarse watershed levels and use it to detect large instance candidates where boundaries are well defined. For smaller instances, whose boundaries are less reliable, BBFNet also predicts instance centers by means of Hough voting followed by mean-shift to reliably detect small objects. A novel triplet loss network helps merging fragmented instances while refining boundary pixels. Our approach is distinct from previous works in panoptic segmentation that rely on a combination of a semantic segmentation network with a computationally costly instance segmentation network based on bounding boxes, such as Mask R-CNN, to guide the prediction of instance labels using a Mixture-of-Expert (MoE) approach. We benchmark our non-MoE method on Cityscapes and Microsoft COCO datasets and show competitive performance with other MoE based approaches while outperfroming exisiting non-proposal based approaches. We achieve this while been computationally more efficient in terms of number of parameters and FLOPs. Video results are provided here https://blog.slamcore.com/reducing-the-cost-of-understanding.

Detailed 3D reconstruction is an important challenge with application to robotics, augmented and virtual reality, which has seen impressive progress throughout the past years. Advancements were driven by the availability of depth cameras (RGB-D), as well as increased compute power, e.g.\ in the form of GPUs -- but also thanks to inclusion of machine learning in the process. Here, we propose X-Section, an RGB-D 3D reconstruction approach that leverages deep learning to make object-level predictions about thicknesses that can be readily integrated into a volumetric multi-view fusion process, where we propose an extension to the popular KinectFusion approach. In essence, our method allows to complete shape in general indoor scenes behind what is sensed by the RGB-D camera, which may be crucial e.g.\ for robotic manipulation tasks or efficient scene exploration. Predicting object thicknesses rather than volumes allows us to work with comparably high spatial resolution without exploding memory and training data requirements on the employed Convolutional Neural Networks. In a series of qualitative and quantitative evaluations, we demonstrate how we accurately predict object thickness and reconstruct general 3D scenes containing multiple objects.

The Mohamed Bin Zayed International Robotics Challenge (MBZIRC) held in spring 2017 was a very successful competition well attended by teams from all over the world. One of the challenges (Challenge 1) required an aerial robot to detect, follow, and land on a moving target in a fully autonomous fashion. In this paper, we present the hardware components of the micro air vehicle (MAV) we built with off the self components alongside the designed algorithms that were developed for the purposes of the competition. We tackle the challenge of landing on a moving target by adopting a generic approach, rather than following one that is tailored to the MBZIRC Challenge 1 setup, enabling easy adaptation to a wider range of applications and targets, even indoors, since we do not rely on availability of global positioning system. We evaluate our system in an uncontrolled outdoor environment where our MAV successfully and consistently lands on a target moving at a speed of up to 5.0 m/s.

One of the main challenges for autonomous aerial robots is to land safely on a target position on varied surface structures in real‐world applications. Most of current aerial robots (especially multirotors) use only rigid landing gears, which limit the adaptability to environments and can cause damage to the sensitive cameras and other electronics onboard. This paper presents a bioinpsired landing system for autonomous aerial robots, built on the inspire–abstract–implement design paradigm and an additive manufacturing process for soft thermoplastic materials. This novel landing system consists of 3D printable Sarrus shock absorbers and soft landing pads which are integrated with an one‐degree‐of‐freedom actuation mechanism. Both designs of the Sarrus shock absorber and the soft landing pad are analyzed via finite element analysis, and are characterized with dynamic mechanical measurements. The landing system with 3D printed soft components is characterized by completing landing tests on flat, convex, and concave steel structures and grassy field in a total of 60 times at different speeds between 1 and 2 m/s. The adaptability and shock absorption capacity of the proposed landing system is then evaluated and benchmarked against rigid legs. It reveals that the system is able to adapt to varied surface structures and reduce impact force by 540N at maximum. The bioinspired landing strategy presented in this paper opens a promising avenue in Aerial Biorobotics, where a cross‐disciplinary approach in vehicle control and navigation is combined with soft technologies, enabled with adaptive morphology.

Datasets have gained an enormous amount of popularity in the computer vision community, from training and evaluation of Deep Learning-based methods to benchmarking Simultaneous Localization and Mapping (SLAM). Without a doubt, synthetic imagery bears a vast potential due to scalability in terms of amounts of data obtainable without tedious manual ground truth annotations or measurements. Here, we present a dataset with the aim of providing a higher degree of photo-realism, larger scale, more variability as well as serving a wider range of purposes compared to existing datasets. Our dataset leverages the availability of millions of professional interior designs and millions of production-level furniture and object assets -- all coming with fine geometric details and high-resolution texture. We render high-resolution and high frame-rate video sequences following realistic trajectories while supporting various camera types as well as providing inertial measurements. Together with the release of the dataset, we will make executable program of our interactive simulator software as well as our renderer available at https://interiornetdataset.github.io. To showcase the usability and uniqueness of our dataset, we show benchmarking results of both sparse and dense SLAM algorithms., British Machine Vision Conference (BMVC) 2018

Incorporating the physical environment is essential for a complete understanding of human behavior in unconstrained every-day tasks. This is especially important in ego-centric tasks where obtaining 3 dimensional information is both limiting and challenging with the current 2D video analysis methods proving insufficient. Here we demonstrate a proof-of-concept system which provides real-time 3D mapping and semantic labeling of the local environment from an ego-centric RGB-D video-stream with 3D gaze point estimation from head mounted eye tracking glasses. We augment existing work in Semantic Simultaneous Localization And Mapping (Semantic SLAM) with collected gaze vectors. Our system can then find and track objects both inside and outside the user field-of-view in 3D from multiple perspectives with reasonable accuracy. We validate our concept by producing a semantic map from images of the NYUv2 dataset while simultaneously estimating gaze position and gaze classes from recorded gaze data of the dataset images.

We present the development process behind AtlantikSolar, a small 6.9 kg hand-launchable low-altitude solar-powered unmanned aerial vehicle (UAV) that recently completed an 81-hour continuous flight and thereby established a new flight endurance world record for all aircraft below 50 kg mass. The goal of our work is to increase the usability of such solar-powered robotic aircraft by maximizing their perpetual flight robustness to meteorological deteriorations such as clouds or winds. We present energetic system models and a design methodology, implement them in our publicly available conceptual design framework for perpetual flight-capable solar-powered UAVs, and finally apply the framework to the AtlantikSolar UAV. We present the detailed AtlantikSolar characteristics as a practical design example. Airframe, avionics, hardware, state estimation, and control method development for autonomous flight operations are described. Flight data are used to validate the conceptual design framework. Flight results from the continuous 81-hour and 2,338 km covered ground distance flight show that AtlantikSolar achieves 39% minimum state-of-charge, 6.8 h excess time and 6.2 h charge margin. These performance metrics are a significant improvement over previous solar-powered UAVs. A performance outlook shows that AtlantikSolar allows perpetual flight in a 6-month window around June 21 at mid-European latitudes, and that multi-day flights with small optical- or infrared-camera payloads are possible for the first time. The demonstrated performance represents the current state-of-the-art in solar-powered low-altitude perpetual flight performance. We conclude with lessons learned from the three-year AtlantikSolar UAV development process and with a sensitivity analysis that identifies the most promising technological areas for future solar-powered UAV performance improvements.

Intensive research on the recent foraminifera in the Gulf of Aqaba- Elat and their physico-chemical environment is being carried out for the last decade by groups of specialists from the Universities of Jerusalem, Basel, and Copenhagen. The results obtained hitherto indicate that this part of the northern Red Sea may represent a model-area of prime interest for a better understanding of foraminiferal distribution and, therefore, for paleontological interpretations of fossil faunas. The involvement in the study of this model area of the Utrecht University group is based on the latter's participation in the International Geological Correlation Program, Project 73/1/1 "Accuracy in Time". Since many biostratigraphical correlations are based on the evolution of measurable parameters of larger foraminiferal species groups, an investigation of possibly environment-dependent morphological changes in living Foraminifera was urgently needed. A recently advanced theory that proloculus-size might increase at greater water-depth was one such topic. Since the thermo-, halo-, and pycnoclines in the Gulf of Elat are rather insignificant, changes with depth in morphology and distribution of foraminiferal species must be governed by depth-related factors other than temperature, salinity or density. Direct or indirect influence of light penetration and types of substrate are thought to control in the Gulf the depthdistribution of the Soritidae and of the investigated species of Borelis, Amphistegina, Planorbulinella, Operculina, Heterostegina, and Heterocylina. Light may equally influence the intraspecific variation in several of these groups. Major changes seem to take place in the uppermost 80 meters of the depth-profiles. For the LG.C.P. project satisfactory estimates can be made now of the range of variation that mean values of such "evolutionary" features as protoconch diameter and nepionic arrangement may show at one time level, as a consequence of environmental control. The volume summarizes the present-day knowledge on the distribution and morphology of the larger Foraminifera of the Gulf, on their reproduction cycles, symbionts, and their possibly symbiont- and depth-related variability in stable oxygen-isotope ratios.

This paper introduces a state estimation framework for legged robots that allows estimating the full pose of the robot without making any assumptions about the geometrical structure of its environment. This is achieved by means of an Observability Constrained Extended Kalman Filter that fuses kinematic encoder data with on-board IMU measurements. By including the absolute position of all footholds into the filter state, simple model equations can be formulated which accurately capture the uncertainties associated with the intermittent ground contacts. The resulting filter simultaneously estimates the position of all footholds and the pose of the main body. In the algorithmic formulation, special attention is paid to the consistency of the linearized filter: it maintains the same observability properties as the nonlinear system, which is a prerequisite for accurate state estimation. The presented approach is implemented in simulation and validated experimentally on an actual quadrupedal robot.