Search

In this paper we introduce a method for multi-class, monocular 3D object detection from a single RGB image, which exploits a novel disentangling transformation and a novel, self-supervised confidence estimation method for predicted 3D bounding boxes. The proposed disentangling transformation isolates the contribution made by different groups of parameters to a given loss, without changing its nature. This brings two advantages: i) it simplifies the training dynamics in the presence of losses with complex interactions of parameters; and ii) it allows us to avoid the issue of balancing independent regression terms. We further apply this disentangling transformation to another novel, signed Intersection-over-Union criterion-driven loss for improving 2D detection results. We also critically review the AP metric used in KITTI3D and resolve a flaw which affected and biased all previously published results on monocular 3D detection. Our improved metric is now used as official KITTI3D metric. We provide extensive experimental evaluations and ablation studies on the KITTI3D and nuScenes datasets, setting new state-of-the-art results. We provide additional results on all the classes of KITTI3D as well as nuScenes datasets to further validate the robustness of our method, demonstrating its ability to generalize for different types of objects.

This thesis explores several practical applications of computer vision using learning based techniques.We begin by exploring the task of single image camera calibration. That is, the prediction of a camera's orientation, zoom and distortion from single images. Our solution is more robust than approaches that rely on detecting geometric elements such as straight lines or vanishing points. Our learning-based solution can harness subtle but important cues available in the images.We then tackle the problems of visual place recognition and visual localization.These tasks deal with the recognition of a camera's pose (position and orientation) using images captured by the camera.This is a difficult task, particularly when the appearance of the locations changes.This technique can complement or replace GPS in situations where it is not precise or robust enough, such as indoors.Finally, we develop two novel general-purpose modules for convolutional neural architectures.We propose the CNN-COSFIRE module for the task of image recognition. It explicitly models the relative in-plane arrangement of convolutional neural network responses, and can be used in detection or classification tasks.We also introduce a new module for convolutional neural networks called 'push-pull' layer that increases their robustness to several types of perturbations of the input images. It is based on a biological phenomenon known as push-pull inhibition.

Lifelong place recognition is an essential and challenging task in computer vision, with vast applications in robust localization and efficient large-scale 3D reconstruction. Progress is currently hindered by a lack of large, diverse, publicly available datasets. We contribute with Mapillary Street-Level Sequences (MSLS), a large dataset for urban and suburban place recognition from image sequences. It contains more than 1.6 million images curated from the Mapillary collaborative mapping platform. The dataset is orders of magnitude larger than current data sources, and is designed to reflect the diversities of true lifelong learning. It features images from 30 major cities across six continents, hundreds of distinct cameras, and substantially different viewpoints and capture times, spanning all seasons over a nine-year period. All images are geo-located with GPS and compass, and feature high-level attributes such as road type. We propose a set of benchmark tasks designed to push state-of-the-art performance and provide baseline studies. We show that current state-of-the-art methods still have a long way to go, and that the lack of diversity in existing datasets has prevented generalization to new environments. The dataset and benchmarks are available for academic research.