Search

Increased use of data and computation have been the main drivers in Deep Learning for improving performance, solving previously unsolvable problems and unlocking new capabilities. At the same time cost of training and obtaining annotations have been increasing exponentially for large scale models where in certain circumstances it is not even feasible for many research organizations to reproduce ground breaking results in the field let alone improve them. This thesis focuses on problems that arise from scaling computational and data requirements for training computer vision algorithms and proposes efficient training techniques in three scenarios. First where data is available but labels are very limited, second where data is abundant but no labels are available and finally where both labels and data is abundant but computation is limited. We provided model-agnostic and efficient solutions for these urgent bottlenecks. Obtaining labeled data is one of the most expensive parts of training deep learning methods which are notoriously data hungry. We have identified that batch normalization statistics calculated from limited labeled data does not reflect the true data distribution and prevent training deep learning models effectively. We proposed a semi-supervised learning method where batch normalization calculations are augmented with unlabeled examples to alleviate this problem and train deep learning models more accurately. Recently self-supervised learning has been used to learn augmentation invariant representations from unlabelled data. These representations have been shown to effectively transfer to tasks where only limited labeled data is available. This eliminates the data bottleneck while exacerbating the computational bottleneck since these methods are much more computationally hungry than their supervised alternatives. In order to accelerate the training for these methods we introduce three strategies: a progressive augmentation and resolution schedule, a fast hard augmentations mining scheme and a matching accelerated learning rate schedule. We show that our training strategies reduce the cost of training for many self-supervised methods while maintaining the same accuracy as standard training. In problems where annotation cost is low and large scale labeled data is available more sophisticated model architectures like Vision transformers can be used which require longer and computationally more demanding training settings. We exploit the intuition that not all the samples carry similar amount of information, and introduce a fast online importance sampling technique that allows faster training. We show that we can accelerate the supervised training on many datasets and backbone architectures while reaching the same level of accuracy as standard training.

The recent progress in deep learning techniques transformed the field of computer vision, with tasks like object classification or segmentation being almost considered solved. This however requires sufficiently many labeled samples to train the system, hence research focus has shifted towards tasks where collecting such data is challenging. Recovering camera poses is one such task, where labels are typically too costly for supervised approaches. This work explores solutions to train camera pose estimation systems without the need for external supervision. Preliminary assessments show that it is possible to formulate this problem as a self supervised reconstruction task. By interpreting a network output as 3D rotation, and using this output to control a differentiable rendering operation, gradient descent can be used to train the network to predict viewpoint information. However, multiple issues arise when applying such a method naively on complex data. Confounding factors of particular importance are symmetries, geometry-breaking rendering pipelines and background induced noise. This leads to a regime where purely self-supervised training breaks, al though semi-supervised approaches are still successful. Specific solutions to the aforementioned problems are therefore studied and evaluated. For symmetries, multiple viewpoint predictions are made, and their distribution is further regulated. Two main rendering pipelines are also compared to improve over naive convolution-based reconstruction: a voxel-based one, and a more recent implicit neural representation. Experimental evidence shows that carefully crafting a system with these improvements allows recovery of poses on many everyday objects, such as cars and chairs, with performances reaching the level of supervised approaches on some categories. In addition, this thesis underlines two potential problems in related approaches. First, an unstable pose retrieval method used in recent implicit representations, that is prohibitively expensive. Second, an insidious issue in unsupervised methods, arising from a combination of dataset biases and naive calibration. As this potentially leads to overestimated performances, it calls for a more robust evaluation standard, as well as more careful data gathering.