Search

We present an approach for encoding visual task relationships to improve model performance in an Unsupervised Domain Adaptation (UDA) setting. Semantic segmentation and monocular depth estimation are shown to be complementary tasks; in a multi-task learning setting, a proper encoding of their relationships can further improve performance on both tasks. Motivated by this observation, we propose a novel Cross-Task Relation Layer (CTRL), which encodes task dependencies between the semantic and depth predictions. To capture the cross-task relationships, we propose a neural network architecture that contains task-specific and cross-task refinement heads. Furthermore, we propose an Iterative Self-Learning (ISL) training scheme, which exploits semantic pseudo-labels to provide extra supervision on the target domain. We experimentally observe improvements in both tasks' performance because the complementary information present in these tasks is better captured. Specifically, we show that: (1) our approach improves performance on all tasks when they are complementary and mutually dependent; (2) the CTRL helps to improve both semantic segmentation and depth estimation tasks performance in the challenging UDA setting; (3) the proposed ISL training scheme further improves the semantic segmentation performance. The implementation is available at https://github.com/susaha/ctrl-uda., 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), ISBN:978-1-6654-4509-2, ISBN:978-1-6654-4510-8

Lecture Notes in Computer Science, 12365, ISSN:0302-9743, ISSN:1611-3349, Computer Vision – ECCV 2020, ISBN:978-3-030-58564-8, ISBN:978-3-030-58565-5

Publisher Summary This chapter covers aspects of the approach to object detection proposed by Viola and Jones. Object detection in still images and video is among the most-demanded techniques that originate from computer vision. Paul Viola and Michael Jones came up with their framework for object detection in early 2001 and since that time, the framework has not changed significantly. The core algorithm of object detection, as described in consists of creation of the object classifier and application of this classifier to an image. Full GPU residency of an algorithm helps to offload the CPU and eliminates excessive memory copies via the PCI-e bus. The implementation and analysis require basic knowledge of C/CCC programming languages and CUDA architecture. When a user has an object classifier trained on images of size M × N and an input image of much higher dimensions, it is possible to detect all instances of the object of interest in the image. This is achieved by applying the classifier to every region of pixels of size M × N on a large set of scales of the input image. The algorithm is discussed along with its well-known implementation from the OpenCV library. The creation process of the GPU-resident object detection pipeline is described step by step and analysis of intermediate results and the pseudo code is provided.