Search

This thesis investigates information sharing for Multi-Task Learning (MTL) in the multimedia and computer vision domains. More specifically, we investigate how, when and where information sharing should occur, in order to maximize the positive, and minimize negative influence between the performed tasks in MTL. In the first chapter, we introduce a benchmark dataset dubbed OmniArt, which contains heterogeneously annotated data from digitized visual artworks. Through OmniArt we first start to observe the relationships that form between the multiple tasks defined on the same dataset. In the second chapter, we investigate how secondary latent features such as back-propagated gradients help in determining between task similarities. In the third chapter of the thesis, we study task relationships through featurewise transformations on the filter level, which allow MTL models to share knowledge between large numbers of tasks. To distinguish performing MTL on a very large number of tasks, in the third chapter we introduce Many-Task Learning (MaTL) which occurs when more than 20 tasks are trained simultaneously. Finally, in the fourth chapter we investigate the relationships between tasks and scales in the form of a universal attention module. These works broaden the applicability scope of MTL approaches. By controlling the way information is processed within MTL, we gain the ability to approach a problem from multiple perspectives and discover relationships hidden in the underlying dataset. As multitasking comes naturally to humans, we hope this work contributes to making MTL the paradigm of choice in general machine learning.

Faces in artistic paintings most often contain the same elements (eyes, nose, mouth...) as faces in the real world, however they are not a photo-realistic transfer of physical visual content. These creative nuances the artists introduce in their work act as interference when facial detection models are used in the artistic domain. In this work we introduce models that can accurately detect, classify and conditionally generate artistically painted eyes in portrait paintings. In addition, we introduce the OmniEyes Dataset that captures the essence of painted eyes with annotated patches from 250 K artistic paintings and their metadata. We evaluate our approach in inpainting, out of context eye generation and classification on portrait paintings from the OmniArt dataset. We conduct a user case study to further study the quality of our generated samples, asses their aesthetic aspects and provide quantitative and qualitative results for our model’s performance.