Search

Deep learning has catalysed progress in tasks such as face recognition and analysis, leading to a quick integration of technological solutions in multiple layers of our society. While such systems have proven to be accurate by standard evaluation metrics and benchmarks, a surge of work has recently exposed the demographic bias that such algorithms exhibit–highlighting that accuracy does not entail fairness. Clearly, deploying biased systems under real-world settings can have grave consequences for affected populations. Indeed, learning methods are prone to inheriting, or even amplifying the bias present in a training set, manifested by uneven representation across demographic groups. In facial datasets, this particularly relates to attributes such as skin tone, gender, and age. In this work, we address the problem of mitigating bias in facial datasets by data augmentation. We propose a multi-attribute framework that can successfully transfer complex, multi-scale facial patterns even if these belong to underrepresented groups in the training set. This is achieved by relaxing the rigid dependence on a single attribute label, and further introducing a tensor-based mixing structure that captures multiplicative interactions between attributes in a multilinear fashion. We evaluate our method with an extensive set of qualitative and quantitative experiments on several datasets, with rigorous comparisons to state-of-the-art methods. We find that the proposed framework can successfully mitigate dataset bias, as evinced by extensive evaluations on established diversity metrics, while significantly improving fairness metrics such as equality of opportunity. © 2021, The Author(s).

Deep learning-based methods have pushed the limits of the state-of-the-art in face analysis. However, despite their success, these models have raised concerns regarding their bias towards certain demographics. This bias is inflicted both by limited diversity across demographics in the training set, as well as the design of the algorithms. In this work, we investigate the demographic bias of deep learning models in face recognition, age estimation, gender recognition and kinship verification. To this end, we introduce the most comprehensive, large-scale dataset of facial images and videos to date. It consists of 40K still images and 44K sequences (14.5M video frames in total) captured in unconstrained, real-world conditions from 1,045 subjects. The data are manually annotated in terms of identity, exact age, gender and kinship. The performance of state-of-the-art models is scrutinized and demographic bias is exposed by conducting a series of experiments. Lastly, a method to debias network embeddings is introduced and tested on the proposed benchmarks. © 2020 Elsevier B.V.

A novel method for active speaker detection and localization in audio-visual recordings is proposed. The method relies on a specifically tailored matrix decomposition that exploits the intrinsic low-dimensional structure of audio-visual data, namely, the low-rank of the background visual/audio information and the sparsity of the correlated foreground components. Concretely, the data matrix of each modality is modeled as a superposition of two terms: 1) a low-rank matrix capturing the background information and 2) a kernelized sparse matrix capturing the non-linear correlated components among the audio and visual modalities and, hence, revealing the active speaker. To this end, we formulate an appropriate optimization problem that involves the minimization of nuclear- and matrix ell-1-norms, and develop an efficient solver. Experimental results on active speaker detection and localization demonstrate the superior performance of the proposed method over other state-of-the-art approaches. © 1994-2012 IEEE.

Speech-driven facial animation is the process which uses speech signals to automatically synthesize a talking character. The majority of work in this domain creates a mapping from audio features to visual features. This often requires post-processing using computer graphics techniques to produce realistic albeit subject dependent results. We present a system for generating videos of a talking head, using a still image of a person and an audio clip containing speech, that doesn't rely on any handcrafted intermediate features. To the best of our knowledge, this is the first method capable of generating subject independent realistic videos directly from raw audio. Our method can generate videos which have (a) lip movements that are in sync with the audio and (b) natural facial expressions such as blinks and eyebrow movements. We achieve this by using a temporal GAN with 2 discriminators, which are capable of capturing different aspects of the video. The effect of each component in our system is quantified through an ablation study. The generated videos are evaluated based on their sharpness, reconstruction quality, and lip-reading accuracy. Finally, a user study is conducted, confirming that temporal GANs lead to more natural sequences than a static GAN-based approach.

Inspired by the recent development of deep network-based methods in semantic image segmentation, we introduce an end-to-end trainable model for face mask extraction in video sequence. Comparing to landmark-based sparse face shape representation, our method can produce the segmentation masks of individual facial components, which can better reflect their detailed shape variations. By integrating Convolutional LSTM (ConvLSTM) algorithm with Fully Convolutional Networks (FCN), our new ConvLSTM-FCN model works on a per-sequence basis and takes advantage of the temporal correlation in video clips. In addition, we also propose a novel loss function, called Segmentation Loss, to directly optimise the Intersection over Union (IoU) performances. In practice, to further increase segmentation accuracy, one primary model and two additional models were trained to focus on the face, eyes, and mouth regions, respectively. Our experiment shows the proposed method has achieved a 16.99% relative improvement (from 54.50% to 63.76% mean IoU) over the baseline FCN model on the 300 Videos in the Wild (300VW) dataset.

Tensors are higher-order extensions of matrices. While matrix methods form the cornerstone of traditional machine learning and data analysis, tensor methods have been gaining increasing traction. However, software support for tensor operations is not on the same footing. In order to bridge this gap, we have developed TensorLy, a Python library that provides a high-level API for tensor methods and deep tensorized neural networks. TensorLy aims to follow the same standards adopted by the main projects of the Python scientific community, and to seamlessly integrate with them. Its BSD license makes it suitable for both academic and commercial applications. TensorLy's backend system allows users to perform computations with several libraries such as NumPy or PyTorch to name but a few. They can be scaled on multiple CPU or GPU machines. In addition, using the deep-learning frameworks as backend allows to easily design and train deep tensorized neural networks. TensorLy is available at https://github.com/tensorly/tensorly

The involvement of the mirror neuron system (MNS) in both imitation and action understanding has been firmly established. Various authors have claimed that the MNS’s function in facilitating imitation builds upon its role in action understanding and is a phylogenetically later development. We argue that this hypothesis lacks sufficient evidence and present support for the reverse: the phylogenetically primary function of the MNS is imitation and the MNS could have evolved in response to a selective pressure for imitative behavior. This hypothesis was tested using evolutionary robotics simulation techniques. The simulation was conducted with embodied and embedded agents with a lifetime-adapting neural network for which the learning parameters were evolutionarily optimized. The agents had to perform an imitation task. Analysis of the resulting controller revealed artificial neurons showing clear mirror characteristics, suggesting that, indeed, mirror neurons evolve due to a selective pressure for imitative behavior.

TERESA is a socially intelligent semi-autonomous telepresence system that is currently being developed as part of an FP7-STREP project funded by the European Union. The ultimate goal of the project is to deploy this system in an elderly day centre to allow elderly people to participate in social events even when they are unable to travel to the centre. In this paper, we present an overview of our progress on TERESA. We discuss the most significant scientific and technical challenges including: understanding and automati-cally recognizing social behaviour; defining social norms for the interaction between a telepresence robot and its users; navigating the environment while taking into account social features and constraints; and learning to estimate the social impact of the robot’s actions from multiple sources of feedback. We report on our current progress on each of these chal-lenges, as well as our plans for future work.

Contains fulltext : 77246.pdf (Publisher’s version ) (Open Access)

Contains fulltext : 72527.pdf (Author’s version preprint ) (Open Access) BNAIC 2008, 30 oktober 2008

Contains fulltext : 73645.pdf (Publisher’s version ) (Open Access) BNAIC 2008, 20th Belgian-Dutch Artificial Intelligence Conference

Contains fulltext : 126051.pdf (Publisher’s version ) (Open Access)

Contains fulltext : 73145.pdf (Publisher’s version ) (Open Access)

Contains fulltext : 57586.pdf (Publisher’s version ) (Closed access) Governments in several European countries, and the EU have set challenging targets for the improvement of road traffic safety by the year 2010. In the Netherlands a program for infrastructure measures was launched, to meet the Dutch targets. The ongoing developments in the field of ITS applications seem however to offer viable alternatives for large-scale infrastructure reconstruction. This paper explores the feasibility of five ADAS applications (navigation, speed assistance, collision avoidance, intersection support and lane keeping) to complement or partly substitute infrastructure measures to reach the stated goals. State-of-the-art and the potential of enabling technologies like positioning, radar, laser, video imaging and communication are analysed from a technical perspective. Technical issues relating to large-scale dedicated ADAS implementation for traffic safety, as well as related policy issues are discussed. 10 oktober 2004 13 p.