Search

This paper presents a novel approach for indoor acoustic source localization using microphone arrays, based on a Convolutional Neural Network (CNN). In the proposed solution, the CNN is designed to directly estimate the three-dimensional position of a single acoustic source using the raw audio signal as the input information and avoiding the use of hand-crafted audio features. Given the limited amount of available localization data, we propose, in this paper, a training strategy based on two steps. We first train our network using semi-synthetic data generated from close talk speech recordings. We simulate the time delays and distortion suffered in the signal that propagate from the source to the array of microphones. We then fine tune this network using a small amount of real data. Our experimental results, evaluated on a publicly available dataset recorded in a real room, show that this approach is able to produce networks that significantly improve existing localization methods based on SRP-PHAT strategies and also those presented in very recent proposals based on Convolutional Recurrent Neural Networks (CRNN). In addition, our experiments show that the performance of our CNN method does not show a relevant dependency on the speaker's gender, nor on the size of the signal window being used. [ABSTRACT FROM AUTHOR]

• A novel CNN based system is proposed for the acoustic source localization task. • The method is largely independent of the room and sensor geometry. • Applicable even with changes in the geometrical conditions of the training set. • It performs consistently better than both classical and deep learning methods. • Performance improvements are more relevant for mismatched training-testing cases. One of the most popular techniques for Acoustic Source Localization is the Generalized Cross Correlation (GCC) and its use in Steered Response Power techniques (SRP). Nowadays, Deep Learning strategies may outperform these classical methods, but they are generally dependent on the room and sensor geometric configuration that are used during the training phases. Hence, they require adaptation and re-training when facing a new environment, which is a problem in practice as re-training requires labelling new data and running a complex training algorithm. In this work we use a Convolutional Deep Neural Network that transforms the GCC between two signals into a Gaussian shaped signal, that we call Deep Generalized Cross Correlation (DeepGCC). We combine DeepGCC estimations to create a 3D acoustic map, similarly to SRP techniques. This acoustic map can be further refined using a sparse generative model to recover the source position. Crucially, we can adapt the acoustic map to different microphone array geometries without retraining the DeepGCC network. We show that our method outperforms both classical approaches and recent Deep Learning strategies in real and simulated challenging scenarios with mismatched training-testing conditions, not requiring re-training with different sensor configurations or room environments. [ABSTRACT FROM AUTHOR]

This paper describes a novel DNN-based system, named PD3net, that detects multiple people from a single depth image, in real time. The proposed neural network processes a depth image and outputs a likelihood map in image coordinates, where each detection corresponds to a Gaussian-shaped local distribution, centered at each person's head. This likelihood map encodes both the number of detected people as well as their position in the image, from which the 3D position can be computed. The proposed DNN includes spatially separated convolutions to increase performance, and runs in real-time with low budget GPUs. We use synthetic data for initially training the network, followed by fine tuning with a small amount of real data. This allows adapting the network to different scenarios without needing large and manually labeled image datasets. Due to that, the people detection system presented in this paper has numerous potential applications in different fields, such as capacity control, automatic video-surveillance, people or groups behavior analysis, healthcare or monitoring and assistance of elderly people in ambient assisted living environments. In addition, the use of depth information does not allow recognizing the identity of people in the scene, thus enabling their detection while preserving their privacy. The proposed DNN has been experimentally evaluated and compared with other state-of-the-art approaches, including both classical and DNN-based solutions, under a wide range of experimental conditions. The achieved results allows concluding that the proposed architecture and the training strategy are effective, and the network generalize to work with scenes different from those used during training. We also demonstrate that our proposal outperforms existing methods and can accurately detect people in scenes with significant occlusions. • Robust system to detect people only using depth information from a depth camera. • System outperforms state-of-the-art methods in different datasets without fine-tuning. • Proposal runs in real time using conventional GPUs. • Computational demands are independent of the number of people in the scene. • Generated database is available to the research community. [ABSTRACT FROM AUTHOR]