Search

Conventional individual head-related transfer function (HRTF) measurements are demanding in terms of measurement time and equipment. For more flexibility, free body movement (FBM) measurement systems provide an easy-to-use way to measure full-spherical HRTF datasets with less effort. However, having no fixed measurement installation implies that the HRTFs are not sampled on a predefined regular grid but rely on the individual movements of the subject. Furthermore, depending on the measurement effort, a rather small number of measurements can be expected, ranging, for example, from 50 to 150 sampling points. Spherical harmonics (SH) interpolation has been extensively studied recently as one method to obtain full-spherical datasets from such sparse measurements, but previous studies primarily focused on regular full-spherical sampling grids. For irregular grids, it remains unclear up to which spatial order meaningful SH coefficients can be calculated and how the resulting interpolation error compares to regular grids. This study investigates SH interpolation of selected irregular grids obtained from HRTF measurements with an FBM system. Intending to derive general constraints for SH interpolation of irregular grids, the study analyzes how the variation of the SH order affects the interpolation results. Moreover, the study demonstrates the importance of Tikhonov regularization for SH interpolation, which is popular for solving ill-posed numerical problems associated with such irregular grids. As a key result, the study shows that the optimal SH order that minimizes the interpolation error depends mainly on the grid and the regularization strength but is almost independent of the selected HRTF set. Based on these results, the study proposes to determine the optimal SH order by minimizing the interpolation error of a reference HRTF set sampled on the sparse and irregular FBM grid. Finally, the study verifies the proposed method for estimating the optimal SH order by comparing interpolation results of irregular and equivalent regular grids, showing that the differences are small when the SH interpolation is optimally parameterized.

A soundscape is the representation of an acoustic environment in a specific context and time. Soundscapes evolve over time according to the changes in social, cultural and political contexts, and they constitute a unique way of experiencing auditory memories of a specific period in history. Considering that the soundscape of a place is ever-changing and it can provide us with important information about a location’s context in time, the study and preservation of soundscapes, in particular historic soundscapes, has gained relevance in the cultural heritage protection and promotion process. Soundscapes enrich the history of places and contribute to enhance the visitors experience in tourism contexts. However, it is impossible to go back in time and record the acoustic experiences from centuries ago. For that reason, the closest we can get to experience an historic soundscape is by trying to recreate it. This can be done with the support of historic documentation and audio immersion technology. We created a solution that allows users to construct scenes that provide immersive auditory experiences. The immersive audio files generated are obtained by combining different current sound recordings, and by composing an immersive auditory scene that can be recorded by the user. Here, we propose Immerscape, a tool that non-expert users can easily use to create immersive historical soundscapes. This work was validated by a user study that evaluated the tool’s functionalities and its acceptance by users. The proposed work is a contribution to the PASEV project, whose main purpose is to preserve and promote Évora’s rich cultural heritage, in particular, their historic soundscapes corresponding to the period between 1540 and 1910. A paisagem sonora é uma representação de um ambiente acústico num contexto e período de tempo específicos. As paisagens sonoras evoluem ao longo do tempo de acordo com mudanças em contextos sociais, culturais e políticos, e constituem uma maneira única de experienciar memórias auditivas de um específico período histórico. Visto que as paisagens sonoras de um local são mutáveis e nos podem fornecer informação importante acerca do contexto do mesmo numa determinada época, o estudo e a preservação de paisagens sonoras, em particular de paisagens sonoras históricas, tem adquirido uma especial relevância no processo de proteção e promoção da herança cultural. Isto acontece visto que as paisagens sonoras enriquecem a cultura e história dos locais e contribuem para melhorar a experiência dos visitantes em ambientes turísticos. Contudo, é impossível voltar atrás no tempo para gravar os contextos sonoros experi- enciado há séculos atrás. Por isso, o mais perto que podemos estar de experienciar uma paisagem sonora histórica passa por tentar recriar essa mesma paisagem. Para isso, deve- mos recorrer a documentação de suporte relativa à época e ao uso de tecnologias de áudio imersivo. A solução apresentada nesta dissertação tem como objetivo permitir aos utilizadores construir cenas que ofereçam uma experiência auditiva imersiva. Os ficheiros imersivos são obtidos através da combinação de diferentes sons atuais, compondo-os numa cena auditória imersiva que pode ser gravada pelo utilizador. É apresentada Immerscape, uma ferramenta que utilizadores não-experientes podem facilmente usar para criar paisagens sonoras históricas. A ferramenta foi avaliada através de um estudo com utilizadores que avaliaram as suas funcionalidades e sua aceitação. O trabalho proposto nesta dissertação é uma contribuição para o projeto PASEV, cujo principal objetivo é preservar e promover a rica herança cultural da cidade de Évora, com especial foco nas suas paisagens sonoras correspondentes aos períodos entre 1540 e 1910.

In binaural synthesis, providing individual HRTFs (head-related transfer functions) to the end user is a key matter, which is addressed in this thesis. On the one hand, we propose a method that consists in the automatic tuning of the weights of a principal component analysis (PCA) statistical model of the HRTF set based on listener localization performance. After having examined the feasibility of the proposed approach under various settings by means of psycho-acoustic simulations of the listening tests, we test it on 12 listeners. We find that it allows considerable improvement in localization performance over non-individual conditions, up to a performance comparable to that reported in the literature for individual HRTF sets. On the other hand, we investigate an underlying question: the dimensionality reduction of HRTF sets. After having compared the PCA-based dimensionality reduction of 9 contemporary HRTF and PRTF (pinna-related transfer function) databases, we propose a dataset augmentation method that relies on randomly generating 3-D pinna meshes and calculating the corresponding PRTFs by means of the boundary element method.; En synthèse binaurale, fournir à l’auditeur des HRTFs (fonctions de transfert relatives à la tête) personnalisées est un problème clef, traité dans cette thèse. D’une part, nous proposons une méthode d’individualisation qui consiste à régler automatiquement les poids d’un modèle statistique ACP (analyse en composantes principales) de jeu d’HRTF à partir des performances de localisation de l’auditeur. Nous examinons la faisabilité de l’approche proposée sous différentes configurations grâce à des simulations psychoacoustiques des tests d’écoute, puis la testons sur 12 auditeurs. Nous constatons qu’elle permet une amélioration considérable des performances de localisation comparé à des conditions d’écoute non-individuelles, atteignant des performances comparables à celles rapportées dans la littérature pour des HRTF individuelles. D’autre part, nous examinons une question sous-jacente : la réduction de dimensionnalité des jeux d’HRTF. Après avoir comparé la réduction de dimensionalité par ACP de 9 bases de données contemporaines d’HRTF et de PRTF (fonctions de transfert relatives au pavillon de l’oreille), nous proposons une méthode d’augmentation de données basée sur la génération aléatoire de formes d’oreilles 3D et sur la simulation des PRTF correspondantes par méthode des éléments frontières.

Digital communications technologies have developed at an increasingly rapid pace, with the COVID-19 pandemic accelerating its recent adoption. This shift over the last few decades has seen a mass migration online, where utilities like video conferencing software have become essential to entire industries and institutions. This thesis proposes the integration of binaural spatialized audio within a web-based video conferencing platform for distributed conversations. The proposed system builds upon findings on the benefits of spatial audio in video conferencing platforms and is guided by the tenets of telepresence. The developed implementation is based on Jitsi Meet, a robust open-source conferencing system. It localizes participant’s voices through sound spatialization methods provided by the Web Audio API. . This project treads new ground in exploring how localized audio can be conceptualized within an accessible telecommunications platform, proposing a novel integration of HRTF-based binaural spatialization within a standard video conferencing layout. System design and experimental questions used in a technical evaluation and user study are informed from a review of audio and video conference systems found in the literature and commercial market. The system evaluation suggests its viability from a compatibility and performance perspective. Perceptual metrics of cognitive load, social presence, and intelligibility are further investigated by a user study where four remote subjects were asked to engage in a short group discussion on a live deployment of the system. Results find support for improvements across all defined metrics as well as increased opinion scores regarding the preference of conferencing with a spatial audio system.

We propose a new method to improve median plane sound localization performance using a nonlinear representation of head-related transfer functions (HRTFs) and a recommender system. First, we reduce the dimensionality of an HRTF data set with multiple subjects using manifold learning in conjunction with a customized intersubject graph which takes into account relevant prior knowledge of HRTFs. Then, we use a sound localization model to estimate a subject's localization performance in terms of polar error and quadrant error rate. These metrics are merged to form a single rating per HRTF pair that we feed into a recommender system. Finally, the recommender system takes the low-dimensional HRTF representation as well as the ratings obtained from the localization model to predict the best HRTF set, possibly constructed by mixing HRTFs from different individuals, that minimizes a subject's localization error. The simulation results show that our method is capable of choosing a set of HRTFs that improves the median plane localization performance with respect to the mean localization performance using non-individualized HRTFs. Moreover, the localization performance achieved by our HRTF recommender system shows no significant difference to the localization performance observed with the best matching non-individualized HRTFs but with the advantage of not having to perform listening tests with all individuals' HRTFs from the database.

The below product proposal is inspired by the Waves NX application and plugin (Waves, 2018). It aims to show the digital audio signal processes that the author has implemented and researched over the Semester for this specific application.

The paper summarizes a number of audio-related studies conducted by the Sound of Vision consortium, which focuses on the construction of a new prototype electronic travel aid for the blind. Different solutions for spatial audio were compared by testing sound localization accuracy in a number of setups, comparing plain stereo panning with generic and individual HRTFs, as well as testing different types of stereo headphones vs custom designed quadrophonic proximaural headphones. A number of proposed sonification approaches were tested by sighted and blind volunteers for accuracy and efficiency in representing simple virtual environments.

Mobile technologies (such as smartphones and tablets) are now common devices of the consumer market. In this PhD we want to use those technologies as the way to introduce tools of sound spatialization into the mass market. Today the size and the number of traducers used to pick-up and to render a spatial sound scene are the main factors which limit the portability of those devices.As a first step, a listening test, based on a spatial audio recording of an opera, let us to evaluate the 3D audio technologies available today for headphone rendering. The results of this test show that, using the appropriate binaural decoding, it is possible to achieve a good binaural rendering using only the four sensors of the Soundfield microphone. Then, the steps of the development of a 3D sound pick-up system are described. Several configurations are evaluated and compared. The device, composed of 3 cardioid micro- phones, was developed following an approach inspired by the sound source localization and by the concept of the "object format encoding". Using the microphone signals and an adapted post-processing it is possible to determine the directions of the sources and to extract a sound signal which is representative of the sound scene. In this way, it is possible to completely describe the sound scene and to compress the audio information. This method offer the advantage of being cross platform compatible. In fact, the sound scene encoded with this method can be rendered over any reproduction system.A second method to extract the spatial information is proposed. It uses the real in situ characteristics of the microphone array to perform the sound scene analysis.Some propositions are made to complement the 3D audio chain allowing to render the result of the sound scene encoding over a binaural system or any king of speaker array using all capabilities of the mobile devices.; Les technologies nomades (smartphones, tablettes, . . . ) étant actuellement très répandues, nous avons souhaité, dans le cadre de cette thèse, les utiliser comme vecteur pour proposer au grand public des outils de spatialisation sonore. La taille et le nombre de transducteurs utilisés pour la captation et la restitution sonore spatialisée sont à ce jour la limitation principale pour une utilisation nomade. Dans une première étape, la captation d’un opéra pour une restitution sur des tablettes tactiles nous a permis d’évaluer les technologies audio 3D disponibles aujourd’hui. Les résultats de cette évaluation ont révélé que l’utilisation des quatre capteurs du microphone Soundfield donne de bons résultats à condition d’effectuer un décodage binaural adapté pour une restitution sur casque. Selon une approche inspirée des méthodes de localisation de source et le concept de format « objet », un prototype de prise de son 3D léger et compact a été développé. Le dispositif microphonique proposé se compose de trois capsules microphoniques cardioïdes. A partir des signaux microphoniques, un algorithme de post-traitement spatial est capable, d’une part, de déterminer la direction des sources et, d’autre part, d’extraire un signal sonore représentatif de la scène spatiale. Ces deux informations permettent ainsi de caractériser complètement la scène sonore 3D en fournissant un encodage spatial offrant le double avantage d’une compression de l’information audio et d’une flexibilité pour le choix du système de reproduction. En effet, la scène sonore ainsi encodée peut être restituée en utilisant un décodage adapté sur n’importe quel type de dispositif.Plusieurs méthodes de localisation et différentes configurations microphoniques (géométrie et directivité) ont été étudiées.Dans une seconde étape, l’algorithme d’extraction de l’information spatiale a été modifié pour prendre en compte les caractéristiques réelles in situ des microphones.Des méthodes pour compléter la chaîne acoustique sont proposées permettant la restitution binaurale ainsi que sur tout autre dispositif de restitution. Elles proposent l’utilisation de capteurs de localisation présents sur les terminaux mobiles afin d’exploiter les capacités qu’ils offrent aujourd’hui.

Mobile technologies (such as smartphones and tablets) are now common devices of the consumer market. In this PhD we want to use those technologies as the way to introduce tools of sound spatialization into the mass market. Today the size and the number of traducers used to pick-up and to render a spatial sound scene are the main factors which limit the portability of those devices. As a first step, a listening test, based on a spatial audio recording of an opera, let us to evaluate the 3D audio technologies available today for headphone rendering. The results of this test show that, using the appropriate binaural decoding, it is possible to achieve a good binaural rendering using only the four sensors of the Soundfield microphone.Then, the steps of the development of a 3D sound pick-up system are described. Several configurations are evaluated and compared. The device, composed of 3 cardioid microphones, was developed following an approach inspired by the sound source localization and by the concept of the "object format encoding". Using the microphone signals and an adapted post-processing it is possible to determine the directions of the sources and to extract a sound signal which is representative of the sound scene. In this way, it is possible to completely describe the sound scene and to compress the audio information.This method offer the advantage of being cross platform compatible. In fact, the sound scene encoded with this method can be rendered over any reproduction system.A second method to extract the spatial information is proposed. It uses the real in situ characteristics of the microphone array to perform the sound scene analysis.Some propositions are made to complement the 3D audio chain allowing to render the result of the sound scene encoding over a binaural system or any king of speaker array using all capabilities of the mobile devices., Les technologies nomades (smartphones, tablettes, . . . ) étant actuellement très répandues,nous avons souhaité, dans le cadre de cette thèse, les utiliser comme vecteur pour proposer au grand public des outils de spatialisation sonore. La taille et le nombre de transducteurs utilisés pour la captation et la restitution sonore spatialisée sont à ce jour la limitation principale pour une utilisation nomade. Dans une première étape, la captation d’un opéra pour une restitution sur des tablettes tactiles nous a permis d’évaluer les technologies audio 3D disponibles aujourd’hui. Les résultats de cette évaluation ont révélé que l’utilisation des quatre capteurs du microphone Soundfield donne de bons résultats à condition d’effectuer un décodage binaural adapté pour une restitution sur casque. Selon une approche inspirée des méthodes de localisation de source et le concept de format « objet », un prototype de prise de son 3D léger et compact a été développé. Le dispositif microphonique proposé se compose de trois capsules microphoniques cardioïdes. A partir des signaux microphoniques, un algorithme de post-traitement spatial est capable, d’une part, de déterminer la direction des sources et, d’autre part, d’extraire un signal sonore représentatif de la scène spatiale. Ces deux informations permettent ainsi de caractérisercomplètement la scène sonore 3D en fournissant un encodage spatial offrant le double avantage d’une compression de l’information audio et d’une flexibilité pour le choix du système de reproduction. En effet, la scène sonore ainsi encodée peut être restituée en utilisant un décodage adapté sur n’importe quel type de dispositif.Plusieurs méthodes de localisation et différentes configurations microphoniques (géométrie et directivité) ont été étudiées.Dans une seconde étape, l’algorithme d’extraction de l’information spatiale a été modifié pour prendre en compte les caractéristiques réelles in situ des microphones.Des méthodes pour compléter la chaîne acoustique sont proposées permettant la restitution binaurale ainsi que sur tout autre dispositif de restitution. Elles proposent l’utilisation de capteurs de localisation présents sur les terminaux mobiles afin d’exploiter les capacités qu’ils offrent aujourd’hui.

Mobile technologies (such as smartphones and tablets) are now common devices of the consumer market. In this PhD we want to use those technologies as the way to introduce tools of sound spatialization into the mass market. Today the size and the number of traducers used to pick-up and to render a spatial sound scene are the main factors which limit the portability of those devices.As a first step, a listening test, based on a spatial audio recording of an opera, let us to evaluate the 3D audio technologies available today for headphone rendering. The results of this test show that, using the appropriate binaural decoding, it is possible to achieve a good binaural rendering using only the four sensors of the Soundfield microphone. Then, the steps of the development of a 3D sound pick-up system are described. Several configurations are evaluated and compared. The device, composed of 3 cardioid micro- phones, was developed following an approach inspired by the sound source localization and by the concept of the "object format encoding". Using the microphone signals and an adapted post-processing it is possible to determine the directions of the sources and to extract a sound signal which is representative of the sound scene. In this way, it is possible to completely describe the sound scene and to compress the audio information. This method offer the advantage of being cross platform compatible. In fact, the sound scene encoded with this method can be rendered over any reproduction system.A second method to extract the spatial information is proposed. It uses the real in situ characteristics of the microphone array to perform the sound scene analysis.Some propositions are made to complement the 3D audio chain allowing to render the result of the sound scene encoding over a binaural system or any king of speaker array using all capabilities of the mobile devices., Les technologies nomades (smartphones, tablettes, . . . ) étant actuellement très répandues, nous avons souhaité, dans le cadre de cette thèse, les utiliser comme vecteur pour proposer au grand public des outils de spatialisation sonore. La taille et le nombre de transducteurs utilisés pour la captation et la restitution sonore spatialisée sont à ce jour la limitation principale pour une utilisation nomade. Dans une première étape, la captation d’un opéra pour une restitution sur des tablettes tactiles nous a permis d’évaluer les technologies audio 3D disponibles aujourd’hui. Les résultats de cette évaluation ont révélé que l’utilisation des quatre capteurs du microphone Soundfield donne de bons résultats à condition d’effectuer un décodage binaural adapté pour une restitution sur casque. Selon une approche inspirée des méthodes de localisation de source et le concept de format « objet », un prototype de prise de son 3D léger et compact a été développé. Le dispositif microphonique proposé se compose de trois capsules microphoniques cardioïdes. A partir des signaux microphoniques, un algorithme de post-traitement spatial est capable, d’une part, de déterminer la direction des sources et, d’autre part, d’extraire un signal sonore représentatif de la scène spatiale. Ces deux informations permettent ainsi de caractériser complètement la scène sonore 3D en fournissant un encodage spatial offrant le double avantage d’une compression de l’information audio et d’une flexibilité pour le choix du système de reproduction. En effet, la scène sonore ainsi encodée peut être restituée en utilisant un décodage adapté sur n’importe quel type de dispositif.Plusieurs méthodes de localisation et différentes configurations microphoniques (géométrie et directivité) ont été étudiées.Dans une seconde étape, l’algorithme d’extraction de l’information spatiale a été modifié pour prendre en compte les caractéristiques réelles in situ des microphones.Des méthodes pour compléter la chaîne acoustique sont proposées permettant la restitution binaurale ainsi que sur tout autre dispositif de restitution. Elles proposent l’utilisation de capteurs de localisation présents sur les terminaux mobiles afin d’exploiter les capacités qu’ils offrent aujourd’hui.

The paper presents a system for customized binaural audio delivery based on the extraction of the relevant features from a 2-D representation of the listener's pinna. A procedure based on multi-flash imaging for recognizing the main contours of the pinna and their position with respect to the ear canal entrance is detailed. The resulting contours drive the parametrization of a structural head-related transfer function model that performs in real time the spatialization of a desired sound file according to the listener's position with respect to the virtual sound source, tracked by sensor-equipped headphones. The low complexity of the model allows smooth implementation and delivery on any mobile device. The purpose of the desired system is to provide low-tech custom binaural audio to any user without the need of tedious and cumbersome subjective measurements.