Search

This dissertation explores the use of Deep Learning-based methodologies aimed at representing and predicting the Head Related Transfer Functions (HRTFs). HRTFs describe the source location-specific acoustic transformations that sound signals undergo due to interactions with the human outer body structures such as the torso, head and pinna. HRTFs are understood to be crucial for sound localization perception, particularly when employing earphones/headphones to replicate directional audio where, non-directional mono sources are filtered using the listener's HRTFs specific to their left and right ears to achieve the intended directionality.As the HRTFs are a function of the individual anthropometry, there arises a pressing need for efficient methods to predict personalized HRTFs, devoid of cumbersome user involvement. The fast developing field of deep learning presents a promising avenue to address this challenge. However, the scarcity of comprehensive HRTF datasets poses a significant challenge. This dissertation work initially delves into techniques for deriving sub-dimensional latent representations from HRTFs, aiming to reduce the data dimensionality and complexity, thereby alleviating the load on prediction models. The latent representations are derived by capitalizing on both spatial and spectral interactions between neighboring HRTFs, in conjunction with contrastive learning methodologies aimed at disentangling pertinent features inherent in HRTFs. Visualizations of the obtained latent space are provided demonstrating the favorable characteristics of the derived representations. Furthermore, this study extends its investigation into utilizing the acquired latent representations for predicting spatially dense HRTFs through upsampling of spatially sparse measurements. The effectiveness of the proposed methodologies is validated using the mean Log Spectral Distortion (LSD) between the ground truth and predicted HRTFs as an objective metric, resulting in state of th

16 personer deltog i en studie vars syfte var att undersöka hur deltagare upplevde olika implementationssätt av spatialt ljud. Studien presenterar tidigare forskning och ett teoretiskt ramverk för att ge läsaren inblick i ämnet, där bland annat tekniska begrepp såsom HRTF och spatialt ljud redovisas. Studiens omfattning är otillräcklig för att etablera konkreta bevis gällande vilket implementationssätt som är bättre eller sämre än något annat i förhållande till vad som föredras av användare. Dock kan man se tendenser gällande specifika attribut i ljudbilden som bidrar till deltagares närvaro i miljön, såsom occlusion och reverb. För framtida arbete kan studien utvecklas genom att inkludera fler parametrar såsom att titta på en specifik genre inom spel samt inkludera insamling av kvantitativa data som används för att utvärdera effektiviteten hos de olika systemen., Det finns övrigt digitalt material (t.ex. film-, bild- eller ljudfiler) eller modeller/artefakter tillhörande examensarbetet som ska skickas till arkivet.There are other digital material (eg film, image or audio files) or models/artifacts that belongs to the thesis and need to be archived.

Virtual reality is a medium where both visual and aural fidelity strive to be comparable to real life. However, effective sound localization is an important part of many virtual game environments. Previous studies on virtual acoustic environments suggest that acoustic fidelity may be at odds with effective sound localization. Therefore, this study examines how one of the more computationally demanding acoustic processes affects sound localization performance in virtual reality. This study compares three different conditions with varying amounts of early reflections in a virtual reality environment. The results showed no statistically significant difference in azimuth errors, elevation errors or response times.

There are different ways of evaluating the sound pressure in a sound field. Doing so is an essential part of several applications, such as active sound control and voice analysis. One way of evaluating the sound pressure in a sound field is by measuring a few discrete points in the field, combined with an interpolation method to find the sound pressure for the remaining field. Recently, it has been shown that kernel ridge regression (KRR) can accurately interpolate a sound field from discrete points in space. This report examines the effect of inaccurate measurements for the resulting estimated sound field by adding noise in all measurements. Further, to allow for accurate estimation of how a sound is perceived at a given location, one not only needs to form an estimate of the expected sound field at the location, but also to examine how the presence of the person's head will shape the resulting field. Thus, the use of a pre-calibrated head-related transfer function (HRTF) in combination with the interpolation technique to interpolate the in-ear sound field is also proposed. Using both simulated and anechoic audio data, the effect of different kernels as well as the added noise for the sound field estimate is studied, which is then illustrating how the combined sound field interpolation technique allows for an improved estimate of the sound field, for all frequencies. Finally, the combined sound field interpolation is shown to be more effective than the KRR method, for both the simulated and the anechoic audio data, with the presence of a person in the sound field., Arbetet presenterar en metod som uppskattar ljud så som en människa uppfattar det i ett rum. Metoden kan användas i exempelvis applikationer med augmented reality och brusreducering. Att kunna uppskatta ljud i ett ljudfält är en nödvändighet för många applikationer. I exempelvis aktivt brusreducerade hörlurar får man in en signal som man vill tysta genom att skicka ut en motsatt signal, som i teorin tar ut den ursprungliga signalen och resulterar i att inget ljud når dina öron. Att så exakt och effektivt som möjligt kunna skatta hur ljudet beter sig i ljudfältet är därför nödvändigt för att få till effektiv brusreducering. I detta arbetet används en ny metod för att skatta ljud i ett ljudfält, där effektiviteten av metoden utvärderas med avseende på brus och mätfel. Utöver detta implementeras även en ny metod som skattar hur närvaron av en människa påverkar det resulterande ljudfältet. Det finns olika sätt att skatta ljudtrycket i ett ljudfält. Ett sätt är att placera ut mikrofoner i fältet som sedan används för att skatta ljudet i övriga fältet, vilket är metoden i denna rapport. Både uppmätt data från ett ekofritt rum och simulerad data utvärderas, där vi i den simulerade datan lagt till både mätfel och brus i alla mätningar. Tanken med projektet är att kunna använda resultaten som en referenspunkt för framtida implementeringar av, för att använda det tidigare exemplet, aktiv brusreducering. I en större skala, där man vill reducera ljudnivån i ett helt rum, blir problemet mer komplicerat. Mellan ett par hörlurar och dina öron är det inte mycket i vägen för ljudet att färdas fritt. I ett rum kan det å andra sidan finnas en mängd olika objekt utspridda; inte minst i form av en människa. En människas närvaro i rummet kommer påverka det resulterande ljudfältet, och av särskilt intresse är ljudfältets påverkan av människans närvaro när det gäller ljudet i människans eget öra. Om ljudet i hela rummet som helhet dämpas men ljudet i människans öra blir högre så blir ljudr

With spatialized audio it lets developers create further experiences and provide accessibility options in video games and interactive media. The technology has evolved during the years, making it easier to use system such as Head-Related Transfer Functions in games. In this study the use of spatialized audio to navigate and locate smaller invisible objects in a larger area is explored, comparing spatialized audio panning with stereo panning., Det finns övrigt digitalt material (t.ex. film-, bild- eller ljudfiler) eller modeller/artefakter tillhörande examensarbetet som ska skickas till arkivet. / There are other digital material (eg film, image or audio files) or models/artifacts that belongs to the thesis and need to be archived.

Spatialization of audio in the vertical plane has historically been limited. Instead, sound designers have used basic DSP to create pseudo height effects to explain the positions of corresponding objects. In recent years, binaural synthesis has become more widespread following an increase in the use of software rendering methods. With these advancements, uncertainty seems to be present around best practices when combining sound design with binaural synthesis for vertical placement of audio cues in games. This thesis compares the vertical localization performance between head related transfer functions (HRTFs) and stereo interaural level difference (ILD), when sounds have been designed with basic DSP to have auditory spatial schema (ASC). A sort of embedment of positional information. It was found that there was no significant time difference between the conditions, while hitcount, the number of correct directions selected, displayed a significant difference in some of the comparisons.

Att använda spatialt ljud när det kommer till utforskning av en spelvärld blir allt vanligare. Hur kan skräckljud påverka intresset av utforskning? Genom att blanda maskinljud med djurläten för att skapa abstrakta ljud med olika fokusområden av frekvensinformation för att se hur spelare reagerar när de utforskar en virtuellstadsmiljö. Beroende på hur frekvensområdet ser ut gör det tydligare att höra ljudet och därefter kunna lokalisera ljudkällan. Genom att använda spatialljudimplementation och ljudocklusion för att skapa en trovärdig lyssnarupplevelse för spelaren samt att kunna se hur navigering och lokalisering påverkas. Användning av spatialt ljud och hur olika frekvensområden kan göra det enklare att navigera kan ha stor nytta inom framtida arbeten för att ge individer med nedsatt syn en bättre spelupplevelse., Det finns övrigt digitalt material (t.ex. film-, bild- eller ljudfiler) eller modeller/artefakter tillhörande examensarbetet som ska skickas till arkivet.

Este producto forma parte de una serie de textos de divulgación científica que buscan reseñar algunas de las investigaciones más importantes en las que ha tenido participación la Universidad EAFIT, publicadas en las revistas especializadas más prestigiosas del mundo

Este producto forma parte de una serie de videos de divulgación científica que buscan reseñar algunas de las investigaciones más importantes en las que ha tenido participación la Universidad EAFIT, publicadas en las revistas especializadas más prestigiosas del mundo

[ES] La reproducción de las propiedades espaciales del sonido es una cuestión cada vez más importante en muchas aplicaciones inmersivas emergentes. Ya sea en la reproducción de contenido audiovisual en entornos domésticos o en cines, en sistemas de videoconferencia inmersiva o en sistemas de realidad virtual o aumentada, el sonido espacial es crucial para una sensación de inmersión realista. La audición, más allá de la física del sonido, es un fenómeno perceptual influenciado por procesos cognitivos. El objetivo de esta tesis es contribuir con nuevos métodos y conocimiento a la optimización y simplificación de los sistemas de sonido espacial, desde un enfoque perceptual de la experiencia auditiva. Este trabajo trata en una primera parte algunos aspectos particulares relacionados con la reproducción espacial binaural del sonido, como son la escucha con auriculares y la personalización de la Función de Transferencia Relacionada con la Cabeza (Head Related Transfer Function - HRTF). Se ha realizado un estudio sobre la influencia de los auriculares en la percepción de la impresión espacial y la calidad, con especial atención a los efectos de la ecualización y la consiguiente distorsión no lineal. Con respecto a la individualización de la HRTF se presenta una implementación completa de un sistema de medida de HRTF y se introduce un nuevo método para la medida de HRTF en salas no anecoicas. Además, se han realizado dos experimentos diferentes y complementarios que han dado como resultado dos herramientas que pueden ser utilizadas en procesos de individualización de la HRTF, un modelo paramétrico del módulo de la HRTF y un ajuste por escalado de la Diferencia de Tiempo Interaural (Interaural Time Difference - ITD). En una segunda parte sobre reproducción con altavoces, se han evaluado distintas técnicas como la Síntesis de Campo de Ondas (Wave-Field Synthesis - WFS) o la panoramización por amplitud. Con experimentos perceptuales se han estudiado la capacidad de estos siste, [CA] La reproducció de les propietats espacials del so és una qüestió cada vegada més important en moltes aplicacions immersives emergents. Ja siga en la reproducció de contingut audiovisual en entorns domèstics o en cines, en sistemes de videoconferència immersius o en sistemes de realitat virtual o augmentada, el so espacial és crucial per a una sensació d'immersió realista. L'audició, més enllà de la física del so, és un fenomen perceptual influenciat per processos cognitius. L'objectiu d'aquesta tesi és contribuir a l'optimització i simplificació dels sistemes de so espacial amb nous mètodes i coneixement, des d'un criteri perceptual de l'experiència auditiva. Aquest treball tracta, en una primera part, alguns aspectes particulars relacionats amb la reproducció espacial binaural del so, com són l'audició amb auriculars i la personalització de la Funció de Transferència Relacionada amb el Cap (Head Related Transfer Function - HRTF). S'ha realitzat un estudi relacionat amb la influència dels auriculars en la percepció de la impressió espacial i la qualitat, dedicant especial atenció als efectes de l'equalització i la consegüent distorsió no lineal. Respecte a la individualització de la HRTF, es presenta una implementació completa d'un sistema de mesura de HRTF i s'inclou un nou mètode per a la mesura de HRTF en sales no anecoiques. A mès, s'han realitzat dos experiments diferents i complementaris que han donat com a resultat dues eines que poden ser utilitzades en processos d'individualització de la HRTF, un model paramètric del mòdul de la HRTF i un ajustament per escala de la Diferencià del Temps Interaural (Interaural Time Difference - ITD). En una segona part relacionada amb la reproducció amb altaveus, s'han avaluat distintes tècniques com la Síntesi de Camp d'Ones (Wave-Field Synthesis - WFS) o la panoramització per amplitud. Amb experiments perceptuals, s'ha estudiat la capacitat d'aquests sistemes per a produir una sensació de distància i l'agudesa espacia, [EN] The reproduction of the spatial properties of sound is an increasingly important concern in many emerging immersive applications. Whether it is the reproduction of audiovisual content in home environments or in cinemas, immersive video conferencing systems or virtual or augmented reality systems, spatial sound is crucial for a realistic sense of immersion. Hearing, beyond the physics of sound, is a perceptual phenomenon influenced by cognitive processes. The objective of this thesis is to contribute with new methods and knowledge to the optimization and simplification of spatial sound systems, from a perceptual approach to the hearing experience. This dissertation deals in a first part with some particular aspects related to the binaural spatial reproduction of sound, such as listening with headphones and the customization of the Head Related Transfer Function (HRTF). A study has been carried out on the influence of headphones on the perception of spatial impression and quality, with particular attention to the effects of equalization and subsequent non-linear distortion. With regard to the individualization of the HRTF a complete implementation of a HRTF measurement system is presented, and a new method for the measurement of HRTF in non-anechoic conditions is introduced. In addition, two different and complementary experiments have been carried out resulting in two tools that can be used in HRTF individualization processes, a parametric model of the HRTF magnitude and an Interaural Time Difference (ITD) scaling adjustment. In a second part concerning loudspeaker reproduction, different techniques such as Wave-Field Synthesis (WFS) or amplitude panning have been evaluated. With perceptual experiments it has been studied the capacity of these systems to produce a sensation of distance, and the spatial acuity with which we can perceive the sound sources if they are spectrally split and reproduced in different positions. The contributions of this research are int

Bachelor End Project, Computer Science and Engineering

This work is focused on the trade-off between sound quality and loudspeaker gain that occurs when using the pressure matching beamforming method to generate stereo crosstalk cancellation with uniform linear arrays.An approach to gain management is proposed that leverages characteristics of subclasses of stereo audio signals (e.g. binaural) to partially overcome this tradeoff by improving both loudspeaker gain and sound quality together.The novelty of the proposed approach is based on an observation that loudspeaker gain management, which is normally accomplished by manipulating beamforming filters, can also be accomplished by manipulating input audio signals.It is shown that decoupling gain management from beamforming filters allows it to be signal-dependent.Also, that signal-dependent gain management gives beamforming filter designers more meaningful methods for controlling both the type and amount of sound quality that is sacrificed to manage gain.A general approach to building a signal-dependent gain management algorithm is suggested.Some specific implementations are given as examples.Finally, a discussion is presented about how signal-dependent gain management potentially establishes a future research project where perceived sound quality, as opposed to numerical sound quality, is maximized.

Denna uppsats faller inom ämnesområdet ljud-&musikproduktion och studiens syfte är att bidra med en djupare förståelse för hur dagens lyssnare upplever binaurala egenskaper i musik. Tidigare forskning föreslår att introducera framtida lyssnare till binaural musik genom så kallade hybridmixar, vanliga stereomixar med vissa binaurala egenskaper. Metodiskt utfördes ett lyssningstest där 15 ljudutbildade personer fick jämföra två låtar, båda mixade i stereo och binauralt. För den första låten applicerades en helt binaural strategi medans den andra låten mixades som en hybridmix. Resultaten visade på att en helt binauralt mixad låt mest bidrog med förvirring och inte var att föredra över en vanlig stereomix. Dock visade sig en blandning av stereo- och binaurala egenskaper i hybridmixen vara något som lyssnarna föredrog över vanlig stereomix då det tillförde något extra och spännande. Sammanfattningsvis tyder resultaten på att hybridmixad musik är något som lyssnarna upplever både nytt och spännande, vilket skulle kunna vara det första steget mot helt binaural musik.

La "head transfer response function" (HTRF) és un funció que permet processar un so de manera que a un oient amb el dispositiu adient (habitualment uns auriculars) li sembli que el so provingui d'un punt en concret. En aquest treball es proposa la realització d'un sistema de computació física on la informació sobre posicions i direccions, ... obtinguda per diversos sensors o bases de dades pugui ser mostrada amb so HTRF, completant la informació que un persona obté mitjançant els seus propis sentits (realitat augmentada).

La creciente popularidad de los sistemas y aplicaciones de realidad virtual, así como las videoconferencias y los videojuegos, exigen grandes esfuerzos en la virtualización de audio en tiempo real, contribuyendo poderosamente al sentido de inmersión del usuario. El posicionamiento del sonido se logra mediante la convolución de las fuentes de audio con las colecciones de Head Related Impulse Responses, cuya medición puede ser tediosa y costosa. La interpolación brinda la oportunidad de ahorrar tiempo y esfuerzo al permitirnos obtener muestras fielmente nuevas de unas pocas. El objetivo de esta tesis es diseñar e implementar algoritmos de interpolación para HRIR y Binaural Room Impulse Responses., Increasing popularity of virtual reality systems and applications, such as videoconferencing and gaming, is demanding vast efforts in real-time audio virtualization, contributing powerfully to the user s sense of immersion. Sound positioning is achieved by means of convolving audio sources with Head Related Impulse Responses collections, whose measurement can be tedious and costly. Interpolation gives the opportunity to save time and efforts by allowing us to obtain faithfully new samples from few ones. The aim of this thesis is to design and implement interpolation algorithms for HRIRs and Binaural Room Impulse Responses.

El futuro del sonido espacial se encuentra ligado al sonido binaural, ya que el consumo de auriculares es cada vez mayor y los contenidos audiovisuales son escuchados mayormente con auriculares. Para mejorar la experiencia del sonido inmersivo, se usan las funciones de transferencia relativas a la cabeza (HRTF) para personalizar el sonido binaural, puesto que mide los efectos que una fuente en condiciones de campo libre tiene en el canal auditivo, siendo específico para cada uno debido a las características antropométricas. Su medida implica condiciones anecoicas y sistemas de posicionamiento, siendo ambos instalaciones caras y complejas que no siempre se encuentran disponibles. La medida en una sala no anecoica con un acondicionamiento acústico mínimo y el uso de arrays de altavoces en diferentes elevaciones sería lo deseado, pero se introducirían reflexiones que no forman parte de la HRTF y necesitan ser canceladas. Por ello, en este proyecto se va a trabajar con un sistema de altavoces ya establecido en una sala no anecoica para la medida de HRTF, empleando diferentes técnicas avanzadas de procesado y métodos de ecualización para corregir los factores en el proceso. El objetivo final es obtener una HRTF pura equiparable a una medida en cámara anecoica. Durante el proyecto, el alumno tendrá un contacto práctico con sistemas de medida de sonido, manejando software y equipos de lo más avanzados utilizados en el campo profesional del sonido., The future of spatial sound is bound up with binaural audio, as headphone market has experienced a remarkable expansion and audiovisual content is being mostly consumed through headphone listening. To provide a better immersive experience, personal Head-Related Transfer Functions (HRTF) are employed for individualizing binaural sound for each subject, as they capture the effects that a source in free-field conditions experiences to the ear canals, being very specific for each one due to the influence of anthropometric characteristics. Its measurement implies anechoic environments and motorized positioning systems, which are expensive and complex facilities not always available for the general public. Measurement in a non-anechoic room with minimum acoustic conditions and the use of different loudspeaker arrays in different elevations would be desirable to avoid those two factors, but reflections which are not part of the HRTF are being introduced, which need to be cancelled. To this effect, in this project we are going to work with an established loudspeaker system in a non-anechoic room for HRTF measurement, employing different advanced processing techniques and equalization methods to deal with factors which comprise the measurement process. The final aim is to obtain a pure HRTF comparable to one measured in anechoic conditions.

En undersökning om HRTFs effektivitet när spelandet sker med ett öra.

Title: Binaural dummy heads measurements and sound spatialization Binaural sound is the one that is played through headphones and provides the sensation of being immersed in the environment where the sound was recorded, creating a spatial sound illusion to the listener. This type of sound is being widely used in the virtual reality field, mostly in videogames, where knowing the sound source¿s position gives advantages to the user. In this study, the procedure and results of the analysis and measurement of two acoustic dummy heads in a non-anechoic chamber are exposed. In order to do it, binaural impulse responses (BIR) of the dummy heads have been measured from multiple 3D positions around them, covering almost the entire surface of a sphere. In addition, speakers with customized features have been built to make these measurements. The most relevant application of this study is the possibility to create a new binaural sound just by using the BIR measurements, giving the immersed sensation to the listener and achieving sound¿s binauralization., Título del TFG: Medidas de maniquís acústicos binaurales y espacialización de sonidos El sonido binaural es aquel que se reproduce mediante auriculares y que proporciona la sensación de estar inmerso en el entorno en el que fue grabado, creando una ¿ilusión¿ sonora espacial al oyente. Este tipo de sonido se está empleando cada vez más en el mundo de la realidad virtual, sobre todo en videojuegos, donde experimentar la posición de una fuente de sonido aporta grandes ventajas. En este trabajo se expone el procedimiento y los resultados del análisis y medida de dos maniquís acústicos (también denominados cabezas binaurales) en una cámara no anecoica. Para ello se han medido las respuestas al impulso binaurales (binaural impulse response, BIR) de los maniquís desde múltiples posiciones en 3D alrededor del mismo, hasta completar casi una esfera. Así mismo, se han construido altavoces especiales ex profeso que reunieran las características deseadas para esta tarea. La aplicación más relevante de este trabajo reside en la posibilidad de auralizar cualquier sonido con las respuestas binaurales medidas, proporcionando la sensación de inmersión al oyente y logrando la binauralización del sonido.

Detta kandidatarbete handlar om hur implementation av binauralt ljud kan användas för att öka precisionen för spelares ljudlokalisering i datorspel. I detta arbete skapades en artefakt som testade spelares förmåga att lokalisera ljudkällor på olika vertikala nivåer. Resultatet analyserades och problematiserades med kvantitativa- och kvalitativa undersökningsmetoder. Arbetet kopplas till tidigare forskning inom binauralt ljud och hur det används inom digitala medier. Studien tar även upp ämnen som involverar den mänskliga hörseln och ljudimplementation i datorspel.

The video game industry is experiencing tremendous growth and developing games that stand out against the competition requires ever increasing production budgets. Graphics have been a selling point for video games for a long time, resulting in huge technical advancements in visuals. Audio has received more attention during the recent years, both in terms of sound-design and technical developments. This study compared two alternatives for spatializing audio over headphones in a game setting, while also comparing them against a more traditional stereo output. A commercial game was used to provide the game setting and participants rated the three different outputs based on three attributes and described their experience using Microsoft Product Reaction Cards. No statistically significant differences were found between the three outputs in any of the six rated attributes when doing a paired t-test or a Wilcoxon Signed-Rank Test. Further analysis of frequency content, participants’ comments and Microsoft Product Reaction Cards were done to find out what sets the spatializing alternatives and stereo apart from each other.

For many application areas the binaural synthesis has become a field of interest. In this paper, we present the basics of bin- aural synthesis for 2 channels -left and right- providing exam- ples and figures, using different approaches: from just delay- ing both signals to the usage of HRIR (Head-Related Impulse Response) and HRTF (Head-Related Transfer Function), and a way to relate spatial information with temporal information of an audio signal, Preprint

The auditory system derives locations of sound sources from spatial cues provided by the interaction of sound with the head and external ears. Those cues are analyzed in specific brainstem pathways and then integrated as cortical representation of locations. The principal cues for horizontal localization are interaural time differences (ITDs) and interaural differences in sound level (ILDs). Vertical and front/back localization rely on spectral-shape cues derived from direction-dependent filtering properties of the external ears. The likely first sites of analysis of these cues are the medial superior olive (MSO) for ITDs, lateral superior olive (LSO) for ILDs, and dorsal cochlear nucleus (DCN) for spectral-shape cues. Localization in distance is much less accurate than that in horizontal and vertical dimensions, and interpretation of the basic cues is influenced by additional factors, including acoustics of the surroundings and familiarity of source spectra and levels. Listeners are quite sensitive to sound motion, but it remains unclear whether that reflects specific motion detection mechanisms or simply detection of changes in static location. Intact auditory cortex is essential for normal sound localization. Cortical representation of sound locations is highly distributed, with no evidence for point-to-point topography. Spatial representation is strictly contralateral in laboratory animals that have been studied, whereas humans show a prominent right-hemisphere dominance.

El objetivo de este trabajo será la creación de una herramienta gráfica con la cual se recree un sistema espacial de audio mediante la escucha con auriculares o altavoces. Esta inclusión de información espacial permitirá al oyente identificar y seleccionar diferentes posiciones virtuales de las fuentes sonoras. La incorporación de esta sensación espacial se obtendrá mediante el filtrado de la señal de audio a través de unos filtros especiales, cuya respuesta en frecuencia se denomina Head-Related Tranfer Functions. Además, para realizar este filtrado en tiempo real, se utilizará el eficiente método llamado Overlap-Save. Finalmente, se realizarán unos test auditivos para verificar el correcto funcionamiento de la herramienta y para analizar la fiabilidad de diferentes HRTF’s y comprobar si tiene algún efecto utilizar una u otra.

This research project examines the scope of the technique of timbre spatialisation in the frequency domain that can be realised and controlled in live performance by a single performer. Existing implementations of timbre spatialisation take either a psychoacoustical approach – employing control rate signals for determining azimuth and distance cues – or an adoption of abstract structures for determining frequency-space modulations. This research project aims to overcome the logistical constraints of real-time multi-parameter mapping by developing an overarching multi-signal framework for control: wave terrain synthesis, an interactive control rate and audio rate system. Due to the precise timing requirements of vectorbased FFT processes, spectral control data are generated in frames. Performed in MaxMSP, the project addresses notions of space and immersion using a practice-led methodology contributing to the creation of a number of compositions, performance software and an accompanying exegesis. In addition, the development and evaluation of timbre spatialisation software by the author is accompanied by a categorical definition of the spatial sound shapes generated., https://ro.ecu.edu.au/theses_ebooks/1003/thumbnail.jpg

Alguna de les aplicacions mèdiques que fan ús de sons amb una determinada localització espacial (com l'ajust dels implants coclears) no permeten l'ús d'auriculars en el subjecte sota estudi pel que han d'usar-se diverses fonts externes amb diferents ubicacions. El treball exposat pretén desenvolupar, basant-se en tècniques de reproducció transaural i usant únicament dos altaveus i senyals tant monofòniques com estereofònics, un generador de sons 3D que pugui usar-se en qualsevol sala sense més recursos que un equip estèreo. Per a això, i ja que els senyals estèreo no poden injectar en el subjecte mitjançant auriculars, sinó amb altaveus col·lats a certa distància de l'oient, s'ha d'implementar un cancel·lador de cross-talk que inverteixi l'efecte de la sala i la influència de cada un dels altaveus en l'oïda contraria al que envia la informació. A més, per crear virtualment l'efecte de localització espacial de la font (en el cas en què el so sigui monofònic i sense informació espacial), els sons generats han d'estar prèviament tractats mitjançant una etapa de filtrat adequada (usant bases de dades de filtres que tenen les funcions de transferència del cap humà o HRTF)., Alguna de las aplicaciones médicas que hacen uso de sonidos con una determinada localización espacial (como el ajuste de los implantes cocleares) no permiten el uso de auriculares en el sujeto bajo estudio por lo que han de usarse diversas fuentes externas con diferentes ubicaciones. El trabajo expuesto pretende desarrollar, basándose en técnicas de reproducción transaural y usando únicamente dos altavoces y señales tanto monofónicas como estereofónicas, un generador de sonidos 3D que pueda usarse en cualquier sala sin más recursos que un equipo estéreo. Para ello, y puesto que las señales estéreo no pueden inyectarse en el sujeto mediante auriculares, sino con altavoces colocados a cierta distancia del oyente, se ha de implementar un cancelador de cross-talk que invierta el efecto de la sala y la influencia de cada uno de los altavoces en el oído contrario al que envía la información. Además, para crear virtualmente el efecto de localización espacial de la fuente (en el caso en el que el sonido sea monofónico y sin información espacial), los sonidos generados han de estar previamente tratados mediante una etapa de filtrado adecuada (usando bases de datos de filtros que poseen las funciones de transferencia de la cabeza humana o HRTF )., Some of the medical applications that use sounds with a determined spatial loca- lization (as the adjustment of cochlear implants) do not allow the use of earphones in the subject under study so that it is required to use several different resources with dissimilar positions. The exposed work expects to develop, basing on transaural play- back techniques and using just two loudspeakers and monophonic and stereophonic signals, a generator of 3D sounds that could be used in any room with the only resour- ce of a stereo system. To do this, and being as the stereo signals cannot be injected in the subject by earphones, except by loudspeakers located away from the listener, it is necessary to implement a crosstalk cancellation that inverts the room’s effect and each loudspeaker’s influence on the ear located on the opposite side to whom it is. Furthermore, in order to create virtually the spatial localization effect of the resour- ce (in the case that the sound is monophonic and without spatial information), the sounds generated must be previously processed by a suitable filter (using databases of filters that own Human Related Transfer Function or HRTFs).

De verstaanbaarheid van de aanwezigen bij teleconferenties is minder dan bij gesprekken waarbij beide partijen zich in \'e\'en ruimte bevinden, aangezien stoorsignalen net zo goed worden opgenomen en verder de ruimtelijke informatie van de spraak verloren gaat. Een systeem werd ontworpen dat real-time bepaalt uit welke richting iemand spreekt, geluid uit overige richtingen onderdrukt en het geluid daarna afspeelt alsof het uit de richting kwam waarvandaan gesproken werd om zo het probleem te verhelpen. Acht microfoons nemen het geluid op, waarna dit geluid in het frequentiedomein wordt verwerkt. Eerst wordt er in vijf richtingen gebeamformd, waarna van elk van deze vijf richtingen de energie wordt bepaald. De richting met de hoogste energie wordt dan gekozen uit als de gewenste. Convolutie met een `head-related transfer function' geeft dan geluid dat lijkt alsof het uit de bepaalde richting komt. Dan wordt door middel van de Overlap-Add-methode het signaal teruggetransformeerd naar het tijdsdomein, waarna het wordt afgespeeld via een koptelefoon. De ontworpen beamformer werkt het beste boven frequenties van 2000 Hz. De richtingbepaling werkt voldoende nauwkeurig en snel. Verder kunnen luisteraars herkennen uit welke richting oorspronkelijk gesproken werd. Aanbevolen wordt in een vervolg een frequentieonafhankelijk beamformalgoritme te gebruiken, een richtingbepaling op basis van kruiscorrelatie toe te passen en een manier te ontwerpen die het geluid ruimtelijk weergeeft door luidsprekers., Multimedia Signal Processing, Electrical Engineering, Mathematics and Computer Science

[ANGLÈS] Binaural technology intends to recreate a 3D sound experience over headphones. This can be achieved by means of binaural recordings (using microphones inserted in the ear canal) or by binaural synthesis. The latter simulates virtual sources and room effect using dedicated filters, namely HRTFs. The Web Audio API is a W3C standard API for processing and synthesizing audio in web applications. As it is still in a process of development and the documentation is scarce, a deep study of the Web Audio API has been done. The aim of this project is to develop a real-time binaural synthesis processor in the browser using the Web Audio API. For the binaural synthesis approach, two different processing structures have been developed: synthesis of measured HRTF filters and synthesis of measured HRTF filters modeled as IIR filters and a delay., [CASTELLÀ] La tecnología binaural tiene como objetivo recrear una experiencia de sonido 3D a través de auriculares. Esto se consigue a través de grabaciones binaurales (usando micrófonos insertados en el conducto auditivo externo) o a través de la síntesis binaural. La síntesis binaural simula fuentes virtuales i el efecto de la sala usando filtros llamados filtros HRTF. La Web Audio API es una API de procesamiento y síntesis de audio para aplicaciones web estandarizada por el consorcio W3C. Como esta tecnología está aún en proceso de desarrollo y la documentación es escasa, se ha hecho un estudio profundo de la tecnología. El objetivo de este trabajo es la realización de un sistema de síntesis binaural en el navegador usando la Web Audio API. En este trabajo se han implementado dos métodos de síntesis binaural: la síntesis de filtros HRTF medidos y la síntesis de filtros HRTF medidos modelados a través de filtros IIR y un delay., [CATALÀ] La tecnologia binaural té com a objectiu recrear una experiència de so 3D a través d'auriculars. Això s'aconsegueix a través de gravacions binaurals (utilitzant micròfons inserits al conducte auditiu extern) o a través de síntesi binaural. La síntesi binaural simula fonts virtuals i l'efecte de sala utilitzant filtres anomenats filtres HRTF. La Web Audio API és una API de processament i síntesi d'àudio per aplicacions web estandarditzada pel consorci W3C. Com aquesta tecnologia està encara en procés de desenvolupament i la documentació és escassa, s'ha fet un estudi profund de la tecnologia. L'objectiu d'aquest treball és la realització d'un sistema de síntesi binaural al navegador utilitzant la Web Audio API. En aquest treball s'han implementat dos mètodes de síntesi binaural: la síntesi de filtres HRTF mesurats i la síntesi de filtres HRTF mesurats modelats a través de filtres IIR i un delay.

[EN] Multichannel audio signal processing has undergone major development in recent years. The incorporation of spatial information into an immersive audiovisual virtual environment or into video games provides a better sense of ¿presence¿ to applications. In a binaural system, spatial sound consists of reproducing audio signals with spatial cues (spatial information embedded in the sound) through headphones. This spatial information allows the listener to identify the virtual positions of the sources corresponding to different sounds. Headphone-based spatial sound is obtained by filtering different sound sources through a collection of special filters (whose frequency responses are called Head-Related Transfer Functions) prior to rendering them through headphones. These filters belong to a database composed by a limited number of spatial fixed positions. A complete audio application that can render multiple sound sources in any position of the space and virtualize movements of sound sources in real time demands high computing needs. Graphics Processing Units (GPUs) are highly parallel programmable coprocessors that provide massive computation when the needed operations are properly parallelized. This paper presents the design of a headphone-based multisource spatial audio application whose main feature is that all required processing is carried out on the GPU. To this end, two solutions have been approached in order to synthesize sound sources in spatial positions that are not included in the database and to virtualize sound sources movements between different spatial positions. The results show that the proposed application is able to move up to 240 sources simultaneously .

A well-known problem for reproductions of binaural recordings and simulations using HRTFs other than those of the actual listener is decreased localization performance and increased front-back confusion. Previous studies have indicated that moving the sound source can reduce these problems. The objective of this study was to measure how a unidirectional sound source movement along an orbit in the horizontal plane (achieved by reproducing sounds with a loudspeaker and turning a recording artificial head with a velocity of 5°/s) affects front-back confusion and localization errors. Three factors were studied: 1) movement angle (no movement, 20°, 30° and 40°) 2) acoustic environment (anechoic and reverberant) and 3) type of stimulus (male speech and recurring 840 Hz sinusoidal tones with a Gaussian envelope). Listening tests showed that front-back confusions decreased for 20° and 40° sound source movements, whereas it increased for 30° movement. The localization errors increased with about 5° when moving the sound sources. The front-back confusions and localization error was not affected by the implementation of reverberation. The frontback confusions occurred generally 3% more often for Gaussian tone bursts than speech, whereas the localization errors were equal., Validerat; 20130624 (global_studentproject_submitter)

[EN] The paper describes the results of virtual sound localization via an object detector in laboratory setting and open environment with blind people. In the tests have been involved two groups of ten blind people. Participants were aged between 25 and 60. The obstacle detection device reproduced 64 sounds at an azimuth of 64° for a distance of between 0.5 m to 5m in a horizontal plane to the level of the eyes. Virtual sounds were reproduced through headphones. For the experiment we used a recorded sound which varies between ¿110dB and ¿23dB with frequency between 86Hz to 18.604Hz. The sound was reproduced 44.1kH frequency and 32 bits, and a duration of 0.04641s. The results show that blind people are able to locate objects by virtually simulated sounds and detect them with high accuracy., [ES] El artículo describe los resultados obtenidos en la localización de sonidos virtuales mediante un dispositivo detector de objetos en entorno de laboratorio y en entorno abierto con personas invidentes. En las pruebas han participado dos grupos de diez personas invidentes. Los participantes tenían edades comprendidas entre 25 y 60 años. El dispositivo detector de obstáculos reproducía 64 sonidos en un azimut de 64º para una distancia de entre 0,5m hasta 5m en un plano horizontal al nivel de los ojos. Los sonidos virtuales se reproducían mediante auriculares. Para el experimento se ha utilizado un sonido grabado que varía entre ¿110dB y ¿23dB con frecuencia entre 86Hz hasta 18.604Hz. El sonido se reprodujo a una frecuencia de 44.1kH y 32 bits, y con una duración de 0.04641s. Los resultados nos muestran que las personas invidentes son capaces de localizar objetos mediante sonidos simulados virtualmente y detectarlos con gran precisión.

Individualization of the head-related transfer function (HRTF) to each listener is important for the listener to localize sound sources accurately. If an inappropriate HRTF is used, the sound localization is incorrect and there is a lower sense of presence. We investigate the admissible ranges for individualization of P1, N1, and N2 frequencies, which are spectral cues of HRTFs to localize sounds, in particular, on the median plane. The admissible ranges for individualization are the valid ranges of the spectral cues on the HRTFs at which each listener can localize presented sounds. We did listening tests to estimate the admissible ranges of P1, N1 and N2. The results suggest that N1 should be accurate, the admissible range is narrow, P1 and N2 are not strict, and also that strict tuning of P1 and N2 is not required. When applying a reconstructed HRTF for a certain listener selected from those of others in the database based on the condition that the N1, N2, and P1 of those HRTFs are limited to the admissible ranges, accuracy of sound localization approaches that applying the most appropriate HRTF for the listener.

La escucha binaural es un recurso que puede ser empleado en aplicaciones audiovisuales para obtener una experiencia inversiva que mejora el estéreo clásico y es comparable al sonido estéreo envolvente multicanal. Para ello es necesario un procesado específico a través de filtrado de tipo HRTF (función de transferencia derivada de la cabeza). Este proceso consiste en aplicar a cualquier sonido monofónico o estereofónico las características que el sistema auditivo necesita para interpretar la localización espacial tridimensional del sonido (indicios binaurales). Estos indicios dependen principalmente de características antropomórficas individuales, lo que introduce ciertos problemas debidos al uso de HRTF no individualizadas, como dificultad para localizar sonidos fuera de nuestra cabeza (internalizaciones) o confusiones entre delante y detrás (inversiones). Una buena comprensión de las características de la audición humana es necesaria para optimizar este procesado. En este trabajo se realiza un estado de la cuestión y un repaso de estudios realizados sobre la materia. Se constata que experimentos previos (Kistler, 1999; Mackensen, 2004; Werséni, 2009) apuntan a una mejora de la localización espacial con el uso de sonidos del mundo real, o a través de la simulación de movimientos relativos fuente sonora-oyente (indicios dinámicos). Se propone un diseño experimental consistente en un test de percepción con el fin de comparar el comportamiento de distintas señales de audio con distintas características temporales y espectrales, la influencia del movimiento y la fiabilidad de distintas herramientas de postproducción y grabación de campo binaurales. Se han probado sobre 11 sujetos, 3 tipos de sonido (espectralmente difuso, impulsivo y voz) y 4 técnicas de recreación binaural: procesado mediante software de postproducción de audio (panorama 5, binaural panner), filtrado con matlab y grabación de campo con microfonía binaural (Soundman OKM) sobre muestras estáticas y con m

Augmented reality, AR, has been popular on mobile devices the past years; these AR's are mostly build on graphics extraneous information. Sound can also be used as a part of an AR. For a sound-based AR to work on a mobile device, it needs to represent sounds in the 3d-space well, so that the users can perceive direction and distance to the source and also experience the virtual environment. The current three-dimensional sound engine on iPhone 4, openAL, lacks of both environmental and spatial clues to fully give a pleasant three-dimensional experience. A 3d-sound engine is proposed, using a generalized or individualized head-related transfer function as a spatial model and an image source method for early reflections and feedback delay networks for late reverberation. The proposed 3d-sound engine is implemented and tested against the current engine, openAL, with an additional reverberation module supported for Mac, which is not supported in the current version of OpenAL in iPhone 4. The test compares both 3d-sound engines of the accuracy in locating a source with respect to azimuth, elevation and distance. The result shows that the two sound engines are fairly equal in all three parameters. It also raised the importance of mapping the mental image to the sound source improve the precision., Validerat; 20110418 (anonymous)

[CA] En aquest treball es proposa el disseny de filtres HRTF per omplir el buit de la reproducció en elevació. S'han dissenyat filtres per diferents mètodes i han sigut avaluats mitjançant tests subjectius que demostren com de complex és el reconeixement o localització de fonts sonores a l'espai en elevació., EN] The study of elevation is an interesting subject of research in many signal processing applications, ranging from location and reproduction of sound sources to object location in electromagnetic exploration systems. Applied to the Head Related Transfer Function HRTF, it can be used to get 3D spatial reproduction systems based on Wave-field Synthesis (WFS). WFS system is a spatial sound reproduction system that can synthesize an acoustic field by means of horizontal loudspeaker arrays. However, even when several horizontal arrays are located in parallel in the vertical dimension, the desired reproduction effect cannot be obtained in elevation. In this work, we propose to design HRTF filters in order to fill the gap in the reproduction in elevation. Some HRTF filters have been designed by different methods, having been evaluated by subjective tests that show the high complexity of recognizing or locating sound sources in elevation in the space

Game players’ audio preference during actual game play was tested for. The goal of the study was to investigate whether or not HRTF encoded audio is preferred over normal stereo sound in a real game situation. The test was done with two sound alternatives, the normal stereo sound output of an Xbox console and the modified surround output of the same Xbox console. The surround output modification consisted of playback over a surround system and then downmix via a KU100 Neumann microphone in order to obtain a HRTF-encoded stereo alternative. The test was conducted with 20 test subjects playing a set of three games from different genres and for each game the subjects had to choose between the two sound alternatives and motivate their choice. For two out of three games the results were inconclusive and the whole groups’ choices of sound alternatives could not be considered statistically significant. The test result for the last game was statistically significant and pointed towards HRTF-encoded stereo sound being preferred. For the study as a whole it was not possible to either prove or reject the investigated thesis. This leads to the conclusion that further study, with an improved method, is needed to answer the original research question., Validerat; 20101217 (root)

The purpose of this project is to desig a systematical method in order to measure human directionality ability in horizontal plane with a single sound source. A completely virtual auditory model has been created in Matlab. The project consists of modeling binaural cues, designing digital filters, designing a test workbench, measuring listener's directionality and analyzing the data. The head related transfer function (HRTF) is computed by calculating the two most important binaural cues, interaural level difference (ILD) and interaural time difference (ITD). The platform is made in Matlab and all results have been shown by plots produced from Matlab code. The directionality test has been done with real human subjects and the results have been analyzed and presented.

This thesis study how hair affects frequencies between 1-10 kHz at different angles of azimuth, in the horizontal plane, when using an artificial head as a model. The azimuth varied from -30º to +180º in steps of 15 degrees. The experiment was conducted in an anechoic chamber. Artificial hair samples were selected after the amount of curliness and divided into groups. The results showed that the level was decreased with 1-8 dB at +0º azimuth. At +/-30º azimuth major notches appeared around 7 kHz. At +75º azimuth the largest decrease between artificial head with hair and without hair is observed between 1,7-3,3 kHz. Several new notches appear between 5-10 kHz for all hair samples when changing the azimuth to +90º. The notches have a level difference up to 20 dB. The conclusion is that the level in dB is decreased between 1,7-10 kHz depending on azimuth. The results also indicate that the amount of curliness could contribute to a decrease in level at +90º azimuth. The level in dB is also decreased at +/-30º azimuth but not depending on the high amount of curliness., Validerat; 20101217 (root)

Validerat; 20101217 (root)

This master thesis examines how the perception of listener envelopment is changed when altering the sidewall material in a computer model of a real concert hall. The research question is based on the lack of information on how the frequency content of the late arriving sound energy affects listener envelopment, but also on the conclusion that envelopment is an affect to large degree of late lateral reflective sounds. A computerized model is used for the purpose to take the research of listener envelopment into a more real acoustical situation than before. Alterations of the late lateral reflections are done by using four side wall materials attaining different sets of absorptions coefficients. A listening test is used for evaluating the change of listener envelopment and 21 listeners conducted the test. Results from earlier research were used to explain the attained results from the ANOVA tests. Sidewall materials which absorbs high frequencies and those that are nonabsorbent creates larger sensations of listener envelopment than materials in which absorbs low, low-mid and mid frequencies in this particular hall., Validerat; 20101217 (root)

This paper reports on the types and magnitudes of localization errors of simulated binaural direction cues generated using non-individualized, head-related transfer functions (HRTFs) with different levels of complexity. Four levels of complexity, as represented by the number of non-zero coefficients of the associated HRTF filters (128, 64, 32, 18 non-zero coefficients), were studied. Experiment I collected 1728 data runs that were exhaustive combinations of the four levels of complexity, nine simulated directions of sound (no direction (i.e., diotical-mono), 0 degrees, 45 degrees, 90 degrees, 135 degrees, 180 degrees, 225 degrees, 270 degrees, and 315 degrees azimuth angles at 0' elevation), two repetitions, and 24 participants). Binaural cues generated from HRTFs of reduced complexity (from 128 to 18 non-zero coefficients) produced significantly higher localization errors for the directions of 45 degrees, 135 degrees, 225 degrees, and 315 degrees azimuth angles (p < 0.01). From the directions of 0 degrees, 90 degrees, and 270 degrees azimuth angles, the cues produced by HRTFs with reduced complexity did not affect the localization error (p > 0.2). Surprisingly, cues produced by HRTFs of 128 non-zero coefficients did not have the lowest number of errors. From 45 degrees, 135 degrees, 225 degrees, and 315 degrees, the lowest numbers of errors were obtained from cues produced by HRTFs of 64, 32, 32, and 64 non-zero coefficients, respectively. Based on these findings, a prototype virtual headphone-based surround-sound (VHSS) system was developed. A double-blind usability experiment with 32 participants indicated that the prototype VHSS system received significantly better surround-sound ratings than did a DolbyTM stereo system (p < 0.02). This paper reports results from an original ergonomics study and the application of these results to the design of a consumer product. (c) 2006 Elsevier Ltd. All rights reserved.

Virtual acoustic prototyping (auralization) is the rendering of a virtual sound field that is created from the calculated acoustic response of a modeled structure. Auralization is useful in the design and subjective evaluation of buildings, automobiles, and aircraft. The virtual acoustic prototyping method in this thesis uses finite element modeling (FEM), the equivalent source method (ESM), and head-related transfer functions (HRTFs). A tradeoff exists between the accuracy of the auralization process and the number of equivalent sources (and thus computational power) that are required. The goal of this research is to validate (numerically and subjectively) a virtual acoustic prototyping method for use in structural design; this thesis illustrates the first attempt to apply the aforementioned methods to a structure that represents a typical building or automobile. The structure's acoustics were modeled using FEM, ESM, and HRTFs. A prototype of the modeled structure was built. A 36% correlation was achieved between the model and prototype. Slight variations in boundary conditions caused significant FEM error, but the data represented a typical structure. Psychoacoustic comparison testing was performed to determine the number of equivalent sources that must be used in an auralization to accurately recreate the sound field. The number was found to be dependent on the type of noise that is played to the test subject. A clear relationship between the numerical correlation of two sounds and the percentage of subjects who could hear a difference between those two sounds was established for impulsive, broadband, and engine noises.

The main goal of this research is to develop the theory and implement practical tools (in both software and hardware) for the capture and recreation of 3D auditory scenes. Our research is expected to have applications in virtual reality, telepresence, film, music, video games, auditory user interfaces, and sound-based surveillance. The first part of our research is concerned with sound capture via a spherical microphone array. The advantage of this array is that it can be steered into any 3D directions digitally with the same beampattern. We develop design methodologies to achieve flexible microphone layouts, optimal beampattern approximation and robustness constraint. We also design novel hemispherical and circular microphone array layouts for more spatially constrained auditory scenes. Using the captured audio, we then propose a unified and simple approach for recreating them by exploring the reciprocity principle that is satisfied between the two processes. Our approach makes the system easy to build, and practical. Using this approach, we can capture the 3D sound field by a spherical microphone array and recreate it using a spherical loudspeaker array, and ensure that the recreated sound field matches the recorded field up to a high order of spherical harmonics. For some regular or semi-regular microphone layouts, we design an efficient parallel implementation of the multi-directional spherical beamformer by using the rotational symmetries of the beampattern and of the spherical microphone array. This can be implemented in either software or hardware and easily adapted for other regular or semi-regular layouts of microphones. In addition, we extend this approach for headphone-based system. Design examples and simulation results are presented to verify our algorithms. Prototypes are built and tested in real-world auditory scenes.