Search

Obstacle avoidance is a major challenge during independent mobility for blind or visually impaired (BVI) people. Typically, BVI people can only perceive obstacles at a short distance (about 1 m, in case they are using the white cane), and some obstacles are hard to detect (e.g . , those elevated from the ground), or should not be hit by the white cane (e.g . , a standing person). A solution to these problems can be found in recent computer-vision techniques that can run on mobile and wearable devices to detect obstacles at a distance. However, in addition to detecting obstacles, it is also necessary to convey information about them in real time. This contribution presents WatchOut , a sonification technique for conveying real-time information about the main properties of an obstacle to a BVI person, who can then use this additional feedback to safely navigate in the environment. WatchOut was designed with a user-centered approach, involving four iterations of online listening tests with BVI participants in order to define, improve and evaluate the sonification technique, eventually obtaining an almost perfect recognition accuracy. WatchOut was also implemented and tested as a module of a mobile app that detects obstacles using state-of-the-art computer vision technology. Results show that the system is considered usable and can guide the users to avoid more than 85% of the obstacles.

This paper presents PhonHarp, a new interface for musical expression. The instrument can be described as a human-device loop, based on an Android app for mobile phones that produces sounds, which are modified by the vocal tract of the musician. After analyzing the state of the art, this work will provide details about functional requirements, design choices, and implementation aspects.

Recent developments in sensor technologies allowed the definition of new human-computer interaction channels, useful for people with very limiting motor disabilities such as quadriplegia. Some of these sensors are available pre-packaged on the mass market, complete with computer interaction softwares, while others are easily achievable at low costs through DIY approaches. In this article we present Resin, an Accessible Digital Musical Instrument dedicated to people with quadriplegic disability. Resin exploits two interaction channels, head movements and the shape of the vocal tract, detected through the corresponding acoustic resonances, to control musical performance parameters. The structure of the instrument is discussed, from both the hardware and software points of view. Feature extraction algorithms for both channels are explained, particularly focusing on the vocal tract resonances interaction paradigm.

Technological development, ubiquity of digital devices and the widespread possibility of Internet access have completely changed the way people approach the musical experience, significantly broadening the audience of producers and users of music. These advances make it important to reflect on how technology has changed the way music is taught and learned, and on the new possibilities offered by tangible, large-scale, augmented, and virtual reality interfaces. The purpose of this chapter is to systematically analyze and discuss digital learning materials explicitly conceived for a wide pedagogical area known as computer-based music education. In order to catch the aspects of heterogeneity on one side, but also, on the other side, make some relevant clusters emerge, the authors introduce a multidimensional classification schema. In particular, the proposed taxonomy considers the axes of cognitive approaches and methods, learning goals, categories of end users, implementation technologies, interfaces, and intended school types and levels. Finally, the most significant trends and practices are evaluated in order to introduce innovation in music education curricula.

Accessible Digital Musical instruments (ADMIs) dedicated to people with motor disabilities represent a relevant niche in accessibility research. The designer is often required to exploit unconventional physical interaction channels, different from hands and fingers. Although comprehensive evaluation methods for Digital Musical Instruments in general are found in literature, little has been done both in ADMIs evaluation and the analysis of suitable interaction channels from a Human-Computer Interaction perspective. In this work the performance of breath, gaze pointing and head movements is analyzed, in terms of movement speed and stability, through a simple experiment. These interaction channels could be exploited in the design of ADMIs dedicated to quadriplegic musicians. The proposed experiment has similarities with past Fitts Law evaluation tests. Results are discussed proposing possible mappings between channels and musical performance parameters. These results could also be useful to inform the design of different interface types.

This paper deals with the issue of individualizing the head-related transfer function (HRTF) rendering process for auditory elevation perception. Is it possible to find a nonindividual, personalized HRTF set that allows a listener to have an equally accurate localization performance than with his/her individual HRTFs? We propose a psychoacoustically motivated, anthropometry based mismatch function between HRTF pairs that exploits the close relation between the listener's pinna geometry and localization cues. This is evaluated using an auditory model that computes a mapping between HRTF spectra and perceived spatial locations. Results on a large number of subjects in the center for image processing and integrated computing (CIPIC) and acoustics research institute (ARI) HRTF databases suggest that there exists a nonindividual HRTF set, which allows a listener to have an equally accurate vertical localization than with individual HRTFs. Furthermore, we find the optimal parameterization of the proposed mismatch function, i.e., the one that best reflects the information given by the auditory model. Our findings show that the selection procedure yields statistically significant improvements with respect to dummy-head HRTFs or random HRTF selection, with potentially high impact from an applicative point of view.

Independent mobility is one of the main challenges for blind or visually impaired (BVI) people. In particular, BVI people often need to identify and avoid nearby obstacles, for example a bicycle parked on the sidewalk. This is generally achieved with a combination of residual vision, hearing and haptic sensing using the white cane. However, in many cases, BVI people can only perceive obstacles at short distance (typically about 1m, i.e., the white cane detection range), in other situations obstacles are hard to detect (e.g., those elevated from the ground), while others should not be hit by the white cane (e.g., a standing person). Thus, some time and effort are required to identify the object in order to understand how to avoid it. A solution to these problems can be found in recent computer vision techniques that can run on mobile and wearable devices to detect obstacles at a distance. However, in addition to detecting obstacles, it is also necessary to convey information about them to a BVI user. This contribution presents WatchOut, a sonification technique for conveying real-time information about the main characteristics of an obstacle to a BVI person, who can then use this additional feedback to safely navigate in the environment. WatchOut was designed with a user-centric approach, involving two iterations of online questionnaires with BVI participants in order to define, improve and evaluate the sonification technique. WatchOut was implemented and tested as a module of a mobile app that detects obstacles using state-of-the-art computer vision technology. Results show that the system is considered usable, and can guide the users to avoid more than 85% of the obstacles.

Mobile virtual reality (VR) is increasingly becoming popular and accessible to everyone that holds a smartphone. In particular, digital didactics can take advantage of natural interaction and immersion in virtual environments, starting from primary education. This paper investigates the problem of enhancing music learning in primary education through the use of mobile VR. To this end, technical and methodological frameworks were developed, and were tested with two classes in the last year of a primary school (10 years old children). The classes were involved in an evaluation study on music genre identification and learning with a multi-platform mobile application called VR4EDU. Students were immersed in music performances of different genres (e.g., classical, country, jazz, and swing), navigating inside several musical rooms. The evaluation of the didactic protocol shows a statistically significant improvement in learning genre characterization (i.e., typical instruments and their spatial arrangements on stage) compared to traditional lessons with printed materials and passive listening. These results show that the use of mobile VR technologies in synergy with traditional teaching methodologies can improve the music learning experience in primary education, in terms of active listening, attention, and time. The inclusion of pupils with certified special needs strengthened our results.

Search and retrieval of multimedia content based on the evoked emotion comprises an interesting scientific field with numerous applications. This paper proposes a method that fuses two heterogeneous modalities, i.e. music and electroencephalographic signals, both for predicting emotional dimensions in the valence-arousal plane and for addressing four binary classification tasks, namely i.e. high/low arousal, positive/negative valence, high/low dominance, high/low liking. The proposed solution exploits Mel-scaled and EEG spectrograms feeding a k-medoids clustering scheme based on canonical correlation analysis. A thorough experimental campaign carried out on a publicly available dataset confirms the efficacy of such an approach. Despite its low computational cost, it was able to surpass state of the art results, and most importantly, in a user-independent manner.

In this paper we explore the potential of 3D printing in the context of preschool music learning, with the aim of making visible its educational value. After analyzing embodied and enactive approaches in preschool and the adoption of tangible interfaces for music education, we review novel technologies in the field of additive fabrication and 3D printing and highlight their potential for music learning. Subsequently, we propose didactic experiences based on a manipulative approach and using already-available or user-customized 3D models. Finally, we present a case study that investigates alternative forms of music notation.

This paper proposes an image-guided HRTF selection procedure that exploits the relation between features of the pinna shape and HRTF notches. Using a 2D image of a user's pinna, the procedure selects from a database the HRTF set that best fits the anthropometry of that user. The proposed procedure is designed to be quickly applied and easy to use for a user without previous knowledge on binaural audio technologies. The entire process is evaluated by means of (i) an auditory model for sound localization in the mid-sagittal plane available from previous literature, and (ii) a short localization test in virtual reality. Using both virtual and real subjects from an HRTF database, predictions and the experimental evaluation aimed to assess the vertical localization performance with HRTF sets selected by the proposed procedure. Our results report a statistically significant improvement in predictions of the auditory model for localization performance with selected HRTFs compared to KEMAR HRTFs, which is a commercial standard in many binaural audio solutions. Moreover, the proposed localization test with human listeners reflect the model's predictions, further supporting the applicability of our perceptually-motivated metrics with anthropometric data extracted by pinna images.

Indoor navigation services for people with visual impairment are being researched in academia, and working systems have already been deployed in public places. While previous research mainly focuses on computing the user’s position with high accuracy, providing non-visual navigation instructions is also a challenge and naive approaches can fail in helping users reach their target destination or even expose them to hazards.In this paper we investigate the problem of guiding users to rotate towards a target direction. We propose three different sonification techniques that provide continuous guidance during rotation, and we compare them to a single-impulse baseline, used in previous works. We also explore three variations that reinforce the proposed techniques by combining them with the baseline. A preliminary study with 10 blind participants highlights two dominant techniques, which we analyze through a follow-up study with 18 participants, from 2 groups with very distant cultural backgrounds. While stark differences emerge in the performance from the two groups, we highlight two clear results common to both: 1) one of the proposed techniques significantly outperforms the baseline, reducing the average rotation error by a factor of 3.5 (from 11° to 3°); 2) the interaction speed of this technique, generally slower than the baseline, significantly improves when combined with the baseline technique.

Towards improving the performance in various music information processing tasks, recent studies exploit different modalities able to capture diverse aspects of music. Such modalities include audio recordings, symbolic music scores, mid-level representations, motion, and gestural data, video recordings, editorial or cultural tags, lyrics and album cover arts. This paper critically reviews the various approaches adopted in Music Information Processing and Retrieval and highlights how multimodal algorithms can help Music Computing applications. First, we categorize the related literature based on the application they address. Subsequently, we analyze existing information fusion approaches, and we conclude with the set of challenges that Music Information Retrieval and Sound and Music Computing research communities should focus in the next years.

A library of piano samples composed of binaural recordings and keyboard vibrations has been built, with the aim of sharing accurate data that in recent years have successfully advanced the knowledge on several aspects about the musical keyboard and its multimodal feedback to the performer. All samples were recorded using calibrated measurement equipment on two Yamaha Disklavier pianos, one grand and one upright model. This paper documents the sample acquisition procedure, with related calibration data. Then, for sound and vibration analysis, it is shown how physical quantities such as sound intensity and vibration acceleration can be inferred from the recorded samples. Finally, the paper describes how the samples can be used to correctly reproduce binaural sound and keyboard vibrations. The library has potential to support experimental research about the psycho-physical, cognitive and experiential effects caused by the keyboard's multimodal feedback in musicians and other users, or, outside the laboratory, to enable an immersive personal piano performance., + ID: 561053 + PeerReviewed

This paper presents a multimodal interactive system for non-visual (auditory-haptic) exploration of virtual maps. The system is able to display haptically the height profile of a map, through a tactile mouse. Moreover, spatial auditory information is provided in the form of virtual anchor sounds located in specific points of the map, and delivered through headphones using customized Head-Related Transfer Functions (HRTFs). The validity of the proposed approach is investigated through two experiments on non-visual exploration of virtual maps. The first experiment has a preliminary nature and is aimed at assessing the effectiveness and the complementarity of auditory and haptic information in a goal reaching task. The second experiment investigates the potential of the system in providing subjects with spatial knowledge: specifically in helping with the construction of a cognitive map depicting simple geometrical objects. Results from both experiments show that the proposed concept, design, and implementation allow to effectively exploit the complementary natures of the "proximal" haptic modality and the "distal" auditory modality. Implications for orientation & mobility (O&M) protocols for visually impaired subjects are discussed. Graphical abstractDisplay Omitted HighlightsWe develop a multimodal interactive system for nonvisual exploration of virtual maps.Ecological and minimalistic principles guide customized binaural audio reproduction.Goal reaching and map reconstruction tasks assess contributions for each modality.The "proximal" haptic modality and the "distal" auditory modality are complementary.3D auditory feedback improves performances with respect to 2D auditory feedback.

Indoor navigation is an important service, currently investigated both in industry and academia. While the main focus of research is the computation of users' position, the additional challenge of conveying guidance instructions arises when the target user is blind or visually impaired (BVI). This contribution presents our ongoing research aimed at adopting sonification techniques to guide a BVI person. In particular we introduce three sonification techniques to guide the user during rotations. Preliminary results, conducted with 7 BVI people, show that some of the proposed sonification technique outperform a benchmark solution adopted in previous contributions.

Achieving a full 3D auditory experience with head-related transfer functions (HRTFs) is still one of the main challenges of spatial audio rendering. HRTFs capture the listener's acoustic effects and personal perception, allowing immersion in virtual reality (VR) applications. This paper aims to investigate the connection between listener sensitivity in vertical localization cues and experienced presence, spatial audio quality, and attention. Two VR experiments with head-mounted display (HMD) and animated visual avatar are proposed: (i) a screening test aiming to evaluate the participants' localization performance with HRTFs for a non-visible spatialized audio source, and (ii) a 2 minute free exploration of a VR scene with five audiovisual sources in a both non-spatialized (2D stereo panning) and spatialized (free-field HRTF rendering) listening conditions. The screening test allows a distinction between good and bad localizers. The second one shows that no biases are introduced in the quality of the experience (QoE) due to different audio rendering methods; more interestingly, good localizers perceive a lower audio latency and they are less involved in the visual aspects.

Repeatability of head-related transfer function (HRTF) measurements is a critical issue in intra- and inter- laboratory setups. In this paper, simulated perceptual variabilities of HRTFs are computed as an attempt to understand if different acquisition methods achieve similar results in terms of psychoacoustic features. We consider 12 HRTF independent measurement sets of a Neumann KU-100 dummy head from the international round-robin study Club Fritz. Our analysis of HRTF variabilities focuses on localization performance in elevation within the mid-sagittal plane. A round robin evaluation is performed by means of an auditory model which is able to predict elevation errors and front-back confusion for a given pair of target and template HRTF sets. Results report comparable localization performances between four HRTF databases, suggesting that these acquisition methods led to similar performances in providing elevation cues. Such findings further emphasize the intrinsic complexity and the sensitivity of the HRTF measurement process. The final aim of this study is to certify the quality and repeatability of a measurement process at perceptual level; this findings could be extended to the acquisition of human head acoustics.

While a standard approach is more or less established for rendering basic vibratory cues in consumer electronics, the implementation of advanced vibrotactile feedback still requires designers and engineers to solve a number of technical issues. Several off-the-shelf vibration actuators are currently available, having different characteristics and limitations that should be considered in the design process. We suggest an iterative approach to design in which vibrotactile interfaces are validated by testing their accuracy in rendering vibratory cues and in measuring input gestures. Several examples of prototype interfaces yielding audio-haptic feedback are described, ranging from open-ended devices to musical interfaces, addressing their design and the characterization of their vibratory output., + ID: 561643 + Reihentitel: Springer Series on Touch and Haptic Systems

Virtual and augmented realities are expected to become more and more important in everyday life in the next future; the role of spatial audio technologies over headphones will be pivotal for application scenarios which involve mobility. This paper introduces the SelfEar project, aimed at low-cost acquisition and personalization of Head-Related Transfer Functions (HRTFs) on mobile devices. This first version focuses on capturing individual spectral features which characterize external ear acoustics, through a self-adjustable procedure which guides users in collecting such information: their mobile device must be held with the stretched arm and positioned at several specific elevation points; acoustic data are acquired by an audio augmented reality headset which embeds a pair of microphones at listener ear-canals. A preliminary measurement session assesses the ability of the system to capture spectral features which are crucial for elevation perception. Moreover, a virtual experiment using a computational auditory model predicts clear vertical localization cues in the measured features.

This paper presents a novel spatial auditory display that combines a virtual environment based on a Digital Waveguide Mesh (DWM) model of a small tubular shape with a binaural rendering system with personalized head-related transfer functions (HRTFs) allowing interactive selection of absolute 3D spatial cues of direction as well as egocentric distance. The tube metaphor in particular minimizes loudness changes with distance, providing mainly direct-to-reverberant and spectral cues. The proposed display was assessed through a target-reaching task where participants explore a 2D virtual map with a pen tablet and hit a sound source (the target) using auditory information only; subjective time to hit and traveled distance were analyzed for three experiments. The first one aimed at assessing the proposed HRTF selection method for personalization and dimensionality of the reaching task, with particular attention to elevation perception; we showed that most subjects performed better when they had to reach a vertically unbounded (2D) rather then an elevated (3D) target. The second experiment analyzed interaction between the tube metaphor and HRTF showing a dominant effect of DWM model over binaural rendering. In the last experiment, participants using absolute distance cues from the tube model performed comparably well to when they could rely on more robust, although relative, intensity cues. These results suggest that participants made proficient use of both binaural and reverberation cues during the task, displayed as part of a coherent 3D sound model, in spite of the known complexity of use of both such cues. HRTF personalization was beneficial for participants who were able to perceive vertical dimension of a virtual sound. Further work is needed to add full physical consistency to the proposed auditory display.

This paper focuses on the localization of footstep sounds interactively generated during walking and provided through headphones. Three distinct experiments were conducted in a laboratory involving a pair of sandals enhanced with pressure sensors and a footstep synthesizer capable of simulating two typologies of surface materials: solid (e.g., wood) and aggregate (e.g., gravel). Different sound delivery methods (mono, stereo, binaural) as well as several surface materials, in the presence or absence of concurrent contextual auditory information provided as soundscapes, were evaluated in a vertical localization task. Results showed that solid surfaces were localized significantly farther from the walker's feet than the aggregate ones. This effect was independent of the used rendering technique, of the presence of soundscapes, and of merely temporal or spectral attributes of sound. The effect is hypothesized to be due to a semantic conflict between auditory and haptic information such that the higher the semantic incongruence the greater the distance of the perceived sound source from the feet. The presented results contribute to the development of further knowledge toward a basis for the design of continuous multimodal feedback in virtual reality applications .

The scattering around the human pinna that is captured by the Head-Related Transfer Functions (HRTFs) is a complex problem that creates uncertainties in both acoustical measurements and simulations. Within the simulation framework of Finite Difference Time Domain (FDTD) with axis-aligned staircase boundaries resulting from a voxelization process, the voxelization-based uncertainty propagating in the HRTF-captured sound field is quantified for one solid and two surface voxelization algorithms. Simulated results utilizing a laser-scanned mesh of Knowles Electronics Manikin for Acoustic Research (KEMAR) show that in the context of complex geometries with local topology comparable to grid spacing such as the human pinna, the voxelization-related uncertainties in simulations emerge at lower frequencies than the generally used accuracy bandwidths. Numerical simulations show that the voxelization process induces both random error and algorithm-dependent bias in the simulated HRTF spectral features. Frequencies fr below which the random error is bounded by various dB thresholds are estimated and predicted. Particular shortcomings of the used voxelization algorithms are identified and the influence of the surface impedance on the induced errors is studied. Simulations are also validated against measurements.

This paper presents a multimodal rendering architecture that integrates physically based sound models with haptic and visual rendering. The proposed sound modeling approach is compared to other existing techniques. An example of implementation of the architecture is presented, that realizes bimodal (auditory and haptic) rendering of contact stiffness. It is shown that the proposed rendering scheme allows tight synchronization of the two modalities, as well as a high degree of interactivity and responsiveness of the sound models to gestures and actions of a user. Finally, an experiment on the relative contributions of haptic and auditory information to bimodal judgments of contact stiffness is presented. Experimental results support the effectiveness of auditory feedback in modulating haptic perception of stiffness. Copyright © 2006 John Wiley & Sons, Ltd.

A novel approach to the selection of generic head-related transfer functions (HRTFs) for binaural audio rendering through headphones is formalized and described in this paper. A reflection model applied to the user’s ear picture facilitates extraction of the relevant anthropometric cues that are used for selecting two HRTF sets in a database fitting that user, whose localization performances are evaluated in a complete psychoacoustic experiment. The proposed selection increases the average elevation performances of 17% (with a peak of 34%) with respect to generic HRTFs from an anthropomorphic mannequin. Italso significantly enhances externalization and reduces the number of up/down reversals.

The auditory perception of materials is a popular topic in the study of non-vocal sound-source perception. In this chapter, we review the empirical evidence on the mechanical and acoustical correlates of the perception of impacted stiff materials, and of the state of matter of sound-generating substances (solids, liquids, gases). As a whole, these studies suggest that recognition abilities are only highly accurate when differentiating between widely diverse materials (e.g. liquids vs. solids or plastics vs. metals) and that limitations in the auditory system, along with the possible internalization of biased statistics in the acoustical environment (e.g. clinking-glass sounds tend to be produced by small objects), might account for the less-than-perfect ability to differentiate between mechanically similar materials. This review is complemented by a summary of studies concerning the perception of deformable materials (fabrics and liquids) and the perceptual and motor-behaviour effects of auditory material-related information in audio-haptic contexts. The results of perceptual studies are the starting point for the development of interactive sound synthesis techniques for rendering the main auditory correlates of material properties, starting from physical models of the involved mechanical interactions. We review the recent literature dealing with contact sound synthesis in such fields as sonic interaction design and virtual reality. Special emphasis is given to softness/hardness correlates in impact sounds, associated with solid object resonances excited through impulsive contact. Synthesis methods for less studied sound-generating systems such as deformable objects (e.g. fabrics and liquids) and aggregate materials are also described.

Purpose – The purpose of this paper is to present a system for customized binaural audio delivery based on the extraction of relevant features from a 2-D representation of the listener’s pinna. Design/methodology/approach – The most significant pinna contours are extracted by means of multi-flash imaging, and they provide values for the parameters of a structural head-related transfer function (HRTF) model. The HRTF model spatializes a given sound file according to the listener’s head orientation, tracked by sensor-equipped headphones, with respect to the virtual sound source. Findings – A preliminary localization test shows that the model is able to statically render the elevation of a virtual sound source better than non-individual HRTFs. Research limitations/implications – Results encourage a deeper analysis of the psychoacoustic impact that the individualized HRTF model has on perceived elevation of virtual sound sources. Practical implications – The model has low complexity and is suitable for implementation on mobile devices. The resulting hardware/software package will hopefully allow an easy and low-tech fruition of custom spatial audio to any user. Originality/value – The authors show that custom binaural audio can be successfully deployed without the need of cumbersome subjective measurements.

The contribution of the external ear to the head-related transfer function (HRTF) heavily depends on the listener's unique anthropometry. In particular, the shape of the most prominent contours of the pinna defines the frequency location of the HRTF spectral notches along the elevation of the sound source. This paper addresses the issue of automatically estimating the location of pinna edges starting from a set of pictures produced by a multi-flash imaging device. A basic image processing algorithm designed to obtain the principal edges and their distance from the ear canal entrance is described. The effectiveness of the developed hardware and software is preliminarily evaluated on a small number of test subjects.

The paper presents a system for customized binaural audio delivery based on the extraction of the relevant features from a 2-D representation of the listener's pinna. A procedure based on multi-flash imaging for recognizing the main contours of the pinna and their position with respect to the ear canal entrance is detailed. The resulting contours drive the parametrization of a structural head-related transfer function model that performs in real time the spatialization of a desired sound file according to the listener's position with respect to the virtual sound source, tracked by sensor-equipped headphones. The low complexity of the model allows smooth implementation and delivery on any mobile device. The purpose of the desired system is to provide low-tech custom binaural audio to any user without the need of tedious and cumbersome subjective measurements.

A novel approach to the modeling of head-related transfer functions (HRTFs) for binaural audio rendering is formalized and described in this paper. Mixed structural modeling (MSM) can be seen as the generalization and extension of the structural modeling approach first defined by Brown and Duda back in 1998. Possible solutions for building partial HRTFs (pHRTFs) of the head, torso, and pinna of a specific listener are first described and then used in the construction of two possible mixed structural models of a KEMAR mannequin. Thanks to the flexibility of the MSM approach, an exponential number of solutions for building custom binaural audio displays can be considered and evaluated, the final aim of the process being the achievement of a HRTF model fully customizable by the listener.

A physically based impact model-already known and exploited in the field of sound synthesis-is studied using both analytical tools and numerical simulations. It is shown that the Hamiltonian of a physical system composed of a mass impacting on a wall can be expressed analytically as a function of the mass velocity during contact. Moreover, an efficient and accurate approximation for the mass outbound velocity is presented, which allows to estimate the Hamiltonian at the end of the contact. Analytical results are then compared to numerical simulations obtained by discretizing the system with several numerical methods. It is shown that, for some regions of the parameter space, the trajectories of the discretized systems may significantly drift from the analytically derived curves. Two approaches, based on enforcing numerical energy consistency, are then proposed to improve the accuracy of numerical simulations.

This paper addresses the problem of modeling head-related transfer functions (HRTFs) for 3-D audio rendering in the front hemisphere. Following a structural approach, we build a model for real-time HRTF synthesis which allows to control separately the evolution of different acoustic phenomena such as head diffraction, ear resonances, and reflections through the design of distinct filter blocks. Parameters to be fed to the model are both derived from mean spectral features in a collection of measured HRTFs and anthropometric features of the specific subject (taken from a photograph of his/her outer ear), hence allowing model customization. Visual analysis of the synthesized HRTFs reveals a convincing correspondence between original and reconstructed spectral features in the chosen spatial range. Furthermore, a possible experimental setup for dynamic psycho acoustical evaluation of such model is depicted.

This paper faces the general problem of modeling pinna-related transfer functions (PRTFs) for 3-D sound rendering. Following a structural approach, we aim at constructing a model for PRTF synthesis which allows to control separately the evolution of ear resonances and spectral notches through the design of two distinct filter blocks. Taking such model as endpoint, we propose a method based on the McAulay-Quatieri partial tracking algorithm to extract the frequencies of the most important spectral notches. Ray-tracing analysis performed on the so obtained tracks reveals a convincing correspondence between extracted frequencies and pinna geometry of a bunch of subjects.

In this paper we will investigate how non---visual senses can be used in toys to enhance and enrich the play experience of all children, while favoring accessibility and inclusion of visually-impaired children. Previous research has shown that --- especially for young children developing sensory-motor skills --- exploration and play are two tightly linked activities: everything is new and needs to be "investigated" and playful behaviors emerge from active exploration. We will propose a new approach in designing and creating objects that elicit this type of behavior and encourage exploration by providing real---time dynamic, haptic, tactile, auditory, and even olfactory feedback depending on children's gestures, movements, and emitted sounds. We believe that this design paradigm is highly innovative with respect to previous research and existing products --- whose interaction is very often based on static feedback. Interactive and dynamic feedback is intrinsically more engaging and allows a variety of quality learning patterns.

This paper reports on a study on the perception and rendering of distance in multimodal virtual environments. A model for binaural sound synthesis is discussed, and its integration in a real-time system with motion tracking and visual rendering is presented. Results from a validation experiment show that the model effectively simulates relevant auditory cues for distance perception in dynamic conditions. The model is then used in a subsequent experiment on the perception of egocentric distance. The design and preliminary result from this experiment are discussed.

This paper presents an experiment on the relative contributions of haptic and auditory information to bimodal judgments of contact stiffness using a rigid probe. Haptic feedback is rendered via a Phantom® OmniTM device, while auditory stimuli are obtained using a physically-based audio model of impact, in which the colliding objects are described as modal resonators that interact through a non-linear impact force. The impact force can be controlled through a stiffness parameter, that influences the contact time of the impact. Previous studies have already indicated that this parameter has a major influence on the auditory perception of hardness/stiffness. In the experiment subjects had to tap on virtual surfaces, and were presented with audio-haptic feedback. In each condition the haptic stiffness had the same value while the acoustic stiffness was varied. Perceived stiffness was determined using an absolute magnitude-estimation procedure: subjects were asked to rate the surfaces on an ordered scale of verbal labels, based on their perceived stiffness. The results indicate that subjects consistently ranked the surfaces according to the auditory stimuli.

Acoustic simulation of friction is a particularly challenging task, because continuous (strong) contact conditions require a tight and veridical integration of the synthesis layer with the control input. This paper presents an algorithmic realization that combines recently proposed physical models of friction with the lumped modal description of resonating bodies. It is shown that the resulting nonlinear dynamical system can be discretized using a numerical technique that allows efficient and accurate simulation. Applications in the context of interactive audio-visual animation on low-cost general-purpose computers are demonstrated, and an approach to joint audio-visual synthesis is proposed that provides fine-scale synchronization and high coherence between the two modalities. The interactive animations show that the model is successful in reproducing several salient everyday sound phenomena, such as rubbing, braking, and squeaky doors.

The paper presents a general procedure for the computation of filter networks made of linear filters and nonlinear non-algebraic (dynamic) elements. The method is developed in the Kirchhoff domain and applies to cases where the network contains an arbitrary number of delay-free paths that involve nonlinear elements. Compared to existing techniques the method does not require a rearrangement of the network structure, instead it subdivides the network into computational substructures specified by appropriate matrices related to the network topology. Sufficient conditions are discussed for the applicability of the method, and results are provided that relate performance of the method to the properties of the nonlinear elements and to the network topology. The last part of the paper discusses applications of the method to the simulation of acoustic systems, including multidimensional wave propagation by means of waveguide-mesh techniques.

The high tides that periodically flood Venice, locally known as acqua alta, are becoming more serious due to recent changes in the surrounding lagoon as well as atmospheric conditions. A special office of the Municipality of Venice, the Center for Tide Prediction and Warning (Centro Previsioni e Segnalazioni Maree-CPSM), provides a continuous tide forecast based on computational models as well as astronomical and meteorological data. When a significant high tide is expected, city authorities activate a network of electromechanical sirens for a few minutes, usually anticipating the tide peak by a few hours. The sirens, however, emit threatening wails reminiscent of air attack warnings, do not convey the gravity of the threat, and may not reach isolated or distant areas. Drawing on a wide range of computing technologies and methodologies, the authors present a new auditory alert system for high tides in Venice designed to replace the existing network of electromechanical sirens. As part of this research effort, our project team first analyzed the current alert system using off-the-shelf acoustic simulation software and a specially designed visualization tool. We then used a form of constraint logic programming to determine the optimal placement of loudspeakers in Venice, a complex task with many physical, economic, and social constraints. Next, we created the alert sounds for our demanding listening environment. The final phase of the project involved iteratively validating and redesigning the alert signals using human testing.

This paper reviews recent developments in physics-based synthesis of piano. The paper considers the main components of the instrument, that is, the hammer, the string, and the soundboard. Modeling techniques are discussed for each of these elements, together with implementation strategies. Attention is focused on numerical issues, and each implementation technique is described in light of its efficiency and accuracy properties. As the structured audio coding approach is gaining popularity, the authors argue that the physical modeling approach will have relevant applications in the field of multimedia communication.

This paper describes an algorithm for real-time synthesis of contact sounds for interactive simulations and animation. The algorithm is derived from a physically-based impact model, and the acoustic characteristics of colliding objects can be realistically simulated by properly adjusting the physical parameters of the model. A technique for describing the spatial dynamics of a resonating object is proposed, which allows simulation of position-dependent interaction. It is shown that the numerical implementation leads to an efficient sound synthesis module, that runs in real-time on low cost platforms. The effectiveness of the model is demonstrated, and its applications are discussed.

The Sonik Spring is a portable and wireless digital instrument, created for real-time synthesis and control of sound. It brings together different types of sensory input, linking gestural motion and kinesthetic feedback to the production of sound. The interface consists of a 15-inch spring with unique flexibility, which allows multiple degrees of variation in its shape and length. The design of the instrument is described and its features discussed. Three performance modes are detailed highlighting the instrument's expressive potential and wide range of functionality.