Search

Inspired by recent developments in data-driven methods for partial differential equation (PDE) estimation, we use sparse modeling techniques to automatically estimate PDEs from data. A dictionary consisting of hypothetical PDE terms is constructed using numerical differentiation. Given data, PDE terms are selected assuming a parsimonious representation, which is enforced using a sparsity constraint. Unlike previous PDE identification schemes, we make no assumptions about which PDE terms are responsible for a given field. The approach is demonstrated on synthetic and real video data, with physical phenomena governed by wave, Burgers, and Helmholtz equations. Codes are available at https://github.com/NoiseLab-RLiu/Automate-PDE-identification.

The synthesis of convincing acoustic drum sounds remains an open problem. In this paper, a method for analysing and synthesising pitch glide in drums is proposed, whereby the discrete cosine transform (DCT) of an unwindowed drum sound is modelled. This is an extension of the scheme initially proposed by Kirby and Sandler [(2020). Proceedings of the 23rd International Conference on Digital Audio Effects, Vienna, Austria, pp. 155-162], which was able to reproduce key components of drum sounds accurately enough that they could not be distinguished from the reference samples. Here, drum modes were analysed in greater detail for a tom-tom struck at 67 different intensities to investigate their evolution with strike velocity. A clear evolution was observed in the DCT features, and interpolation was used to synthesise the modes of intermediate velocity. These synthesised modes were evaluated objectively through null testing, which showed that a continuous blending of strike velocities could be achieved throughout the data set. An AB listening test was also performed, where 20 participants attempted to distinguish between pairs of real and synthesised sounds. Exactly 50% accuracy was achieved overall, which demonstrates that the synthesised samples were deemed to sound as realistic as genuine samples. These results demonstrate that the DCT representation is a valuable framework for analysis and synthesis of drum sounds. It is also likely that this approach could be applied to other instruments.

The two-dimensional (2D) grid-free compressive beamforming based on atomic norm minimization is a promising solution for estimating the source direction-of-arrival and quantifying the source strength. However, the existing method is limited to measuring with rectangular grid arrays. To overcome this limitation, a 2D grid-free compressive beamforming strategy for arbitrary planar array geometries is proposed, where the microphone pressure is transformed into a representation of 2D Fourier series expansion. The term truncation method of Fourier series polynomials is established. The effectiveness and feasibility of the proposed method in the acoustic source identification are preliminarily verified by an experimental case, and topics worthy of further study are discussed based on the numerical simulations.

While source localization and seabed classification are often approached separately, the convolutional neural networks (CNNs) in this paper simultaneously predict seabed type, source depth and speed, and the closest point of approach. Different CNN architectures are applied to mid-frequency tonal levels from a moving source recorded on a 16-channel vertical line array (VLA). After training each CNN on synthetic data, a statistical representation of predictions on test cases is presented. The performance of a single regression-based CNN is compared to a multitask CNN in which regression is used for the source parameters and classification for the seabed type. The impact of water sound speed profile and seabed variations on the predictions is evaluated using simulated test cases. Environmental mismatch between the training and testing data has a negative impact on source depth estimates, while the remaining labels are estimated tolerably well but with a bias towards shorter ranges. Similar results are found for data measured on two VLAs during Seabed Characterization Experiment 2017. This work shows the superiority of multitask learning and the potential for using a CNN to localize an acoustic source and detect the surficial seabed properties from mid-frequency sounds.

Critical acoustical systems operating in complex environments contaminated with disturbances and noise offer an extreme challenge when excited by out-of-the-ordinary, impulsive, transient events that can be undetected and seriously affect their overall performance. Transient impulse excitations must be detected, extracted, and evaluated to determine any potential system damage that could have been imposed; therefore, the problem of recovering the excitation in an uncertain measurement environment becomes one of multichannel deconvolution. Recovering a transient and its initial energy has not been solved satisfactorily, especially when the measurement has been truncated and only a small segment of response data is available. The development of multichannel deconvolution techniques for both complete and incomplete excitation data is discussed, employing a model-based approach based on the state-space representation of an identified acoustical system coupled to a forward modeling solution and a Kalman-type processor for enhancement and extraction. Synthesized data are utilized to assess the feasibility of the various approaches, demonstrating that reasonable performance can be achieved even in noisy environments.

The speech signal is inherently variable and listeners need to recalibrate when local, short-term distributions of acoustic dimensions deviate from long-term representation. The present experiment investigated the specificity of this perceptual adjustment, addressing whether the perceptual system is capable of tracking differing simultaneous short-term acoustic distributions of the same speech categories, conditioned by context. The results indicated that instead of aggregating over the contextual variation, listeners tracked separate distributional statistics for instances of speech categories experienced in different phonetic/lexical contexts, suggesting that perceptual learning is not only influenced by distributional statistics, but also by external factors such as contextual information.

In this paper, we introduce an extension of the image method for generating room impulse responses in a structure with more than a single confined space, namely, the structure image method (StIM). The proposed method, StIM, can efficiently generate a large number of environmental examples for a structure impulse response, which is required by current deep-learning methods for many tasks, while maintaining low computational complexity. We address the integration of the environment representation, produced by StIM, into the training process, and present a framework for training deep models. We demonstrate the usage of StIM when training an audio classification model and testing with real recordings acquired by accessible day-to-day devices. StIM shows promising results for indoors audio classification, where the target sound source is not located in the same room as the microphones. StIM enables large scale simulations of multi-room acoustics with low computational complexity which is mostly beneficial for training of deep learning networks.

In this work, we explore machine learning through a model-agnostic feature representation known as braiding, that employs braid manifolds to interpret multipath ray bundles. We generate training and testing data using the well-known BELLHOP model to simulate shallow water acoustic channels across a wide range of multipath scattering activity. We examine three different machine learning techniques—k-nearest neighbors, random forest tree ensemble, and a fully connected neural network—as well as two machine learning applications. The first application applies known physical parameters and braid information to determine the number of reflections the acoustic signal may undergo through the environment. The second application applies braid path information to determine if a braid is an important representation of the channel (i.e., evolving across bands of higher amplitude activity in the channel). Testing accuracy of the best trained machine learning algorithm in the first application was 86.70% and the testing accuracy of the second application was 99.94%. This work can be potentially beneficial in examining how the reflectors in the environment changeover time while also determining relevant braids for faster channel estimation.

A method to retrieve the parameters used to create a multitrack mix using only raw tracks and the stereo mixdown is presented. This method is able to model linear time-invariant effects such as gain, pan, equalisation, delay, and reverb. Nonlinear effects, such as distortion and compression, are not considered in this work. The optimization procedure used is the stochastic gradient descent with the aid of differentiable digital signal processing modules. This method allows for a fully interpretable representation of the mixing signal chain by explicitly modelling the audio effects rather than using differentiable blackbox modules. Two reverb module architectures are proposed, a “stereo reverb” model and an “individual reverb” model, and each is discussed. Objective feature measures are taken of the outputs of the two architectures when tasked with estimating a target mix and compared against a stereo gain mix baseline. A listening study is performed to measure how closely the two architectures can perceptually match a reference mix when compared to a stereo gain mix. Results show that the stereo reverb model performs best on objective measures and there is no statistically significant difference between the participants' perception of the stereo reverb model and reference mixes.

Synthetic aperture sonar (SAS) provides high-resolution acoustic imaging by processing coherently the backscattered acoustic signal recorded over consecutive pings. Traditionally, object detection and classification tasks rely on high-resolution seafloor mapping achieved with widebeam, broadband SAS systems. However, aspect- or frequency-specific information is crucial for improving the performance of automatic target recognition algorithms. For example, low frequencies can be partly transmitted through objects or penetrate the seafloor providing information about internal structure and buried objects, while multiple views provide information about the object's shape and dimensions. Sub-band and limited-view processing, though, degrades the SAS resolution. In this paper, SAS imaging is formulated as an l1-norm regularized least-squares optimization problem which improves the resolution by promoting a parsimonious representation of the data. The optimization problem is solved in a distributed and computationally efficient way with an algorithm based on the alternating direction method of multipliers. The resulting SAS image is the consensus outcome of collaborative filtering of the data from each ping. The potential of the proposed method for high-resolution, narrowband, and limited-aspect SAS imaging is demonstrated with simulated and experimental data.

The recent explosion in the availability of echosounder data from diverse ocean platforms has created unprecedented opportunities to observe the marine ecosystems at broad scales. However, the critical lack of methods capable of automatically discovering and summarizing prominent spatio-temporal echogram structures has limited the effective and wider use of these rich datasets. To address this challenge, a data-driven methodology is developed based on matrix decomposition that builds compact representation of long-term echosounder time series using intrinsic features in the data. In a two-stage approach, noisy outliers are first removed from the data by principal component pursuit, then a temporally smooth nonnegative matrix factorization is employed to automatically discover a small number of distinct daily echogram patterns, whose time-varying linear combination (activation) reconstructs the dominant echogram structures. This low-rank representation provides biological information that is more tractable and interpretable than the original data, and is suitable for visualization and systematic analysis with other ocean variables. Unlike existing methods that rely on fixed, handcrafted rules, this unsupervised machine learning approach is well-suited for extracting information from data collected from unfamiliar or rapidly changing ecosystems. This work forms the basis for constructing robust time series analytics for large-scale, acoustics-based biological observation in the ocean.

Auditory evoked potentials (AEPs) include the auditory brainstem response (ABR), middle latency response (MLR), and cortical auditory evoked potentials (CAEPs), each one covering a specific latency range and frequency band. For this reason, ABR, MLR, and CAEP are usually recorded separately using different protocols. This article proposes a procedure providing a latency-dependent filtering and down-sampling of the AEP responses. This way, each AEP component is appropriately filtered, according to its latency, and the complete auditory pathway response is conveniently represented (with the minimum number of samples, i.e., without unnecessary redundancies). The compact representation of the complete response facilitates a comprehensive analysis of the evoked potentials (keeping the natural continuity related to the neural activity transmission along the auditory pathway), which provides a new perspective in the design and analysis of AEP experiments. Additionally, the proposed compact representation reduces the storage or transmission requirements when large databases are manipulated for clinical or research purposes. The analysis of the AEP responses shows that a compact representation with 40 samples/decade (around 120 samples) is enough for accurately representing the response of the complete auditory pathway and provides appropriate latency-dependent filtering. MatLab/Octave code implementing the proposed procedure is included in the supplementary materials.

Boundary integral analysis of scattering from rigid bodies is well known. Analysis often proceeds along the following lines: representation of the geometry using a collection of triangles, representation of physics using low order ansatz functions defined on each triangle, and then solving the resulting discrete system. This prescription for the common solution stands out in terms of the low-order approximation of both geometry and representation of physics; specifically, both are C0. Taking inspiration from computer graphics literature, a framework wherein continuity of representation (both geometry and physics) can be as high as C2 is developed. In this paper, the steps necessary to develop such a iso-geometric (i.e., using the same basis functions for representing both geometry and physics) boundary integral solver are elucidated. In doing so, an efficient method based on a wideband fast multipole method to evaluate the required inner products and matrix vector products is proposed and demonstrated. Numerous examples are presented to highlight the benefits of the proposed approach.

Solving the linearized equations of continuum mechanics to model the acoustic scattering of targets close to the seafloor is a challenging task. Among the traditional algorithms, staggered second-order finite-difference time-domain (FDTD) schemes combine simplicity and ease of implementation and parallelization, especially on modern GPUs. They have been successfully applied in two-dimensional configurations. Their major drawback is their inherent low resolution, due to their formal low order and the stair-step representation of the interfaces between media (e.g., target/water). To avoid excessive computational costs in three-dimensional configurations, while maintaining high numerical accuracy, we propose a FDTD multi-grid method. The mesh is refined locally near the interfaces, while a coarse grid is employed in homogeneous regions. In the fine grids, the aforesaid FDTD is employed, whereas, in the coarse grid, higher-order staggered schemes are used to compensate for the decrease in resolution due to the larger spacing. High-order interpolation allows matching the solutions at the grid interfaces. In this presentation, the performances of the new method on classical benchmark tests are analyzed and discussed. The numerical and computational capabilities are then illustrated by applying the proposed technique to the scattering of buried objects.

Estimating the pitch, or fundamental frequency, of a monophonic audio signal is a fundamental task in computational audio analysis with many downstream applications such as automatic transcription. The current general state of the art method for pitch detection is CRePE: a Convolutional Representation for Pitch Estimation. CRePE is a deep convolutional neural network designed to estimate the fundamental frequency of an audio signal directly from the waveform. However, CRePE, and other general pitch detection methods, do not perform well on steelpan audio. This is likely due to the steelpan's complex spectral characteristics that differentiate it timbrally from other sound sources. We combine a deep convolutional neural network architecture based on CRePE with a training dataset of steelpan audio from several distinct sounding tenor steelpans to achieve improved tenor steelpan pitch detection directly from the audio signal. We assess our model's ability to generalize by evaluating it with a test dataset that includes audio samples from steelpans that have no samples as part of the training set and compare these results to CRePE's performance on the same test dataset.

This study presents a promising data-driven approach for approximating and minimizing acoustic wave scattering in 2D acoustic cloak design, in which the optimal arrangement of cylindrical structures is found via a combination of generative deep learning models and optimization based on data generated by multiple scattering theory. In forward design, a fully connected neural network and convolutional neural network is trained with binary images of planar configurations of rigid cylinders to predict the total scattering cross section (TSCS) at discrete values of normalized wavenumber. In inverse design, generative deep learning models are developed that encode cylindrical structures into a continuous representation called a latent vector, and new structures are generated by decoding random latent values in this representation. In these models, the variational autoencoders, supervised and unsupervised learning, and Gaussian process are combined to predict optimal arrangements of scatterers for the desired minimal TSCS. Comparison of the scattered wavefield between the generated configurations and the numerical result shows the validity and effectiveness of the proposed method.

Existing auralization frameworks for interactive virtual environments have found applications in simulating acoustic conditions for binaural listening and real-time audiovisual navigation. This work, situated at the Collaborative-Research Augmented Immersive Virtual Environment Laboratory (CRAIVE-Lab), extends elements of these frameworks for human-scale interactive audiovisual display with movement of virtual sound sources and non-static virtual representation of the facility’s footprint. The work involves the development of an adaptive acoustic ray-tracing prototype that is capable of generating impulse responses for individual virtual loudspeaker representation based upon changes of room orientation in virtual space at runtime. Through the integrated use of game engines (i.e., Unity and Unreal Engine), the prototype is presented in the context of dynamic audiovisual display, and actively analyzes the virtual scene geometries using a multi-detailed rendering approach. With both reconstructed high-resolution 3D models of existing spaces, and automatically generated virtual landscapes from geo-spatial data, the developed system is evaluated both in terms of computational efficiency, and in terms of conventional room acoustics parameters using model-based acoustic energy decay analysis across listening region. [Work supported by the Cognitive Immersive Systems Laboratory (CISL) and NSF IIS-1909229.]

We present a morphological representation of the underwater acoustic channel that employs geometric braids to gain real-time knowledge of the dominant multipath activity. The key idea is to incorporate braid manifolds in tandem with non-convex mixed norm optimization techniques to track the temporal evolution of multipath arrivals. Specifically, we focus on channel braids that manifest as rapidly fluctuating high-energy taps in the delay spread as well as the delay-Doppler scattering function. Multiple braids can also be topologically combined using braid operations to interpret oceanic phenomena such as caustics and surface wave focusing, among others. We present techniques for adaptively updating the channel braids and their overlap patterns to reflect the temporal evolution of the shallow water acoustic channel. We evaluate the performance of proposed morphological channel estimation technique in terms of the normalized prediction error and computational time. Results based on numerical channel simulations based on diverse oceanic and experimental conditions as well as experimental field data from the SPACE08 experiment will be presented.

Autonomous detection and classification of species through bioacoustics sound has been an ongoing and challenging problem due to noise, overlapping sound, and varying frequency components. This work assesses bioacoustic sound data through speech patterns similarities by utilizing graphical representations of features found in spectrograms. Speech within the received sound can be characterized as individual components within the spectrogram that can be used to identify species. In this work, individual components are connected high PSD dB values in the spectrogram that form unique shapes. By representing these components through graphical representations, similarities of speech can be seen as repeating patterns across larger collected data sets that can be associated with certain behaviors of species. An example resulting graphical representation is presented through a sample humpback whale speech collected. [This research is funded by the Office of Naval Research under Grant No. N00014-19-1-2609.]

This paper applies Bayesian geoacoustic inversion with a hybrid seabed model parameterization to modal-dispersion data from the New England Mud Patch to attempt to estimate gradient structure in the upper mud layer. The hybrid seabed parameterization is based on an upper layer with a general representation of gradients based on Bernstein-polynomial basis functions, above an unknown number of discrete (uniform) layers formulated trans-dimensionally. The dataset, collected during the 2017 Seabed Characterization Experiment (SBCEX17), involves high-order modes (up to mode 21) extracted via warping time-frequency analysis from recordings of a combustive sound source on a vertical hydrophone array. The inversion results are compared to those from a trans-dimensional inversion of the same data with no gradient layer, and to results from other acoustic data sets collected in the region as well as to nearby core measurements.

Sound source components are a crucially important aspect in soundscape, since humans’ perception and evaluation of soundscape strongly depend on the recognition of the constituent sound sources of the acoustic environment. Over the last few years, numbers of studies examined the methods of automated recognition of environmental sound sources/events using machine learning technology. However, the recognition of sound source components in soundscapes remains a challenge for real applications. This present study utilizes the deep learning, a latest developed and advanced machine learning technology, to develop an algorithm to solve this challenge. Without manually extracted acoustic or psychoacoustic features based on expertise, this algorithm automatically extracts features from the spectrogram representation of the sound signals and makes the recognition. It produces the sound source components recognized in the acoustic environment and their respective present time. The algorithm achieves high recognition accuracy, based on binaural recordings of outdoor acoustic environment. For new recordings, i.e., test samples, the recognition accuracies vary from 81.3% to 100%, depending on the specific sound source. This algorithm developed can greatly help soundscape research, and can be applied in automatic soundscape description of sound source components, prediction of soundscape perception and evaluation, and soundscape management.

Conclusions about variation and change of vowels based on the first (F1) and second (F2) formant frequencies may be challenged by at least two major drawbacks intrinsic to this simple 2-D representation. First, F1 and F2 measured at one or two representative points can capture only limited amount of vowel dynamics. The numeric values of F1 and F2 are also prone to errors due to aperiodicity in speech signal. In this study, we explore alternative ways for more informative and robust vowel representation. We examine both LPC and MFCC based features using two encoding methods. Frequency domain features are first estimated at each time step for the entire vowel duration. The resulting t feature vectors are then passed sequentially through either a set of t independent autoencoders or a recurrent neural network (RNN), both with k hidden units. In case of the autoencoders, each k-by-t vowel matrix is further projected to a lower dimensional space using PCA. In case of the RNN, the k-dimensional hidden vectors are used for cross-condition comparisons. Compared to the simple F1-F2 measurement, our methods are able to capture further nuanced within-category variations across dialect varieties in TIMIT. Results from LPC and MFCC based representations are also compared.Conclusions about variation and change of vowels based on the first (F1) and second (F2) formant frequencies may be challenged by at least two major drawbacks intrinsic to this simple 2-D representation. First, F1 and F2 measured at one or two representative points can capture only limited amount of vowel dynamics. The numeric values of F1 and F2 are also prone to errors due to aperiodicity in speech signal. In this study, we explore alternative ways for more informative and robust vowel representation. We examine both LPC and MFCC based features using two encoding methods. Frequency domain features are first estimated at each time step for the entire vowel duration. The resulting t feature vectors are then passed sequentially through either a set of t independent autoencoders or a recurrent neural network (RNN), both with k hidden units. In case of the autoencoders, each k-by-t vowel matrix is further projected to a lower dimensional space using PCA. In case of the RNN, the k-dimensional hidden vectors a...

Recent work [1,2] has shown that Convolutional Neural Networks (CNNs) trained on spectrograms of acoustic signals are capable of learning high-level latent representations for the purpose of detecting and classifying the vocalizations of endangered baleen whales. The aforementioned latent representations were used in the development of an automated system that was capable of detecting the vocalizations of blue, fin, and sei whales against non-biological and ambient noise sources to a high degree of accuracy (0.961, F-1 Score = 0.899). In this work, we conduct an exploratory analysis of the same latent representations using statistical machine learning approaches as well as by visualizing the convolutional feature maps learned by the CNN. Through this analysis we attempt to interpret what properties of a spectrogram are easily and/or most often exploited by the CNN during training in order to improve upon the state-of-the-art and develop more robust detection systems going forward. [1] M. Thomas, B. Martin, K. Kowarski, B. Gaudet, and S. Matwin, Marine Mammal Species Classification using Convolutional Neural Networks and a Novel Acoustic Representation, ECML PKDD 2019 (Springer, Cham, 2019). [2] M. Thomas, "Towards a novel data representation for classifying acoustic signals," in Canadian Conference on Artificial Intelligence (Springer, Cham, 2019).

When medium property variation is not smooth and slow or perturbative, multiple reflections are non-negligible. We model multiples and develop associated methods for estimating medium parameters from received signals. We focus on 1-D which is fundamental theoretically and has direct applications to layered media, e.g., nondestructive testing for delamination in layered composites. We account for all reflection multiples: intra and inter-layer including sub and supra-source layer multiples. We represent the system in two ways, and develop associated model expressions and estimation methods. For the block diagram representation, we employ system functionals to develop closed-from expressions of the transfer function that do not require integration. Using this expression, we can estimate parameters by optimization. For the tree-diagram representation, we develop expressions that construct the response semi-incrementally in time. We develop non-optimization-based methods to estimate parameters from theseexpressions without prior knowledge of the number of layers. For broad-band signals, this method runs in real-time. Our models and methodsaccount for various multiples, are easily modifiable for various configurations (of pre-source layers and location of source-receiver) and source radiation patterns (uni/bi-directionality). Configurational flexibility, diagrammatic representations, simple closed-form expression, interpretable models, and form of estimation methods are strengths of our models and methods.

Auditory models have been developed for decades to simulate characteristics of the human auditory system, but it is often unknown how well auditory models compare to each other or perform in tasks they were not primarily designed for. This study systematically analyzes predictions of seven publicly-available cochlear filter models in response to a fixed set of stimuli to assess their capabilities of reproducing key aspects of human cochlear mechanics. The following features were assessed at frequencies of 0.5, 1, 2, 4, and 8 kHz: cochlear excitation patterns, nonlinear response growth, frequency selectivity, group delays, signal-in-noise processing, and amplitude modulation representation. For each task, the simulations were compared to available physiological data recorded in guinea pigs and gerbils as well as to human psychoacoustics data. The presented results provide application-oriented users with comprehensive information on the advantages, limitations and computation costs of these seven mainstream cochlear filter models.

Spatial descriptions of the head-related transfer function (HRTF) using spherical harmonics, which is commonly used for the purpose, consider all directions simultaneously. However, in perceptual studies, it is necessary to model HRTFs with different angular resolutions at different directions. To this end, an alternative spatial representation of the HRTF, based on local analysis functions, is introduced. The proposal is shown to have the potential to describe the local features of the HRTF. This is verified by comparing the reconstruction error achieved by the proposal to that of the spherical harmonic decomposition when reconstructing the HRTF inside a spherical cap.

A framework for simulating auditory discrimination experiments, based on an approach from Schädler, Warzybok, Hochmuth, and Kollmeier [(2015). Int. J. Audiol. 54, 100-107] which was originally designed to predict speech recognition thresholds, is extended to also predict psychoacoustic thresholds. The proposed framework is used to assess the suitability of different auditory-inspired feature sets for a range of auditory discrimination experiments that included psychoacoustic as well as speech recognition experiments in noise. The considered experiments were 2 kHz tone-in-broadband-noise simultaneous masking depending on the tone length, spectral masking with simultaneously presented tone signals and narrow-band noise maskers, and German Matrix sentence test reception threshold in stationary and modulated noise. The employed feature sets included spectro-temporal Gabor filter bank features, Mel-frequency cepstral coefficients, logarithmically scaled Mel-spectrograms, and the internal representation of the Perception Model from Dau, Kollmeier, and Kohlrausch [(1997). J. Acoust. Soc. Am. 102(5), 2892-2905]. The proposed framework was successfully employed to simulate all experiments with a common parameter set and obtain objective thresholds with less assumptions compared to traditional modeling approaches. Depending on the feature set, the simulated reference-free thresholds were found to agree with-and hence to predict-empirical data from the literature. Across-frequency processing was found to be crucial to accurately model the lower speech reception threshold in modulated noise conditions than in stationary noise conditions.

This paper proposes a multi-layer alternating sparse−dense framework for bird species identification. The framework takes audio recordings of bird vocalizations and produces compressed convex spectral embeddings (CCSE). Temporal and frequency modulations in bird vocalizations are ensnared by concatenating frames of the spectrogram, resulting in a high dimensional and highly sparse super-frame-based representation. Random projections are then used to compress these super-frames. Class-specific archetypal analysis is employed on the compressed super-frames for acoustic modeling, obtaining the convex-sparse CCSE representation. This representation efficiently captures species-specific discriminative information. However, many bird species exhibit high intra-species variations in their vocalizations, making it hard to appropriately model the whole repertoire of vocalizations using only one dictionary of archetypes. To overcome this, each class is clustered using Gaussian mixture models (GMM), and for each cluster, one dictionary of archetypes is learned. To calculate CCSE for any compressed super-frame, one dictionary from each class is chosen using the responsibilities of individual GMM components. The CCSE obtained using this GMM-archetypal analysis framework is referred to as local CCSE. Experimental results corroborate that local CCSE either outperforms or exhibits comparable performances to existing methods including support vector machine powered by dynamic kernels and deep neural networks.

Speaker identification (SI) techniques has been used in numerous commercial products over the last decades. In SI, the main purpose is to match a voice sample from an unknown speaker to one of the labeled speaker models. To be able accomplish this task, there are two operational phases, training (can be also termed as enrollment) and testing. In both phases, feature extraction and feature matching are the two key steps. In this work, we have extracted features with Mel-Frequency Cepstrum Coefficients (MFCC) because MFCC has accurate representation of the vocal tract, and a Spectral-Flux based Voice Activity Detector (VAD) is implemented to extract features from the speech segments. In feature matching task, we build a Convolutional Long short-term memory (LSTM) Neural Network for the speaker models. We examine main performance of the system in terms of identification rate and compare the proposed method with other SI methods under several noisy conditions at different signal to noise ratio (SNR) levels.Speaker identification (SI) techniques has been used in numerous commercial products over the last decades. In SI, the main purpose is to match a voice sample from an unknown speaker to one of the labeled speaker models. To be able accomplish this task, there are two operational phases, training (can be also termed as enrollment) and testing. In both phases, feature extraction and feature matching are the two key steps. In this work, we have extracted features with Mel-Frequency Cepstrum Coefficients (MFCC) because MFCC has accurate representation of the vocal tract, and a Spectral-Flux based Voice Activity Detector (VAD) is implemented to extract features from the speech segments. In feature matching task, we build a Convolutional Long short-term memory (LSTM) Neural Network for the speaker models. We examine main performance of the system in terms of identification rate and compare the proposed method with other SI methods under several noisy conditions at different signal to noise ratio (SNR) levels.

Acoustic cues are robust elements that can be used to infer information contained in the speech signal, such as underlying linguistic distinctive features and the words intended by the speaker (Stevens JASA 2002). Yet, most current automatic speech recognition systems do not take advantage of a feature-cue-based framework for signal analysis. In this project, a set of common acoustic cues has been explicitly modeled by Gaussian mixture models. This set of acoustic cues can provide evidence for the overall phoneme and word sequences of an utterance. The extracted cues and their values can also determine a speaker’s linguistic production pattern, i.e., the systematic context-governed modifications in surface-phonetic form that occur pervasively in conversational speech. The simple Gaussian mixture model representation structure reduces the need for extensive amounts of training data, in contrast to conventional schemes based on large neural networks.Acoustic cues are robust elements that can be used to infer information contained in the speech signal, such as underlying linguistic distinctive features and the words intended by the speaker (Stevens JASA 2002). Yet, most current automatic speech recognition systems do not take advantage of a feature-cue-based framework for signal analysis. In this project, a set of common acoustic cues has been explicitly modeled by Gaussian mixture models. This set of acoustic cues can provide evidence for the overall phoneme and word sequences of an utterance. The extracted cues and their values can also determine a speaker’s linguistic production pattern, i.e., the systematic context-governed modifications in surface-phonetic form that occur pervasively in conversational speech. The simple Gaussian mixture model representation structure reduces the need for extensive amounts of training data, in contrast to conventional schemes based on large neural networks.

Hindustani classical music is an ancient improvisational form of music based on ragas, melodic frameworks which have no written representation and are passed down through a fading oral tradition. The proposed system, PhonoNet, aims to preserve Hindustani classical music by providing computational prediction of ragas, so singers can receive live feedback when learning. PhonoNet also creates a visual format for documenting and preserving raga information. First, the system computes the short-term Fourier Transform of the input audio data to form a chromagram representation of the notes being sung. These data are then augmented using a transpositional data augmentation algorithm and split into chunks for use as training inputs for a deep convolutional neural network. The convolutional network's filters are analyzed using a saliency visualization algorithm and modified with a recurrent layer to allow processing of full-length songs. The convolutional system achieves 78.9% validation accuracy for raga prediction on 150 second audio chunks. The joint raga prediction system achieves a new state-of-the-art 98.9% accuracy for raga prediction on full-length songs. Future work can extend the proposed hierarchical system to other tasks with long temporal sequences and extend the data augmentation and visualization algorithms to different applications of audio processing.Hindustani classical music is an ancient improvisational form of music based on ragas, melodic frameworks which have no written representation and are passed down through a fading oral tradition. The proposed system, PhonoNet, aims to preserve Hindustani classical music by providing computational prediction of ragas, so singers can receive live feedback when learning. PhonoNet also creates a visual format for documenting and preserving raga information. First, the system computes the short-term Fourier Transform of the input audio data to form a chromagram representation of the notes being sung. These data are then augmented using a transpositional data augmentation algorithm and split into chunks for use as training inputs for a deep convolutional neural network. The convolutional network's filters are analyzed using a saliency visualization algorithm and modified with a recurrent layer to allow processing of full-length songs. The convolutional system achieves 78.9% validation accuracy for raga predicti...

Recent literature established the highly adaptive nature of echolocation: both toothed whales and bats modify their signals and sonar beam movements dynamically based on echoes from previous emissions in a task-depending manner. However, no theoretical framework systematically interprets these search processes across varying taxa and experimental conditions. To address this need, we cast the “infotaxis” model originally proposed for simulating moth odor tracking into the context of echolocation-based target search. Infotaxis posits that the animal chooses their next action to maximize the expected information gain. By doing so, the searching animal balances the exploration of new information about its environment with exploitation of existing information. The echolocator chooses the next sonar beam location and signal content based on its internal representations of the search space and sensory uncertainty. We model the echolocator’s internal representation of the search space, the echo signatures of the target and clutter objects, and information acquired from echoes as probabilistic distributions and employ a Bayesian update of the target distribution after each echolocation reception. Simulations show the repeated inspection and switching among objects commonly observed in behavioral experiments of echolocation search. [Work supported by ONR MURI program.]

Deep neural networks (DNNs) are advantageous for representing complex nonlinear relationships. This paper applies DNNs to source localization in a shallow water environment. Two methods are proposed to estimate the range and depth of a broadband source through different neural network architectures. The first adopts the classical two-stage scheme, in which feature extraction and DNN analysis are independent steps. The eigenvectors associated with the modal signal space are extracted as the input feature. Then, the time delay neural network is exploited to model the long term feature representation and constructs the regression model. The second concerns a convolutional neural network-feed-forward neural network (CNN-FNN) architecture, which trains the network directly by taking the raw multi-channel waveforms as input. The CNNs are expected to perform spatial filtering for multi-channel signals, in an operation analogous to time domain filters. The outputs of CNNs are summed as the input to FNN. Several experiments are conducted on the simulated and experimental data to evaluate the performance of the proposed methods. The results demonstrate that DNNs are effective for source localization in complex and varied water environments, especially when there is little precise environmental information.

Audio classification techniques often depend on the availability of a large labeled training dataset for successful performance. However, in many application domains of audio classification (e.g., wildlife monitoring), obtaining labeled data is still a costly and laborious process. Motivated by this observation, a technique is proposed to efficiently learn a clean template from a few labeled, but likely corrupted (by noise and interferences), data samples. This learning can be done efficiently via tensorial dynamic time warping on the articulation index-based time-frequency representations of audio data. The learned template can then be used in audio classification following the standard template-based approach. Experimental results show that the proposed approach outperforms both (1) the recurrent neural network approach and (2) the state-of-the-art in the template-based approach on a wildlife detection application with few training samples.

Cochlear implants have improved speech recognition for deaf individuals, but further modifications are required before performance will match that of normal-hearing listeners. In this study, the hypotheses were tested that (1) implant processing would benefit from efforts to preserve the structure of the low-frequency formants and (2) time-varying aspects of that structure would be especially beneficial. Using noise-vocoded and sine-wave stimuli with normal-hearing listeners, two experiments examined placing boundaries between static spectral channels to optimize representation of the first two formants and preserving time-varying formant structure. Another hypothesis tested in this study was that children might benefit more than adults from strategies that preserve formant structure, especially time-varying structure. Sixty listeners provided data to each experiment: 20 adults and 20 children at each of 5 and 7 years old. Materials were consonant-vowel-consonant words, four-word syntactically correct, meaningless sentences, and five-word syntactically correct, meaningful sentences. Results showed that listeners of all ages benefited from having channel boundaries placed to optimize information about the first two formants, and benefited even more from having time-varying structure. Children showed greater gains than adults only for time-varying formant structure. Results suggest that efforts would be well spent trying to design processing strategies that preserve formant structure.

In this letter, authors propose an auditory feature representation technique with the filterbank learned using an annealing dropout convolutional restricted Boltzmann machine (ConvRBM) and noise-robust energy estimation using the Teager energy operator (TEO). TEO is applied on each subband of ConvRBM filterbank and pooled later to get the short-term spectral features. Experiments on AURORA 4 database show that the proposed features perform better than the Mel filterbank features. The relative improvement of 2.59%–11.63% and 1.26%–6.87% in word error rate is achieved using the time delay neural network and the bidirectional long short-term memory models, respectively.

Damping in vibration models is often treated on an empirical basis based upon experimental measurement of damping in the eigenmodes of the model. Thus if one starts with a finite element model, for example, the undamped eigenmodes are found and then the damping is inserted into the eigenmode model of the structure. However sometimes one wishes to return to the finite element model representation and then the question arises as to how to represent the damping consistent with the eigenmode representation. This note provides an answer to that question.

The spike representation of multi-pitch sounds leaving the cochlear is discussed using a Finite-Difference Time Domain (FDTD) physical model of the cochlear with musical sounds as input. Previously, the model has lead to a new pitch theory, where it was found that the interspike intervals (ISI) of the fundamental periodicity of a single-pitch sound are present at multiple Bark bands in the cochlear. This is due to drop-outs of spikes in Bark bands of higher harmonics. Pitch is therefore represented as the most prominent periodicity within the auditory nerve. In the multi-pitch case, the model shows several cases of such multi-Bark periodicities. With multi-pitch sounds having a residual pitch, the strongest of these multi-Bark cases is that of the residual periodicity. With several pitches perceptually clearly distinguishable one from another, each pitch periodicity appears as a multi-Bark periodicity. Still with sounds where the single pitches are not so clear and separating them needs musical training and intense and multiple listening of the sound, the multi-Bark cases are blurred. Therefore, the model represents the perception of multi-pitch sounds during immediate perception and identification of multiple pitches needing musical training is no longer present at this low-level sound representation at the cochlear output, as expected.

The shallow water acoustic channel is well-known to exhibit rapid temporal fluctuations due to multipath effects, fluid motion as well as focusing events in the ocean. These events often overlap temporally thus contributing in unpredictable ways to the channel delay spread. We will present recent results from employing geometric signal processing techniques to shallow water channel estimation and will introduce new ways to identify and disambiguate overlapping oceanic phenomena within the channel delay spread. Specifically, we will explore how geometric braid theory and related operations can be employed to disentangle overlapping channel braids, and explore the efficacy of such signal processing techniques across diverse channel representations. Finally, we will discuss the importance of capturing oceanic phenomena at the appropriate resolution of time, frequency, and sparsity and the trade-offs in accuracy involved based on the chosen channel representation(s).

Horseshoe bats have to navigate through complex environments such as dense forests and structure-rich vegetation relying on input from their highly sophisticated biosonar systems. One of the key components of these bats' ability to obtain high-quality acoustic information is to alter the shape of their outer ears rapidly. In prior work, the authors have shown that by mimicking the horseshoe bat rapid ear movements, a bat-inspired robotic dynamic periphery for recording speech signals could enhance speech recognition for limited dataset and also provide estimates for the speaker's direction along with speech recognition. In our current study, we continued to investigate how speech datasets processed by the dynamic periphery may be enhanced compared to a reference by extracting acoustical features through Mel frequency cepstral coefficient (MFCC) transform, Lyon's cochlear bandpass filters, and a neural spike representation, respectively. This study aims to characterize the detailed acoustical differences and quantify the improved speaker intelligence with noise robustness through the dynamic periphery. The ultimate goal of this research is to identify a signal representation that is well suited to capitalize on the time-variant properties of the biomimetic recording periphery and make the dynamic information-bearing features accessible for the classification stages.Horseshoe bats have to navigate through complex environments such as dense forests and structure-rich vegetation relying on input from their highly sophisticated biosonar systems. One of the key components of these bats' ability to obtain high-quality acoustic information is to alter the shape of their outer ears rapidly. In prior work, the authors have shown that by mimicking the horseshoe bat rapid ear movements, a bat-inspired robotic dynamic periphery for recording speech signals could enhance speech recognition for limited dataset and also provide estimates for the speaker's direction along with speech recognition. In our current study, we continued to investigate how speech datasets processed by the dynamic periphery may be enhanced compared to a reference by extracting acoustical features through Mel frequency cepstral coefficient (MFCC) transform, Lyon's cochlear bandpass filters, and a neural spike representation, respectively. This study aims to characterize the detailed acoustical differences a...

Three-dimensional exterior acoustic problems with irregular domains are solved using a hypersingular meshless method. In particular, the method of fundamental solutions (MFS) is used to formulate and analyze such acoustic problems. It is well known that source points for MFS cannot be located on the real boundary due to the singularity of the kernel functions. Thus, the diagonal terms of the influence matrices are unobtainable when source points are located on the boundary. An efficient approach is proposed to overcome such difficulties, when the MFS is used for three-dimensional exterior acoustic problems. This work is an extension of previous research on two-dimensional problems. The solution of the problem is expressed in terms of a double-layer potential representation on the physical boundary. Three examples are presented in which the proposed method is compared to the MFS and boundary element method. Good numerical performance is demonstrated by the proposed hypersingular meshless method.

Boundary integral equations (BIEs) find applications in problems ranging from sonar to medical diagnostics. The two ingredients of the BIE solution technique are (1) representation of the domain and (2) design of approximation spaces to represent physical quantities on the domain. These, in concert, affect accuracy and convergence of the simulation. This paper presents a framework that permits the development of a scheme for refinement (of size and order) in both geometry and function representations. Further, this permits flexibility in the types of basis functions that can be used. Capabilities of the proposed framework are shown via a number of numerical examples.

As an animal moves in its natural environment, to seek food, track targets, and steer around obstacles, its distance and direction to objects continuously change, invoking dynamic feedback between 3D scene representation, attention, and action-selection. Animals that rely on active sensing provide powerful systems to investigate neural underpinnings of sensory-guided behaviors, as they produce the very signals that inform motor actions. Echolocating bats, for example, transmit sonar signals and process auditory information carried by echoes to guide behavioral decisions for spatial orientation. Further, the bat adapts its echolocation signal design in response to 3D spatial information computed from echo returns, and therefore, the directional aim and temporal patterning of its calls provide a window into the animal’s attention to objects in its surroundings. In addition, the bat actively controls pinna position and head movements to enhance auditory cues about 3D target position. These adaptive behaviors...

Wavepacket models provide a convenient representation of jet noise source phenomena because of the extended, partially correlated nature of the turbulent mixing noise. When treated as an equivalent source model, they are useful to estimate features of both the radiated noise as well as the source characteristics to assist in jet noise reduction efforts. In this study, advanced cross-beamforming techniques are applied to measurements in the vicinity of a high-performance military aircraft. These results are then decomposed into an azimuthally-averaged multi-wavepacket representation of the data, which can then be treated as an equivalent source. Estimates of the field levels and coherence properties using the equivalent source are compared with measurements, and results from the multi-wavepacket model show good agreement with benchmark measurements over a range of frequencies that contribute significantly to the overall radiation. The capabilities and limitations of the model to estimate field properties a...

In recent years, there is an expanding enthusiasm for studying the seabed structure. The investigation of seabed has numerous applications, for example, protest recognition, exploration of natural resources, angling, analyzing the composition of residues, etc. In oceanic examinations, submerged objects location is a standout amongst the most basic errands. This emphasizes the necessity of image segmentation, which divides an image into parts that have strong correlations with objects to reflect the actual information collected from the real world. Image segmentation is the most practical approach to virtually all automated image recognition systems. Clustering of numerical data forms the basis of many classification and system modelling algorithms. The purpose of clustering is to identify natural groupings of data from a large data set to produce a concise representation of a system's behaviour. In this paper, Empirical Mode Decomposition (EMD) picture upgrade procedure has been utilized to enhance the nature of sonar pictures and Fuzzy C means (FCM) clustering method is used for underwater image segmentation. The result of EMD is compared with histogram equalization and Contrast stretching. EMD with K-means and EMD with FCM is compared and the exploratory outcomes demonstrate that the proposed technique can acquire exact outcomes and enhances operational productivity.

The shallow water acoustic channel is well-known to exhibit rapid fluctuations in delay spread, which poses a fundamental roadblock to real-time channel estimation, particularly under moderate to rough sea conditions. This talk will summarize recent research on multiple representations of the shallow water acoustic channel and present the relative merits of each channel representation in localizing different channel effects. In particular, we will examine the efficacy of employing geometric signal processing techniques to adaptively track the shallow water acoustic channel in each representation. Results based on experimental field data as well as channel simulations under varying environmental conditions will be presented.The shallow water acoustic channel is well-known to exhibit rapid fluctuations in delay spread, which poses a fundamental roadblock to real-time channel estimation, particularly under moderate to rough sea conditions. This talk will summarize recent research on multiple representations of the shallow water acoustic channel and present the relative merits of each channel representation in localizing different channel effects. In particular, we will examine the efficacy of employing geometric signal processing techniques to adaptively track the shallow water acoustic channel in each representation. Results based on experimental field data as well as channel simulations under varying environmental conditions will be presented.

One of the most serious remaining challenges in speech recognition is dealing with corruption of speech signal by other nuisance speech ("babble"). A promising approach to solving the problem of separating the signal of interest from the detractors is to inject direction dependent signatures into all signals received, which has been realized by bat-inspired biomimetic pinna—dynamic periphery. Changing the shape of a biomimetic pinna during the recordings introduces substantial time-variant signatures into speech signals. To investigate the utility of these signatures, we have used bioinspired signal representations (cochleagram and spikegram) as input for speech classifiers based on Gaussian mixture models (GMM) and hidden Markov models (HMM). The speech samples used were obtained from open source databases: spoken digits and alphabets from Carnegie Mellon University were mixed with babble or noise samples from Columbia University. Since the time-variant signatures were found to depend strongly on the direction of the sound source, we attempted to include datasets from different directions for training and testing to feed into the classifiers. The results indicate that dynamic periphery can substantially improve recognition and that these effects depend on the signal representation as well as the angular composition of the training dataset.One of the most serious remaining challenges in speech recognition is dealing with corruption of speech signal by other nuisance speech ("babble"). A promising approach to solving the problem of separating the signal of interest from the detractors is to inject direction dependent signatures into all signals received, which has been realized by bat-inspired biomimetic pinna—dynamic periphery. Changing the shape of a biomimetic pinna during the recordings introduces substantial time-variant signatures into speech signals. To investigate the utility of these signatures, we have used bioinspired signal representations (cochleagram and spikegram) as input for speech classifiers based on Gaussian mixture models (GMM) and hidden Markov models (HMM). The speech samples used were obtained from open source databases: spoken digits and alphabets from Carnegie Mellon University were mixed with babble or noise samples from Columbia University. Since the time-variant signatures were found to depend strongly on the dir...

Acoustic color features that specify a sonar target in the time-frequency spectral domain may be broadly classified into two groups: (i) elastic wave features that depend on the material composition of the target, and (ii) wave features that depend on the specific geometry of the sonar target. Typically these two types of target features overlap intricately in the acoustic color domain that renders them difficult to separate using traditional signal processing techniques. Furthermore, field experimental conditions such as target orientation, proud, buried or semi-buried state of the target as well as sediment characteristics, influence the relative overlap and morphology of these acoustic color features. A key bottleneck to robust target feature representation and subsequent classification is how these potentially overlapping features may change in the field from what is measured under laboratory conditions or observed through model-based simulations. We will present some relevant results from our ongoing...

While graphics and visual representations for Virtual Reality (VR) systems are very well developed, the manner in which audio signals and acoustic environments are recreated in a VR system is not. In the case of audio spatialization, the current approach makes use of a library of standard head related transfer functions (HRTFs), i.e., a user selects a generic HRTF from a library, with limited personal information. It is essentially a “best-guess” representation of that individual’s HRTF. This limits the accuracy of audio developments for virtual reality. This paper reports on results from localization tests to determine the capabilities of a generic HRTF used in a VR environment. Volunteers entered a VR world, and an invisible sound source made a short bursts of white noise at various position in the room. Volunteers were asked to point to the location of the sound source, and results were captured to the nearest millimeter using the VR’s motion tracking system. It is proposed that future versions of this experimental methodology will enable the development of a pseudo-personalized HRTF, unique to each individual VR user.While graphics and visual representations for Virtual Reality (VR) systems are very well developed, the manner in which audio signals and acoustic environments are recreated in a VR system is not. In the case of audio spatialization, the current approach makes use of a library of standard head related transfer functions (HRTFs), i.e., a user selects a generic HRTF from a library, with limited personal information. It is essentially a “best-guess” representation of that individual’s HRTF. This limits the accuracy of audio developments for virtual reality. This paper reports on results from localization tests to determine the capabilities of a generic HRTF used in a VR environment. Volunteers entered a VR world, and an invisible sound source made a short bursts of white noise at various position in the room. Volunteers were asked to point to the location of the sound source, and results were captured to the nearest millimeter using the VR’s motion tracking system. It is proposed that future versions of this...

It is shown that Buckingham's grain-shearing (GS) model [JASA (2000)] as well as its improved viscous-GS (VGS) model [JASA (2007)] can be expressed using the mathematical framework of fractional calculus. The fractional version of the standard fluid model is adopted to independently arrive at the wave equations and dispersion relations derived by Buckingham. The fractional-order wave equations obtained for the compressional waves and shear waves are relatively easier to analyse due to their closed-form representation in the fractional framework. It is also shown that the fractional calculus approach may help in bridging the disparate fields of non-Newtonian rheology and sediment acoustics, which may have actually developed independently of each other. Further, the experimental data relating wave dispersion and attenuation in marine sediments is found to match with the predictions from the fractional framework. The overall goal is to show that fractional calculus is not just a mathematical framework that can only be applied to curve-fit the observational data for complex media. In fact, it has an inherent connection to real physical processes that needs to be explored more.