Search

Open-source software libraries have a significant impact on the development of Audio Signal Processing and Music Information Retrieval (MIR) systems. Despite the abundance of such tools, there is a lack of an extensive and easy-to-use reference library for audio feature extraction on Web clients. In this article, we present Essentia.js, an open-source JavaScript (JS) library for audio and music analysis on both web clients and JS engines. Along with the Web Audio API, it can be used for both offline and real-time audio feature extraction on web browsers. Essentia.js is modular, lightweight, and easy-to-use, deploy, maintain, and integrate into the existing plethora of JS libraries and web technologies. It is powered by a WebAssembly back end cross-compiled from the Essentia C++ library, which facilitates a JS interface to a wide range of low-level and high-level audio features, including signal processing MIR algorithms as well as pre-trained TensorFlow.js machine learning models. It also provides a higher-level JS API and add-on MIR utility modules along with extensive documentation, usage examples, and tutorials. We benchmark the proposed library on two popular web browsers and the Node.js engine, and four devices, including mobile Android and iOS, comparing it to the native performance of Essentia and the Meyda JS library. The work on Essentia.js has been partially funded by the Ministry of Science and Innovation of the Spanish Government under the grant agreement PID2019-111403GB-I00 (Musical AI).

Η Ημιτονοειδής Μοντελοποίηση είναι μια από τις πιο ευρέως χρησιμοποιούμενες παραμετρικές μεθόδους για την επεξεργασία σήματος φωνής και ήχου. Η ακριβής εκτίμηση των ημιτονοειδών παραμέτρων (πλάτη, συχνότητες, και φάσεις) είναι ένα κρίσιμο σημείο για τη ακριβή αναπαράσταση των σημάτων που αναλύονται. Στην παρούσα εργασία, με βάση τις πρόσφατες εξελίξεις στην ημιτονοειδή ανάλυση, προτείνουμε υψηλής ανάλυσης, προσαρμόσιμα ημιτονοειδή μοντέλα για συστήματα ανάλυσης, σύνθεσης, και τροποποίησης ομιλίας. Στόχος μας είναι να προσφέρουμε συστήματα που αναπαριστούν σήματα φωνής με εξαιρετικά ακριβή και συμπαγή τρόπο.Εμπνευσμένοι από πρόσφατα προταθέντα μοντέλα, όπως το προσαρμόσιμο Σχεδον - Αρμονικό Μοντέλο (aQHM) και το προσαρμόσιμο Αρμονικό Μοντέλο (aHM), διατυπώνουμε τη θεωρία της προσαρμόσιμης Ημιτονοειδούς Μοντελοποίησης και προτείνουμε ένα μοντέλο που ονομάζεται εκτεταμένο προσαρμόσιμο Σχεδον - Αρμονικό Μοντέλο (eaQHM), το οποίο είναι ένα μη παραμετρικό μοντέλο, ικανό να προσαρμόσει τα στιγμιαία πλάτη και φάσεις των συναρτήσεων βάσης του στα τοπικά χρονικά μεταβαλλόμενα χαρακτηριστικά του σήματος της φωνής, αμβλύνοντας έτσι τη γνωστή υπόθεση της τοπικής στασιμότητας. Αποδεικνύεται ότι το eaQHM παρουσιάζει υψηλότερες επιδόσεις από το aQHM στην ανάλυση και ανασύνθεση των έμφωνων τμημάτων φωνής. Με βάση το eaQHM, ένα υβριδικό σύστημα ανάλυσης / σύνθεσης ομιλίας παρουσιάζεται (eaQHNM), μαζί με μια υβριδική έκδοση του του aHM (aHNM). Επιπλέον, παρουσιάζουμε κίνητρα για μια αναπαράσταση του σήματος της φωνής σε όλο το φάσμο και σε όλη τη διάρκεια του, χρησιμοποιώντας το eaQHM, αναπαριστώντας έτσι όλα τα μέρη του σήματος της φωνής, με υψηλής ανάλυσης AM-FM ημίτονα. Η αξιολόγηση δείχνει ότι η προσαρμοσιμότητα και η σχεδόν-αρμονικότητα είναι αρκετή για να παράξει πολύ υψηλή ποιότητα στην ανασύνθεση των άφωνων τμημάτων της φωνής. Στη συνέχεια, παρουσιάζεται το σύστημα πλήρους φάσματος ανάλυσης και σύνθεσης βασισμένο στο eaQHM, το οποίο υπερτερεί συστημάτων που θεωρούνται state-of-the-art, υβριδικά ή πλήρους ανάλυσης, στην ανάλυση και ανασύνθεση φωνής. Η υπεροχή του στην ποιότητα ανασύνθεσης επιβεβαιώθηκε με αντικειμενικές και υποκειμενικές αξιολογήσεις.Όσον αφορά τις εφαρμογές, το eaQHM και το aHM εφαρμόζονται σε μετασχηματισμούς φωνής (κλιμάκωση χρόνου και κλιμάκωση θεμελιώδους συχνότητας). Οι μετασχηματισμοί που προκύπτουν είναι υψηλής ποιότητας, ακολουθώντας πολύ απλούς κανόνες, σε σύγκριση με άλλα συστήματα state-of-the-art. Οι έννοιες της σχετικής φάσης και της καθυστέρησης σχετικής φάσης είναι ζωτικής σημασίας για την ανάπτυξη μετασχηματισμένου σήματος με χαρακτηριστικά αναλλοίωτου σχήματος, χωρίς τεχνικά ελαττώματα, και υψηλής ποιότητας. Τα αποτελέσματα δείχνουν ότι τα συστηματα βασισμένα στην αρμονικότητα προτιμούνται έναντι αυτών της σχεδόν-αρμονικότητας, λόγω της απλότητας της αναπραστάσης. Επιπλέον, το eaQHM εφαρμόζεται στο πρόβλημα της μοντελοποίησης σημάτων ήχου, και συγκεκριμένα ήχων μουσικών οργάνων. Το eaQHM αξιολογείται και σύγκρινεται με state-of-the-art συστήματα, και έχει υψηλές επιδόσεις όσον αφορά την ποιότητα επανασύνθεσης, αναπαριστωντας με επιτυχία τα στάδια της επίθεσης, της μετάβασης, και της στατικότητας ενός ήχου μουσικού οργάνου. Τέλος, μια άλλη προτεινόμενη εφαρμογή έγκειται στην ανάλυση και ταξινόμηση της εκφραστικής ομιλίας. Το eaQHM εφαρμόζεται στην ανάλυση της εκφραστικής ομιλίας, παρέχοντας τις στιγμιαίες παραμέτρους του ως χαρακτηριστικά που μπορούν να χρησιμοποιηθούν στην αναγνώριση και ταξινόμηση, βασισμένη σε διανυσματικούς κβαντιστές, εκφραστικής ομιλίας. Αν και τα ημιτονοειδή μοντέλα δεν χρησιμοποιούνται συνήθως σε τέτοιες εφαρμογές, τα αποτελέσματα είναι ελπιδοφόρα.

Music is one of the most pious cultures that exist throughout the world. Almost every country has their own rich musical culture, most of them are with strong classical backgrounds. India is one such land, rich in various types of music. This research paper is about the study of the identification of Raga. ‘Raga’ is said to be the melody of a song. We have used the Audio Signal Processing (ASP) techniques for feature extraction of two very similar ragas from Carnatic Music (CM) by analyzing the ‘RagaChhaya’ swaras commonly known as ‘RagaLakshna’ swaras of these ragas using Essentia and SMS tools. Essentia is an open-source library based on C++ with python and JavaScript wrappers for extensive audio analysis and MIR studies. The ‘SMS-tools’ is an application called ‘Spectral Modelling and Synthesis’ used to analyze sound and music applications. We proposed a supervised machine learning model trained with the features extracted from the two ragas considered, using the SMS-tools, for testing and training data that compares the given input piece of the audio signal to correctly identify based on the trained model.

Background and objectives Diplophonia is a common symptom in voice disorders. Depending on the underlying aetiology, diplophonic patients typically need treatment such as phonosurgery or speech therapy. In current clinical practice, the presence of diplophonia is assessed by auditive rating. To avoid subjectivity in voice assessment and to follow principles of evidence based medicine, objective instrumental assessment methods are needed. In order to gain insight into instrumental assessment of diplophonic voice, comparisons between different assessment approaches are necessary. The aim of the study is to compare the performance of two independent objective approaches on their ability to detect diplophonia. The compared approaches are the formerly published degree of subharmonics (DSH), and a newly proposed measure for spatial bimodality of the vocal fold vibration. Material and methods From a clinical database of 352 laryngeal high-speed videos with synchronous audio recordings, 60 phonation segments (20 euphonic, twenty diplophonic and twenty non-diplophonic dysphonic) were auditively selected. For all phonation segments, the DSH and the newly proposed measure for spatial bimodality were determined. The DSH is the occurrence rate of audio analysis blocks with ambiguous fundamental frequency in percent. The bimodality measure quantifies the spatial occurrence of secondary oscillation frequencies along the vocal folds’ edges. Both the DSH and the bimodality measure are evaluated on their ability to detect diplophonia by means of cut off threshold classification. Results and conclusions The DSH showed excellent classification rates for separating diplophonic from euphonic phonation (sensitivity: 98.4%, specificity: 100%). In separating diplophonic from non-diplophonic dysphonic phonation, the bimodality measure slightly outperforms the DSH approach (sensitivity: 54.6%, specificity: 92.7%). The separation of diplophonia from other kinds of dysphonia is challenging, and more sophisticated methods are needed. It is concluded that auditive and glottal diplophonia must be distinguished. As the clinical assessment of diplophonia primarily aims at determining glottal conditions, the video-based approach might deliver clinically more relevant data than the auditive approach.

Both the collection and analysis of large music repertoires constitute major challenges within musicological disciplines such as jazz research. Automatic methods of music analysis based on audio signal processing have the potential to assist researchers and to accelerate the transcription and analysis of music recordings significantly. In this paper, we propose a framework for analyzing improvised monophonic solos in multi-instrumental jazz recordings with special focus on reed and brass instruments. The analysis algorithms rely on prior score-information, which is taken from high quality manual solo transcriptions. Following an initial solo and accompaniment source separation, we propose algorithms for tone-wise extraction of fundamental frequency and intensity contours. Based on this fine-grained representation of recorded jazz solos, we perform several exploratory experiments motivated by questions relating to jazz research in order to analyze the use of expressive stylistic devices such as intonation, pitch modulation, and dynamics in jazz solos. The results show that a score-informed audio analysis of jazz recordings can provide valuable insights into the individual stylistic characteristics of jazz musicians.

Today, pulmonary diseases are one of the major causes of mortality in the world. Even though there are different diagnostic tests available such as X-ray and tomography, the stethoscope is still the first, cheapest and the most frequently used diagnostic device for the physicians. In this paper, a smart electronic stethoscope has been designed to help physicians with the diagnosis of the disease using Machine Learning. In order to create a design that satisfies all the needs of the physicians, 15 doctors and medical students from several hospitals have been contacted and interviewed. The developed system has been tested with different machine learning techniques and its efficiency has been shown by obtaining 84% accuracy while classifying respiratory audio.

Like the concept of free-viewpoint TV (FTV), the audio data should be rendered according to video data. However, on condition that minimum numbers of microphone are used, it is difficult to acquire accurate audio signal for rendering audio data to the image with the choice of view point. Especially, degradation of high frequency components (HFC) happens due to the characteristic of polar pattern for microphone. The degradation of HFC causes imperfection of signal restoration and leads to the degradation of clarity for hearing. In this paper, a compensation method for the degradation of HFC audio signal is proposed for producing an immersive audio effect at realistic media. Our experimental results show that low frequency components (LFC) of audio signal had a little directional degradation in spite of effect of the polar patterns of microphone and the compensation of HFC can be realized with adapting the attenuation inclination of LFC. This research is expected to be helpful for producing an immersive audio effect for a realistic media.

© 2018 by the author. Licensee MDPI, Basel, Switzerland. This work investigates the problem of detecting hazardous events on roads by designing an audio surveillance system that automatically detects perilous situations such as car crashes and tire skidding. In recent years, research has shown several visual surveillance systems that have been proposed for road monitoring to detect accidents with an aim to improve safety procedures in emergency cases. However, the visual information alone cannot detect certain events such as car crashes and tire skidding, especially under adverse and visually cluttered weather conditions such as snowfall, rain, and fog. Consequently, the incorporation of microphones and audio event detectors based on audio processing can significantly enhance the detection accuracy of such surveillance systems. This paper proposes to combine time-domain, frequency-domain, and joint time-frequency features extracted from a class of quadratic time-frequency distributions (QTFDs) to detect events on roads through audio analysis and processing. Experiments were carried out using a publicly available dataset. The experimental results conform the effectiveness of the proposed approach for detecting hazardous events on roads as demonstrated by 7% improvement of accuracy rate when compared against methods that use individual temporal and spectral features. Funding text #1 The ability to detect hazardous events on the road can be a matter of life and death, or it can emxpeearnimthenetadtiifofne,regnenceerbateetdwteheenreasunlotsramnadlwliafes raenspdoansliibfelewfoirthwraitminagjothrehpaanpderi.caSp.A..I-Mn .thanisalwyzoerdk,thweerehsualvtse proposedasystemthatcombinest-,f-and(t,f)-domainfeaturestosimultaneouslyconsiderthenon-approach and the results to further improve the quality of the paper. stationary,instantaneous,andlong-termpropertiesofaudiosignalstofacilitateautomaticdetection and classificationofaudio anomalies. The results of experiments performed on a publiclyavailable National Research Fund (a member of the Qatar Foundation). The statements made herein are solely the rbesponsibilityenchmark dofattheaseauthors.t demonstrate the robustness of the proposed method against background noise compared with state-of-the-art approaches for road surveillance applications. Our audio classification system is confirmed to be effective indetecting hazardous...View all Funding text #2 Acknowledgments: This publication was made possible by NPRP grant # NPRP8-140-2-065 from the Qatar National Research Fund (a member of the Qatar Foundation). The statements made herein are solely the respoDatansibifrliomtyothefthFatalityeauthoAnalysisrs. Reporting System and the General Estimates System; National Highway Traffic Safety Administration: Washington, DC, USA, 2015. 5.ConfEvanco,lictsofInWt.eMre.sThet:ThImpacteauthoofrsRapiddeclareIncidentthattherDetectionearenocononfFrliceewaytsofinAccidentterest. Fatalities. Mitretek: McLean

Artificial neural networks are increasingly being introduced into modern life, finding application in different areas of activity. One of these uses is audio processing. Analysis of audio files is based on extracting of big amount of various characteristics which could be used in different ways. This paper is devoted to development of algorithm of collecting biometric voice data and neural network which uses these data to classify speech emotions. The main objective of this work was to create classifier of emotions which uses audio file as input data.

In this study the movement patterns of ten expert musicians are quantitatively related to expressive timing patterns and the music structure during performances. The hypothesis is that ancillary gestures recurrently employed are closely related to expressive intentions, and that the expressive content imposed in key musical passages is thus reflected in the patterns of gestural recurrence. A movement and an audio analysis of 30 clarinet performances of a Brahms' excerpt are compared. Results show direct correlations between the recurrence pattern of clarinetists' ancillary movements and expressive bar duration manipulations employed by them, associated with melodic phrasing and harmonic transitions.

We present a complete, fully functional energy-autonomous audio sensor node with 6×5×4mm3 form factor. The system uses a new audio processing IC integrated with a MEMS microphone, general purpose 32-bit processor, 8Mb Flash, RF transceiver with custom 3D antenna, PV cells for energy harvesting and battery. The 4.7μW audio processing IC performs audio acquisition with 4–32× compression. The complete stand-alone system achieves 38mins of speech recording and energy-autonomous operation in room light.

In the current scenario, there exist huge amount of audio files with mixed sound sources. These mixed sounds consist of different frequencies viz. high frequency (string instruments like guitar), low frequencies (base instrument like drums, tabla) and intermediate human speech frequency. All these frequencies makes a music signal which is required for pleasure. The music creation is achieved by mixing of multiple signals using mixer. Our aim is to reverse the process of mixing and extract an audio signals so that they can be used in applications like karaoke, remix, instrumental music, audio restoration etc. One of the major application is noise cancellation and music transcription. In this paper we have demonstrated separation of single channel separation from mixed audio signal with the accuracy of 93% and average extraction speed of 20 seconds per minute of audio.

Continuous Condition Monitoring of the machine is importan to improve efficiency of the machine, to avoid unexpected accidents and financial losses. Condition monitoring using audio signal processing is focused in this paper. The sound of the machine carries information about the machine condition. The change in features of audio signal are observed using the Peak Variation Response (PVR) of the Fast Fourier Transform (FFT) of signal. In this paper, the conventional FFT method has been used to prove that, it is sufficient for the detection of abnormalities due to different loading condition of the machine using audio signature. And a study on PVR of Intrinsic Mode Function (IMF) has been carried out to present the cause of variation in the energy and frequency of the oscillations of audio frequency (AF) modulated signals.

This page proposed an audio indicator which designed by using C9014, LED by operational amplifier level indicator, the decimal count/distributor of CD4017. The experimental can control audibly neon and holiday lights through the signal. Input audio signal after C9014 composed of operational amplifier for power amplifier, the adjust potentiometer extraction amplification signal input voltage CD4017 distributors make its drive to count, then connect the LED display running situation of the circuit. This simple audio indicator just use only U1 and can produce two colors LED with the audio signal tandem come pursuit of the running effect, from LED display the running of the situation takes can understand the general audio signal. The variation in the audio and the frequency of the signal and the corresponding level size. In this light can achieve jump to change, slowly, atlas, lighting four forms, used in home, hotel, discos, theater, advertising and other fields, and a wide range of USES, rU1h life in a mode...

This paper presents a system of audio signal processing based on FPGA,the system uses audio codec chip LM4550 to A/D transform and D/A transform the input analog audio signal and output digital audio signal.Using FPGA as the high speed signal processor to realize volume adjustment and audio effect control,so it can output different style music.Meantime, the system designs a FFT computing module and control system of VGA display interface,to compute the digital audio signal which is A/D transformed,and real-time display the frequency spectrum of audio signal on VGA.

The real-time processing for the input analog audio signal, audio processing program is proposed based on DSP. The system use FFT algorithm as the core, first, the input analog audio signal is sampled and A/D conversion using TLV320AIC23, and then use high speed digital signal processor to make real-time processing for the signal. Theoretical and experimental results show that the system can meet the design requirements, it has the advantage of high real-time and simple structure. The system has a good application and reference value for the development and design of data collecting and remote monitoring.

An audio recording is subject to a number of possible distortions and artifacts. Consider, for example, artifacts due to acoustic reverberation and background noise. The acoustic reverberation depends on the shape and the composition of a room, and it causes temporal and spectral smearing of the recorded sound. The background noise, on the other hand, depends on the secondary audio source activities present in the evidentiary recording. Extraction of acoustic cues from an audio recording is an important but challenging task. Temporal changes in the estimated reverberation and background noise can be used for dynamic acoustic environment identification (AEI), audio forensics, and ballistic settings. We describe a statistical technique to model and estimate the amount of reverberation and background noise variance in an audio recording. An energy-based voice activity detection method is proposed for automatic decaying-tail-selection from an audio recording. Effectiveness of the proposed method is tested using a data set consisting of speech recordings. The performance of the proposed method is also evaluated for both speaker-dependent and speaker-independent scenarios.

Recommending TV content to groups of viewers is best carried out when relevant information such as the demographics of the group is available. However, it can be difficult and time consuming to extract information for every user in the group. This paper shows how an audio analysis of the age and gender of a group of users watching the TV can be used for recommending a sequence of N short TV content items for the group. First, a state of the art audio-based classifier determines the age and gender of each user in an M-user group and creates a group profile. A genetic recommender algorithm then selects for each user in the profile, a single personalized multimedia item for viewing. When the number of items to be presented is different to the number of viewers in the group, i.e. M ≠ N, a novel adaptation algorithm is proposed that first converts the M-user group profile to an N-slot content profile, thus ensuring that items are proportionally allocated to users with respect to their demographic categorization. The proposed system is compared to an ideal system where the group demographics are provided explicitly. Results using real speaker utterances show that, in spite of the inaccuracies of state-of-the-art age-and-gender detection systems, the proposed system has a significant ability to predict an item with a matching age and gender category. User studies were conducted where subjects were asked to rate a sequence of advertisements, where half of the advertisements were randomly selected, and the other half were selected using the audio-derived demographics. The recommended advertisements received a significant higher median rating of 7.75, as opposed to 4.25 for the randomly selected advertisements 1. Recommending TV content to groups of viewers is best carried out when relevant information such as the demographics of the group is available. However, it can be difficult and time consuming to extract information for every user in the group. This paper shows how an audio analysis of the age and gender of a group of users watching the TV can be used for recommending a sequence of N short TV content items for the group. First, a state of the art audio-based classifier determines the age and gender of each user in an M-user group and creates a group profile. A genetic recommender algorithm then selects for each user in the profile, a single personalized multimedia item for viewing. When the number of items to be presented is different to the number of viewers in the group, i.e. M ≠ N, a novel adaptation algorithm is proposed that first converts the M-user group profile to an N-slot content profile, thus ensuring that items are proportionally allocated to users with respect to their demographic categorization. The proposed system is compared to an ideal system where the group demographics are provided explicitly. Results using real speaker utterances show that, in spite of the inaccuracies of state-of-the-art age-and-gender detection systems, the proposed system has a significant ability to predict an item with a matching age and gender category. User studies were conducted where subjects were asked to rate a sequence of advertisements, where half of the advertisements were randomly selected, and the other half were selected using the audio-derived demographics. The recommended advertisements received a significant higher median rating of 7.75, as opposed to 4.25 for the randomly selected advertisements 1.

Robustness against noise and interfering audio signals is one of the challenges in speech recognition and audio analysis technology. One avenue to approach this challenge is single-channel multiple-source modeling. Factorial hidden Markov models (FHMMs) are capable of modeling acoustic scenes with multiple sources interacting over time. While these models reach good performance on specific tasks, there are still serious limitations restricting the applicability in many domains. In this paper, we generalize these models and enhance their applicability. In particular, we develop an EM-like iterative adaptation framework which is capable to adapt the model parameters to the specific situation (e.g. actual speakers, gain, acoustic channel, etc.) using only speech mixture data. Currently, source-specific data is required to learn the model. Inference in FHMMs is an essential ingredient for adaptation. We develop efficient approaches based on observation likelihood pruning. Both adaptation and efficient inference are empirically evaluated for the task of multipitch tracking using the GRID corpus.

The use of a transparent thin film speaker in a home window can provide both an invisible audio playback device and an active noise cancellation system. However, several key challenges need to be addressed for development of such a system. A traditional feedforward active noise cancellation system uses direct microphone measurements for both reference and error signals. Such a system can degrade both the audio playback and the noise cancellation performance. Hence a wave separation algorithm is used in this paper for separation of the audio playback component from the external noise component for the reference signal. Further, an online adaptive secondary transfer function estimation method is used for accurate removal of the audio component from the error signal. Two approaches are attempted for the secondary transfer function estimation – use of source audio signals and use of additive white noise. The developed algorithms are evaluated in a scaled cabin equipped with a window and a transparent thin film actuator. Experimental results show that the developed system can preserve the auxiliary audio sound while cancelling external noise effectively. The system based on the use of source audio signals for secondary path estimation is found to yield the largest cancellation of external noise while producing the least amplitude of control inputs that cancel desired audio signals.

A audio compressor is an important part of a audio processing system. The compression algorithm designed in this paper will be applied to audio compressor in the digital audio broadcasting transmitter. In this paper, an effective algorithm for dynamic control of the gain by taking mean square value is proposed, and two different audio signal from two channel processing schemes will be compared to get better one on the impact of compression. Then the algorithm is evaluated by the effect of compression of signal amplitude in the MATLAB simulation, and a scheme selected that signal is firstly dealt with to enhance the effect of the dual channel stereo and then is compressed is more suitable for the two channel audio compression from two schemes.

We propose a design of visible light communication which is based on the white LED power source. The design possesses effective software flowchart and the optimized circuit of the hardware modules which include power supply, signal amplification, LED driver, Audio signal transmission, Audio signal reception, filter and amplifier and Audio output. The testing of the design shows our system can realize the short-distance transmission of MP3 audio signal. Due to the advantages of stable performance, little distortion, low cast, simplicity and feasibility, our design owns a strong practical significance in the promotion of visible light communication technology.

In this paper, we present madmom, an open-source audio processing and music information retrieval (MIR) library written in Python. madmom features a concise, NumPy-compatible, object oriented design with simple calling conventions and sensible default values for all parameters, which facilitates fast prototyping of MIR applications. Prototypes can be seamlessly converted into callable processing pipelines through madmom's concept of Processors, callable objects that run transparently on multiple cores. Processors can also be serialised, saved, and re-run to allow results to be easily reproduced anywhere. Apart from low-level audio processing, madmom puts emphasis on musically meaningful high-level features. Many of these incorporate machine learning techniques and madmom provides a module that implements some in MIR commonly used methods such as hidden Markov models and neural networks. Additionally, madmom comes with several state-of-the-art MIR algorithms for onset detection, beat, downbeat and meter tracking, tempo estimation, and piano transcription. These can easily be incorporated into bigger MIR systems or run as stand-alone programs.

Enhancing speech from audio signals recorded using several microphones is of interest to many audio applications such as human-computer interfaces and teleconference systems. With an increasing availability of low-cost micro-electro-mechanical systems (MEMS) and powerful processing units, embedded systems are capable of recording and processing several audio signals in realtime. In this paper, an overview of the entire signal processing chain from audio signal acquisition with an array of MEMS microphones to the signal enhancement using directional filtering is provided. The presented results indicate that good noise and interference suppression can be achieved in realtime using an embedded system equipped with low-cost microphones.

This paper presents a novel BigEAR big data framework that employs psychological audio processing chain (PAPC) to process smartphone-based acoustic big data collected when the user performs social conversations in naturalistic scenarios. The overarching goal of BigEAR is to identify moods of the wearer from various activities such as laughing, singing, crying, arguing, and sighing. These annotations are based on ground truth relevant for psychologists who intend to monitor/infer the social context of individuals coping with breast cancer. We pursued a case study on couples coping with breast cancer to know how the conversations affect emotional and social well being. In the state-of-the-art methods, psychologists and their team have to hear the audio recordings for making these inferences by subjective evaluations that not only are time-consuming and costly, but also demand manual data coding for thousands of audio files. The BigEAR framework automates the audio analysis. We computed the accuracy of BigEAR with respect to the ground truth obtained from a human rater. Our approach yielded overall average accuracy of 88.76% on real-world data from couples coping with breast cancer., 6 pages, 10 equations, 1 Table, 5 Figures, IEEE International Workshop on Big Data Analytics for Smart and Connected Health 2016, June 27, 2016, Washington DC, USA

Technology to record sound, available in personal devices such as smartphones or video recording devices, is now ubiquitous. However, the production quality of the sound on this user-generated content is often very poor: distorted, noisy, with garbled speech or indistinct music. Our interest lies in the causes of the poor recording, especially what happens between the sound source and the electronic signal emerging from the microphone, and finding an automated method to warn the user of such problems. Typical problems, such as distortion, wind noise, microphone handling noise and frequency response, were tested. A perceptual model has been developed from subjective tests on the perceived quality of such errors and data measured from a training dataset composed of various audio files. It is shown that perceived quality is associated with distortion and frequency response, with wind and handling noise being just slightly less important. In addition, the contextual content of the audio sample was found to modulate perceived quality at similar levels to degradations such as wind and rendering those introduced by handling noise negligible.

Nowadays, audio channels in musical concerts (as well as in cabins as in live concerts) start from analogical audio signals generated at microphones, continue with signal transmission to a central audio mixer through wire connections (a wire for each microphone) and end at its digital processing and posterior reproduction through audio system speakers. Algorithm proposed by the authors suggests the signal digitalization directly at microphones and the introduction of virtual audio mixers, which simplifies audio channels management, and reproduces equalization and volume control functions from physical ones, adding control mechanisms over output channels. Noise and attenuation levels are measured and compared with traditional systems, to demonstrate the advantages of using the proposed algorithm.

Audio information plays a rather important role in the increasing digital content that is available today, resulting in a need for methodologies that automatically analyze such content: audio event recognition for home automations and surveillance systems, speech recognition, music information retrieval, multimodal analysis (e.g. audio-visual analysis of online videos for content based recommendation), etc. This paper presents pyAudioAnalysis, an open-source Python library that provides a wide range of audio analysis procedures including: feature extraction, classification of audio signals, supervised and unsupervised segmentation and content visualization. pyAudioAnalysis is licensed under the Apache License and is available at GitHub (https://github.com/tyiannak/pyAudioAnalysis/). Here we present the theoretical background behind the wide range of the implemented methodologies, along with evaluation metrics for some of the methods. pyAudioAnalysis has been already used in several audio analysis research applications: smart-home functionalities through audio event detection, speech emotion recognition, depression classification based on audio- visual features, music segmentation, multimodal content-based movie recommendation and health applications (e.g. monitoring eating habits). The feedback provided from all these particular audio applications has led to practical enhancement of the library.

In this paper, the subjective audio quality of various digital radio systems is evaluated. In order to estimate the true audio quality of a commercial digital radio service, test audio signals were acquired directly from the digital radio equipment in the field. Twenty-one subjects participated in the test after pre-screening. The evaluation process consisted of two phases. In the first phase, the audio quality of each system was evaluated as a function of the bit rates, and the relationship between the audio quality and the channel capacity of each system was obtained. In order to confirm the evaluation consistency, in the second phase, the audio quality as a function of the systems was evaluated. Assuming that the digital radio equipment tested in this paper is used for service, these evaluation results provide the true audio quality of commercial digital radio services for various types of systems and bit rates, and can be used as a reference to determine the optimal channel capacity of digital radio service for a given target audio quality.

In this paper, a terrestrial digital multimedia broadcasting (T-DMB) multichannel audio broadcasting system based on spatial audio coding is presented. The proposed system provides realistic multichannel audio service via T-DMB with a small increase of data rate as well as backward compatibility with the conventional stereo-based T-DMB player. To reduce the data rate for additional multichannel audio signals, we compress the multichannel audio signals using the sound source location cue coding algorithm, which is an efficient parametric multichannel audio compression technique. For compatibility, we use the dependent property of an elementary stream descriptor, and this property should be ignored in a conventional T-DMB player. To verify the feasibility of the proposed system, we implement the T-DMB multichannel audio encoder and a prototype player. We perform a compatibility test using the T-DMB multichannel audio encoder and conventional T-DMB players. The test demonstrates that the proposed system is compatible with a conventional T-DMB player and that it can provide a promisingly rich audio service.

In this article, we present CLAM, a C++ software framework, that offers a complete development and research platform for the audio and music domain. It offers an abstract model for audio systems and includes a repository of processing algorithms and data types as well as all the necessary tools for audio and control input/output. The framework offers tools that enable the exploitation of all these features to easily build cross-platform applications or rapid prototypes for media processing algorithms and systems. Furthermore, included ready-to-use applications can be used for tasks such as audio analysis/synthesis, plug-in development, feature extraction or metadata annotation. CLAM represents a step forward over other similar existing environments in the multimedia domain. Nevertheless, it also shares models and constructs with many of those. These commonalities are expressed in the form of a metamodel for multimedia processing systems and a design pattern language.

We have extended our sports video browsing framework for personal video recorders, such as recordable-DVD recorders, blu-ray disc recorders and/or hard disc recorders, to music segment detection. Our extension to Japanese broadcast music video programs consists of detecting audio segment boundaries such as conversations with guests followed by music/song etc. Our proposed system first identifies the music/song scenes using audio analysis, and then adjusts the start/end position by detecting video shot changes, so as to achieve accurate detection of the music segment thus enabling rapid browsing. Our preliminary results indicate that our audio-only summarization with scene change support works well for music video content. We can therefore integrate the enhancement into our product at a low computational cost.

This paper will describe how audio amplifiers specifications can be used as a pedagogical tool to help explain distortion concepts in signal processing theory. Audio entertainment systems are an ideal and practical vehicle for this subject since these are ubiquitous among students, and audio quality or lack thereof is readily perceived. We will consider various distortion types including amplitude, phase, and nonlinear. In the nonlinear distortion category we present harmonic, and intermodulation. Also discussed are distortions due to limits of dynamic range, and quantization errors. We then present how distortion concepts can be integrated into a signals and systems course.

This paper proposes dictionary learning with archetypes for audio processing. Archetypes refer to so-called pure types, which are a combination of a few data points and which can be combined to obtain a data point. The concept has been found useful in various problems, but it has not yet been applied for audio analysis. The algorithm performs archetypal analysis that minimises the generalised Kullback-Leibler divergence, shown suitable for audio, between an observation and the model. The methodology is evaluated in a source separation scenario (mixtures of speech) and shows results, which are comparable to the state-of-the-art, with perceptual measures indicating its superiority over all of the competing methods in the case of medium-size dictionaries.

Spatial audio coding is a technique that capable of representing multichannel audio signals as a lower number of audio channels accompanied by spatial parameters and residual signal which will be useful for recreating the original multi-channel audio signals. Moving Picture Expert Group (MPEG) Surround, an international standard developed based on spatial audio coding, specifies Reverse Two-To-Three (R-TTT) module to extend stereo audio, consisted of left and right channels, into three audio channels: left, centre, and right channels based on Channel Prediction Coefficient (CPC) as spatial parameter and residual signal. In this paper, a modified residual signal is proposed to provide a better audio waveform reconstruction in the decoder side by minimising distortion caused by quantisation of CPC. Our experiments show that the waveform accuracy in terms of Signal-to-Noise Ratio (SNR) gets improved as high as 11 dB while the subjective test shows that the proposed method does not reduce perceptual quality, in terms of Subjective Difference Grade (SDG) score, of the reconstructed audio signals.

We describe Lamello, an approach for creating tangible in- put components that recognize user interaction via passive acoustic sensing. Lamello employs comb-like structures with varying-length tines at interaction points (e.g., along slider paths). Moving a component generates tine strikes; a real- time audio processing pipeline analyzes the resultant sounds and emits high-level interaction events. Our main contributions are in the co-design of the tine structures, information encoding schemes, and audio analysis. We demonstrate 3D printed Lamello-powered buttons, sliders, and dials.

The reversal of the current directions in audio circuit elements causes polarity inversion of the acquired audio signal with respect to the reference input signal. The objective of the work presented in this paper is to implement a simple polarity detection circuit in audio preamplifiers which provides an indication of the signal polarity inversion. The present work also demonstrates the possibilities of polarity inversion in audio circuits of audio data acquisition devices. Inputs fed in the inverting/noninverting terminals of audio operational amplifiers (Op-Amps) cause polarity reversal of the amplitude values of the speech/audio signals. Even though, polarity inversion in audio circuits are perceptually indistinguishable, provides inaccurate values of speech parameters estimated by processing the speech. The work presented in this paper discusses, how polarity inversion is introduced at the circuit level and proposes a polarity detection circuit which provides an indication of polarity reversal after the pre amplification. The effectiveness of the proposed polarity inversion circuit is confirmed by 100 % polarity detection rate for the 100 randomly selected audio files of the CMU-Arctic database when simulated using Proteus 8.0. The paper is concluded by discussing the significance of VLSI implementation proposed polarity detection circuit in the most commonly used audio preamplifier systems.

This paper proposes a new blind method of spectral (high frequency) enhancement of MP3 audio. Due to the protocol constraint, the audio bandwidth of MP3 is restricted to 16 kHz at 128 kbps and 12 kHz at 64 kbps. Although band-restricted MP3 audio provides savings of storage space and network bandwidth, it suffers a major problem of a loss in high frequency fidelity such as localization, ambient information, and bright nature of audio. In contrast to previous works, this paper suggests a couple of new approaches such as dynamic cut-off finding, harmonic compensation and the gain-controlled high frequency compensation. Furthermore, the proposed algorithm requires less computational power with better high frequency enhancement capability compared to the previous works. The proposed algorithm is implemented on TI TMS6713 DSP platform and tested with IFP 800 model MP3 player. Audio quality of the proposed algorithm is reported and compared using spectrum analysis.

One of the major issues in multimedia forensics is the identification of video acquisition devices. Most of the relevant state-of-the-art solutions rely on either visual or audio analysis, using feature arrays that are highly correlated with the characteristics of the respective camera or microphone. In this work, we present a multi-modal approach that uses both video and audio information to improve the detection accuracy. For this purpose, microphone detection based on the blind estimation of the frequency response is complemented with a video camera detection based on a set of video features related to the Color Filter Array interpolation. Experimental results show that the combined approach results in an improved overall classification accuracy over the mono-modal cases.

This paper proposes a serial transmission technique of signals for multi-channel audio systems. The technique converts analog audio signals of multiple channels of an audio unit into digital signals, and transfers the digital signals to the speakers through a serial line sequentially. Each speaker is allowed to select its corresponding digital audio signals and to transform the signals into analog audio signals to make sounds for the speaker. The functions of the proposed technique are modeled in VHDL, implemented with FPGA chips, and tested for actual audio systems.

In this paper, we study coding artifacts in MPEG-compressed scalable audio. Specifically, we consider the MPEG advanced audio coder (AAC) using bit slice scalable arithmetic coding (BSAC) as implemented in the MPEG-4 reference software. First we perform human subjective testing using the comparison category rating (CCR) approach, quantitatively comparing the performance of scalable BSAC with the nonscaled TwinVQ and AAC algorithms. This testing indicates that scalable BSAC performs very poorly relative to TwinVQ at the lowest bitrate considered (16 kb/s) largely because of an annoying and seemingly random mid-range tonal signal that is superimposed onto the desired output. In order to better understand and quantify the distortion introduced into compressed audio at low bit rates, we apply two analysis techniques: Reng bifrequency probing and time-frequency decomposition. Using Reng probing, we conclude that aliasing is most likely not the cause of the annoying tonal signal; instead, time-frequency or spectrogram analysis indicates that its cause is most likely suboptimal bit allocation. Finally, we describe the energy equalization quality metric (EEQM) for predicting the relative perceptual performance of the different coding algorithms and compare its predictive ability with that of ITU Recommendation ITU-R BS.1387-1.

The problem of auditory scene segmentation plays important role in many audio analysis and processing tasks. The accuracy and robustness of segmentation step have influence on the remaining stages in audio processing chain. In the work a dedicated system for segmentation of the audio stream is presented. The segmentation scheme uses Delta-BIC and metric-based techniques to determine the change-point in audio data. For this purpose a dedicated auditory feature has been proposed, which is based on the gammatone filter bank. The proposed feature (GTEAD) has been designed using inter-channel analysis of the auditory filter bank outputs. For each channel, the temporal envelope and its periodic self-similarities have been calculated. Then, the distances between obtained signals from the neighbouring channels have been computed resulting in the final feature vector. The performance of the GTEAD feature has been compared to the popular MFCC feature using database of audio streams with defined single change-point in each example. The obtained results show that GTEAD feature outperforms MFCC feature in terms of accuracy and the number of detected points.

This paper presents a compression technique to improve the quality of three-dimensional (3D) audio produced by multiple loudspeaker channels or by headphone. The approach is based on extracting the side information of spatial sound sources within the three-dimensional space when capturing the sound sources. Different from other compression technique, the distances of sound sources are included in the side information. The separated signals of different sound sources are downmixed into one mono or stereo audio signal with the side information. The resulting downmixed signal is then compressed with traditional audio coder, resulting in a better perceptual quality of 3D audio by adding the distance parameter in the side information, and maintaining a low bit rates comparable with directional audio coding (DirAC).

The paper deals with the development of an efficient real time system for the reproduction of a spatialized audio field taking into account the listeners position. The system is composed of two parts: a sound rendering system based on a crosstalk canceller that is required in order to have a spatialized audio reproduction and a listener position tracking system in order to model the crosstalk canceller parameters. Then, an efficient implementation of a time domain crosstalk cancellation algorithm is presented considering an improved version of the recursive ambiophonics crosstalk elimination algorithm. A real time application is proposed introducing a Kinect control, capable to accurately track the listener position and changing the crosstalk parameters related to its position. Several results are presented comparing the proposed approach with the state of the art in order to confirm its validity.

The Philips audio fingerprint[1] has been used for years, but its robustness against external noise has not been studied accurately. This paper shows the Philips fingerprint is noise resistant, and is capable of recognizing music that is corrupted by noise at a -4 to -7 dB signal to noise ratio. In addition, the drawbacks of the Philips fingerprint are addressed by utilizing a “Power Mask” in conjunction with the Philips fingerprint during the matching process. This Power Mask is a weight matrix given to the fingerprint bits, which allows mismatched bits to be penalized according to their relevance in the fingerprint. The effectiveness of the proposed fingerprint was evaluated by experiments using a database of 1030 songs and 1184 query files that were heavily corrupted by two types of noise at varying levels. Our experiments show the proposed method has significantly improved the noise resistance of the standard Philips fingerprint.

Three-dimensional (3D) audio technologies are booming with the success of 3D video technology. The surge in audio channels makes its huge data unacceptable for transmitting bandwidth and storage media, and the signal compression algorithm for 3D audio systems becomes an important task. This paper investigates the conventional mid/side (M/S) coding method and discusses the signal correlation property of three-dimensional multichannel systems. Then based on the channel triple, a three-channel dependent M/S coding (3D-M/S) method is proposed to reduce interchannel redundancy and corresponding transform matrices are presented. Furthermore, a framework is proposed to enable 3D-M/S compress any number of audio channels. Finally, the masking threshold of the perceptual audio core codec is modified, which guarantees the final coding noise to meet the perceptual threshold constraint of the original channel signals. Objective and subjective tests with panning signals indicate an increase in coding efficiency compared to Independent channel coding and a moderate complexity increase compared to a PCA method.

The audio archives of the CNRS-Musee de l'Homme are among the most important collections of ethnomusicological recordings in Europe. As the number of collections increase and as new audio technologies arise, questions about the best approaches for preserving, archiving and providing public access to these materials have arisen. With this in mind, ethnomusicologists and engineers have joined efforts since 2007 to develop a scalable and collaborative web platform for managing and increasing access to digitized sound archives. This web platform is based on Telemeta, an open-source web audio framework dedicated to digital sound archives. Since 2011, the Telemeta framework has been deployed to hold the platform of the CNRS-Musee de l'Homme's audio archives, which are managed by the Research Center for Ethnomusicology. This framework focuses on facilitating collaboration and user access to audio items and their associated metadata. The architecture of Telemeta relies on TimeSide, an open-source audio processing framework written in Python and JavaScript languages, which provides decoding, encoding and streaming capabilities together with a smart embeddable HTML audio player. TimeSide can also produce various automatic annotation, segmentation and musicological analysis that have been developed in the interdisciplinary research project called DIADEMS. Furthermore it includes a set of audio analysis plug-ins and wraps several audio feature extraction libraries. This paper introduces the Telemeta framework and discusses how cutting-edge tools are being implemented that provide new ways to archive and analyze sound libraries.

The purpose of this chapter is to provide basic knowledge and techniques related to the creation, representation, playback, recording and storing of audio signals using MATLAB. In addition, short-term audio analysis is introduced here.

For real-time audio streaming services over next-generation mobile phone systems, such as live concert broadcasting and radio relaying, we have developed a realtime IMT-2000 audio transmission system. This system consists of a real-time encoder and a portable real-time decoder, which are both based on fixed-point DSP chips. TWINVQ technology is used for audio compression, and a scheme to protect audio data against bit-errors was designed based on error sensitivity observations. Self-synchronization and adaptive error concealment are carried out at the decoder. We confirmed that the system can transmit high-quality stereo audio samples at 32 kHz with a total bit rate of 64 kbit/s over a simulated IMT-2000 channel. We also checked that this system can connect to a real IMT-2000 experimental network, and found that it is easily applicable to various audio streaming services.

With stereo audio stepping into people's daily lives, spatial parametric technology comes to its broad prospects for development. This paper provides the audio bandwidth extension encoding and decoding devices, the bitrate of less high-frequency signal reconstruction, in order to improve the quality of the output audio signal. This research makes a contribution to the lack of existing technology, audio bandwidth extension encoding and decoding devices, digital audio decoding to take advantage of a lower bit rate to restore the original audio signal in the digital encoding process the loss of high-frequency components, thereby enhancing the sound playback quality.