Search

The present work approaches intelligent traffic evaluation and congestion detection using sound sensors and machine learning. For this, two important problems are addressed: traffic condition assessment from audio data, and analysis of audio under uncontrolled environments. By modeling the traffic parameters and the sound generation from passing vehicles and using the produced audio as a source of data for learning the traffic audio patterns, we provide a solution that copes with the time, the cost and the constraints inherent to the activity of traffic monitoring. External noise sources were introduced to produce more realistic acoustic scenes and to verify the robustness of the methods presented. Audio-based monitoring becomes a simple and low-cost option, comparing to other methods based on detector loops, or GPS, and as good as camera-based solutions, without some of the common problems of image-based monitoring, such as occlusions and light conditions. The approach is evaluated with data from audio analysis of traffic registered in locations around the city of Sao Jose dos Campos, Brazil, and audio files from places around the world, downloaded from YouTube. Its validation shows the feasibility of traffic automatic audio monitoring as well as using machine learning algorithms to recognize audio patterns under noisy environments.

Issues that are increasingly arising in criminal proceedings are highlighted, namely: use of data obtained as a result of diagnostic researches on video, sound recordings that allows to ensure a higher level of completeness, objectivity and comprehensiveness of research. The purpose of the article is to analyze the possibilities of forensic diagnostic researches on video and sound recordings of both oral speech and the sound environment of an offense event. Various modern possibilities of researches on video and sound recordings are considered. Attention is paid both to the specifics of diagnostic speech researches in order to obtain information about personality of the unknown speaker and to information that can be obtained through research and diagnosis of non-speech information. Analyzed the possibilities (within the framework of the examination of video, sound recording) diagnostic researches on directly speech information, including the definition of: oral speech form, nature of relations between the interlocutors, conversation meaning (definition of the meaningful situation: consent vs disagreement; permission vs prohibition; understanding vs misunderstanding; request; advice; promise; assurance; gratitude; threat; clarification; order; question; message, etc.), rate of speech and the emotional state of the interlocutors, as well as expert diagnostics of biological parameters of the speakers. Possibilities of diagnostic studies of technical studies of recording media and recording equipment, that can contain information about the technology of obtaining/fixing/saving video, sound recordings, properties and features of the media itself are considered. Non-speech sounds are classified according to their belonging to certain sources. The article highlights effectiveness of using the forensic information obtained through forensic examination in practice of investigating crimes.

Style transfer is one of the most popular uses of neural networks. It has been thoroughly researched, such as extracting the style from famous paintings and applying it to other images thus creating synthetic paintings. Generative adversarial networks (GANs) are used to achieve this. This paper explores the many ways in which the same results can be achieved with audio related tasks, for which a plethora of new applications can be found. Analysis of different techniques used to transfer styles of audios, specifically changing the gender of the audio is implemented. The Crowd sourced high-quality UK and Ireland English Dialect speech data set was used. In this paper, the input is the male or female wave form and the opposite gender’s waveform is synthesized by the network, with the content spoken remaining the same. Different architectures are explored, from naive techniques and directly training audio waveforms against convolution neural networks (CNN) to using extensive algorithms researched for image style conversion and generation of spectrograms (using GANs) to be trained on CNNs. This research has a broader scope when used in converting music from one genre to another, identification of synthetic voices, curating voices for AIs based on preference etc.

Open-source software libraries have a significant impact on the development of Audio Signal Processing and Music Information Retrieval (MIR) systems. Despite the abundance of such tools, there is a lack of an extensive and easy-to-use reference library for audio feature extraction on Web clients. In this article, we present Essentia.js, an open-source JavaScript (JS) library for audio and music analysis on both web clients and JS engines. Along with the Web Audio API, it can be used for both offline and real-time audio feature extraction on web browsers. Essentia.js is modular, lightweight, and easy-to-use, deploy, maintain, and integrate into the existing plethora of JS libraries and web technologies. It is powered by a WebAssembly back end cross-compiled from the Essentia C++ library, which facilitates a JS interface to a wide range of low-level and high-level audio features, including signal processing MIR algorithms as well as pre-trained TensorFlow.js machine learning models. It also provides a higher-level JS API and add-on MIR utility modules along with extensive documentation, usage examples, and tutorials. We benchmark the proposed library on two popular web browsers and the Node.js engine, and four devices, including mobile Android and iOS, comparing it to the native performance of Essentia and the Meyda JS library. The work on Essentia.js has been partially funded by the Ministry of Science and Innovation of the Spanish Government under the grant agreement PID2019-111403GB-I00 (Musical AI).

The article deals with some problems related to the production of forensic speech and audio analysis, taking into account global digitalization of all spheres of human life. Voicechanger, technology of voice synthesis signifi cantly complicate forensic phonoscopic examinations and put before the forensic experts a new, interesting challenges. We consider the algorithm for changing the voice using modern software, as well as features of voice synthesis technologies. Some variants of studying such phonograms are proposed.

Research on self-efficacy and confidence has spread across several subfields of psychology and neuroscience. The role of one’s confidence is very crucial in the formation of attitude and communication skills. The importance of differentiating the levels of confidence is quite visible in this domain. With the recent advances in extracting behavioral insight from a signal in multiple applications, detecting confidence is found to have great importance. One such prominent application is detecting confidence in interview conversations. We have collected an audiovisual data set of interview conversations with 34 candidates. Every response (from each of the candidate) of this data set is labeled with three levels of confidence: high, medium, and low. Furthermore, we have also developed algorithms to efficiently compute such behavioral confidence from speech and video. A deep learning architecture is proposed for detecting confidence levels (high, medium, and low) from an audiovisual clip recorded during an interview. The achieved unweighted average recall (UAR) reaches 85.9% on audio data and 73.6% on video data captured from an interview session.

Nearly half of the world languages are considered endangered and need to be documented, analyzed, and revitalized. However, existing linguistics tools lack the accessibility to effectively analyze languages such as Blackfoot in which relative pitch movement is significant, e.g., words with the same sound sequence but convey different meanings when changing in pitches. To address this issue, we present a novel form of audio analysis with perceptual scale, and develop a consolidated and interactive toolset called MeTILDA (Melodic Transcription in Language Documentation and Analysis) to effectively capture perceived changes in pitch movement and to host other existing desktop-based linguistic tools on the cloud to enable collaboration, data-sharing, and data reuse among multiple linguistic tools.

Η Ημιτονοειδής Μοντελοποίηση είναι μια από τις πιο ευρέως χρησιμοποιούμενες παραμετρικές μεθόδους για την επεξεργασία σήματος φωνής και ήχου. Η ακριβής εκτίμηση των ημιτονοειδών παραμέτρων (πλάτη, συχνότητες, και φάσεις) είναι ένα κρίσιμο σημείο για τη ακριβή αναπαράσταση των σημάτων που αναλύονται. Στην παρούσα εργασία, με βάση τις πρόσφατες εξελίξεις στην ημιτονοειδή ανάλυση, προτείνουμε υψηλής ανάλυσης, προσαρμόσιμα ημιτονοειδή μοντέλα για συστήματα ανάλυσης, σύνθεσης, και τροποποίησης ομιλίας. Στόχος μας είναι να προσφέρουμε συστήματα που αναπαριστούν σήματα φωνής με εξαιρετικά ακριβή και συμπαγή τρόπο.Εμπνευσμένοι από πρόσφατα προταθέντα μοντέλα, όπως το προσαρμόσιμο Σχεδον - Αρμονικό Μοντέλο (aQHM) και το προσαρμόσιμο Αρμονικό Μοντέλο (aHM), διατυπώνουμε τη θεωρία της προσαρμόσιμης Ημιτονοειδούς Μοντελοποίησης και προτείνουμε ένα μοντέλο που ονομάζεται εκτεταμένο προσαρμόσιμο Σχεδον - Αρμονικό Μοντέλο (eaQHM), το οποίο είναι ένα μη παραμετρικό μοντέλο, ικανό να προσαρμόσει τα στιγμιαία πλάτη και φάσεις των συναρτήσεων βάσης του στα τοπικά χρονικά μεταβαλλόμενα χαρακτηριστικά του σήματος της φωνής, αμβλύνοντας έτσι τη γνωστή υπόθεση της τοπικής στασιμότητας. Αποδεικνύεται ότι το eaQHM παρουσιάζει υψηλότερες επιδόσεις από το aQHM στην ανάλυση και ανασύνθεση των έμφωνων τμημάτων φωνής. Με βάση το eaQHM, ένα υβριδικό σύστημα ανάλυσης / σύνθεσης ομιλίας παρουσιάζεται (eaQHNM), μαζί με μια υβριδική έκδοση του του aHM (aHNM). Επιπλέον, παρουσιάζουμε κίνητρα για μια αναπαράσταση του σήματος της φωνής σε όλο το φάσμο και σε όλη τη διάρκεια του, χρησιμοποιώντας το eaQHM, αναπαριστώντας έτσι όλα τα μέρη του σήματος της φωνής, με υψηλής ανάλυσης AM-FM ημίτονα. Η αξιολόγηση δείχνει ότι η προσαρμοσιμότητα και η σχεδόν-αρμονικότητα είναι αρκετή για να παράξει πολύ υψηλή ποιότητα στην ανασύνθεση των άφωνων τμημάτων της φωνής. Στη συνέχεια, παρουσιάζεται το σύστημα πλήρους φάσματος ανάλυσης και σύνθεσης βασισμένο στο eaQHM, το οποίο υπερτερεί συστημάτων που θεωρούνται state-of-the-art, υβριδικά ή πλήρους ανάλυσης, στην ανάλυση και ανασύνθεση φωνής. Η υπεροχή του στην ποιότητα ανασύνθεσης επιβεβαιώθηκε με αντικειμενικές και υποκειμενικές αξιολογήσεις.Όσον αφορά τις εφαρμογές, το eaQHM και το aHM εφαρμόζονται σε μετασχηματισμούς φωνής (κλιμάκωση χρόνου και κλιμάκωση θεμελιώδους συχνότητας). Οι μετασχηματισμοί που προκύπτουν είναι υψηλής ποιότητας, ακολουθώντας πολύ απλούς κανόνες, σε σύγκριση με άλλα συστήματα state-of-the-art. Οι έννοιες της σχετικής φάσης και της καθυστέρησης σχετικής φάσης είναι ζωτικής σημασίας για την ανάπτυξη μετασχηματισμένου σήματος με χαρακτηριστικά αναλλοίωτου σχήματος, χωρίς τεχνικά ελαττώματα, και υψηλής ποιότητας. Τα αποτελέσματα δείχνουν ότι τα συστηματα βασισμένα στην αρμονικότητα προτιμούνται έναντι αυτών της σχεδόν-αρμονικότητας, λόγω της απλότητας της αναπραστάσης. Επιπλέον, το eaQHM εφαρμόζεται στο πρόβλημα της μοντελοποίησης σημάτων ήχου, και συγκεκριμένα ήχων μουσικών οργάνων. Το eaQHM αξιολογείται και σύγκρινεται με state-of-the-art συστήματα, και έχει υψηλές επιδόσεις όσον αφορά την ποιότητα επανασύνθεσης, αναπαριστωντας με επιτυχία τα στάδια της επίθεσης, της μετάβασης, και της στατικότητας ενός ήχου μουσικού οργάνου. Τέλος, μια άλλη προτεινόμενη εφαρμογή έγκειται στην ανάλυση και ταξινόμηση της εκφραστικής ομιλίας. Το eaQHM εφαρμόζεται στην ανάλυση της εκφραστικής ομιλίας, παρέχοντας τις στιγμιαίες παραμέτρους του ως χαρακτηριστικά που μπορούν να χρησιμοποιηθούν στην αναγνώριση και ταξινόμηση, βασισμένη σε διανυσματικούς κβαντιστές, εκφραστικής ομιλίας. Αν και τα ημιτονοειδή μοντέλα δεν χρησιμοποιούνται συνήθως σε τέτοιες εφαρμογές, τα αποτελέσματα είναι ελπιδοφόρα.

Music is one of the most pious cultures that exist throughout the world. Almost every country has their own rich musical culture, most of them are with strong classical backgrounds. India is one such land, rich in various types of music. This research paper is about the study of the identification of Raga. ‘Raga’ is said to be the melody of a song. We have used the Audio Signal Processing (ASP) techniques for feature extraction of two very similar ragas from Carnatic Music (CM) by analyzing the ‘RagaChhaya’ swaras commonly known as ‘RagaLakshna’ swaras of these ragas using Essentia and SMS tools. Essentia is an open-source library based on C++ with python and JavaScript wrappers for extensive audio analysis and MIR studies. The ‘SMS-tools’ is an application called ‘Spectral Modelling and Synthesis’ used to analyze sound and music applications. We proposed a supervised machine learning model trained with the features extracted from the two ragas considered, using the SMS-tools, for testing and training data that compares the given input piece of the audio signal to correctly identify based on the trained model.

The focus of this research is to analyze the results of encrypting audio using various authenticated encryption algorithms implemented in the Python cryptography library for ensuring authenticity and confidentiality of the original contents. The Advanced Encryption Standard (AES) is used as the underlying cryptographic primitive in conjunction with various modes including Galois Counter Mode (GCM), Counter with Cipher Block Chaining Message Authentication Code (CCM), and Cipher Block Chaining (CBC) with Keyed-Hashing for encrypting a relatively small audio file. The resulting encrypted audio shows similarity in the variance when encrypting using AES-GCM and AES-CCM. There is a noticeable reduction in variance of the performed encodings and an increase in the amount of time it takes to encrypt and decrypt the same audio file using AES-CBC with Keyed-Hashing. In addition, the corresponding encrypted using this mode audio spans a longer duration. As a result, AES should either have GCM or CCM for an efficient and reliable authenticated encryption integration within a workflow.

As the COVID-19 pandemic continues, rapid non-invasive testing has become essential. Recent studies and benchmarks motivates the use of modern artificial intelligence (AI) tools that utilize audio waveform spectral features of coughing for COVID-19 diagnosis. In this paper, we describe the system we developed for COVID-19 cough detection. We utilize features directly extracted from the coughing audio and use deep learning algorithms to develop automated diagnostic tools for COVID-19. In particular, we develop a unique modification of the VGG13 deep learning architecture for audio analysis that uses log-mel spectrograms and a combination of binary cross entropy and focal losses. This unique modification enabled the model to achieve highly robust classification of the DiCOVA 2021 COVID-19 data. We also explore the use of data augmentation and an ensembling strategy to further improve the performance on the validation and the blind test datasets. Our model achieved an average validation AUROC of 82.23% and a test AUROC of 78.3% at a sensitivity of 80.49%.

In this paper, we propose a system that will analyze the speech signals and gather the emotion from the same efficient solution based on combinations. This system solely served to identify emotions present in the signal or speech using concepts of deep learning and algorithms of machine learning (ML). Using the above mentioned, the system will determine the eight emotions present in the speech signal; anger, sad, happy, neutral, calm, fearful, disgust and surprised. The system is built with the language python and librosa, sound file libraries, which are part of the more extensive scikit library used for specific applications of audio analysis. The system will receive the sound files from the dataset present on the internet called RAVDESS. It will then analyze the audio files' spectrograms in WAV format and return us the efficiency of the system, which is the intended Outcome. We have achieved an efficiency rate of 81.82%.

Sound is a ubiquitous natural phenomenon that contains a wealth of information that constantly enhances our understanding of the objective world. With the continuous development of computer network technology and communication technology, audio information has become a very important part. Audio is a non-semantic symbolic representation and an unstructured binary stream. Because the audio itself lacks the description of content semantics and structured organization, it brings great difficulty to the audio classification work. The research of digital audio classification will become more and more important with the increasing number of digital audio resources in the network. Digital audio classification technology is the key technology to solve this problem. It is the key to solve the problem of audio structure and extract audio structured information and content semantics. It is a research hot spot in the field of audio analysis. It has important application value in many fields, such as audio retrieval, video summary and auxiliary video analysis. This paper studies the structure of audio, the analysis and extraction of audio features, the digital audio classifier based on support vector machines (SVM) and the audio segmentation technology based on BCI. SVM is an important achievement of machine learning research in recent years. As a new machine learning method, SVM can solve practical problems such as small sample, nonlinearity and high dimension, so it has become a new research hot spot after the study of neural network. Experiments show that the SVM-based audio classification algorithm has good classification effect, and the smoothed audio segmentation results are more accurate. With the further development of the research, the research results will be well applied in practice.

The effectiveness of pain management relies on the choice and the correct use of suitable pain assessment tools. In the case of newborns, some of the most common tools are human-based and observational, thus affected by subjectivity and methodological problems. Therefore, in the last years there has been an increasing interest in developing an automatic machine-based pain assessment tool. This research is a preliminary investigation towards the inclusion of a scoring system for the vocal expression of the infant into an automatic tool. To this aim we present a method to compute three correlated indicators which measure three distress-related features of the cry: duration, dysphonantion and fundamental frequency of the first cry. In particular, we propose a new method to measure the dysphonantion of the cry via spectral entropy analysis, resulting in an indicator that identifies three well separated levels of distress in the vocal expression. These levels provide a classification that is highly correlated with the human-based assessment of the cry.

The audio content of the movie trailers carries various prominent characteristics that can be exploited to predict the genres of the movie. None of the previous approaches have focused on extracting the audio features from the trailers using a clustering-based unsupervised learning approach followed by distance-based supervised learning. Hence, in this paper, we propose a novel framework for movie genre classification using audio features of movie trailers. Movie trailers belonging to the five most generic and popular genres (i.e., Action, Romance, Horror, Science Fiction, and Comedy) are considered in the work. The proposed AFAnet architecture is trained on a total of 78 features including 68 audio features and 10 distance-based features extracted after the k-means clustering on the audio chunks. The cross-dataset validation is performed using the standard LMTD dataset to validate the performance of our proposed framework. The results obtained depict that our model has performed excellently and show the robustness of the proposed approach.

In recent times, technologies such as machine learning and deep learning have played a vital role in providing assistive solutions to a medical domain’s challenges. They also improve predictive accuracy for early and timely disease detection using medical imaging and audio analysis. Due to the scarcity of trained human resources, medical practitioners are welcoming such technology assistance as it provides a helping hand to them in coping with more patients. Apart from critical health diseases such as cancer and diabetes, the impact of respiratory diseases is also gradually on the rise and is becoming life-threatening for society. The early diagnosis and immediate treatment are crucial in respiratory diseases, and hence the audio of the respiratory sounds is proving very beneficial along with chest X-rays. The presented research work aims to apply Convolutional Neural Network based deep learning methodologies to assist medical experts by providing a detailed and rigorous analysis of the medical respiratory audio data for Chronic Obstructive Pulmonary detection. In the conducted experiments, we have used a Librosa machine learning library features such as MFCC, Mel-Spectrogram, Chroma, Chroma (Constant-Q) and Chroma CENS. The presented system could also interpret the severity of the disease identified, such as mild, moderate, or acute. The investigation results validate the success of the proposed deep learning approach. The system classification accuracy has been enhanced to an ICBHI score of 93%. Furthermore, in the conducted experiments, we have applied K-fold Cross-Validation with ten splits to optimize the performance of the presented deep learning approach.

International audience; Since 2017, the goal of the two-million song WASABI database has been to build a knowledge graph linking collected metadata (artists,discography, producers, dates, etc.) with metadata generated by the analysis of both the songs’ lyrics (topics, places, emotions, structure, etc.) and audio signal (chords, sound, etc.). It relies on natural language processing and machine learning methods for extraction, and semantic Web frameworks for representation and integration. It describes more than 2 millions commercial songs, 200K albums and 77K artists. It can be exploited by music search engines, music professionals (e.g. journalists, radio presenters, music teachers) or scientists willing to analyze popular music published since 1950. It is available under an open license, inmultiple formats and with online and open source services including aninteractive navigator, a REST API and a SPARQL endpoint.

In this paper, we discuss the proliferation of household smart devices and review the literature to explore whether the implementation characteristics of such systems may provide avenues of attack to obtain private biometric data. Examples include the use of mechanical hard drives as audio microphones and interception of soft-keyboard input through audio analysis of haptic feedback. As the use of biometric data increases in casual environments, the opportunity for it to be stolen in unexpected ways is also increasing. There are many examples of the technology being utilised by hackers to enable unexpected use such as spoofing. We examine the importance and sanctity of biometric data in the modern world and posit that manufacturers must avoid complacency and advocate secure design, to ensure security and privacy of users.

The automatic identification and verification of speakers through representative audio continue to gain the attention of many researchers with diverse domains of applications. Despite this diversity, the availability of classified and categorized multi-purpose Arabic audio libraries is scarce. Therefore, we introduce a large Arabic-based audio clips dataset (15810 clips) of 30 popular reciters cantillating 37 chapters from the Holy Quran. These chapters have a variable number of verses saved to different subsequent folders, where each verse is allocated one folder containing 30 audio clips for the declared reciters covering the same textual content. An additional 397 audio clips for 12 competent imitators of the top reciters are collected based on popularity and number of views/downloads to allow for cross-comparison of text, reciters, and authenticity. Based on the volume, quality, and rich diversity of this dataset we anticipate a wide range of deployments for speaker identification, in addition to setting a new direction for the structure and organization of similar large audio clips dataset.

Audio analysis is fast becoming a requirement of digital media for analysing multiple frequencies of sound at a time and also reduce the background noise.The urban sound challenge is increasing day by day and the problem is meant to introduce for audio analysis and processing in the usual classification scenario. Model is implemented on the basis of keras framework and librosalibrary. Keras is capable of running of the algorithm tensorflow. Kerascan be described as an interface rather than a standalone machine learning tools. Librosa is one of the python library for music and audio analysis. It helps us with necessary music information retrieval systems. The authorcollect the database and use the data and also use the graph for a better understanding of audio data analysis.

Automatic tagging of music is an important research topic in Music Information Retrieval and audio analysis algorithms proposed for this task have achieved improvements with advances in deep learning. In particular, many state-of-the-art systems use Convolutional Neural Networks and operate on mel-spectrogram representations of the audio. In this paper, we compare commonly used mel-spectrogram representations and evaluate model performances that can be achieved by reducing the input size in terms of both lesser amount of frequency bands and larger frame rates. We use the MagnaTagaTune dataset for comprehensive performance comparisons and then compare selected configurations on the larger Million Song Dataset. The results of this study can serve researchers and practitioners in their trade-off decision between accuracy of the models, data storage size and training and inference times., The 28th European Signal Processing Conference (EUSIPCO)

International audience; Cough is a symptom in over a hundred respiratory diseases. The audio features in cough signals contain erudition about the predicament of the respiratory system. Using deep learning or signal processing, these features can be used to build an effective disease prediction system. However, cough analysis remains an area that has received scant attention from machine learning researchers. This can be attributed to several factors such as inefficient ancillary systems, high expenses in obtaining datasets, or difficulty in building classifiers. This paper categorized and reviewed the current progress on cough audio analysis for the classification of pulmonary diseases. It also explored potential future issues in research. Additionally, it proposed a model for the classification of ten serious pulmonary ailments commonly seen in Indian adolescents. The proposed model is evaluated against four existing state of the art techniques in the literature.

Producers of Electronic Dance Music (EDM) typically spend more time creating, shaping, mixing and mastering sounds, than with aspects of composition and arrangement. They analyze the sound by close listening and by leveraging audio metering and audio analysis tools, until they successfully created the desired sound aesthetics. DJs of EDM tend to play sets of songs that meet their sound ideal. We therefore suggest using audio metering and monitoring tools from the recording studio to analyze EDM, instead of relying on conventional low-level audio features. We test our novel set of features by a simple classification task. We attribute songs to DJs who would play the specific song. This new set of features and the focus on DJ sets is targeted at EDM as it takes the producer and DJ culture into account. With simple dimensionality reduction and machine learning these features enable us to attribute a song to a DJ with an accuracy of 63%. The features from the audio metering and monitoring tools in the recording studio could serve for many applications in Music Information Retrieval, such as genre, style and era classification and music recommendation for both DJs and consumers of electronic dance music.

We introduce Surfboard, an open-source Python library for extracting audio features with application to the medical domain. Surfboard is written with the aim of addressing pain points of existing libraries and facilitating joint use with modern machine learning frameworks. The package can be accessed both programmatically in Python and via its command line interface, allowing it to be easily integrated within machine learning workflows. It builds on state-of-the-art audio analysis packages and offers multiprocessing support for processing large workloads. We review similar frameworks and describe Surfboard's architecture, including the clinical motivation for its features. Using the mPower dataset, we illustrate Surfboard's application to a Parkinson's disease classification task, highlighting common pitfalls in existing research. The source code is opened up to the research community to facilitate future audio research in the clinical domain., Comment: 5 pages. 0 figures. Under review

Background and objectives Diplophonia is a common symptom in voice disorders. Depending on the underlying aetiology, diplophonic patients typically need treatment such as phonosurgery or speech therapy. In current clinical practice, the presence of diplophonia is assessed by auditive rating. To avoid subjectivity in voice assessment and to follow principles of evidence based medicine, objective instrumental assessment methods are needed. In order to gain insight into instrumental assessment of diplophonic voice, comparisons between different assessment approaches are necessary. The aim of the study is to compare the performance of two independent objective approaches on their ability to detect diplophonia. The compared approaches are the formerly published degree of subharmonics (DSH), and a newly proposed measure for spatial bimodality of the vocal fold vibration. Material and methods From a clinical database of 352 laryngeal high-speed videos with synchronous audio recordings, 60 phonation segments (20 euphonic, twenty diplophonic and twenty non-diplophonic dysphonic) were auditively selected. For all phonation segments, the DSH and the newly proposed measure for spatial bimodality were determined. The DSH is the occurrence rate of audio analysis blocks with ambiguous fundamental frequency in percent. The bimodality measure quantifies the spatial occurrence of secondary oscillation frequencies along the vocal folds’ edges. Both the DSH and the bimodality measure are evaluated on their ability to detect diplophonia by means of cut off threshold classification. Results and conclusions The DSH showed excellent classification rates for separating diplophonic from euphonic phonation (sensitivity: 98.4%, specificity: 100%). In separating diplophonic from non-diplophonic dysphonic phonation, the bimodality measure slightly outperforms the DSH approach (sensitivity: 54.6%, specificity: 92.7%). The separation of diplophonia from other kinds of dysphonia is challenging, and more sophisticated methods are needed. It is concluded that auditive and glottal diplophonia must be distinguished. As the clinical assessment of diplophonia primarily aims at determining glottal conditions, the video-based approach might deliver clinically more relevant data than the auditive approach.

Both the collection and analysis of large music repertoires constitute major challenges within musicological disciplines such as jazz research. Automatic methods of music analysis based on audio signal processing have the potential to assist researchers and to accelerate the transcription and analysis of music recordings significantly. In this paper, we propose a framework for analyzing improvised monophonic solos in multi-instrumental jazz recordings with special focus on reed and brass instruments. The analysis algorithms rely on prior score-information, which is taken from high quality manual solo transcriptions. Following an initial solo and accompaniment source separation, we propose algorithms for tone-wise extraction of fundamental frequency and intensity contours. Based on this fine-grained representation of recorded jazz solos, we perform several exploratory experiments motivated by questions relating to jazz research in order to analyze the use of expressive stylistic devices such as intonation, pitch modulation, and dynamics in jazz solos. The results show that a score-informed audio analysis of jazz recordings can provide valuable insights into the individual stylistic characteristics of jazz musicians.

Video lectures are part of our daily lives. Whether to learn something new, review content for exams or just out of curiosity. People are increasingly looking for video lectures that address what they are looking for. Unfortunately, finding specific content in this type of video is not an easy task. Many video lectures are extensive and cover several topics, and not all of these topics are relevant to the user who has found the video. The result is that the user spends so much time trying to find topic of interest in the middle of content irrelevant to him. The temporal segmentation of video lectures in topics can solve this problem allowing users to navigate of a non-linear way through all topics of a video lecture. However, temporal video lecture segmentation is not an easy task and needs to be automatized. For this reason, in this paper we propose an optimization model for the temporal video lecture segmentation problem. This model uses as features the Word2Vec representation of video lecture's audio transcripts and low-level acoustic characteristics. To find the best video partition, an genetic algorithm with local search is used. We have performed experiments in two data sets and results showed that our proposal is able to overcome state-of-the-art methods and achieve good results for different kinds of video lectures.

This paper introduces the alpha version of a Python library called Iracema, which aims to provide models for the extraction of meaningful information from recordings of monophonic pieces of music, for purposes of research in music performance. With this objective in mind, we propose an architecture that will provide to users an abstraction level that simplifies the manipulation of different kinds of time series, as well as the extraction of segments from them. In this paper we: (1) introduce some key concepts at the core of the proposed architecture; (2) list the current functionalities of the package; (3) give some examples of the application programming interface; and (4) give some brief examples of audio analysis using the system.

The field of Text-to-Speech has experienced huge improvements last years benefiting from deep learning techniques. Producing realistic speech becomes possible now. As a consequence, the research on the control of the expressiveness, allowing to generate speech in different styles or manners, has attracted increasing attention lately. Systems able to control style have been developed and show impressive results. However the control parameters often consist of latent variables and remain complex to interpret. In this paper, we analyze and compare different latent spaces and obtain an interpretation of their influence on expressive speech. This will enable the possibility to build controllable speech synthesis systems with an understandable behaviour.

Today, pulmonary diseases are one of the major causes of mortality in the world. Even though there are different diagnostic tests available such as X-ray and tomography, the stethoscope is still the first, cheapest and the most frequently used diagnostic device for the physicians. In this paper, a smart electronic stethoscope has been designed to help physicians with the diagnosis of the disease using Machine Learning. In order to create a design that satisfies all the needs of the physicians, 15 doctors and medical students from several hospitals have been contacted and interviewed. The developed system has been tested with different machine learning techniques and its efficiency has been shown by obtaining 84% accuracy while classifying respiratory audio.

Soundscape can be regarded as the auditory landscape, conceived individually or at collaborative level. This paper presents ATHUS (ATHens Urban Soundscape), a dataset of audio recordings of ambient urban sounds, which has been annotated in terms of the corresponding perceived soundscape quality. To build our dataset, several users have recorded sounds using a simple smartphone application, which they also used to annotate the recordings, in terms of the perceived quality of the soundscape (i.e. level of “pleasantness”), in a range of 1 (unbearable) to 5 (optimal). The dataset has been made publicly available (in http://users.iit.demokritos.gr/~tyianak/soundscape) as an audio feature representation form, so that it can directly be used in a supervised machine learning pipeline without need for feature extraction. In addition, this paper presents and publicly provides (https://github.com/tyiannak/soundscape_quality) a baseline approach, which demonstrates how the dataset can be used to train a supervised model to predict soundscape quality levels. Experiments under various setups using this library have demonstrated that Support Vector Machine Regression outperforms SVM Classification for the particular task, which is something expected if we consider the gradual nature of the soundscape quality labels. The goal of this paper is to provide to machine learning engineers, working on audio analytics, a first step towards the automatic recognition of soundscape quality in urban spaces, which could lead to powerful assessment tools in the hands of policy makers with regards to noise pollution and sustainable urban living.

Acknowledging the advantages as well as the dangers of the internet content on kids education and entertainment, YouTube Kids was created. Based on regulations for child-friendly programs, several violations are identified and restricted from viewable content. When a child surfs the Internet, the same regulations could be automatically detected and filtered. However, current YouTube Kids content filtering relies on meta-data attributes, where inappropriate content could pass the filtering mechanism. This research, propose an advanced real-time content filtering approach using automated video and audio analysis as an extra layer for kids safety. The proposed method utilizes the thin-slicing theory, where several one second slices are selected randomly from the clip and extracted. The use of a one-second slice will assure a temporal analysis of the clip content, and ensures a real-time content analysis. For each slice, audio is automatically transcribed using automatic speech recognition techniques to be further analysed for its linguistic content. Furthermore, the audio signal is analysed to detect event and scenes (e.g. explosion). The image frames extracted from the slices are also inspected for its content to avoid inappropriate scenes, such as violence. Upon the success of this approach on YouTube Kids application, investigation of its generalizability to other video applications, and other languages could be performed.

This paper presents the novel sound-field localization and visualization techniques for audiovisual spatial audio analysis, content description, and management. The method focuses on topographic analysis and mapping of gastro-intestinal motility (GIM) through multichannel recording of abdominal sounds (AS). Related research has attempted to study GIM physiology patterns and diagnose specific abnormalities or diseases. In this context, a new AS source localization method is implemented and evaluated. Sound imaging and spatiotemporal mapping utilities are deployed next by means of sound level distribution images. Novel audio description and management is introduced for the analysis automation of prolonged AS recordings, facilitating smart content browsing with audiovisual summarization and highlighting. The proposed modalities can be utilized in multimedia healthcare applications, where multimodal monitoring of GIM and other psychophysiological parameters can be combined for the study of human digestive patterns and their relation to other factors (i.e., subjects’ medical history, nutrition, medication, psychological state, and others). The adopted methodology of spatial audio analysis introduces a generic framework that can be efficiently applied to various sound localization and imaging tasks, which are encountered both in bioacoustics and in contemporary multimedia involving multichannel/3D audio.

Like the concept of free-viewpoint TV (FTV), the audio data should be rendered according to video data. However, on condition that minimum numbers of microphone are used, it is difficult to acquire accurate audio signal for rendering audio data to the image with the choice of view point. Especially, degradation of high frequency components (HFC) happens due to the characteristic of polar pattern for microphone. The degradation of HFC causes imperfection of signal restoration and leads to the degradation of clarity for hearing. In this paper, a compensation method for the degradation of HFC audio signal is proposed for producing an immersive audio effect at realistic media. Our experimental results show that low frequency components (LFC) of audio signal had a little directional degradation in spite of effect of the polar patterns of microphone and the compensation of HFC can be realized with adapting the attenuation inclination of LFC. This research is expected to be helpful for producing an immersive audio effect for a realistic media.

A curious divide characterizes the usage of audio descriptors for timbre research in music information research (MIR) and music psychology. While MIR uses a multitude of audio descriptors for tasks such as automatic instrument classification, only a highly constrained set is used to describe the physical correlates of timbre perception in parts of music psychology. We argue that this gap is not coincidental and results from the differences in the two fields’ methodologies, their epistemic groundwork, and research goals. This paper lays out perspectives on the emergence of the divide and reviews studies in both fields with regards to divergences in research methods and goals. We discuss new representations for spectro-temporal modulations in MIR and psychology, and compare approaches to spectral envelope description in depth. Finally, we will propose that the interdisciplinary discourse on the computational modelling of music requires negotiations about the roles of scientific evaluation criteria.

The mining of data streams has been attracting much attention in the recent years, specially from Machine Learning researchers. One important task in learning from data streams is to correctly detect changing data characteristics over time, since this is critical to the correct modeling of data behavior. With the understanding that many applications generate unlabeled streams, different algorithms have been proposed to approach unsupervised change detection. These algorithms implement different strategies, from simple incremental methods that monitor data statistics, to more advanced techniques based on divergences of clustering models. In recent studies, however, authors pointed out those algorithms lack in learning guarantees, meaning that results obtained by these methods could be due to model parameterization. These observations led to the development of a new stability concept that is suitable for unsupervised streams. This stability concept motivated a new change detection algorithm which ensures model modifications corresponding to actual data changes. Previous results on artificial scenarios have confirmed this algorithm’s ability to correctly detect changes. However, the requirement of assessing the algorithm’s performance on real-world data remained, which is essential to the understanding of the algorithm’s capabilities. Motivated by this observation, this work applied this algorithm to the domain of audio analysis, more specifically, in music change detection. Results obtained in different music tracks provide interesting insights on the types of changes that produce a more significant impact on the algorithm’s decisions, allowing for a better understanding about its underlying dynamics.

This paper intends to analyze metadata and audio analysis features from a random sample of one million popular tracks, available in the Million Song Dataset (MSD), and assess their potential of making it into the Billboard hot 100 song list. Through the use of various machine learning different models, we can determine the interaction and importance of different variables over time and their effects on the success on the Billboard charts. With such knowledge, we can access and identify the past music trends and help producers with the steps to create the perfect commercially successful song.

© 2018 by the author. Licensee MDPI, Basel, Switzerland. This work investigates the problem of detecting hazardous events on roads by designing an audio surveillance system that automatically detects perilous situations such as car crashes and tire skidding. In recent years, research has shown several visual surveillance systems that have been proposed for road monitoring to detect accidents with an aim to improve safety procedures in emergency cases. However, the visual information alone cannot detect certain events such as car crashes and tire skidding, especially under adverse and visually cluttered weather conditions such as snowfall, rain, and fog. Consequently, the incorporation of microphones and audio event detectors based on audio processing can significantly enhance the detection accuracy of such surveillance systems. This paper proposes to combine time-domain, frequency-domain, and joint time-frequency features extracted from a class of quadratic time-frequency distributions (QTFDs) to detect events on roads through audio analysis and processing. Experiments were carried out using a publicly available dataset. The experimental results conform the effectiveness of the proposed approach for detecting hazardous events on roads as demonstrated by 7% improvement of accuracy rate when compared against methods that use individual temporal and spectral features. Funding text #1 The ability to detect hazardous events on the road can be a matter of life and death, or it can emxpeearnimthenetadtiifofne,regnenceerbateetdwteheenreasunlotsramnadlwliafes raenspdoansliibfelewfoirthwraitminagjothrehpaanpderi.caSp.A..I-Mn .thanisalwyzoerdk,thweerehsualvtse proposedasystemthatcombinest-,f-and(t,f)-domainfeaturestosimultaneouslyconsiderthenon-approach and the results to further improve the quality of the paper. stationary,instantaneous,andlong-termpropertiesofaudiosignalstofacilitateautomaticdetection and classificationofaudio anomalies. The results of experiments performed on a publiclyavailable National Research Fund (a member of the Qatar Foundation). The statements made herein are solely the rbesponsibilityenchmark dofattheaseauthors.t demonstrate the robustness of the proposed method against background noise compared with state-of-the-art approaches for road surveillance applications. Our audio classification system is confirmed to be effective indetecting hazardous...View all Funding text #2 Acknowledgments: This publication was made possible by NPRP grant # NPRP8-140-2-065 from the Qatar National Research Fund (a member of the Qatar Foundation). The statements made herein are solely the respoDatansibifrliomtyothefthFatalityeauthoAnalysisrs. Reporting System and the General Estimates System; National Highway Traffic Safety Administration: Washington, DC, USA, 2015. 5.ConfEvanco,lictsofInWt.eMre.sThet:ThImpacteauthoofrsRapiddeclareIncidentthattherDetectionearenocononfFrliceewaytsofinAccidentterest. Fatalities. Mitretek: McLean

Artificial neural networks are increasingly being introduced into modern life, finding application in different areas of activity. One of these uses is audio processing. Analysis of audio files is based on extracting of big amount of various characteristics which could be used in different ways. This paper is devoted to development of algorithm of collecting biometric voice data and neural network which uses these data to classify speech emotions. The main objective of this work was to create classifier of emotions which uses audio file as input data.

This chapter explains the basic concepts in computational methods used for analysis of sound scenes and events. Even though the analysis tasks in many applications seem different, the underlying computational methods are typically based on the same principles. We explain the commonalities between analysis tasks such as sound event detection, sound scene classification, or audio tagging. We focus on the machine learning approach, where the sound categories (i.e., classes) to be analyzed are defined in advance. We explain the typical components of an analysis system, including signal pre-processing, feature extraction, and pattern classification. We also preset an example system based on multi-label deep neural networks, which has been found to be applicable in many analysis tasks discussed in this book. Finally, we explain the whole processing chain that involves developing computational audio analysis systems.

The current paper proposes a unique approach by examining the ancient Greek literature and antique black figure amphorae’s representations (also known as melanomorpha) for the reconstruction, after digital design processing, of a modern top-quality replica of an ancient tortoise lyre. Through the review of certain ancient Greek documents, the observation of the amphorae’s relevant representations, 3D scanning and reverse engineering as well as 3D design using advanced Computer Aided Design (CAD) software, this study illustrates the detailed drawings and the fabrication procedures followed for a modern version of the ancient musical instrument. By using only materials available in antiquity such as specific kinds of wood, tortoise carcasses and sheep strings, as well as modern carpentry technology, a high-quality musical instrument was produced suited for use by today’s professional musicians. Two variations were produced and tested utilizing the Phrygian and Lydian ancient Greek music scales in a speciali...

In this study the movement patterns of ten expert musicians are quantitatively related to expressive timing patterns and the music structure during performances. The hypothesis is that ancillary gestures recurrently employed are closely related to expressive intentions, and that the expressive content imposed in key musical passages is thus reflected in the patterns of gestural recurrence. A movement and an audio analysis of 30 clarinet performances of a Brahms' excerpt are compared. Results show direct correlations between the recurrence pattern of clarinetists' ancillary movements and expressive bar duration manipulations employed by them, associated with melodic phrasing and harmonic transitions.

© 2015 Taylor & Francis. This paper presents a Bayesian probabilistic framework for real-time alignment of a recording or score with a live performance using an event-based approach. Multitrack audio files are processed using existing onset detection and harmonic analysis algorithms to create a representation of a musical performance as a sequence of time-stamped events. We propose the use of distributions for the position and relative speed which are sequentially updated in real-time according to Bayes’ theorem. We develop the methodology for this approach by describing its application in the case of matching a single MIDI track and then extend this to the case of multitrack recordings. An evaluation is presented that contrasts ourmultitrack alignment method with state-of-the-art alignment techniques.

Many machine learning tasks have been shown solvable with impressive levels of success given large amounts of training data and computational power. For the problems which lack data sufficient to achieve high performance, methods for transfer learning can be applied. These refer to performing the new task while having prior knowledge of the nature of the data, gained by first performing a different task, for which training data is abundant. Shown successful for other machine learning tasks, transfer learning is now investigated in audio analysis. We propose to solve the weakly labelled problem of sound event tagging with small amounts of training data by transferring the abstract knowledge about the nature of audio data from another tagging task. The proposed methods constitute pre-training of a recurrent neural network or its parts to perform one tagging task given abundant and diverse training data, and then using it or its parts for a new task of tagging sound events of different nature, for which the data is limited. Several architectures for such transfer are proposed and evaluated, showing impressive classification accuracy of 83.4% with gains of up to 20 percentage points over the baseline given as little as 36 training samples for the target task.

3D video is introducing great changes in many health related areas. The realism of such information provides health professionals with strong evidence analysis tools to facilitate clinical decision processes. Speech and language therapy aims to help subjects in correcting several disorders. The assessment of the patient by the speech and language therapist (SLT), requires several visual and audio analysis procedures that can interfere with the patient's production of speech. In this context, the main contribution of this paper is a 3D video system to improve health information management processes in speech and language therapy. The 3D video retrieval and management system supports multimodal health records and provides the SLTs with tools to support their work in many ways: (i) it allows SLTs to easily maintain a database of patients' orofacial and speech exercises; (ii) supports three-dimensional orofacial measurement and analysis in a non-intrusive way; and (iii) search patient speech-exercises by similar facial characteristics, using facial image analysis techniques. The second contribution is a dataset with 3D videos of patients performing orofacial speech exercises. The whole system was evaluated successfully in a user study involving 22 SLTs. The user study illustrated the importance of the retrieval by similar orofacial speech exercise. info:eu-repo/semantics/publishedVersion

We present a complete, fully functional energy-autonomous audio sensor node with 6×5×4mm3 form factor. The system uses a new audio processing IC integrated with a MEMS microphone, general purpose 32-bit processor, 8Mb Flash, RF transceiver with custom 3D antenna, PV cells for energy harvesting and battery. The 4.7μW audio processing IC performs audio acquisition with 4–32× compression. The complete stand-alone system achieves 38mins of speech recording and energy-autonomous operation in room light.

As multicore computers become widespread, the need for software programmers to decide on the most effective parallelization techniques becomes very prominent. In this case study, we examined a competition in which four teams of graduate students parallelized two sequential audio analysis applications. The students were introduced with PThreads, OpenMP and TBB parallel programming models. Use of different profiling and debugging tools was also taught during this course. Two of the teams parallelized libVorbis audio encoder and the other two parallelized the LAME encoding engine. The strategies used by the four teams to parallelize these applications included the use of taught programming models, focusing on both fine-grained and coarse-grained parallelism. These strategies are discussed in detail along with the tools utilized for the development and profiling. An analysis of the results obtained is also performed to discuss speedups and audio quality of the encoded output. A list of the lessons to be remembered while parallelizing an application has been provided as well. These lessons include best pedagogical methods, importance of understanding the program before choosing a programming model, concentrating on coarse-grained parallelism first, looking for dependency relaxation, parallelism beyond the predefined language constructs, the need of practice or prior experience in parallel programming and the need for assisting tools in parallelization.