Search

Adapting speaker recognition systems to new environments is a widely-used technique to improve a well-performing model learned from large-scale data towards a task-specific small-scale data scenarios. However, previous studies focus on single domain adaptation, which neglects a more practical scenario where training data are collected from multiple acoustic domains needed in forensic scenarios. Audio analysis for forensic speaker recognition offers unique challenges in model training with multi-domain training data due to location/scenario uncertainty and diversity mismatch between reference and naturalistic field recordings. It is also difficult to directly employ small-scale domain-specific data to train complex neural network architectures due to domain mismatch and performance loss. Fine-tuning is a commonly-used method for adaptation in order to retrain the model with weights initialized from a well-trained model. Alternatively, in this study, three novel adaptation methods based on domain adversarial training, discrepancy minimization, and moment-matching approaches are proposed to further promote adaptation performance across multiple acoustic domains. A comprehensive set of experiments are conducted to demonstrate that: 1) diverse acoustic environments do impact speaker recognition performance, which could advance research in audio forensics, 2) domain adversarial training learns the discriminative features which are also invariant to shifts between domains, 3) discrepancy-minimizing adaptation achieves effective performance simultaneously across multiple acoustic domains, and 4) moment-matching adaptation along with dynamic distribution alignment also significantly promotes speaker recognition performance on each domain, especially for the LENA-field domain with noise compared to all other systems., Comment: IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING

The present work approaches intelligent traffic evaluation and congestion detection using sound sensors and machine learning. For this, two important problems are addressed: traffic condition assessment from audio data, and analysis of audio under uncontrolled environments. By modeling the traffic parameters and the sound generation from passing vehicles and using the produced audio as a source of data for learning the traffic audio patterns, we provide a solution that copes with the time, the cost and the constraints inherent to the activity of traffic monitoring. External noise sources were introduced to produce more realistic acoustic scenes and to verify the robustness of the methods presented. Audio-based monitoring becomes a simple and low-cost option, comparing to other methods based on detector loops, or GPS, and as good as camera-based solutions, without some of the common problems of image-based monitoring, such as occlusions and light conditions. The approach is evaluated with data from audio analysis of traffic registered in locations around the city of Sao Jose dos Campos, Brazil, and audio files from places around the world, downloaded from YouTube. Its validation shows the feasibility of traffic automatic audio monitoring as well as using machine learning algorithms to recognize audio patterns under noisy environments.

Las medidas de oído real ha permitido al audioprotesista disponer de un criterio para la valoración de la adaptación de audífonos fiable y válido. El uso de estas medidas en la estimación de la bondad de la adaptación nos permite, entre otras ventajas, tener en cuenta las diferencias individuales al facilitarnos parámetros referidos al rendimiento del audífono en oído real. En el presente artículo llevaremos a cabo una revisión de las principales medidas que pueden registrarse con audioanalizador y sonda microfónica en oído real y sus principales aplicaciones.

Issues that are increasingly arising in criminal proceedings are highlighted, namely: use of data obtained as a result of diagnostic researches on video, sound recordings that allows to ensure a higher level of completeness, objectivity and comprehensiveness of research. The purpose of the article is to analyze the possibilities of forensic diagnostic researches on video and sound recordings of both oral speech and the sound environment of an offense event. Various modern possibilities of researches on video and sound recordings are considered. Attention is paid both to the specifics of diagnostic speech researches in order to obtain information about personality of the unknown speaker and to information that can be obtained through research and diagnosis of non-speech information. Analyzed the possibilities (within the framework of the examination of video, sound recording) diagnostic researches on directly speech information, including the definition of: oral speech form, nature of relations between the interlocutors, conversation meaning (definition of the meaningful situation: consent vs disagreement; permission vs prohibition; understanding vs misunderstanding; request; advice; promise; assurance; gratitude; threat; clarification; order; question; message, etc.), rate of speech and the emotional state of the interlocutors, as well as expert diagnostics of biological parameters of the speakers. Possibilities of diagnostic studies of technical studies of recording media and recording equipment, that can contain information about the technology of obtaining/fixing/saving video, sound recordings, properties and features of the media itself are considered. Non-speech sounds are classified according to their belonging to certain sources. The article highlights effectiveness of using the forensic information obtained through forensic examination in practice of investigating crimes.

The concept and essence of distortion are considered from a technical stand-point and criminal law perspective. The most common ways of distorting a voice and a speech are provided, as well as certain methods of detecting an intentional change in a voice and human speech by computer tools and linguistic analysis. Some software and hardware tools changing a speech signal both in real time and in a pre-prepared recording are analyzed. To solve diagnostic and identification tasks, a pressing issue in forensic video and audio analysis is studied which is addressed to forensic experts in this field more often.

Style transfer is one of the most popular uses of neural networks. It has been thoroughly researched, such as extracting the style from famous paintings and applying it to other images thus creating synthetic paintings. Generative adversarial networks (GANs) are used to achieve this. This paper explores the many ways in which the same results can be achieved with audio related tasks, for which a plethora of new applications can be found. Analysis of different techniques used to transfer styles of audios, specifically changing the gender of the audio is implemented. The Crowd sourced high-quality UK and Ireland English Dialect speech data set was used. In this paper, the input is the male or female wave form and the opposite gender’s waveform is synthesized by the network, with the content spoken remaining the same. Different architectures are explored, from naive techniques and directly training audio waveforms against convolution neural networks (CNN) to using extensive algorithms researched for image style conversion and generation of spectrograms (using GANs) to be trained on CNNs. This research has a broader scope when used in converting music from one genre to another, identification of synthetic voices, curating voices for AIs based on preference etc.

Open-source software libraries have a significant impact on the development of Audio Signal Processing and Music Information Retrieval (MIR) systems. Despite the abundance of such tools, there is a lack of an extensive and easy-to-use reference library for audio feature extraction on Web clients. In this article, we present Essentia.js, an open-source JavaScript (JS) library for audio and music analysis on both web clients and JS engines. Along with the Web Audio API, it can be used for both offline and real-time audio feature extraction on web browsers. Essentia.js is modular, lightweight, and easy-to-use, deploy, maintain, and integrate into the existing plethora of JS libraries and web technologies. It is powered by a WebAssembly back end cross-compiled from the Essentia C++ library, which facilitates a JS interface to a wide range of low-level and high-level audio features, including signal processing MIR algorithms as well as pre-trained TensorFlow.js machine learning models. It also provides a higher-level JS API and add-on MIR utility modules along with extensive documentation, usage examples, and tutorials. We benchmark the proposed library on two popular web browsers and the Node.js engine, and four devices, including mobile Android and iOS, comparing it to the native performance of Essentia and the Meyda JS library. The work on Essentia.js has been partially funded by the Ministry of Science and Innovation of the Spanish Government under the grant agreement PID2019-111403GB-I00 (Musical AI).

The article deals with some problems related to the production of forensic speech and audio analysis, taking into account global digitalization of all spheres of human life. Voicechanger, technology of voice synthesis signifi cantly complicate forensic phonoscopic examinations and put before the forensic experts a new, interesting challenges. We consider the algorithm for changing the voice using modern software, as well as features of voice synthesis technologies. Some variants of studying such phonograms are proposed.

Vibrato is one of the most common musical techniques used for the enrichment of vocal and musical instrument sounds. We propose a method that can analyse the intonation, vibrato rate, and vibrato e...

Many audio analysis systems employ peak picking procedure to produce the final decision. A typical scheme uses a thresholding function to minimise detection errors where its form depends on the structure of the input signal. The paper covers the problem of an adaptive thresholding function estimation. Using the genetic algorithm to optimise the components of the thresholding function we have determined the level of importance of individual local statistics on the final function representation. The proposed method has been used to tune the peak detection procedure to identify the change points in an audio signal. In the result of the heuristic configuration, the best accuracy of segment boundaries have been obtained for thresholding function built on top of two local statistics of the detection function and constant value. Finally, as an example, a comparison with the state–of–the–art scheme for audio segmentation was performed.

With the fast development of computer and information technology, multimedia data has become the most important form of information media. Auditory information plays an important role in information location, this comes from the fact that it can be difficult to find useful information. Thus audio classification becomes more important in audio analysis as it prepares for content-based audio retrieval. There is quite a bit of research on the topic of audio classification methods, audio feature analysis, and extraction based on audio classification. Many works of literature extract features of audio signals based on time or Fourier transform frequency domain. The emergence of the wavelet theory provides a time-frequency analysis tool for signal analysis. Wavelet transformation is a local transformation of the signal in time and frequency which can effectively extract information from the signal, and perform multi-scale refinement analysis on functions or signals through operations such as stretching and translation instead of the traditional Fourier transformation. In the time-frequency analysis of the signal, the wavelet analysis captures the local time and frequency characters of the signal which can improve the ability of signal analysis. It can also change certain locals of the signal without affecting other aspects of it. In this paper, the frequency domain features are combined with the wavelet domain features. At the same time that the MFCC features are extracted, the discrete wavelet transform is used to extract the features of the wavelet domain. Then the statistical features are extracted for each audio example, and the SVM model is used to realize the different forms of audio classification identification.

The automatic analysis of students’ in-classroom behavior is valuable to evaluate the effect of teaching. Recent studies of in-classroom video analysis mainly focus on lecture content, positions and identities of students. In this paper, we propose to analyze the students’ concentration degree to the teacher or teaching content. Specifically, we detect students’ faces, track faces, and analyze the students’ behavior, i.e. raising or downing faces and corresponding head orientations to the teacher, teaching content or not. Besides, texts are obtained from the teacher’s speech and the course topics taught in the class are extracted. Audio features of the teacher’s speech are extracted and analyzed. Finally, the correlation of the students’ concentration degree with the course topics, audio features are analyzed. This analysis can help teachers find the effective teaching characteristic to better improve students’ concentration degree.

Recently, deep neural networks (DNNs) have shown the remarkable success of feature representations in computer vision, audio analysis, and natural language processing. Furthermore, DNNs have been used for electroencephalography (EEG) signal classification in recent studies on brain–computer interface. However, most works use one-dimensional EEG features to learn DNNs that ignores the local information within multichannel or multiple frequency bands in the EEG signals. In this paper, we propose a novel emotion recognition method using a convolutional neural network (CNN) while preventing the loss of local information. The proposed method consists of two parts. The first part generates topology-preserving differential entropy features while keeping the distance from the center electrode to other electrodes. The second part learns the proposed CNN to estimate three-class emotional states (positive, neutral, negative). We evaluate our work on SEED dataset, including 62-channel EEG signals recorded from 15 subjects. Our experimental results demonstrate that the proposed method achieved superior performance on SEED dataset with an average accuracy of 90.41% with the visualization of extracted features from the proposed CNN using t-SNE to show our representation outperforms the other representations based on standard features for EEG analysis. Besides, with the additional experiment on VIG dataset to estimate the vigilance of EEG dataset, we show the off-the-shelf availability of the proposed method.

Research on self-efficacy and confidence has spread across several subfields of psychology and neuroscience. The role of one’s confidence is very crucial in the formation of attitude and communication skills. The importance of differentiating the levels of confidence is quite visible in this domain. With the recent advances in extracting behavioral insight from a signal in multiple applications, detecting confidence is found to have great importance. One such prominent application is detecting confidence in interview conversations. We have collected an audiovisual data set of interview conversations with 34 candidates. Every response (from each of the candidate) of this data set is labeled with three levels of confidence: high, medium, and low. Furthermore, we have also developed algorithms to efficiently compute such behavioral confidence from speech and video. A deep learning architecture is proposed for detecting confidence levels (high, medium, and low) from an audiovisual clip recorded during an interview. The achieved unweighted average recall (UAR) reaches 85.9% on audio data and 73.6% on video data captured from an interview session.

This paper presents a novel approach to the acquisition of musical gestures on guitar based on Tunneling Magnetoresistance (TMR) sensing. With this minimally invasive setup, tracking of the horizontal displacement of the strings is used to capture gestures related to left and right-hand techniques. A pitch-based calibration is suggested to map the sensed displacement to pitch shifts so that the acquired signals can be directly used to estimate pitch produced by string bending in real-time. Some of the performer’s gestures, despite corresponding to different physical interactions, might produce a similar sonic output, as is the case of upward and downward string bends on the guitar. The proposed technology can be used to disambiguate between these gestures whether that is for automatic transcription purposes or for crafting instrument augmentations that build upon the performer’s existing expertise.

Nearly half of the world languages are considered endangered and need to be documented, analyzed, and revitalized. However, existing linguistics tools lack the accessibility to effectively analyze languages such as Blackfoot in which relative pitch movement is significant, e.g., words with the same sound sequence but convey different meanings when changing in pitches. To address this issue, we present a novel form of audio analysis with perceptual scale, and develop a consolidated and interactive toolset called MeTILDA (Melodic Transcription in Language Documentation and Analysis) to effectively capture perceived changes in pitch movement and to host other existing desktop-based linguistic tools on the cloud to enable collaboration, data-sharing, and data reuse among multiple linguistic tools.

Η Ημιτονοειδής Μοντελοποίηση είναι μια από τις πιο ευρέως χρησιμοποιούμενες παραμετρικές μεθόδους για την επεξεργασία σήματος φωνής και ήχου. Η ακριβής εκτίμηση των ημιτονοειδών παραμέτρων (πλάτη, συχνότητες, και φάσεις) είναι ένα κρίσιμο σημείο για τη ακριβή αναπαράσταση των σημάτων που αναλύονται. Στην παρούσα εργασία, με βάση τις πρόσφατες εξελίξεις στην ημιτονοειδή ανάλυση, προτείνουμε υψηλής ανάλυσης, προσαρμόσιμα ημιτονοειδή μοντέλα για συστήματα ανάλυσης, σύνθεσης, και τροποποίησης ομιλίας. Στόχος μας είναι να προσφέρουμε συστήματα που αναπαριστούν σήματα φωνής με εξαιρετικά ακριβή και συμπαγή τρόπο.Εμπνευσμένοι από πρόσφατα προταθέντα μοντέλα, όπως το προσαρμόσιμο Σχεδον - Αρμονικό Μοντέλο (aQHM) και το προσαρμόσιμο Αρμονικό Μοντέλο (aHM), διατυπώνουμε τη θεωρία της προσαρμόσιμης Ημιτονοειδούς Μοντελοποίησης και προτείνουμε ένα μοντέλο που ονομάζεται εκτεταμένο προσαρμόσιμο Σχεδον - Αρμονικό Μοντέλο (eaQHM), το οποίο είναι ένα μη παραμετρικό μοντέλο, ικανό να προσαρμόσει τα στιγμιαία πλάτη και φάσεις των συναρτήσεων βάσης του στα τοπικά χρονικά μεταβαλλόμενα χαρακτηριστικά του σήματος της φωνής, αμβλύνοντας έτσι τη γνωστή υπόθεση της τοπικής στασιμότητας. Αποδεικνύεται ότι το eaQHM παρουσιάζει υψηλότερες επιδόσεις από το aQHM στην ανάλυση και ανασύνθεση των έμφωνων τμημάτων φωνής. Με βάση το eaQHM, ένα υβριδικό σύστημα ανάλυσης / σύνθεσης ομιλίας παρουσιάζεται (eaQHNM), μαζί με μια υβριδική έκδοση του του aHM (aHNM). Επιπλέον, παρουσιάζουμε κίνητρα για μια αναπαράσταση του σήματος της φωνής σε όλο το φάσμο και σε όλη τη διάρκεια του, χρησιμοποιώντας το eaQHM, αναπαριστώντας έτσι όλα τα μέρη του σήματος της φωνής, με υψηλής ανάλυσης AM-FM ημίτονα. Η αξιολόγηση δείχνει ότι η προσαρμοσιμότητα και η σχεδόν-αρμονικότητα είναι αρκετή για να παράξει πολύ υψηλή ποιότητα στην ανασύνθεση των άφωνων τμημάτων της φωνής. Στη συνέχεια, παρουσιάζεται το σύστημα πλήρους φάσματος ανάλυσης και σύνθεσης βασισμένο στο eaQHM, το οποίο υπερτερεί συστημάτων που θεωρούνται state-of-the-art, υβριδικά ή πλήρους ανάλυσης, στην ανάλυση και ανασύνθεση φωνής. Η υπεροχή του στην ποιότητα ανασύνθεσης επιβεβαιώθηκε με αντικειμενικές και υποκειμενικές αξιολογήσεις.Όσον αφορά τις εφαρμογές, το eaQHM και το aHM εφαρμόζονται σε μετασχηματισμούς φωνής (κλιμάκωση χρόνου και κλιμάκωση θεμελιώδους συχνότητας). Οι μετασχηματισμοί που προκύπτουν είναι υψηλής ποιότητας, ακολουθώντας πολύ απλούς κανόνες, σε σύγκριση με άλλα συστήματα state-of-the-art. Οι έννοιες της σχετικής φάσης και της καθυστέρησης σχετικής φάσης είναι ζωτικής σημασίας για την ανάπτυξη μετασχηματισμένου σήματος με χαρακτηριστικά αναλλοίωτου σχήματος, χωρίς τεχνικά ελαττώματα, και υψηλής ποιότητας. Τα αποτελέσματα δείχνουν ότι τα συστηματα βασισμένα στην αρμονικότητα προτιμούνται έναντι αυτών της σχεδόν-αρμονικότητας, λόγω της απλότητας της αναπραστάσης. Επιπλέον, το eaQHM εφαρμόζεται στο πρόβλημα της μοντελοποίησης σημάτων ήχου, και συγκεκριμένα ήχων μουσικών οργάνων. Το eaQHM αξιολογείται και σύγκρινεται με state-of-the-art συστήματα, και έχει υψηλές επιδόσεις όσον αφορά την ποιότητα επανασύνθεσης, αναπαριστωντας με επιτυχία τα στάδια της επίθεσης, της μετάβασης, και της στατικότητας ενός ήχου μουσικού οργάνου. Τέλος, μια άλλη προτεινόμενη εφαρμογή έγκειται στην ανάλυση και ταξινόμηση της εκφραστικής ομιλίας. Το eaQHM εφαρμόζεται στην ανάλυση της εκφραστικής ομιλίας, παρέχοντας τις στιγμιαίες παραμέτρους του ως χαρακτηριστικά που μπορούν να χρησιμοποιηθούν στην αναγνώριση και ταξινόμηση, βασισμένη σε διανυσματικούς κβαντιστές, εκφραστικής ομιλίας. Αν και τα ημιτονοειδή μοντέλα δεν χρησιμοποιούνται συνήθως σε τέτοιες εφαρμογές, τα αποτελέσματα είναι ελπιδοφόρα.

Music is one of the most pious cultures that exist throughout the world. Almost every country has their own rich musical culture, most of them are with strong classical backgrounds. India is one such land, rich in various types of music. This research paper is about the study of the identification of Raga. ‘Raga’ is said to be the melody of a song. We have used the Audio Signal Processing (ASP) techniques for feature extraction of two very similar ragas from Carnatic Music (CM) by analyzing the ‘RagaChhaya’ swaras commonly known as ‘RagaLakshna’ swaras of these ragas using Essentia and SMS tools. Essentia is an open-source library based on C++ with python and JavaScript wrappers for extensive audio analysis and MIR studies. The ‘SMS-tools’ is an application called ‘Spectral Modelling and Synthesis’ used to analyze sound and music applications. We proposed a supervised machine learning model trained with the features extracted from the two ragas considered, using the SMS-tools, for testing and training data that compares the given input piece of the audio signal to correctly identify based on the trained model.

Blackfoot language is an endangered language that needs to be documented, analyzed, and preserved. Blackfoot is challenging to learn and teach because it is a pitch accent language whose words with same characters can take on different meanings when changing in pitch. Linguistics researchers are working to create visual aids, called Pitch Art, to teach the nuance in pitch changes. However, the existing techniques used to create Pitch Art fail to accurately indicate changes in pitch and require time-consuming work across multiple applications. To address this issue, this project proposes a system called MeTILDA (Melodic Transcription in Language Documentation and Application) to provide new forms of audio analysis and to automate the process of creating Pitch Art. MeTILDA provides value to a variety of stakeholders. Linguistics researchers are provided with tools to analyze and compare Blackfoot speeches. Teachers are given collections of words and recordings from native speakers to teach students. Students are given the ability to compare their own pronunciation of Blackfoot words to that of native speakers. We also present a new form of audio analysis, called perceptual scale, to provide more effective visuals of perceived changes in pitch movement. By collaborating with domain experts in this field, we have validated the effectiveness of MeTILDA in creating Pitch Art using the perceptual scale.

The task of automatically identifying and/or verifying the identity of a speaker from a recording of a speech sample, known as automatic speaker recognition, has been studied for many years and automatic speaker recognition technologies have improved recently and becoming inexpensive and reliable methods for identifying and verifying people. Although automatic speaker recognition research has now spanned over 50 years, there is not adequate research done with regards to low-resourced South African indigenous languages. In this paper, a multi-layer perceptron (MLP) classifier model is trained and deployed on a graphical user interface for real time identification and verification of Sepedi native speakers. Sepedi is a low-resourced language spoken by the majority of residents in the Limpopo province of South Africa. The data used to train the speaker recognition system is obtained from the NCHLT (National Centre for Human Language Technology) project. A total of 34 short-term acoustic features of speech are extracted with the use of py Audio Analysis library and Sklearn is used to train the MLP classifier model which performs well with an accuracy of 95%. The GUI is developed with QT Creator and PyQT4 and it has obtained a true acceptance rate (TAR) of 66.67% and a true rejection rate of (TRR) 13.33%.

As the COVID-19 pandemic continues, rapid non-invasive testing has become essential. Recent studies and benchmarks motivates the use of modern artificial intelligence (AI) tools that utilize audio waveform spectral features of coughing for COVID-19 diagnosis. In this paper, we describe the system we developed for COVID-19 cough detection. We utilize features directly extracted from the coughing audio and use deep learning algorithms to develop automated diagnostic tools for COVID-19. In particular, we develop a unique modification of the VGG13 deep learning architecture for audio analysis that uses log-mel spectrograms and a combination of binary cross entropy and focal losses. This unique modification enabled the model to achieve highly robust classification of the DiCOVA 2021 COVID-19 data. We also explore the use of data augmentation and an ensembling strategy to further improve the performance on the validation and the blind test datasets. Our model achieved an average validation AUROC of 82.23% and a test AUROC of 78.3% at a sensitivity of 80.49%.

Because of the rise of music songs, both online and offline, genre differentiation of music is increasingly important in today's environment. This increases the need to properly catalog and get more access. Music classification is important when searching for music in a large set. Machine learning methods are used in the bulk of modern type classification techniques known as KNN, SVM, etc. This article presents the GT-ZAN music dataset. It has ten dissimilar musical genres. Deep learning technique is encoded to coach and identify the method. For training and classification, a convolution neural network is used. The most important role in the audio analysis is feature extraction. As a function vector, the Mel Frequency Cepstral constant (MFCC) is utilized for the sound sample. By extracting the feature vector, the planned framework categorizes music into varied genres. Our study found that our project has an accuracy level of about 97% for training and 74% for testing, which will significantly boost and encourage the classification of music genres.

In this paper, we propose a system that will analyze the speech signals and gather the emotion from the same efficient solution based on combinations. This system solely served to identify emotions present in the signal or speech using concepts of deep learning and algorithms of machine learning (ML). Using the above mentioned, the system will determine the eight emotions present in the speech signal; anger, sad, happy, neutral, calm, fearful, disgust and surprised. The system is built with the language python and librosa, sound file libraries, which are part of the more extensive scikit library used for specific applications of audio analysis. The system will receive the sound files from the dataset present on the internet called RAVDESS. It will then analyze the audio files' spectrograms in WAV format and return us the efficiency of the system, which is the intended Outcome. We have achieved an efficiency rate of 81.82%.

Autonomously generating artificial soundscapes for video games, virtual reality, and sound art presents several non-trivial challenges. We outline a system called Audio Metaphor that is built upon ...

Sound is a ubiquitous natural phenomenon that contains a wealth of information that constantly enhances our understanding of the objective world. With the continuous development of computer network technology and communication technology, audio information has become a very important part. Audio is a non-semantic symbolic representation and an unstructured binary stream. Because the audio itself lacks the description of content semantics and structured organization, it brings great difficulty to the audio classification work. The research of digital audio classification will become more and more important with the increasing number of digital audio resources in the network. Digital audio classification technology is the key technology to solve this problem. It is the key to solve the problem of audio structure and extract audio structured information and content semantics. It is a research hot spot in the field of audio analysis. It has important application value in many fields, such as audio retrieval, video summary and auxiliary video analysis. This paper studies the structure of audio, the analysis and extraction of audio features, the digital audio classifier based on support vector machines (SVM) and the audio segmentation technology based on BCI. SVM is an important achievement of machine learning research in recent years. As a new machine learning method, SVM can solve practical problems such as small sample, nonlinearity and high dimension, so it has become a new research hot spot after the study of neural network. Experiments show that the SVM-based audio classification algorithm has good classification effect, and the smoothed audio segmentation results are more accurate. With the further development of the research, the research results will be well applied in practice.

The effectiveness of pain management relies on the choice and the correct use of suitable pain assessment tools. In the case of newborns, some of the most common tools are human-based and observational, thus affected by subjectivity and methodological problems. Therefore, in the last years there has been an increasing interest in developing an automatic machine-based pain assessment tool. This research is a preliminary investigation towards the inclusion of a scoring system for the vocal expression of the infant into an automatic tool. To this aim we present a method to compute three correlated indicators which measure three distress-related features of the cry: duration, dysphonantion and fundamental frequency of the first cry. In particular, we propose a new method to measure the dysphonantion of the cry via spectral entropy analysis, resulting in an indicator that identifies three well separated levels of distress in the vocal expression. These levels provide a classification that is highly correlated with the human-based assessment of the cry.

Low-level audio features are commonly used in many audio analysis tasks, such as audio scene classification or acoustic event detection. Due to the variable length of audio signals, it is a common approach to create fixed-length feature vectors consisting of a set of statistics that summarize the temporal variability of such short-term features. To avoid the loss of temporal information, the audio event can be divided into a set of mid-term segments or texture windows. However, such an approach requires to estimate accurately the onset and offset times of the audio events in order to obtain a robust mid-term statistical description of their temporal evolution. This paper proposes the use of an alternative event representation based on nonlinear time normalization prior to the extraction of mid-term statistics. The short-term features are transformed into a new fixed-length representation that considers uniform distance subsampling over a defined feature space in contrast to the classical short-term temporal framing. The results show that the use of distance-based texture windows provides an improved statistical description of the event robust to errors in the event segmentation stage under noisy conditions.

The audio content of the movie trailers carries various prominent characteristics that can be exploited to predict the genres of the movie. None of the previous approaches have focused on extracting the audio features from the trailers using a clustering-based unsupervised learning approach followed by distance-based supervised learning. Hence, in this paper, we propose a novel framework for movie genre classification using audio features of movie trailers. Movie trailers belonging to the five most generic and popular genres (i.e., Action, Romance, Horror, Science Fiction, and Comedy) are considered in the work. The proposed AFAnet architecture is trained on a total of 78 features including 68 audio features and 10 distance-based features extracted after the k-means clustering on the audio chunks. The cross-dataset validation is performed using the standard LMTD dataset to validate the performance of our proposed framework. The results obtained depict that our model has performed excellently and show the robustness of the proposed approach.

Real-ear measurements has allowed the audiologist to have a criterion for assessing the adaptation of reliable and valid instruments. The use of these measures in estimating the goodness of adaptation allows us, among other advantages, to take into account individual differences by providing parameters related to the performance of the hearing aid in real ear. In this article we will carry out a review of the main measurements that can be recorded with a real ear audio analyzer and microphone probe and their main applications. Las medidas de oído real ha permitido al audioprotesista disponer de un criterio para la valoración de la adaptación de audífonos fiable y válido. El uso de estas medidas en la estimación de la bondad de la adaptación nos permite, entre otras ventajas, tener en cuenta las diferencias individuales al facilitarnos parámetros referidos al rendimiento del audífono en oído real. En el presente artículo llevaremos a cabo una revisión de las principales medidas que pueden registrarse con audioanalizador y sonda microfónica en oído real y sus principales aplicaciones.

International audience; Since 2017, the goal of the two-million song WASABI database has been to build a knowledge graph linking collected metadata (artists,discography, producers, dates, etc.) with metadata generated by the analysis of both the songs’ lyrics (topics, places, emotions, structure, etc.) and audio signal (chords, sound, etc.). It relies on natural language processing and machine learning methods for extraction, and semantic Web frameworks for representation and integration. It describes more than 2 millions commercial songs, 200K albums and 77K artists. It can be exploited by music search engines, music professionals (e.g. journalists, radio presenters, music teachers) or scientists willing to analyze popular music published since 1950. It is available under an open license, inmultiple formats and with online and open source services including aninteractive navigator, a REST API and a SPARQL endpoint.

Speaker Recognition is included in pattern recognition, where one of the most critical parts is the process of data classification. In the classification, the built system must estimate the classification of data into a classroom dimension closest to the training set. The speaker's introduction aims to identify evidence of speech recording by a handheld telephone that involves comparing one or more unidentified sound samples with one or more known sound samples. In this research, the data used in the form of evidence of recording conversation by telephone and recording of comparison of some unexpected. The part that is done is to classify speaker recognition with the Support Vector Machine (SVM) classification method to recognize the speaker. Using the SVM method, the accuracy of classifying the speaker's introduction is excellent. From the test results, the SVM method's use resulted in an accuracy rate of 86.67% for the test with the same sentence and up to 67% for different sentences to recognize the speaker with the values of C 0.01 and $\boldsymbol{\gamma}$ (Gamma) 0.0001.

Background: The global cases of Covid-19 increasing day by day. On Nov. 25, 2020, a total of 59,850,910 cases reported globally with a 1,411,216 global death. In India, total cases in the country now stand at 91,77,841 including 86,04,955 recoveries and 4,38,667 active cases as of Nov. 24, 2020, as per data issued by ICMR. A new generation of voice/audio analysis application which can tell whether the person is suffering from COVID-19 or not. Aims: To describe how to establish a new generation of voice/audio analysis applications to identify the suspected covid-19 hidden cases in hotspot areas with the help of an audio sample of the general public. Materials & Methods: The different patents and data available as literature on the internet are evaluated to make a new generation of voice/audio analysis application with the help of an audio sample of the general public. Results: The collection of the audio sample will be done from the already suffered covid-19 patients in (.Wave files) personally or through phone calls. The audio samples like the sound of the cough, the pattern of breathing, respiration rate, and way of speech will be recorded. The parameters will be evaluated for loudness, articulation, tempo, rhythm, melody, and timbre. The analysis and interpretation of the parameters can be made through machine learning and artificial intelligence to detect corona cases with an audio sample. Discussion: The voice/audio application current project can be merged with a mobile App called “Aarogya Setu” by Govt. of India. The project can be implemented in the high-risk area of Covid-19 in the country. Conclusion: This new method of detecting cases will decrease the workload in the covid-19 laboratory.

Deforestation is exponentially depleting the planet's biodiversity and natural ecosystems at an alarming rate. This research aims to address illegal logging through realtime alerting and monitoring of suspected gas-fueled chainsaw sounds in the forest. Features were extracted from a collated nature sound dataset and trained on a supervised machine learning algorithm. The model is deployed through a microcomputer to process the chainsaw sounds through radio frequency transmission. The system has a desktop application that triggers an alarm and visualizes relevant information from the detected illegal logging activity location. The device prototype is easily-replaceable, modular, and portable and can be reconfigured to large-scale domains such as rainforests. The main contributions of this research are the improvement of alert and monitoring of illegal logging through (1) real-time and online audio analysis and detection of gas-powered chainsaws sounds through k-nearest neighbors; (2) a deployable prototype capable of listening to chainsaw sounds in the forest while buried, and (3) development of a graphical user interface for monitoring of module feedback and responses. The experimental results show that our system has an accuracy of 96.00% an F1-score of 94.34%.

In this paper, we discuss the proliferation of household smart devices and review the literature to explore whether the implementation characteristics of such systems may provide avenues of attack to obtain private biometric data. Examples include the use of mechanical hard drives as audio microphones and interception of soft-keyboard input through audio analysis of haptic feedback. As the use of biometric data increases in casual environments, the opportunity for it to be stolen in unexpected ways is also increasing. There are many examples of the technology being utilised by hackers to enable unexpected use such as spoofing. We examine the importance and sanctity of biometric data in the modern world and posit that manufacturers must avoid complacency and advocate secure design, to ensure security and privacy of users.

Beamforming of miniature and MEMS microphone arrays often relies on only the array factor, with the microphone elements assumed to be ideal, i.e., identical, with linear phase and constant omnidirectional magnitude response over all frequencies. However, real-world element responses are not ideal and there is a shortfall of measurements of array-embedded microphone directional responses. This is a significant gap for estimating a real-world acoustic microphone array response. This paper presents a procedure for physically evaluating the phase and amplitude of miniature microphones using a two-port audio analyzer. While the phase is often the essence of array beamforming (hence the term phased array) the amplitudes are also weighted in general beamforming. Directional responses, or patterns, are perhaps the most difficult sensor parameter to measure, requiring specialized equipment; currently there are no accurate commercial systems readily available for 3D patterns. Our method estimates embedded microphone directional response samples, and shows that while the relative phase can behave similarly to that of isolated, electrically small sensors, the amplitudes differ significantly, which compromises accurate beamforming. The phase differences between a pair of embedded microphones can be modelled using basic geometric spacing. Comparisons between the measured and modelled signals of the microphone elements indicate that the individual embedded element responses should be included for accurate estimation of the array response, rather than just using the array factor.

Audio signals generated by the human body (e.g., sighs, breathing, heart, digestion, vibration sounds) have routinely been used by clinicians as indicators to diagnose disease or assess disease progression. Until recently, such signals were usually collected through manual auscultation at scheduled visits. Research has now started to use digital technology to gather bodily sounds (e.g., from digital stethoscopes) for cardiovascular or respiratory examination, which could then be used for automatic analysis. Some initial work shows promise in detecting diagnostic signals of COVID-19 from voice and coughs. In this paper we describe our data analysis over a large-scale crowdsourced dataset of respiratory sounds collected to aid diagnosis of COVID-19. We use coughs and breathing to understand how discernible COVID-19 sounds are from those in asthma or healthy controls. Our results show that even a simple binary machine learning classifier is able to classify correctly healthy and COVID-19 sounds. We also show how we distinguish a user who tested positive for COVID-19 and has a cough from a healthy user with a cough, and users who tested positive for COVID-19 and have a cough from users with asthma and a cough. Our models achieve an AUC of above 80% across all tasks. These results are preliminary and only scratch the surface of the potential of this type of data and audio-based machine learning. This work opens the door to further investigation of how automatically analysed respiratory patterns could be used as pre-screening signals to aid COVID-19 diagnosis., Comment: 9 pages, 6 figures, 2 tables, Accepted for publication at KDD'20 (Health Day)

In this paper, a novel phase-locking technique is used to interface a Programmable Josephson Voltage Standard (PJVS) with a Keysight U8930B Performance Audio Analyzer. This technique uses both hardware and software to phase-lock the PJVS to the analyzer via a master clock. Both long term stability and nulling measurements were performed on the analyzer. The long term stability results indicated that the lowest Allan deviation was achieved after 500 s of sampling. The long term stability provides a springboard for the nulling measurements. The nulling measurements allows the possibility for the U8903B to be used as a reliable transfer of the ac quantum voltage standard.

The robustness and vulnerability of Deep Neural Networks (DNN) are quickly becoming a critical area of interest since these models are in widespread use across real-world applications (i.e., image and audio analysis, recommendation system, natural language analysis, etc.). A DNN’s vulnerability is exploited by an adversary to generate data to attack the model; however, the majority of adversarial data generators have focused on image domains with far fewer work on audio domains. More recently, audio analysis models were shown to be vulnerable to adversarial audio examples (e.g., speech command classification, automatic speech recognition, etc.). Thus, one urgent open problem is to detect adversarial audio reliably. In this contribution, we incorporate a separate and yet related DNN technique to detect adversarial audio, namely model quantization. Then we propose an algorithm to detect adversarial audio by using a DNN’s quantization error. Specifically, we demonstrate that adversarial audio typically exhibits a larger activation quantization error than benign audio. The quantization error is measured using character error rates. We use the difference in errors to discriminate adversarial audio. Experiments with three the-state-of-the-art audio attack algorithms against the DeepSpeech model show our detection algorithm achieved high accuracy on the Mozilla dataset.

Objective Polysomnography is typically used to evaluate the severity of obstructive sleep apnea (OSA) but the inconvenience of application and high cost considerably affect the diagnostics. In this study, sleep sound signals are used to detect OSA in patients. Approach A deep feature transfer-based OSA detection approach is proposed. First, a deep convolutional neural network is trained on large-scale labeled audio data sets to distinguish respiration sounds from environmental noise. Second, the trained model is transferred to recognize respiration sounds in sleep sound signals. Third, the deep features of the detected respiration sounds are used to train a logistic regression classifier to identify OSA patients from potential patients. Polysomnography-based diagnosis is used as a reference. Main results A self-collected data set of 132 potential OSA patients is applied in OSA detection experiments. The OSA detection performances are tested on four models for different apnea-hypopnea index thresholds and sexes resulting in accuracies of 80.17%, 80.21%, 81.63% and 77.22%. The corresponding areas under the receiver operating characteristic curves are 0.82, 0.80, 0.81 and 0.79. In addition, the proposed method presented a significant performance improvement compared with the state-of-the-art methods. Significance Big data, deep learning and transfer learning can be successfully applied to improve diagnostic accuracy in OSA detection. The performance of the proposed approach is superior to that of traditional audio analysis technology. The proposed method significantly reduces difficulties in OSA detection and diagnosis, such that potential OSA patients can perform initial inspections by themselves at home.

This is an overview paper on an interactive music exploration interface for music collections. This interface is meant to help explore the cross-cultural similarities, interactions, and patterns of music excerpts from different regions and understand the similarities by employing computational audio analysis, machine learning, and visualization techniques. In our computational analysis, we used standard audio features that capture timbre information and projected them onto a lower-dimensional space for visualizing the (dis)similarity. There are two collections of non-Eurogenetic music under study. The 2-D and 3-D mappings are visualized through a dashboard application and also rendered in Virtual Reality space where users can interact and explore to get meaningful insights about the structural (dis)similarities of the music collections.

Audio analysis is fast becoming a requirement of digital media for analysing multiple frequencies of sound at a time and also reduce the background noise.The urban sound challenge is increasing day by day and the problem is meant to introduce for audio analysis and processing in the usual classification scenario. Model is implemented on the basis of keras framework and librosalibrary. Keras is capable of running of the algorithm tensorflow. Kerascan be described as an interface rather than a standalone machine learning tools. Librosa is one of the python library for music and audio analysis. It helps us with necessary music information retrieval systems. The authorcollect the database and use the data and also use the graph for a better understanding of audio data analysis.

The explosion of machine learning and artificial intelligence research along with expanding computing capabilities in the last decade has made accurate applications possible and accessible to everyday life to the degree where we are now seeing massive adoption by individuals (smart devices) and industries (retail, marketing, real estate, etc.). Healthcare tends to lag in adoption of new technology and applications to existing solutions due to several barriers including regulatory, reimbursement, and medical security. One area we have observed growth outside of healthcare with now parallel work for medical applications is voice computing. In recent years voice recognition technology has achieved significant milestones, offering users highly accurate performance leading to widespread adoption, now with over half of mobile device users engaging with voice assistant regularly. In healthcare we have seen a number of voice computing companies emerge including NeuroLex Laboratories and Lyssn who aim to use machine learning to process the voice signal and provide a clinical measure for patients and providers. Our work here demonstrates the application of machine and deep learning tools to a dataset of voice recordings from several thousand subjects, including a cohort with Parkinson’s disease. Our goal is to demonstrate the feasibility and performance of voice computing to detect the Parkinson’s disease phenotype using only voice. This may enable future use of voice as a digital biomarker for Parkinson’s disease with benefits including improved access to screening and diagnosis, symptom tracking, decreased cost, and increased accuracy of diagnosis. From this work, we demonstrate the application of voice data to diagnose Parkinson’s disease accurately. The methodology demonstrated here can be extended to diagnose any illness that physiologically affects the vocal tract using voice as a digital biomarker.

Attachment is a deep and enduring emotional bond that connects one person to another across time and space. Our early attachment styles are established in childhood through the interaction between infants and caregivers. There are two attachment types, secure and insecure. The attachment experience affects personality development, particularly a sense of security, and research shows that it influences the ability to form stable relationships throughout life. It is also an important aspect of assessing the quality of parenting. Therefore, attachment has been widely studied in psychology research. It’s usually acquired by Ainsworth’s Strange Situation Assessment (SSA) through tedious observation. As far as we know, there is no computational method to predict infant attachment type. We try to use the Still-Face Paradigm (SFP) video and audio as input to predict attachment types through machine learning methods. In the present work, we recruited 64 infant-mother participants, collected videos of SFP when babies are 5–8 months of age and identified their attachment types including secure and insecure by SSA when those infants are almost 2 years old. For the visual part, we extract motion features and apply a RNN network with LSTM units model for classification. For the audio part, speech enhancement is conducted as data pre-processing, pitch frequency, short-time energy and Mel Frequency Cepstral Coefficient feature sequences are extracted. Then SVM is deployed to explore the patterns in it. The experiments show that our method is able to discriminate between the 2 classes of subjects with a good accuracy.

The incidence of mosquito-borne diseases is significant in under-developed regions, mostly due to the lack of resources to implement aggressive control measurements against mosquito proliferation. A potential strategy to raise community awareness regarding mosquito proliferation is building a live map of mosquito incidences using smartphone apps and crowdsourcing. In this paper, we explore the possibility of identifying Aedes aegypti mosquitoes using machine learning techniques and audio analysis captured from commercially available smartphones. In summary, we downsampled Aedes aegypti wingbeat recordings and used them to train a convolutional neural network (CNN) through supervised learning. As a feature, we used the recording spectrogram to represent the mosquito wingbeat frequency over time visually. We trained and compared three classifiers: a binary, a multiclass, and an ensemble of binary classifiers. In our evaluation, the binary and ensemble models achieved accuracy of 97.65% ($\pm$ 0.55) and 94.56% ($\pm$ 0.77), respectively, whereas the multiclass had an accuracy of 78.12% ($\pm$ 2.09). The best sensitivity was observed in the ensemble approach (96.82% $\pm$ 1.62), followed by the multiclass for the particular case of Aedes aegypti (90.23% $\pm$ 3.83) and the binary (88.49% $\pm$ 6.68). The binary classifier and the multiclass classifier presented the best balance between precision and recall, with F1-measure close to 90%. Although the ensemble classifier achieved the lowest precision, thus impairing its F1-measure (79.95% $\pm$ 2.13), it was the most powerful classifier to detect Aedes aegypti in our dataset., Comment: Published in Computers in Biology and Medicine