Search

The present work approaches intelligent traffic evaluation and congestion detection using sound sensors and machine learning. For this, two important problems are addressed: traffic condition assessment from audio data, and analysis of audio under uncontrolled environments. By modeling the traffic parameters and the sound generation from passing vehicles and using the produced audio as a source of data for learning the traffic audio patterns, we provide a solution that copes with the time, the cost and the constraints inherent to the activity of traffic monitoring. External noise sources were introduced to produce more realistic acoustic scenes and to verify the robustness of the methods presented. Audio-based monitoring becomes a simple and low-cost option, comparing to other methods based on detector loops, or GPS, and as good as camera-based solutions, without some of the common problems of image-based monitoring, such as occlusions and light conditions. The approach is evaluated with data from audio analysis of traffic registered in locations around the city of Sao Jose dos Campos, Brazil, and audio files from places around the world, downloaded from YouTube. Its validation shows the feasibility of traffic automatic audio monitoring as well as using machine learning algorithms to recognize audio patterns under noisy environments.

The concept and essence of distortion are considered from a technical stand-point and criminal law perspective. The most common ways of distorting a voice and a speech are provided, as well as certain methods of detecting an intentional change in a voice and human speech by computer tools and linguistic analysis. Some software and hardware tools changing a speech signal both in real time and in a pre-prepared recording are analyzed. To solve diagnostic and identification tasks, a pressing issue in forensic video and audio analysis is studied which is addressed to forensic experts in this field more often.

Style transfer is one of the most popular uses of neural networks. It has been thoroughly researched, such as extracting the style from famous paintings and applying it to other images thus creating synthetic paintings. Generative adversarial networks (GANs) are used to achieve this. This paper explores the many ways in which the same results can be achieved with audio related tasks, for which a plethora of new applications can be found. Analysis of different techniques used to transfer styles of audios, specifically changing the gender of the audio is implemented. The Crowd sourced high-quality UK and Ireland English Dialect speech data set was used. In this paper, the input is the male or female wave form and the opposite gender’s waveform is synthesized by the network, with the content spoken remaining the same. Different architectures are explored, from naive techniques and directly training audio waveforms against convolution neural networks (CNN) to using extensive algorithms researched for image style conversion and generation of spectrograms (using GANs) to be trained on CNNs. This research has a broader scope when used in converting music from one genre to another, identification of synthetic voices, curating voices for AIs based on preference etc.

Open-source software libraries have a significant impact on the development of Audio Signal Processing and Music Information Retrieval (MIR) systems. Despite the abundance of such tools, there is a lack of an extensive and easy-to-use reference library for audio feature extraction on Web clients. In this article, we present Essentia.js, an open-source JavaScript (JS) library for audio and music analysis on both web clients and JS engines. Along with the Web Audio API, it can be used for both offline and real-time audio feature extraction on web browsers. Essentia.js is modular, lightweight, and easy-to-use, deploy, maintain, and integrate into the existing plethora of JS libraries and web technologies. It is powered by a WebAssembly back end cross-compiled from the Essentia C++ library, which facilitates a JS interface to a wide range of low-level and high-level audio features, including signal processing MIR algorithms as well as pre-trained TensorFlow.js machine learning models. It also provides a higher-level JS API and add-on MIR utility modules along with extensive documentation, usage examples, and tutorials. We benchmark the proposed library on two popular web browsers and the Node.js engine, and four devices, including mobile Android and iOS, comparing it to the native performance of Essentia and the Meyda JS library. The work on Essentia.js has been partially funded by the Ministry of Science and Innovation of the Spanish Government under the grant agreement PID2019-111403GB-I00 (Musical AI).

The article deals with some problems related to the production of forensic speech and audio analysis, taking into account global digitalization of all spheres of human life. Voicechanger, technology of voice synthesis signifi cantly complicate forensic phonoscopic examinations and put before the forensic experts a new, interesting challenges. We consider the algorithm for changing the voice using modern software, as well as features of voice synthesis technologies. Some variants of studying such phonograms are proposed.

Purpose Minimally invasive surgery (MIS) has become the standard for many surgical procedures as it minimizes trauma, reduces infection rates and shortens hospitalization. However, the manipulation of objects in the surgical workspace can be difficult due to the unintuitive handling of instruments and limited range of motion. Apart from the advantages of robot-assisted systems such as augmented view or improved dexterity, both robotic and MIS techniques introduce drawbacks such as limited haptic perception and their major reliance on visual perception. Methods In order to address the above-mentioned limitations, a perception study was conducted to investigate whether the transmission of intra-abdominal acoustic signals can potentially improve the perception during MIS. To investigate whether these acoustic signals can be used as a basis for further automated analysis, a large audio data set capturing the application of electrosurgery on different types of porcine tissue was acquired. A sliding window technique was applied to compute log-mel-spectrograms, which were fed to a pre-trained convolutional neural network for feature extraction. A fully connected layer was trained on the intermediate feature representation to classify instrument–tissue interaction. Results The perception study revealed that acoustic feedback has potential to improve the perception during MIS and to serve as a basis for further automated analysis. The proposed classification pipeline yielded excellent performance for four types of instrument–tissue interaction (muscle, fascia, liver and fatty tissue) and achieved top-1 accuracies of up to 89.9%. Moreover, our model is able to distinguish electrosurgical operation modes with an overall classification accuracy of 86.40%. Conclusion Our proof-of-principle indicates great application potential for guidance systems in MIS, such as controlled tissue resection. Supported by a pilot perception study with surgeons, we believe that utilizing audio signals as an additional information channel has great potential to improve the surgical performance and to partly compensate the loss of haptic feedback.

Many audio analysis systems employ peak picking procedure to produce the final decision. A typical scheme uses a thresholding function to minimise detection errors where its form depends on the structure of the input signal. The paper covers the problem of an adaptive thresholding function estimation. Using the genetic algorithm to optimise the components of the thresholding function we have determined the level of importance of individual local statistics on the final function representation. The proposed method has been used to tune the peak detection procedure to identify the change points in an audio signal. In the result of the heuristic configuration, the best accuracy of segment boundaries have been obtained for thresholding function built on top of two local statistics of the detection function and constant value. Finally, as an example, a comparison with the state–of–the–art scheme for audio segmentation was performed.

With the fast development of computer and information technology, multimedia data has become the most important form of information media. Auditory information plays an important role in information location, this comes from the fact that it can be difficult to find useful information. Thus audio classification becomes more important in audio analysis as it prepares for content-based audio retrieval. There is quite a bit of research on the topic of audio classification methods, audio feature analysis, and extraction based on audio classification. Many works of literature extract features of audio signals based on time or Fourier transform frequency domain. The emergence of the wavelet theory provides a time-frequency analysis tool for signal analysis. Wavelet transformation is a local transformation of the signal in time and frequency which can effectively extract information from the signal, and perform multi-scale refinement analysis on functions or signals through operations such as stretching and translation instead of the traditional Fourier transformation. In the time-frequency analysis of the signal, the wavelet analysis captures the local time and frequency characters of the signal which can improve the ability of signal analysis. It can also change certain locals of the signal without affecting other aspects of it. In this paper, the frequency domain features are combined with the wavelet domain features. At the same time that the MFCC features are extracted, the discrete wavelet transform is used to extract the features of the wavelet domain. Then the statistical features are extracted for each audio example, and the SVM model is used to realize the different forms of audio classification identification.

Recently, deep neural networks (DNNs) have shown the remarkable success of feature representations in computer vision, audio analysis, and natural language processing. Furthermore, DNNs have been used for electroencephalography (EEG) signal classification in recent studies on brain–computer interface. However, most works use one-dimensional EEG features to learn DNNs that ignores the local information within multichannel or multiple frequency bands in the EEG signals. In this paper, we propose a novel emotion recognition method using a convolutional neural network (CNN) while preventing the loss of local information. The proposed method consists of two parts. The first part generates topology-preserving differential entropy features while keeping the distance from the center electrode to other electrodes. The second part learns the proposed CNN to estimate three-class emotional states (positive, neutral, negative). We evaluate our work on SEED dataset, including 62-channel EEG signals recorded from 15 subjects. Our experimental results demonstrate that the proposed method achieved superior performance on SEED dataset with an average accuracy of 90.41% with the visualization of extracted features from the proposed CNN using t-SNE to show our representation outperforms the other representations based on standard features for EEG analysis. Besides, with the additional experiment on VIG dataset to estimate the vigilance of EEG dataset, we show the off-the-shelf availability of the proposed method.

Research on self-efficacy and confidence has spread across several subfields of psychology and neuroscience. The role of one’s confidence is very crucial in the formation of attitude and communication skills. The importance of differentiating the levels of confidence is quite visible in this domain. With the recent advances in extracting behavioral insight from a signal in multiple applications, detecting confidence is found to have great importance. One such prominent application is detecting confidence in interview conversations. We have collected an audiovisual data set of interview conversations with 34 candidates. Every response (from each of the candidate) of this data set is labeled with three levels of confidence: high, medium, and low. Furthermore, we have also developed algorithms to efficiently compute such behavioral confidence from speech and video. A deep learning architecture is proposed for detecting confidence levels (high, medium, and low) from an audiovisual clip recorded during an interview. The achieved unweighted average recall (UAR) reaches 85.9% on audio data and 73.6% on video data captured from an interview session.

This paper presents a novel approach to the acquisition of musical gestures on guitar based on Tunneling Magnetoresistance (TMR) sensing. With this minimally invasive setup, tracking of the horizontal displacement of the strings is used to capture gestures related to left and right-hand techniques. A pitch-based calibration is suggested to map the sensed displacement to pitch shifts so that the acquired signals can be directly used to estimate pitch produced by string bending in real-time. Some of the performer’s gestures, despite corresponding to different physical interactions, might produce a similar sonic output, as is the case of upward and downward string bends on the guitar. The proposed technology can be used to disambiguate between these gestures whether that is for automatic transcription purposes or for crafting instrument augmentations that build upon the performer’s existing expertise.

Nearly half of the world languages are considered endangered and need to be documented, analyzed, and revitalized. However, existing linguistics tools lack the accessibility to effectively analyze languages such as Blackfoot in which relative pitch movement is significant, e.g., words with the same sound sequence but convey different meanings when changing in pitches. To address this issue, we present a novel form of audio analysis with perceptual scale, and develop a consolidated and interactive toolset called MeTILDA (Melodic Transcription in Language Documentation and Analysis) to effectively capture perceived changes in pitch movement and to host other existing desktop-based linguistic tools on the cloud to enable collaboration, data-sharing, and data reuse among multiple linguistic tools.

Music is one of the most pious cultures that exist throughout the world. Almost every country has their own rich musical culture, most of them are with strong classical backgrounds. India is one such land, rich in various types of music. This research paper is about the study of the identification of Raga. ‘Raga’ is said to be the melody of a song. We have used the Audio Signal Processing (ASP) techniques for feature extraction of two very similar ragas from Carnatic Music (CM) by analyzing the ‘RagaChhaya’ swaras commonly known as ‘RagaLakshna’ swaras of these ragas using Essentia and SMS tools. Essentia is an open-source library based on C++ with python and JavaScript wrappers for extensive audio analysis and MIR studies. The ‘SMS-tools’ is an application called ‘Spectral Modelling and Synthesis’ used to analyze sound and music applications. We proposed a supervised machine learning model trained with the features extracted from the two ragas considered, using the SMS-tools, for testing and training data that compares the given input piece of the audio signal to correctly identify based on the trained model.

The focus of this research is to analyze the results of encrypting audio using various authenticated encryption algorithms implemented in the Python cryptography library for ensuring authenticity and confidentiality of the original contents. The Advanced Encryption Standard (AES) is used as the underlying cryptographic primitive in conjunction with various modes including Galois Counter Mode (GCM), Counter with Cipher Block Chaining Message Authentication Code (CCM), and Cipher Block Chaining (CBC) with Keyed-Hashing for encrypting a relatively small audio file. The resulting encrypted audio shows similarity in the variance when encrypting using AES-GCM and AES-CCM. There is a noticeable reduction in variance of the performed encodings and an increase in the amount of time it takes to encrypt and decrypt the same audio file using AES-CBC with Keyed-Hashing. In addition, the corresponding encrypted using this mode audio spans a longer duration. As a result, AES should either have GCM or CCM for an efficient and reliable authenticated encryption integration within a workflow.

As the COVID-19 pandemic continues, rapid non-invasive testing has become essential. Recent studies and benchmarks motivates the use of modern artificial intelligence (AI) tools that utilize audio waveform spectral features of coughing for COVID-19 diagnosis. In this paper, we describe the system we developed for COVID-19 cough detection. We utilize features directly extracted from the coughing audio and use deep learning algorithms to develop automated diagnostic tools for COVID-19. In particular, we develop a unique modification of the VGG13 deep learning architecture for audio analysis that uses log-mel spectrograms and a combination of binary cross entropy and focal losses. This unique modification enabled the model to achieve highly robust classification of the DiCOVA 2021 COVID-19 data. We also explore the use of data augmentation and an ensembling strategy to further improve the performance on the validation and the blind test datasets. Our model achieved an average validation AUROC of 82.23% and a test AUROC of 78.3% at a sensitivity of 80.49%.

Because of the rise of music songs, both online and offline, genre differentiation of music is increasingly important in today's environment. This increases the need to properly catalog and get more access. Music classification is important when searching for music in a large set. Machine learning methods are used in the bulk of modern type classification techniques known as KNN, SVM, etc. This article presents the GT-ZAN music dataset. It has ten dissimilar musical genres. Deep learning technique is encoded to coach and identify the method. For training and classification, a convolution neural network is used. The most important role in the audio analysis is feature extraction. As a function vector, the Mel Frequency Cepstral constant (MFCC) is utilized for the sound sample. By extracting the feature vector, the planned framework categorizes music into varied genres. Our study found that our project has an accuracy level of about 97% for training and 74% for testing, which will significantly boost and encourage the classification of music genres.

In this paper, we propose a system that will analyze the speech signals and gather the emotion from the same efficient solution based on combinations. This system solely served to identify emotions present in the signal or speech using concepts of deep learning and algorithms of machine learning (ML). Using the above mentioned, the system will determine the eight emotions present in the speech signal; anger, sad, happy, neutral, calm, fearful, disgust and surprised. The system is built with the language python and librosa, sound file libraries, which are part of the more extensive scikit library used for specific applications of audio analysis. The system will receive the sound files from the dataset present on the internet called RAVDESS. It will then analyze the audio files' spectrograms in WAV format and return us the efficiency of the system, which is the intended Outcome. We have achieved an efficiency rate of 81.82%.

Sound analysis research has mainly been focused on speech and music processing. The deployed methodologies are not suitable for analysis of sounds with varying background noise, in many cases with very low signal-to-noise ratio (SNR). In this paper, we present a method for the detection of patterns of interest in audio signals. We propose novel trainable feature extractors, which we call COPE (Combination of Peaks of Energy). The structure of a COPE feature extractor is determined using a single prototype sound pattern in an automatic configuration process, which is a type of representation learning. We construct a set of COPE feature extractors, configured on a number of training patterns. Then we take their responses to build feature vectors that we use in combination with a classifier to detect and classify patterns of interest in audio signals. We carried out experiments on four public data sets: MIVIA audio events, MIVIA road events, ESC-10 and TU Dortmund data sets. The results that we achieved (recognition rate equal to 91.71% on the MIVIA audio events, 94% on the MIVIA road events, 81.25% on the ESC-10 and 94.27% on the TU Dortmund) demonstrate the effectiveness of the proposed method and are higher than the ones obtained by other existing approaches. The COPE feature extractors have high robustness to variations of SNR. Real-time performance is achieved even when the value of a large number of features is computed.

Sound is a ubiquitous natural phenomenon that contains a wealth of information that constantly enhances our understanding of the objective world. With the continuous development of computer network technology and communication technology, audio information has become a very important part. Audio is a non-semantic symbolic representation and an unstructured binary stream. Because the audio itself lacks the description of content semantics and structured organization, it brings great difficulty to the audio classification work. The research of digital audio classification will become more and more important with the increasing number of digital audio resources in the network. Digital audio classification technology is the key technology to solve this problem. It is the key to solve the problem of audio structure and extract audio structured information and content semantics. It is a research hot spot in the field of audio analysis. It has important application value in many fields, such as audio retrieval, video summary and auxiliary video analysis. This paper studies the structure of audio, the analysis and extraction of audio features, the digital audio classifier based on support vector machines (SVM) and the audio segmentation technology based on BCI. SVM is an important achievement of machine learning research in recent years. As a new machine learning method, SVM can solve practical problems such as small sample, nonlinearity and high dimension, so it has become a new research hot spot after the study of neural network. Experiments show that the SVM-based audio classification algorithm has good classification effect, and the smoothed audio segmentation results are more accurate. With the further development of the research, the research results will be well applied in practice.

The effectiveness of pain management relies on the choice and the correct use of suitable pain assessment tools. In the case of newborns, some of the most common tools are human-based and observational, thus affected by subjectivity and methodological problems. Therefore, in the last years there has been an increasing interest in developing an automatic machine-based pain assessment tool. This research is a preliminary investigation towards the inclusion of a scoring system for the vocal expression of the infant into an automatic tool. To this aim we present a method to compute three correlated indicators which measure three distress-related features of the cry: duration, dysphonantion and fundamental frequency of the first cry. In particular, we propose a new method to measure the dysphonantion of the cry via spectral entropy analysis, resulting in an indicator that identifies three well separated levels of distress in the vocal expression. These levels provide a classification that is highly correlated with the human-based assessment of the cry.

Low-level audio features are commonly used in many audio analysis tasks, such as audio scene classification or acoustic event detection. Due to the variable length of audio signals, it is a common approach to create fixed-length feature vectors consisting of a set of statistics that summarize the temporal variability of such short-term features. To avoid the loss of temporal information, the audio event can be divided into a set of mid-term segments or texture windows. However, such an approach requires to estimate accurately the onset and offset times of the audio events in order to obtain a robust mid-term statistical description of their temporal evolution. This paper proposes the use of an alternative event representation based on nonlinear time normalization prior to the extraction of mid-term statistics. The short-term features are transformed into a new fixed-length representation that considers uniform distance subsampling over a defined feature space in contrast to the classical short-term temporal framing. The results show that the use of distance-based texture windows provides an improved statistical description of the event robust to errors in the event segmentation stage under noisy conditions.

The audio content of the movie trailers carries various prominent characteristics that can be exploited to predict the genres of the movie. None of the previous approaches have focused on extracting the audio features from the trailers using a clustering-based unsupervised learning approach followed by distance-based supervised learning. Hence, in this paper, we propose a novel framework for movie genre classification using audio features of movie trailers. Movie trailers belonging to the five most generic and popular genres (i.e., Action, Romance, Horror, Science Fiction, and Comedy) are considered in the work. The proposed AFAnet architecture is trained on a total of 78 features including 68 audio features and 10 distance-based features extracted after the k-means clustering on the audio chunks. The cross-dataset validation is performed using the standard LMTD dataset to validate the performance of our proposed framework. The results obtained depict that our model has performed excellently and show the robustness of the proposed approach.

In recent times, technologies such as machine learning and deep learning have played a vital role in providing assistive solutions to a medical domain’s challenges. They also improve predictive accuracy for early and timely disease detection using medical imaging and audio analysis. Due to the scarcity of trained human resources, medical practitioners are welcoming such technology assistance as it provides a helping hand to them in coping with more patients. Apart from critical health diseases such as cancer and diabetes, the impact of respiratory diseases is also gradually on the rise and is becoming life-threatening for society. The early diagnosis and immediate treatment are crucial in respiratory diseases, and hence the audio of the respiratory sounds is proving very beneficial along with chest X-rays. The presented research work aims to apply Convolutional Neural Network based deep learning methodologies to assist medical experts by providing a detailed and rigorous analysis of the medical respiratory audio data for Chronic Obstructive Pulmonary detection. In the conducted experiments, we have used a Librosa machine learning library features such as MFCC, Mel-Spectrogram, Chroma, Chroma (Constant-Q) and Chroma CENS. The presented system could also interpret the severity of the disease identified, such as mild, moderate, or acute. The investigation results validate the success of the proposed deep learning approach. The system classification accuracy has been enhanced to an ICBHI score of 93%. Furthermore, in the conducted experiments, we have applied K-fold Cross-Validation with ten splits to optimize the performance of the presented deep learning approach.

Background: The global cases of Covid-19 increasing day by day. On Nov. 25, 2020, a total of 59,850,910 cases reported globally with a 1,411,216 global death. In India, total cases in the country now stand at 91,77,841 including 86,04,955 recoveries and 4,38,667 active cases as of Nov. 24, 2020, as per data issued by ICMR. A new generation of voice/audio analysis application which can tell whether the person is suffering from COVID-19 or not. Aims: To describe how to establish a new generation of voice/audio analysis applications to identify the suspected covid-19 hidden cases in hotspot areas with the help of an audio sample of the general public. Materials & Methods: The different patents and data available as literature on the internet are evaluated to make a new generation of voice/audio analysis application with the help of an audio sample of the general public. Results: The collection of the audio sample will be done from the already suffered covid-19 patients in (.Wave files) personally or through phone calls. The audio samples like the sound of the cough, the pattern of breathing, respiration rate, and way of speech will be recorded. The parameters will be evaluated for loudness, articulation, tempo, rhythm, melody, and timbre. The analysis and interpretation of the parameters can be made through machine learning and artificial intelligence to detect corona cases with an audio sample. Discussion: The voice/audio application current project can be merged with a mobile App called “Aarogya Setu” by Govt. of India. The project can be implemented in the high-risk area of Covid-19 in the country. Conclusion: This new method of detecting cases will decrease the workload in the covid-19 laboratory.

Deforestation is exponentially depleting the planet's biodiversity and natural ecosystems at an alarming rate. This research aims to address illegal logging through realtime alerting and monitoring of suspected gas-fueled chainsaw sounds in the forest. Features were extracted from a collated nature sound dataset and trained on a supervised machine learning algorithm. The model is deployed through a microcomputer to process the chainsaw sounds through radio frequency transmission. The system has a desktop application that triggers an alarm and visualizes relevant information from the detected illegal logging activity location. The device prototype is easily-replaceable, modular, and portable and can be reconfigured to large-scale domains such as rainforests. The main contributions of this research are the improvement of alert and monitoring of illegal logging through (1) real-time and online audio analysis and detection of gas-powered chainsaws sounds through k-nearest neighbors; (2) a deployable prototype capable of listening to chainsaw sounds in the forest while buried, and (3) development of a graphical user interface for monitoring of module feedback and responses. The experimental results show that our system has an accuracy of 96.00% an F1-score of 94.34%.

The automatic identification and verification of speakers through representative audio continue to gain the attention of many researchers with diverse domains of applications. Despite this diversity, the availability of classified and categorized multi-purpose Arabic audio libraries is scarce. Therefore, we introduce a large Arabic-based audio clips dataset (15810 clips) of 30 popular reciters cantillating 37 chapters from the Holy Quran. These chapters have a variable number of verses saved to different subsequent folders, where each verse is allocated one folder containing 30 audio clips for the declared reciters covering the same textual content. An additional 397 audio clips for 12 competent imitators of the top reciters are collected based on popularity and number of views/downloads to allow for cross-comparison of text, reciters, and authenticity. Based on the volume, quality, and rich diversity of this dataset we anticipate a wide range of deployments for speaker identification, in addition to setting a new direction for the structure and organization of similar large audio clips dataset.

Objective Polysomnography is typically used to evaluate the severity of obstructive sleep apnea (OSA) but the inconvenience of application and high cost considerably affect the diagnostics. In this study, sleep sound signals are used to detect OSA in patients. Approach A deep feature transfer-based OSA detection approach is proposed. First, a deep convolutional neural network is trained on large-scale labeled audio data sets to distinguish respiration sounds from environmental noise. Second, the trained model is transferred to recognize respiration sounds in sleep sound signals. Third, the deep features of the detected respiration sounds are used to train a logistic regression classifier to identify OSA patients from potential patients. Polysomnography-based diagnosis is used as a reference. Main results A self-collected data set of 132 potential OSA patients is applied in OSA detection experiments. The OSA detection performances are tested on four models for different apnea-hypopnea index thresholds and sexes resulting in accuracies of 80.17%, 80.21%, 81.63% and 77.22%. The corresponding areas under the receiver operating characteristic curves are 0.82, 0.80, 0.81 and 0.79. In addition, the proposed method presented a significant performance improvement compared with the state-of-the-art methods. Significance Big data, deep learning and transfer learning can be successfully applied to improve diagnostic accuracy in OSA detection. The performance of the proposed approach is superior to that of traditional audio analysis technology. The proposed method significantly reduces difficulties in OSA detection and diagnosis, such that potential OSA patients can perform initial inspections by themselves at home.

The field of Soundscape Ecology refers to the study of sounds produced in natural environments and how they can provide important information about the state of the environment, as well as on the potential impacts caused by changes due to external influences. The analysis and visualization of large amounts of ecological recordings, as well as the development of appropriate tools for audio analysis contitute a major challenge. Mechanisms for extracting audio features, as well as the characterization of acoustic events of interest, resulting in datasets that capture the frequency variations and the occurrence of acoustic events in the recordings, still constitute a problem due to available solutions do not prove adequate for data analysis in acoustic ecology research, involving domain-specific issues and voluminous amounts of audio records collected over long periods of time. This work aims to address problems related to the extraction of audio features, providing assistance through visualization to the selection of the most significants, that could represent the subtle variations in ecological recordings, as well as assisting specialists in the generation of annotated dtasets by the characterization of acoustic events through exploratory visualizations, and methods for detecting vessels in underwater recordings. A framework named Seecology is presented, encompassing suitable methods and tools to supporting specialists and scholars of environmental analysis. Case studies were carried out with the framework in terrestrial and underwater recordings provided by acoustic ecology researchers, by producing datasets from the custom feature extractor included in the framework, and in the case of the method developed for detecting boats in underwater recordings, a comparative study to another method was conducted to determine its accuracy, in addition to the case study to determine its effectiveness. The presented methods for extracting characteristics, characterizing acoustic events through exploratory visualization and boat detection, demonstrated their effectiveness for applications in acoustic ecology, with the framework containing the methods capable of producing multidimensional datasets without excessive computational costs, allowing the user to easily generate annotations on this data through the included visualizations. The boat detection method performed better than the one it was compared, both in speed and accuracy, being able to detect weak signals from boats even under extreme noise. A área de Ecologia de Paisagens Sonoras (Soundscape Ecology) refere-se ao estudo de sons produzidos em ambientes naturais e como eles podem fornecer informações importantes sobre o meio ambiente, bem como possíveis impactos causados por alterações devido a influências externas. A análise e visualização de grandes quantidades de gravações ecológicas, juntamente com a produção de recursos para análises constituem um grande desafio. Meios para a extração de características dos áudios, bem como a caracterização de eventos acústicos de interesse, produzindo conjuntos de dados que representem as variações de frequências e eventos acústicos capturados nas gravações, ainda consituem um problema devido às soluções disponíveis não se mostrarem adequadas para análises de dados em pesquisas de ecologia acústica, envolvendo questões específicas do domínio e quantidades volumosas de registros de áudio coletados por longos períodos de tempo. Faz-se necessário o desenvolvimento de métodos e ferramental para extrair e representar a grande quantidade dos dados produzidos a partir de estudos de ecologia. Este trabalho tem por objetivo abordar problemas relacionados à extração de características dos áudios, auxiliando na seleção das mais significativas que representem as sutis variações nas gravações ecológicas, bem como auxiliar especialistas na geração de conjuntos de dados anotados pela caracterização de eventos acústicos por meio de visualizações exploratórias, e métodos para um problema específico, que é a detecção de embarcações em gravações subaquáticas. Um arcabouço nomeado Seecology é apresentado, englobando métodos e ferramentas adequados para dar suporte aos especialistas e estudiosos de análise ambiental. Estudos de caso foram realizados com o arcabouço em gravações terrestres e subaquáticas fornecidos por pesquisadores da área, produzindo conjuntos de dados a partir do extrator de características personalizado incluso no arcabouço. No caso do método desenvolvido para detecção de barcos em gravações subaquáticas, um estudo comparativo a outro método foi conduzido para determinar sua acurácia, além do estudo de caso para determinar sua eficácia. Os métodos propostos para extração de características, caracterização de eventos acústicos por meio de visualização exploratória e detecção de barcos, demonstraram sua eficácia para aplicações em ecologia acústica, sendo o arcabouço capaz de produzir conjuntos de dados multidimensionais sem custos computacionais excessivos. Dessa forma o usuário é capaz de gerar anotações nestes dados facilmente por meio das visualizações inclusas. O método de detecção de barcos obteve desempenho superior ao que foi comparado, tanto em velocidade quanto em acurácia, sendo capaz de detectar sinais fracos de barcos mesmo sob ruído extremo.

Audio analysis is fast becoming a requirement of digital media for analysing multiple frequencies of sound at a time and also reduce the background noise.The urban sound challenge is increasing day by day and the problem is meant to introduce for audio analysis and processing in the usual classification scenario. Model is implemented on the basis of keras framework and librosalibrary. Keras is capable of running of the algorithm tensorflow. Kerascan be described as an interface rather than a standalone machine learning tools. Librosa is one of the python library for music and audio analysis. It helps us with necessary music information retrieval systems. The authorcollect the database and use the data and also use the graph for a better understanding of audio data analysis.

Automatic tagging of music is an important research topic in Music Information Retrieval and audio analysis algorithms proposed for this task have achieved improvements with advances in deep learning. In particular, many state-of-the-art systems use Convolutional Neural Networks and operate on mel-spectrogram representations of the audio. In this paper, we compare commonly used mel-spectrogram representations and evaluate model performances that can be achieved by reducing the input size in terms of both lesser amount of frequency bands and larger frame rates. We use the MagnaTagaTune dataset for comprehensive performance comparisons and then compare selected configurations on the larger Million Song Dataset. The results of this study can serve researchers and practitioners in their trade-off decision between accuracy of the models, data storage size and training and inference times., The 28th European Signal Processing Conference (EUSIPCO)

The explosion of machine learning and artificial intelligence research along with expanding computing capabilities in the last decade has made accurate applications possible and accessible to everyday life to the degree where we are now seeing massive adoption by individuals (smart devices) and industries (retail, marketing, real estate, etc.). Healthcare tends to lag in adoption of new technology and applications to existing solutions due to several barriers including regulatory, reimbursement, and medical security. One area we have observed growth outside of healthcare with now parallel work for medical applications is voice computing. In recent years voice recognition technology has achieved significant milestones, offering users highly accurate performance leading to widespread adoption, now with over half of mobile device users engaging with voice assistant regularly. In healthcare we have seen a number of voice computing companies emerge including NeuroLex Laboratories and Lyssn who aim to use machine learning to process the voice signal and provide a clinical measure for patients and providers. Our work here demonstrates the application of machine and deep learning tools to a dataset of voice recordings from several thousand subjects, including a cohort with Parkinson’s disease. Our goal is to demonstrate the feasibility and performance of voice computing to detect the Parkinson’s disease phenotype using only voice. This may enable future use of voice as a digital biomarker for Parkinson’s disease with benefits including improved access to screening and diagnosis, symptom tracking, decreased cost, and increased accuracy of diagnosis. From this work, we demonstrate the application of voice data to diagnose Parkinson’s disease accurately. The methodology demonstrated here can be extended to diagnose any illness that physiologically affects the vocal tract using voice as a digital biomarker.

Attachment is a deep and enduring emotional bond that connects one person to another across time and space. Our early attachment styles are established in childhood through the interaction between infants and caregivers. There are two attachment types, secure and insecure. The attachment experience affects personality development, particularly a sense of security, and research shows that it influences the ability to form stable relationships throughout life. It is also an important aspect of assessing the quality of parenting. Therefore, attachment has been widely studied in psychology research. It’s usually acquired by Ainsworth’s Strange Situation Assessment (SSA) through tedious observation. As far as we know, there is no computational method to predict infant attachment type. We try to use the Still-Face Paradigm (SFP) video and audio as input to predict attachment types through machine learning methods. In the present work, we recruited 64 infant-mother participants, collected videos of SFP when babies are 5–8 months of age and identified their attachment types including secure and insecure by SSA when those infants are almost 2 years old. For the visual part, we extract motion features and apply a RNN network with LSTM units model for classification. For the audio part, speech enhancement is conducted as data pre-processing, pitch frequency, short-time energy and Mel Frequency Cepstral Coefficient feature sequences are extracted. Then SVM is deployed to explore the patterns in it. The experiments show that our method is able to discriminate between the 2 classes of subjects with a good accuracy.

The incidence of mosquito-borne diseases is significant in under-developed regions, mostly due to the lack of resources to implement aggressive control measurements against mosquito proliferation. A potential strategy to raise community awareness regarding mosquito proliferation is building a live map of mosquito incidences using smartphone apps and crowdsourcing. In this paper, we explore the possibility of identifying Aedes aegypti mosquitoes using machine learning techniques and audio analysis captured from commercially available smartphones. In summary, we downsampled Aedes aegypti wingbeat recordings and used them to train a convolutional neural network (CNN) through supervised learning. As a feature, we used the recording spectrogram to represent the mosquito wingbeat frequency over time visually. We trained and compared three classifiers: a binary, a multiclass, and an ensemble of binary classifiers. In our evaluation, the binary and ensemble models achieved accuracy of 97.65% ($\pm$ 0.55) and 94.56% ($\pm$ 0.77), respectively, whereas the multiclass had an accuracy of 78.12% ($\pm$ 2.09). The best sensitivity was observed in the ensemble approach (96.82% $\pm$ 1.62), followed by the multiclass for the particular case of Aedes aegypti (90.23% $\pm$ 3.83) and the binary (88.49% $\pm$ 6.68). The binary classifier and the multiclass classifier presented the best balance between precision and recall, with F1-measure close to 90%. Although the ensemble classifier achieved the lowest precision, thus impairing its F1-measure (79.95% $\pm$ 2.13), it was the most powerful classifier to detect Aedes aegypti in our dataset., Comment: Published in Computers in Biology and Medicine

International audience; Cough is a symptom in over a hundred respiratory diseases. The audio features in cough signals contain erudition about the predicament of the respiratory system. Using deep learning or signal processing, these features can be used to build an effective disease prediction system. However, cough analysis remains an area that has received scant attention from machine learning researchers. This can be attributed to several factors such as inefficient ancillary systems, high expenses in obtaining datasets, or difficulty in building classifiers. This paper categorized and reviewed the current progress on cough audio analysis for the classification of pulmonary diseases. It also explored potential future issues in research. Additionally, it proposed a model for the classification of ten serious pulmonary ailments commonly seen in Indian adolescents. The proposed model is evaluated against four existing state of the art techniques in the literature.

We introduce Surfboard, an open-source Python library for extracting audio features with application to the medical domain. Surfboard is written with the aim of addressing pain points of existing libraries and facilitating joint use with modern machine learning frameworks. The package can be accessed both programmatically in Python and via its command line interface, allowing it to be easily integrated within machine learning workflows. It builds on state-of-the-art audio analysis packages and offers multiprocessing support for processing large workloads. We review similar frameworks and describe Surfboard's architecture, including the clinical motivation for its features. Using the mPower dataset, we illustrate Surfboard's application to a Parkinson's disease classification task, highlighting common pitfalls in existing research. The source code is opened up to the research community to facilitate future audio research in the clinical domain., Comment: 5 pages. 0 figures. Under review

One of the more complex issues facing natural language processing (NLP) is how to deal with overlapped speech, i.e., when two or more speakers interfere with or talk over each other, and the more general case of co-channel speech, i.e., when two or more speakers are present in an audio stream regardless of interference. Frequently, one speaker is selected as a primary speaker for the purpose of analysis with other speakers relegated to the category of interfering speakers. Despite the breadth of research into overlapped speech detection, few endeavors have been made into preserving the speech of so-called interfering speakers. A compelling case can be made for a more comprehensive analysis of co-channel speech in the fields of computational linguistics, accessibility automation, and entertainment, particularly under real-time constraints. Currently available open-source audio libraries, while technically capable of supporting such research endeavors, are cumbersome to work with. To this end, the work introduces the Adaptive Real-Time Imprecise Computation (ARTIC) pipeline for audio analysis, a simple but flexible approach to stream processing that tracks computation times and deadlines for the various pipeline stages and affords the user the ability to specify automatic precision reductions to avoid projected deadline misses as well as automatic precision increases to combat underutilization. A proof of concept is tested with the intent to build upon this groundwork for a more comprehensive project having the goal of multi-speaker interference detection and eventually speaker separation.

We present a method for the analysis of the finger-string interaction in guitar performances and the computation of fine actions during the plucking gesture. The method is based on Motion Capture using high-speed cameras that can track the position of reflective markers placed on the guitar and fingers, in combination with audio analysis. A major problem inherent in optical motion capture is that of marker occlusion and, in guitar playing, it is the right hand of the guitarist that is extremely difficult to capture, especially during the plucking process, where the track of the markers at the fingertips is lost very frequently. This work presents two models that allow the reconstruction of the position of occluded markers: a rigid-body model to track the motion of the guitar strings and a flexible-body model to track the motion of the hands. In combination with audio analysis (onset and pitch detection), the method can estimate a comprehensive set of sound control features that include the plucked string, the plucking finger, and the characteristics of the plucking gesture in the phases of contact, pressure and release (e.g., position, timing, velocity, direction, or string displacement). This work has been sponsored by the European Union Horizon 2020 research and innovation program under grant agreement No. 688269 (TELMI project) and Beatriu de Pinos grant 2010 BP-A 00209 by the Catalan Research Agency (AGAUR).

Farm management practices done by machinery generate a high acoustical impact on animals. The acoustic variations in terms of equivalent level (Leq) and the different types of noise can affect the well-being of broilers by means of reducing the food and water ingest. In this work, we create a dataset in which we conduct a preliminary analysis of the acoustical impact generated by the farm management in an intensive broiler poultry farm of 25,000 birds. The project collects acoustic data during the first two weeks of the birds life, focusing the study on the first week. To create the dataset, we randomly select some files from each day of the study and they are analysed and labelled manually using an audio analysis software. The acoustical events defined in collaboration with the farmer and vet are the fan and the food and water supply, and definitions are based on duration, impact, and Signal to Noise Ratio (SNR). The analysis concludes that the main acoustical source in a broilers’ farm is the fan, and that it has a non-negligible acoustical impact. Nevertheless, the most frequent acoustical noise source active is food supply, but with less Leq impact.

Deep learning methods have recently made significant contributions to sound event detection. These methods either use a block-level approach to distinguish parts of audio containing the event, or analyze the small frames of the audio separately. In this paper, we introduce a new method, AudioMask, for rare sound event detection by combining these two approaches. AudioMask first applies Mask R-CNN, a state-of-the-art algorithm for detecting objects in images, to the log mel-spectrogram of the audio files. Mask R-CNN detects audio segments that might contain the target event by generating bounding boxes around them in time-frequency domain. Then we use a frame-based audio event classifier trained independently from Mask R-CNN, to analyze each individual frame in the candidate segments proposed by Mask R-CNN. A post-processing step combines the outputs of the Mask R-CNN and the frame-level classifier to identify the true events. By evaluating AudioMask over the data sets from 2017 Detection and Classification of Acoustic Scenes and Events (DCASE) Challenge Task 2, We show that our algorithm performs better than the baseline models by 13.3% in the average F-score and achieves better results compared to the other non-ensemble methods in the challenge.

In the field of Automatic Music Transcription, note tracking systems constitute a key process in the overall success of the task as they compute the expected note-level abstraction out of a frame-based pitch activation representation. Despite its relevance, note tracking is most commonly performed using a set of hand-crafted rules adjusted in a manual fashion for the data at issue. In this regard, the present work introduces an approach based on machine learning, and more precisely supervised classification, that aims at automatically inferring such policies for the case of piano music. The idea is to segment each pitch band of a frame-based pitch activation into single instances which are subsequently classified as active or non-active note events. Results using a comprehensive set of supervised classification strategies on the MAPS piano data-set report its competitiveness against other commonly considered strategies for note tracking as well as an improvement of more than +10% in terms of F-measure when compared to the baseline considered for both frame-level and note-level evaluations. This research work is partially supported by Universidad de Alicante through the FPU program [UAFPU2014–5883] and the Spanish Ministerio de Economía y Competitividad through project TIMuL [No. TIN2013–48152–C2–1–R, supported by EU FEDER funds]. EB is supported by a UK RAEng Research Fellowship [grant number RF/128].

This study has successfully developed an audio analyzer system to identify the stroke position on table tennis racket instantaneously, which can effectively help the technical training. It’s well known, a good stroke is often determined by the exact stroke position on table tennis racket during the play. Therefore, instantaneous identification of the stroke position on table tennis racket during play can provide good adjusting reference of action and effectively help the technical training. In this research, this developed audio analyzer consists of a microphone and the analysing program with fast Fourier transform (FFT) using LabVIEW software. The face of table tennis racket is divided into four blocks which are the center, the left-of-center, the right-of-center, and the trail. The stroke sound generated between table tennis racket and ball during play is captured by the microphone and synchronously analyzed through FFT to determine the principal frequency, where the different stroke position on the table tennis racket yields different principal frequency. The results of calibration show that every table tennis racket has three characteristic principal frequencies on the four blocks, which can distinguish center, right/left, and trail respectively. The positional verification rate of the center position identification is all higher than 90% even up to 100% which can assist technical training.

The estimation of fundamental frequency, or pitch, is a fundamental task in computational audio analysis with a variety of applications. Steelpan audio has proven difficult for general pitch detection methods CRePE and pYin. CRePE is a method that uses a deep convolutional neural network to perform pitch estimation directly from the audio signal while pYIN is a digital signal processing-based approach. Audio feature extraction is the process of using digital signal processing techniques to extract low level audio information from signals. Combining audio feature extraction with logistic regression is currently the best performing steelpan pitch estimation method, but the efficacy of using deep neural networks in lieu of traditional machine learning algorithms has yet to be determined. This paper compares the performance and computational requirements of a deep neural network-based architecture against logistic regression as well as the established pYIN and CRePE pitch detection methods to determine which method is the most accurate and efficient. All of these methods are evaluated on a test dataset containing one-hit audio samples from several distinct sounding steelpans. Generalization to other steelpans is assessed by including samples in the test dataset from steelpans for which no samples are in the training dataset.

Continuous audio analysis from embedded and mobile devices is an increasingly important application domain. More and more, appliances like the Amazon Echo, along with smartphones and watches, and even research prototypes seek to perform multiple discriminative tasks simultaneously from ambient audio; for example, monitoring background sound classes (e.g., music or conversation), recognizing certain keywords (‘Hey Siri' or ‘Alexa'), or identifying the user and her emotion from speech. The use of deep learning algorithms typically provides state-of-the-art model performances for such general audio tasks. However, the large computational demands of deep learning models are at odds with the limited processing, energy and memory resources of mobile, embedded and IoT devices. In this paper, we propose and evaluate a novel deep learning modeling and optimization framework that specifically targets this category of embedded audio sensing tasks. Although the supported tasks are simpler than the task of speech recognition, this framework aims at maintaining accuracies in predictions while minimizing the overall processor resource footprint. The proposed model is grounded in multi-task learning principles to train shared deep layers and exploits, as input layer, only statistical summaries of audio filter banks to further lower computations. We find that for embedded audio sensing tasks our framework is able to maintain similar accuracies, which are observed in comparable deep architectures that use single-task learning and typically more complex input layers. Most importantly, on an average, this approach provides almost a 2.1× reduction in runtime, energy, and memory for four separate audio sensing tasks, assuming a variety of task combinations.

Accurate frequency tracking of harmonic signals is an essential process for analysis of musical instrument and singing sounds. For fine frequency estimation, we propose two methods for enhancement ...

The article offers a method for determining vehicle speed from the audio track recorded by the dashboard camera mounted on the car in question. The author analyzed the operation of a four-stroke internal combustion engine, and established a dependency between the number of rotations (rpm) of the engine crankshaft and the number of rotations transferred to the wheel by the manual transmission. The author presents a formula that helps to determine the static radius of the wheel from its marking symbols on the tire. He looks at examples from forensic casework and offers options for determining the ID number of an engaged gearbox. Furthermore, he reviews the possibilities of applying this method to vehicles with an automatic transmission. In conclusion, the article presents the results of tests of the proposed method conducted by leading experts from various forensic institutions of the Russian Ministry of Justice. They point to the consistency of results when engine rpm is calculated based on cepstral analysis and engine tachometer readings.

Video lectures are part of our daily lives. Whether to learn something new, review content for exams or just out of curiosity. People are increasingly looking for video lectures that address what they are looking for. Unfortunately, finding specific content in this type of video is not an easy task. Many video lectures are extensive and cover several topics, and not all of these topics are relevant to the user who has found the video. The result is that the user spends so much time trying to find topic of interest in the middle of content irrelevant to him. The temporal segmentation of video lectures in topics can solve this problem allowing users to navigate of a non-linear way through all topics of a video lecture. However, temporal video lecture segmentation is not an easy task and needs to be automatized. For this reason, in this paper we propose an optimization model for the temporal video lecture segmentation problem. This model uses as features the Word2Vec representation of video lecture's audio transcripts and low-level acoustic characteristics. To find the best video partition, an genetic algorithm with local search is used. We have performed experiments in two data sets and results showed that our proposal is able to overcome state-of-the-art methods and achieve good results for different kinds of video lectures.

The field of Text-to-Speech has experienced huge improvements last years benefiting from deep learning techniques. Producing realistic speech becomes possible now. As a consequence, the research on the control of the expressiveness, allowing to generate speech in different styles or manners, has attracted increasing attention lately. Systems able to control style have been developed and show impressive results. However the control parameters often consist of latent variables and remain complex to interpret. In this paper, we analyze and compare different latent spaces and obtain an interpretation of their influence on expressive speech. This will enable the possibility to build controllable speech synthesis systems with an understandable behaviour.

This work describes and discusses an algorithm submitted to the Sound Event Localization and Detection Task of DCASE2019 Challenge. The proposed methodology relies on parametric spatial audio analysis for source localization and detection, combined with a deep learning-based monophonic event classifier. The evaluation of the proposed algorithm yields overall results comparable to the baseline system. The main highlight is a reduction of the localization error on the evaluation dataset by a factor of 2.6, compared with the baseline performance., 5 pages, 5 figures, submitted to DCASE2019 Workshop