Search

SecureVision is an advanced and trustworthy deepfake detection system created to tackle the growing threat of 'deepfake' movies that tamper with media, undermine public trust, and jeopardize cybersecurity. We present a novel approach that combines big data analytics with state-of-the-art deep learning algorithms to detect altered information in both audio and visual domains. One of SecureVision's primary innovations is the use of multi-modal analysis, which improves detection capabilities by concurrently analyzing many media forms and strengthening resistance against advanced deepfake techniques. The system's efficacy is further enhanced by its capacity to manage large datasets and integrate self-supervised learning, which guarantees its flexibility in the ever-changing field of digital deception. In the end, this study helps to protect digital integrity by providing a proactive, scalable, and efficient defense against the ubiquitous threat of deepfakes, thereby establishing a new benchmark for privacy and security measures in the digital era. [ABSTRACT FROM AUTHOR]

PurposeDesign/methodology/approachFindingsResearch implicationsPractical implicationsOriginalityThis study investigates the sound features of short video advertisements (ads) released by enterprises entering the social e-commerce market and tests whether they are potential drivers of enterprise performance. Based on these findings, this study provides practical references for the design of enterprise ads and short-video social e-commerce platforms.An integrated approach comprising machine learning algorithms and various statistical analysis techniques was employed to quantify sound features. Subsequently, we conducted correlation tests, regression analyses, and robustness tests to further verify our measurements and hypotheses.Sound features in advertisements directly influence an enterprise’s market entry performance. We found that speech rate and voice quality positively affected market entry performance, while loudness and voice pitch negatively impacted performance. In addition, the tempo of the background music positively influences performance.Operating within a cohesive research framework, this study extensively examines the effects of key sound features on enterprise performance. This study also contributes to broadening the academic literature related to the heuristic-systematic model theory and extends audio analysis methods to the social e-commerce market entry field, providing empirical evidence of the impact of sound features on the market entry performance of enterprises.The findings emphasize that enterprises should comprehend the role of sound features and utilize them appropriately when designing ads to be released upon entering the short video social e-commerce market. Furthermore, by considering the findings and audio analysis technologies, platforms can identify user references for sound more precisely and provide enterprises with more guidelines for sound design.This is the first empirical study to discuss the impact of sound features in short video ads on enterprises’ market-entry performance. Additionally, it combines the social e-commerce platform context and expands enterprise market entry to the most popular short video market. [ABSTRACT FROM AUTHOR]

Respiratory conditions have been a focal point in recent medical studies. Early detection and timely treatment are crucial factors in improving patient outcomes for any medical condition. Traditionally, doctors diagnose respiratory conditions through an investigation process that involves listening to the patient's lungs. This study explores the potential of combining audio analysis with convolutional neural networks to detect respiratory conditions in patients. Given the significant impact of proper hyperparameter selection on network performance, contemporary optimizers are employed to enhance efficiency. Moreover, a modified algorithm is introduced that is tailored to the specific demands of this study. The proposed approach is validated using a real-world medical dataset and has demonstrated promising results. Two experiments are conducted: the first tasked models with respiratory condition detection when observing mel spectrograms of patients' breathing patterns, while the second experiment considered the same data format for multiclass classification. Contemporary optimizers are employed to optimize the architecture selection and training parameters of models in both cases. Under identical test conditions, the best models are optimized by the introduced modified metaheuristic, with an accuracy of 0.93 demonstrated for condition detection, and a slightly reduced accuracy of 0.75 for specific condition identification. [ABSTRACT FROM AUTHOR]

Different sounds have various effects on human health, and by introducing the ones that are therapeutic, a healing environment can be created. This paper describes the process to train and test a machine learning algorithm to describe and explore the therapeutic nature of Quranic verse. Using a dataset containing four emotional states namely happy, sad, angry, and relaxed, we trained a model and classified different recitations of the Quran into one of these states. This paper proposes the use of Mel-frequency cepstral coefficients (MFCC) to extract features from Quranic audio and classify it with respect to a known dataset. Based on the experiments conducted on Quranic verses, we summarize our results. [ABSTRACT FROM AUTHOR]

The detection and identification of impulsive sounds applied in a specific context, particularly in sports events, enables the analysis and synthesis of various metrics and statistics associated with the game or even the player's performance. In this context, the automatic identification of an impulsive sound, such as a ball being hit by a player, is a major contribution to the construction of a data source on which game-specific analysis can be performed. Considering all the characteristics (features) of a particular type of impulsive sound in various conditions/environments involves dealing with numerous variables, making it equally challenging to efficiently find the values of the hyperparameters that allow obtaining the best configuration for a given algorithm to be executed in a machine learning process. The contribution of this work is to explore the hyperparameter space in search of values that optimize the performance of the entire process of automatic impulsive sound classification. This process begins with the generation of the dataset to be processed, continues with the training of classification models, and ends with the evaluation of the learned models. The experiments consider a binary classification problem, where a distinction must be made between the intended event and noise. The validation of the process resorts to an audio extracted from videos of sports event competitions, specifically in tennis and padel, where the goal is to identify the sounds of racket hits on the ball. This work is currently in progress, but the preliminary results already enable to evidence the impact of hyperparameter optimization on the accuracy of the overall learning process. [ABSTRACT FROM AUTHOR]

Auditory data has become ubiquitous in tourism and hospitality, especially with digitalization. However, a lack of feasible frameworks for analysing auditory data creates barriers to tourism and hospitality research using auditory data. This study proposed a framework to address the methodological challenges of analysing auditory data in tourism and hospitality. The framework is applied in a digital audio interpretation setting to demonstrate its feasibility and the value of audio features. The proposed framework offers new directions for advancing tourism and hospitality research using unstructured auditory data in various contexts. [ABSTRACT FROM AUTHOR]

Respiratory diseases represent a significant global burden, necessitating efficient diagnostic methods for timely intervention. Digital biomarkers based on audio, acoustics, and sound from the upper and lower respiratory system, as well as the voice, have emerged as valuable indicators of respiratory functionality. Recent advancements in machine learning (ML) algorithms offer promising avenues for the identification and diagnosis of respiratory diseases through the analysis and processing of such audio-based biomarkers. An ever-increasing number of studies employ ML techniques to extract meaningful information from audio biomarkers. Beyond disease identification, these studies explore diverse aspects such as the recognition of cough sounds amidst environmental noise, the analysis of respiratory sounds to detect respiratory symptoms like wheezes and crackles, as well as the analysis of the voice/speech for the evaluation of human voice abnormalities. To provide a more in-depth analysis, this review examines 75 relevant audio analysis studies across three distinct areas of concern based on respiratory diseases' symptoms: (a) cough detection, (b) lower respiratory symptoms identification, and (c) diagnostics from the voice and speech. Furthermore, publicly available datasets commonly utilized in this domain are presented. It is observed that research trends are influenced by the pandemic, with a surge in studies on COVID-19 diagnosis, mobile data acquisition, and remote diagnosis systems. [ABSTRACT FROM AUTHOR]

Using automated models to analyze classroom discourse is a valuable tool for educators to improve their teaching methods. In this paper, we focus on exploring alternatives to ensure the generalizability of models for identifying teaching practices across diverse teaching contexts. Our proposal utilizes artificial intelligence to analyze audio recordings of classroom activities. By leveraging deep learning for speaker diarization and traditional machine learning algorithms for classifying teaching practices, we extract features from the audio diarization using a processing pipeline to provide detailed insights into teaching dynamics. These features enable the classification of three distinct teaching practices: lectures, group discussions, and the use of audience response systems. Our findings demonstrate that these features effectively capture the nuances of teacher-student interactions, allowing for a refined analysis of teaching styles. To enhance the robustness and generalizability of our model, we explore various pipelines for audio processing, evaluating the model’s performance across diverse contexts involving different teachers and students. By comparing these practices and their associated features, we illustrate how AI-driven tools can support teachers in reflecting on and improving their teaching strategies.

SecureVision is an advanced and trustworthy deepfake detection system created to tackle the growing threat of ‘deepfake’ movies that tamper with media, undermine public trust, and jeopardize cybersecurity. We present a novel approach that combines big data analytics with state-of-the-art deep learning algorithms to detect altered information in both audio and visual domains. One of SecureVision’s primary innovations is the use of multi-modal analysis, which improves detection capabilities by concurrently analyzing many media forms and strengthening resistance against advanced deepfake techniques. The system’s efficacy is further enhanced by its capacity to manage large datasets and integrate self-supervised learning, which guarantees its flexibility in the ever-changing field of digital deception. In the end, this study helps to protect digital integrity by providing a proactive, scalable, and efficient defense against the ubiquitous threat of deepfakes, thereby establishing a new benchmark for privacy and security measures in the digital era.

Social robots in the home will need to solve audio identification problems to better interact with their users. This article focuses on the classification between (a) natural conversation that includes at least one co-located user and (b) media that is playing from electronic sources and does not require a social response, such as television shows. This classification can help social robots detect a user's social presence using sound. Social robots that are able to solve this problem can apply this information to assist them in making decisions, such as determining when and how to appropriately engage human users. We compiled a dataset from a variety of acoustic environments that contained either natural or media audio, including audio that we recorded in our own homes. Using this dataset, we performed an experimental evaluation on a range of traditional machine learning classifiers and assessed the classifiers' abilities to generalize to new recordings, acoustic conditions, and environments. We conclude that a C-Support Vector Classification (SVC) algorithm outperformed other classifiers. Finally, we present a classification pipeline that in-home robots can utilize, and we discuss the timing and size of the trained classifiers as well as privacy and ethics considerations. [ABSTRACT FROM AUTHOR]

Stress can be considered a mental/physiological reaction in conditions of high discomfort and challenging situations. The levels of stress can be reflected in both the physiological responses and speech signals of a person. Therefore the study of the fusion of the two modalities is of great interest. For this cause, public datasets are necessary so that the different proposed solutions can be comparable. In this work, a publicly available multimodal dataset for stress detection is introduced, including physiological signals and speech cues data. The physiological signals include electrocardiograph (ECG), respiration (RSP), and inertial measurement unit (IMU) sensors equipped in a smart vest. A data collection protocol was introduced to receive physiological and audio data based on alterations between well-known stressors and relaxation moments. Five subjects participated in the data collection, where both their physiological and audio signals were recorded by utilizing the developed smart vest and audio recording application. In addition, an analysis of the data and a decision-level fusion scheme is proposed. The analysis of physiological signals includes a massive feature extraction along with various fusion and feature selection methods. The audio analysis comprises a state-of-the-art feature extraction fed to a classifier to predict stress levels. Results from the analysis of audio and physiological signals are fused at a decision level for the final stress level detection, utilizing a machine learning algorithm. The whole framework was also tested in a real-life pilot scenario of disaster management, where users were acting as first responders while their stress was monitored in real time. [ABSTRACT FROM AUTHOR]

Forensic audio analysis is a foundation stone of many crime investigations. In forensic evidence; the audio file of the human voice is analyzed to extract much information in addition to the content of the speech, such as the speaker's identity, emotions, gender, origin, etc. The accurate determination of individuals into groups based on their age development stage and their gender are often used as early investigations to differentiate them and determine the legal rights and responsibilities associated with them. This work introduces a light CNN model with a new architecture to detect the human age-evolution being's stage (kids or adults) at the same time the gender of the adult one (male or female) based on the individual's voice characteristics, which offers a balance between computational efficiency and model accuracy. The temporal information in the audio file is prepared by scaled and normalized. Then this information is exploited to extract and track the unique and salient audio features that make up the pattern of the feature map for each target class through some convolutional layers followed by maxpooling layers. Finally, The decision is made based on these feature maps by some fully connected layers. Successful and promising results are accomplished in terms of accuracy and loss functions which realize 0.99 and 0.017 respectively over the riched Voxceleb2 dataset. The proposed model underscores the importance of leveraging Light DNNs for gender and age-evolution detection, offering a robust and ethically sound solution for real-world applications in the field of audio forensics such as span speaker identification, victim profiling, deception detection, and more, contributing to the advancement of audio forensic analysis. [ABSTRACT FROM AUTHOR]

Affected by the COVID-19 epidemic, the final examinations at many universities and the recruitment interviews of enterprises were forced to be transferred to online remote video invigilation, which undoubtedly improves the space and possibility of cheating. To solve these problems, this paper proposes an intelligent invigilation system based on the EfficientDet target detection network model combined with a centroid tracking algorithm. Experiments show that cheating behavior detection model proposed in this paper has good detection, tracking and recognition effects in remote testing scenarios. Taking the EfficientDet network as the detection target, the average detection accuracy of the network is 81%. Experiments with real online test videos show that the cheating behavior detection accuracy can reach 83.1%. In addition, to compensate for the shortage of image detection, we also design an audio detection module to carry out auxiliary detection and forensics. The audio detection module is used to continuously detect the environmental sound of the examination room, save suspicious sounds and provide evidence for judging cheating behavior. [ABSTRACT FROM AUTHOR]

Respiratory conditions have been a focal point in recent medical studies. Early detection and timely treatment are crucial factors in improving patient outcomes for any medical condition. Traditionally, doctors diagnose respiratory conditions through an investigation process that involves listening to the patient’s lungs. This study explores the potential of combining audio analysis with convolutional neural networks to detect respiratory conditions in patients. Given the significant impact of proper hyperparameter selection on network performance, contemporary optimizers are employed to enhance efficiency. Moreover, a modified algorithm is introduced that is tailored to the specific demands of this study. The proposed approach is validated using a real-world medical dataset and has demonstrated promising results. Two experiments are conducted: the first tasked models with respiratory condition detection when observing mel spectrograms of patients’ breathing patterns, while the second experiment considered the same data format for multiclass classification. Contemporary optimizers are employed to optimize the architecture selection and training parameters of models in both cases. Under identical test conditions, the best models are optimized by the introduced modified metaheuristic, with an accuracy of 0.93 demonstrated for condition detection, and a slightly reduced accuracy of 0.75 for specific condition identification.

Respiratory diseases represent a significant global burden, necessitating efficient diagnostic methods for timely intervention. Digital biomarkers based on audio, acoustics, and sound from the upper and lower respiratory system, as well as the voice, have emerged as valuable indicators of respiratory functionality. Recent advancements in machine learning (ML) algorithms offer promising avenues for the identification and diagnosis of respiratory diseases through the analysis and processing of such audio-based biomarkers. An ever-increasing number of studies employ ML techniques to extract meaningful information from audio biomarkers. Beyond disease identification, these studies explore diverse aspects such as the recognition of cough sounds amidst environmental noise, the analysis of respiratory sounds to detect respiratory symptoms like wheezes and crackles, as well as the analysis of the voice/speech for the evaluation of human voice abnormalities. To provide a more in-depth analysis, this review examines 75 relevant audio analysis studies across three distinct areas of concern based on respiratory diseases’ symptoms: (a) cough detection, (b) lower respiratory symptoms identification, and (c) diagnostics from the voice and speech. Furthermore, publicly available datasets commonly utilized in this domain are presented. It is observed that research trends are influenced by the pandemic, with a surge in studies on COVID-19 diagnosis, mobile data acquisition, and remote diagnosis systems.

Predictive Maintenance (PdM) has a main role in the Fourth Industrial Revolution; its goal is to design models that can safely detect failure in systems before they fail, aiming to reduce financial, environmental, and operational costs. A brushless DC (BLDC) electric motors have increasingly become more popular and been gaining popularity in industrial applications, so their analysis for PdM applications is only a natural progression; audio analysis proves to be a useful method to achieve this and rises as a very pragmatic case of study of the characteristics of the motors. The main goal of this paper is to showcase sound-based behavior of BLDC motors in different failure modes as result of an experiment led by researchers at Universidad del Cauca in Colombia. This dataset may provide researchers with useful information regarding signal processing and the development of Machine Learning applications that would achieve an improvement within Predictive Maintenance and I4.0.Predictive Maintenance (PdM) has a main role in the Fourth Industrial Revolution; its goal is to design models that can safely detect failure in systems before they fail, aiming to reduce financial, environmental, and operational costs. A brushless DC (BLDC) electric motors have increasingly become more popular and been gaining popularity in industrial applications, so their analysis for PdM applications is only a natural progression; audio analysis proves to be a useful method to achieve this and rises as a very pragmatic case of study of the characteristics of the motors. The main goal of this paper is to showcase sound-based behavior of BLDC motors in different failure modes as result of an experiment led by researchers at Universidad del Cauca in Colombia. This dataset may provide researchers with useful information regarding signal processing and the development of Machine Learning applications that would achieve an improvement within Predictive Maintenance and I4.0.

Several excavations at the Hemudu site in Zhejiang Province have led to the discovery of more than 100 pieces of bird bones that were likely used to create a kind of musical instrument. However, in contrast to the Jiahu bone flute, it is not clear for what the Hemudu bone whistle was used, although it was probably used for musical purposes. In this paper, the 7000-year-old Hemudu bone whistle is reproduced with 3D scanning and printing technology. Therefore, we propose a robust scan point detection and tracking method that includes three main stages. First, we conducted robust point detection and selection where noise and irrelevant points were removed. Then, we propose to track the key points and conduct 3D model optimization. With the above efforts, we obtained the 3D model of the bone whistle, and the audio of the bone whistle is replicated. The purpose of the bone whistles is inferred and evaluated through audio analyses. The experimental results show that the method is feasible.

Microphone identification is a crucial challenge in the field of digital audio forensics. The ability to accurately identify the type of microphone used to record a piece of audio can provide important information for forensic analysis and crime investigations. In recent years, transformer-based deep-learning models have been shown to be effective in many different tasks. This paper proposes a system based on a transformer for microphone identification based on recorded audio. Two types of experiments were conducted: one to identify the model of the microphones and another in which identical microphones were identified within the same model. Furthermore, extensive experiments were performed to study the effects of different input types and sub-band frequencies on system accuracy. The proposed system is evaluated on the Audio Forensic Dataset for Digital Multimedia Forensics (AF-DB). The experimental results demonstrate that our model achieves state-of-the-art accuracy for inter-model and intra-model microphone classification with 5-fold cross-validation. [ABSTRACT FROM AUTHOR]

Asthma is a common, usually long-term respiratory disease with negative impact on global society and economy. Treatment involves using medical devices (inhalers) that distribute medication to the airways and its efficiency depends on the precision of the inhalation technique. There is a clinical need for objective methods to assess the inhalation technique, during clinical consultation. Integrated health monitoring systems, equipped with sensors, enable the recognition of drug actuation, embedded with sound signal detection, analysis and identification from intelligent structures, that could provide powerful tools for reliable content management. Health monitoring systems equipped with sensors, embedded with sound signal detection, enable the recognition of drug actuation and could be used for effective audio content analysis. This paper revisits sound pattern recognition with machine learning techniques for asthma medication adherence assessment and presents the Respiratory and Drug Actuation (RDA) Suite (https://gitlab.com/vvr/monitoring-medication-adherence/rda-benchmark) for benchmarking and further research. The RDA Suite includes a set of tools for audio processing, feature extraction and classification procedures and is provided along with a dataset, consisting of respiratory and drug actuation sounds. The classification models in RDA are implemented based on conventional and advanced machine learning and deep networks’ architectures. This study provides a comparative evaluation of the implemented approaches, examines potential improvements and discusses on challenges and future tendencies.

In this paper, we advocate in favor of smartphone apps as low-cost, easy-to-deploy solutions for raising awareness among the population on the proliferation of Aedes aegypti mosquitoes. Nevertheless, devising such a smartphone app is challenging for many reasons, including the required maturity level of techniques for identifying mosquitoes based on features that can be captured using smartphone resources. In this paper, we identify a set of (non-exhaustive) requirements that smartphone apps must meet to become an effective tooling in the fight against Ae. aegypti , and advance the state-of-the-art with (i) a dataset for benchmarking solutions for detecting Ae. aegypti mosquitoes from wingbeat sound recordings collected with smartphones, and (ii) a residual convolutional neural network for classifying Ae. aegypti mosquitoes from their wingbeat sound, considering a training methodology for reducing the influence of background noise in the classification process. From the analysis of precision, accuracy, recall, and F1-score, we provide evidence that residual convolutional neural networks have potential as a cornerstone for tracking mosquito apps for smartphones. We demonstrate the proposed solution's feasibility by implementing a prototype capable of detecting mosquitoes in a controlled environment. • Residual CNN-based models running in a Smartphone App can identify Ae. aegypti. • Residual and classification blocks made classification suitable for Smartphones. • The proposed topology reached an accuracy as high as 98% and F1-measure of 90%. • These results suggest the potential of detecting Ae. aegypti using smartphone apps. • Such apps could raise awareness and help fight nearby Ae. aegypti breeding sites. [ABSTRACT FROM AUTHOR]

With the emergence of new digital technologies, a significant surge has been seen in the volume of multimedia data generated from various smart devices. Several challenges for data analysis have emerged to extract useful information from multimedia data. One such challenge is the early and accurate detection of anomalies in multimedia data. This study proposes an efficient technique for anomaly detection and classification of rare events in audio data. In this paper, we develop a vast audio dataset containing seven different rare events (anomalies) with 15 different background environmental settings (e.g., beach, restaurant, and train) to focus on both detection of anomalous audio and classification of rare sound (e.g., events—baby cry, gunshots, broken glasses, footsteps) events for audio forensics. The proposed approach uses the supreme feature extraction technique by extracting mel-frequency cepstral coefficients (MFCCs) features from the audio signals of the newly created dataset and selects the minimum number of best-performing features for optimum performance using principal component analysis (PCA). These features are input to state-of-the-art machine learning algorithms for performance analysis. We also apply machine learning algorithms to the state-of-the-art dataset and realize good results. Experimental results reveal that the proposed approach effectively detects all anomalies and superior performance to existing approaches in all environments and cases.

We introduce a method for computer-assisted analysis of orchestration. We also look into the role that texture and orchestration have in structuring musical form. The method comprises a numerical representation, a hierarchy of ‘textural situations’ and measures for heterogeneity, diversity and complexity of orchestral-textural configurations.

As the voice disorder is a typical early symptom of Parkinson, some researchers attempt to diagnose this disease based on voice data collected from suspected patients. Although existing methods can provide acceptable results, they just work in partial scenarios. In other words, they are not generable and robust enough. To this end, we present a Parkinson’s auxiliary diagnosis system based on human speech, which can adaptively build a suitable deep neural network based on sound features. The system includes two modules: hybrid features extraction and adaptive network construction. We extract kinds of information from the voice data to form a new compound feature. Furthermore, particle swarm optimization (PSO) algorithm is employed to build the corresponding 1D convolution network for features classification. Extensive experiments on two datasets consisting of English and Italian are conducted for evaluation purposes. Experimental results show that our method improves the accuracy of voice-based Parkinson’s disease detection to some extent.

Recently, attention of the emotional speech signals research has been boosted in human machine interfaces due to availability of high computation capability. Selection of suitable feature sets, design of a proper classification methods and prepare an appropriate dataset are the main key issues of speech emotion recognition systems. Detecting emotions is one of the most important marketing strategies in today’s world. You could personalize different things for an individual specifically to suit their interest. Once machines have the capability of understanding the emotions of a person, it will greatly enhance the user experience. In this report, we will try to classify the emotion of a voice (audio clip) using different algorithms of Artificial Intelligence (AI). Based on the accuracy rates, a suitable choice could be made to make such application in the future. [ABSTRACT FROM AUTHOR]

On the one hand facial expression recognition (FER) of a human to determine his emotional state is a wellstudied task. On the other hand, most of the existing approaches are focused on recognition of basic emotions or binary labels of emotional tone (positive or negative). However, facial micro-expression (FMiE) recognition (FMiER) is an understudied but growing area. The interest of researchers in this area is due to the unsolved problems of determining the subtle intentions of humans in situations with high stakes, such as lie detection, state of mind recognition, negotiation techniques recognition, human behavior patterns prediction. In contrast to facial macro-expressions FMiEs are involuntary and transient facial expressions capable of revealing the genuine feelings that humans attempt to hide. These differences are reasons why FMiE detection relies heavily on expert experience and why FMiER is a difficult task which still requires additional research. In addition to FER, the human voice also contains a lot of information about the internal state of a human and speech emotion recognition is an actively studied task as well. In this paper we consider the problem of lie detection, however we soften it to the problem of assessing the confidence of a person in a video. We hypothesized that the use of information from two modalities and the joint exploring of this information through modalities can allow to predict the subtle feelings of a human with high accuracy. We introduce a solution that extracts and analyzes person's facial and speech features on a video sequence using attention mechanisms in the task of truth determination. Our approach uses state-of-the-art methods and achieves 84.91% accuracy. The survey also includes a comparison of various audio and visual features combinations and shows that our approach shows comparable accuracy-performance ratio. [ABSTRACT FROM AUTHOR]

Microphone identification is a crucial challenge in the field of digital audio forensics. The ability to accurately identify the type of microphone used to record a piece of audio can provide important information for forensic analysis and crime investigations. In recent years, transformer-based deep-learning models have been shown to be effective in many different tasks. This paper proposes a system based on a transformer for microphone identification based on recorded audio. Two types of experiments were conducted: one to identify the model of the microphones and another in which identical microphones were identified within the same model. Furthermore, extensive experiments were performed to study the effects of different input types and sub-band frequencies on system accuracy. The proposed system is evaluated on the Audio Forensic Dataset for Digital Multimedia Forensics (AF-DB). The experimental results demonstrate that our model achieves state-of-the-art accuracy for inter-model and intra-model microphone classification with 5-fold cross-validation.

We introduce a method for computer-assisted analysis of orchestration. We also look into the role that texture and orchestration have in structuring musical form. The method comprises a numerical representation, a hierarchy of ‘textural situations’ and measures for heterogeneity, diversity and complexity of orchestral-textural configurations. [ABSTRACT FROM AUTHOR]

Background: Respiratory sounds have been recognized as a possible indicator of behavior and health. Computer analysis of these sounds can indicate characteristic sound changes caused by COVID-19 and can be used for diagnostics of this illness. Objective: The aim of the study is to develop 2 fast, remote computer-assisted diagnostic methods for specific acoustic phenomena associated with COVID-19 based on analysis of respiratory sounds. Methods: Fast Fourier transform (FFT) was applied for computer analysis of respiratory sound recordings produced by hospital doctors near the mouths of 14 patients with COVID-19 (aged 18-80 years) and 17 healthy volunteers (aged 5-48 years). Recordings for 30 patients and 26 healthy persons (aged 11-67 years, 34, 60%, women), who agreed to be tested at home, were made by the individuals themselves using a mobile telephone; the records were passed for analysis using WhatsApp. For hospitalized patients, the illness was diagnosed using a set of medical methods; for outpatients, polymerase chain reaction (PCR) was used. The sampling rate of the recordings was from 44 to 96 kHz. Unlike usual computer-assisted diagnostic methods for illnesses based on respiratory sound analysis, we proposed to test the high-frequency part of the FFT spectrum (2000-6000 Hz). Results: Comparing the FFT spectra of the respiratory sounds of patients and volunteers, we developed 2 computer-assisted methods of COVID-19 diagnostics and determined numerical healthy-ill criteria. These criteria were independent of gender and age of the tested person. Conclusions: The 2 proposed computer-assisted diagnostic methods, based on the analysis of the respiratory sound FFT spectra of patients and volunteers, allow one to automatically diagnose specific acoustic phenomena associated with COVID-19 with sufficiently high diagnostic values. These methods can be applied to develop noninvasive screening self-testing kits for COVID-19. [ABSTRACT FROM AUTHOR]

This text describes the musical evaluation of a hybrid music notation system that combines traditional notation with symbols and concepts from spectromorphological analysis. During three academic years from 2017 to 2020, three groups of composition students learned to work with sound notation, recreating and interpreting short electroacoustic music sketches based solely on their notation transcriptions – they had not heard the original sketches. The students' score interpretations bore obvious similarities to the original music sketches and their written reflections showed that there were no major difficulties in understanding the notation although some difficulties existed concerning finding suitable sounds, especially sounds with stable pitch. [ABSTRACT FROM AUTHOR]

The quantitative assessment of the musical timbre in an audio record is still an open-ended issue. Evaluating the musical timbre allows not only to establish precise musical parameters but also the recognition, classification of musical instruments, and assessment of the musical quality of a sound record. In this paper, we present a minimum set of dimensionless descriptors, motivated by musical acoustics, using the spectra obtained by the Fast Fourier Transform (FFT), which allows describing the timbre of wooden aerophones (Bassoon, Clarinet, Transverse Flute, and Oboe) using individual sound recordings of the musical tempered scale. We postulate that the proposed descriptors are sufficient to describe the timbral characteristics in the aerophones studied, allowing their recognition using the acoustic spectral signature. We believe that this approach can be further extended to use multidimensional unsupervised machine learning techniques, such as clustering, to obtain new insights into timbre characterization. [ABSTRACT FROM AUTHOR]

Barking and other dog vocalizations have acoustic properties related to emotions, physiological reactions, attitudes, or some particular internal states. In the field of intelligent audio analysis, researchers use methods based on signal processing and machine learning to analyze the digitized acoustic signals' properties and obtain relevant information. The present work describes a method to classify the identity, breed, age, sex, and context associated with each bark. This information can support the decisions of people who regularly interact with animals, such as dog trainers, veterinarians, rescuers, police, people with visual impairment. Our approach uses deep neural networks to generate trained models for each classification task. We worked with 19,643 barks recorded from 113 dogs of different breeds, ages and sexes. Our methodology consists of three stages. First, the pre-processing stage prepares the data and transforms it into the appropriate format for each classification model. Second, the characterization stage evaluates different representation models to identify the most suitable for each task. Third, the classification stage trains each classification model and selects the best hyperparameters. After tuning and training each model, we evaluated its performance. We analyzed the most relevant features extracted from the audio and the most appropriate deep neural network architecture for that feature type. Even if the application of our method is not ready for being used in ethological practice, our evaluation showed an outstanding performance of the proposed method, surpassing previous research results on this topic, providing the basis for further technological development. • Analyzed diverse dog breed data, optimizing computational efficiency. • Largest variety & number of bark samples in barking classification. • Employed efficient characterization techniques, reducing computational cost. • Achieved classifier success with tailored deep neural network architecture. • Optimized classification with neural network hyperparameter tuning. [ABSTRACT FROM AUTHOR]

Identification of appropriate content-based features for the description of audio signals can provide a better repre-sentation of naturalistic music stimuli which, in recent years, have been used to understand how the human brain processes such information. In this work, an extensive clustering analysis has been carried out on a large and benchmark audio dataset to assess whether features commonly extracted in the literature are in fact statistically relevant. Our results show that not all of these well-known acoustic features might be statistically necessary. We also demonstrate quantitatively that, regardless of the musical genre, the same acoustic feature is selected to represent each cluster. This finding discloses that there is a general redundancy among the set of audio descriptors used, that does not depend on a particular music track or genre, allowing an expressive reduction of the number of features necessary to identify apropriate time instants on the audio for further brain signal processing of music stimuli. [ABSTRACT FROM AUTHOR]

Pain is a complex phenomenon, the experience of which varies widely across individuals. At worst, chronic pain can lead to anxiety and depression. Cost-effective strategies are urgently needed to improve the treatment of pain, and thus we propose a novel home-based pain measurement system for the longitudinal monitoring of pain experience and variation in different patients with chronic low back pain. The autonomous nervous system and audio-visual features are analyzed from heart rate signals, voice characteristics and facial expressions using a unique measurement protocol. Self-reporting is utilized for the follow-up of changes in pain intensity, induced by well-designed physical maneuvers, and for studying the consecutive trends in pain. We describe the study protocol, including hospital measurements and questionnaires and the implementation of the home measurement devices. We also present different methods for analyzing the multimodal data: electroencephalography, audio, video and heart rate. Our intention is to provide new insights using technical methodologies that will be beneficial in the future not only for patients with low back pain but also patients suffering from any chronic pain. [ABSTRACT FROM AUTHOR]

This work aims to demonstrate the feasibility that haptic information can be acquired from a da Vinci robotic tool using audio sensing according to sensor placement requirements in a real clinical scenario. For that, two potential audio sensor locations were studied using an experimental setup for performing, in a repeatable way, interactions of a da Vinci forceps with three different tissues. The obtained audio signals were assessed in terms of their resulting signal-to-noise-ratio (SNR) and their capability to distinguish between different tissues. A spectral energy distribution analysis using Discrete Wavelet Transformation was performed to extract signal signatures from the tested tissues. Results show that a high SNR was obtained in most of the audio recordings acquired from both studied positions. Additionally, evident spectral energy-related patterns could be extracted from the audio signals allowing us to distinguish between different palpated tissues.

Open-source software libraries have a significant impact on the development of Audio Signal Processing and Music Information Retrieval (MIR) systems. Despite the abundance of such tools, there is a lack of an extensive and easy-to-use reference library for audio feature extraction on Web clients. In this article, we present 'Essentia.js', an open-source JavaScript (JS) library for audio and music analysis on both web clients and JS engines. Along with the Web Audio API, it can be used for both offline and real-time audio feature extraction on web browsers. 'Essentia.js' is modular, lightweight, and easy-to-use, deploy, maintain, and integrate into the existing plethora of JS libraries and web technologies. It is powered by a WebAssembly back end cross-compiled from the Essentia C++ library, which facilitates a JS interface to a wide range of low-level and high-level audio features, including signal processing MIR algorithms as well as pre-trained TensorFlow.js machine learning models. It also provides a higher-level JS API and add-on MIR utility modules along with extensive documentation, usage examples, and tutorials. We benchmark the proposed library on two popular web browsers and the Node.js engine, and four devices, including mobile Android and iOS, comparing it to the native performance of Essentia and the Meyda JS library.

'Covering' previously recorded tracks is a central aspect of popular music. Despite an abundance of examples, discourse related to cover-version tracks is limited and includes questionable application of dichotomizing terminology ('cover', 'version') that can cause problems for interpreting the cover-version genre and diminish the nuanced approaches taken by artists who produce cover-versions. Examining recordings from the Punk Goes... album series reveals many examples that go beyond the either–or cover/version dichotomy generally presented in musicology. As such, this article has three main aims: to investigate the ways in which Punk Goes... tracks innovatively move between the poles of cover and version, thus requiring a new analytical framework; to explore the connection between imitation and transformation to the ascription of cover and version; and to move beyond prior musicological scholarship that emphasizes formal elements and composer intention. This article presents a reclassification of the terms 'cover' and 'version' as extreme poles of a 'cover-version spectrum' dependent on the listener's interpretation of the performative character. Analyses challenge assumptions that equate musical phrases with fixed musical meanings, suggesting instead that the interpreted manner of performance is more significant than syntactical concerns. This provides a structure for clearer examination of cover-versions and supports the view that semiotic methods are a vital tool in the analysis of recorded popular music. [ABSTRACT FROM AUTHOR]

Research on self-efficacy and confidence has spread across several subfields of psychology and neuroscience. The role of one’s confidence is very crucial in the formation of attitude and communication skills. The importance of differentiating the levels of confidence is quite visible in this domain. With the recent advances in extracting behavioral insight from a signal in multiple applications, detecting confidence is found to have great importance. One such prominent application is detecting confidence in interview conversations. We have collected an audiovisual data set of interview conversations with 34 candidates. Every response (from each of the candidate) of this data set is labeled with three levels of confidence: high, medium, and low. Furthermore, we have also developed algorithms to efficiently compute such behavioral confidence from speech and video. A deep learning architecture is proposed for detecting confidence levels (high, medium, and low) from an audiovisual clip recorded during an interview. The achieved unweighted average recall (UAR) reaches 85.9% on audio data and 73.6% on video data captured from an interview session.

Amidst discussions regarding the nature of a musical work, tensions within and between score- and performance-based approaches often increase ideological entrenchment. Opera's textual and visual elements, along with its inherently social nature, both simultaneously complicate understanding of a work's nature and provide interdisciplinary analytical inroads. Analysis of operatic performance faces challenges of how to interrogate onstage musical behaviors, and how they relate to both dramatic narrative and an opera's identity. This article applies Martin Heidegger's dichotomy of technē and poiēsis to relationships between scores, performances, and works, characterizing works as conceptual, and scores and performances as tangible embodiments. Opera scholarship relies primarily upon scores, creating a lacuna of sound-based examinations. Adapting analyses developed in CHARM's "Mazurka Project," this essay incorporates textual considerations into tempo hierarchy as a means of asserting each performance's nuanced uniqueness and thus provides a window into performers' creative processes. Interviews with the performers considered in this study confirmed postulations derived from analyzing temporal-textual emphases. This approach is adaptable to other expressive elements employed in creating a role onstage. In addition to the hybrid empirical-hermeneutic application, the datasets created in this approach could be valuable as ground truth in machine learning applications.

Vibrato is one of the most common musical techniques used for the enrichment of vocal and musical instrument sounds. We propose a method that can analyse the intonation, vibrato rate, and vibrato extent of a vibrato tone as a function of time, which is based on the discrete Fourier transform of its fundamental frequency trajectory. According to experimental results, the proposed method is robust to the irregularities in the fundamental frequency trajectory. In addition, the proposed method provides different results for intonation and vibrato extent from those of Prame's method when the fundamental frequency trajectory is not perfectly sinusoidal. [ABSTRACT FROM AUTHOR]

Purpose: Minimally invasive surgery (MIS) has become the standard for many surgical procedures as it minimizes trauma, reduces infection rates and shortens hospitalization. However, the manipulation of objects in the surgical workspace can be difficult due to the unintuitive handling of instruments and limited range of motion. Apart from the advantages of robot-assisted systems such as augmented view or improved dexterity, both robotic and MIS techniques introduce drawbacks such as limited haptic perception and their major reliance on visual perception. Methods: In order to address the above-mentioned limitations, a perception study was conducted to investigate whether the transmission of intra-abdominal acoustic signals can potentially improve the perception during MIS. To investigate whether these acoustic signals can be used as a basis for further automated analysis, a large audio data set capturing the application of electrosurgery on different types of porcine tissue was acquired. A sliding window technique was applied to compute log-mel-spectrograms, which were fed to a pre-trained convolutional neural network for feature extraction. A fully connected layer was trained on the intermediate feature representation to classify instrument–tissue interaction. Results: The perception study revealed that acoustic feedback has potential to improve the perception during MIS and to serve as a basis for further automated analysis. The proposed classification pipeline yielded excellent performance for four types of instrument–tissue interaction (muscle, fascia, liver and fatty tissue) and achieved top-1 accuracies of up to 89.9%. Moreover, our model is able to distinguish electrosurgical operation modes with an overall classification accuracy of 86.40%. Conclusion: Our proof-of-principle indicates great application potential for guidance systems in MIS, such as controlled tissue resection. Supported by a pilot perception study with surgeons, we believe that utilizing audio signals as an additional information channel has great potential to improve the surgical performance and to partly compensate the loss of haptic feedback. [ABSTRACT FROM AUTHOR]

• Analyze the students' behavior together with teaching contents and characteristic. • Focus on the ratio of students paying attention to the teaching contents. • Extract the teaching characteristic and topics from in-classroom videos. • Represent the teaching characteristic by extracting features from teacher's audio. • Analyze the students' behavior on different topics extracted from audio information. The automatic analysis of students' in-classroom behavior is valuable to evaluate the effect of teaching. Recent studies of in-classroom video analysis mainly focus on lecture content, positions and identities of students. In this paper, we propose to analyze the students' concentration degree to the teacher or teaching content. Specifically, we detect students' faces, track faces, and analyze the students' behavior, i.e. raising or downing faces and corresponding head orientations to the teacher, teaching content or not. Besides, texts are obtained from the teacher's speech and the course topics taught in the class are extracted. Audio features of the teacher's speech are extracted and analyzed. Finally, the correlation of the students' concentration degree with the course topics, audio features are analyzed. This analysis can help teachers find the effective teaching characteristic to better improve students' concentration degree. [ABSTRACT FROM AUTHOR]

Currently, VoIP company technicians conduct tests to classify call quality as good or bad. Even though, there are automatic platforms that make test VoIP calls to classify them, they do not perform audio processing to detect False Answer Supervision (FAS), which is a common and undesirable feature of VoIP calls. In this paper, a Vector Support Machine (SVM) along with several functions used in voice recognition were implemented to emulate the human decision procedure (the task of audio classification and analysis performed by technicians). The experiments were based on the comparison between the results obtained from the current classification methods and those derived from the SVM. A 10-fold cross-validation was used to evaluate the system performance. The tests results from the proposed methodology show a better percentage of successful classification compared to a selected automatic platform called CheckMyRoutes.