Search

SecureVision is an advanced and trustworthy deepfake detection system created to tackle the growing threat of 'deepfake' movies that tamper with media, undermine public trust, and jeopardize cybersecurity. We present a novel approach that combines big data analytics with state-of-the-art deep learning algorithms to detect altered information in both audio and visual domains. One of SecureVision's primary innovations is the use of multi-modal analysis, which improves detection capabilities by concurrently analyzing many media forms and strengthening resistance against advanced deepfake techniques. The system's efficacy is further enhanced by its capacity to manage large datasets and integrate self-supervised learning, which guarantees its flexibility in the ever-changing field of digital deception. In the end, this study helps to protect digital integrity by providing a proactive, scalable, and efficient defense against the ubiquitous threat of deepfakes, thereby establishing a new benchmark for privacy and security measures in the digital era. [ABSTRACT FROM AUTHOR]

PurposeDesign/methodology/approachFindingsResearch implicationsPractical implicationsOriginalityThis study investigates the sound features of short video advertisements (ads) released by enterprises entering the social e-commerce market and tests whether they are potential drivers of enterprise performance. Based on these findings, this study provides practical references for the design of enterprise ads and short-video social e-commerce platforms.An integrated approach comprising machine learning algorithms and various statistical analysis techniques was employed to quantify sound features. Subsequently, we conducted correlation tests, regression analyses, and robustness tests to further verify our measurements and hypotheses.Sound features in advertisements directly influence an enterprise’s market entry performance. We found that speech rate and voice quality positively affected market entry performance, while loudness and voice pitch negatively impacted performance. In addition, the tempo of the background music positively influences performance.Operating within a cohesive research framework, this study extensively examines the effects of key sound features on enterprise performance. This study also contributes to broadening the academic literature related to the heuristic-systematic model theory and extends audio analysis methods to the social e-commerce market entry field, providing empirical evidence of the impact of sound features on the market entry performance of enterprises.The findings emphasize that enterprises should comprehend the role of sound features and utilize them appropriately when designing ads to be released upon entering the short video social e-commerce market. Furthermore, by considering the findings and audio analysis technologies, platforms can identify user references for sound more precisely and provide enterprises with more guidelines for sound design.This is the first empirical study to discuss the impact of sound features in short video ads on enterprises’ market-entry performance. Additionally, it combines the social e-commerce platform context and expands enterprise market entry to the most popular short video market. [ABSTRACT FROM AUTHOR]

Respiratory conditions have been a focal point in recent medical studies. Early detection and timely treatment are crucial factors in improving patient outcomes for any medical condition. Traditionally, doctors diagnose respiratory conditions through an investigation process that involves listening to the patient's lungs. This study explores the potential of combining audio analysis with convolutional neural networks to detect respiratory conditions in patients. Given the significant impact of proper hyperparameter selection on network performance, contemporary optimizers are employed to enhance efficiency. Moreover, a modified algorithm is introduced that is tailored to the specific demands of this study. The proposed approach is validated using a real-world medical dataset and has demonstrated promising results. Two experiments are conducted: the first tasked models with respiratory condition detection when observing mel spectrograms of patients' breathing patterns, while the second experiment considered the same data format for multiclass classification. Contemporary optimizers are employed to optimize the architecture selection and training parameters of models in both cases. Under identical test conditions, the best models are optimized by the introduced modified metaheuristic, with an accuracy of 0.93 demonstrated for condition detection, and a slightly reduced accuracy of 0.75 for specific condition identification. [ABSTRACT FROM AUTHOR]

Different sounds have various effects on human health, and by introducing the ones that are therapeutic, a healing environment can be created. This paper describes the process to train and test a machine learning algorithm to describe and explore the therapeutic nature of Quranic verse. Using a dataset containing four emotional states namely happy, sad, angry, and relaxed, we trained a model and classified different recitations of the Quran into one of these states. This paper proposes the use of Mel-frequency cepstral coefficients (MFCC) to extract features from Quranic audio and classify it with respect to a known dataset. Based on the experiments conducted on Quranic verses, we summarize our results. [ABSTRACT FROM AUTHOR]

The detection and identification of impulsive sounds applied in a specific context, particularly in sports events, enables the analysis and synthesis of various metrics and statistics associated with the game or even the player's performance. In this context, the automatic identification of an impulsive sound, such as a ball being hit by a player, is a major contribution to the construction of a data source on which game-specific analysis can be performed. Considering all the characteristics (features) of a particular type of impulsive sound in various conditions/environments involves dealing with numerous variables, making it equally challenging to efficiently find the values of the hyperparameters that allow obtaining the best configuration for a given algorithm to be executed in a machine learning process. The contribution of this work is to explore the hyperparameter space in search of values that optimize the performance of the entire process of automatic impulsive sound classification. This process begins with the generation of the dataset to be processed, continues with the training of classification models, and ends with the evaluation of the learned models. The experiments consider a binary classification problem, where a distinction must be made between the intended event and noise. The validation of the process resorts to an audio extracted from videos of sports event competitions, specifically in tennis and padel, where the goal is to identify the sounds of racket hits on the ball. This work is currently in progress, but the preliminary results already enable to evidence the impact of hyperparameter optimization on the accuracy of the overall learning process. [ABSTRACT FROM AUTHOR]

Auditory data has become ubiquitous in tourism and hospitality, especially with digitalization. However, a lack of feasible frameworks for analysing auditory data creates barriers to tourism and hospitality research using auditory data. This study proposed a framework to address the methodological challenges of analysing auditory data in tourism and hospitality. The framework is applied in a digital audio interpretation setting to demonstrate its feasibility and the value of audio features. The proposed framework offers new directions for advancing tourism and hospitality research using unstructured auditory data in various contexts. [ABSTRACT FROM AUTHOR]

Respiratory diseases represent a significant global burden, necessitating efficient diagnostic methods for timely intervention. Digital biomarkers based on audio, acoustics, and sound from the upper and lower respiratory system, as well as the voice, have emerged as valuable indicators of respiratory functionality. Recent advancements in machine learning (ML) algorithms offer promising avenues for the identification and diagnosis of respiratory diseases through the analysis and processing of such audio-based biomarkers. An ever-increasing number of studies employ ML techniques to extract meaningful information from audio biomarkers. Beyond disease identification, these studies explore diverse aspects such as the recognition of cough sounds amidst environmental noise, the analysis of respiratory sounds to detect respiratory symptoms like wheezes and crackles, as well as the analysis of the voice/speech for the evaluation of human voice abnormalities. To provide a more in-depth analysis, this review examines 75 relevant audio analysis studies across three distinct areas of concern based on respiratory diseases' symptoms: (a) cough detection, (b) lower respiratory symptoms identification, and (c) diagnostics from the voice and speech. Furthermore, publicly available datasets commonly utilized in this domain are presented. It is observed that research trends are influenced by the pandemic, with a surge in studies on COVID-19 diagnosis, mobile data acquisition, and remote diagnosis systems. [ABSTRACT FROM AUTHOR]

Using automated models to analyze classroom discourse is a valuable tool for educators to improve their teaching methods. In this paper, we focus on exploring alternatives to ensure the generalizability of models for identifying teaching practices across diverse teaching contexts. Our proposal utilizes artificial intelligence to analyze audio recordings of classroom activities. By leveraging deep learning for speaker diarization and traditional machine learning algorithms for classifying teaching practices, we extract features from the audio diarization using a processing pipeline to provide detailed insights into teaching dynamics. These features enable the classification of three distinct teaching practices: lectures, group discussions, and the use of audience response systems. Our findings demonstrate that these features effectively capture the nuances of teacher-student interactions, allowing for a refined analysis of teaching styles. To enhance the robustness and generalizability of our model, we explore various pipelines for audio processing, evaluating the model’s performance across diverse contexts involving different teachers and students. By comparing these practices and their associated features, we illustrate how AI-driven tools can support teachers in reflecting on and improving their teaching strategies.

SecureVision is an advanced and trustworthy deepfake detection system created to tackle the growing threat of ‘deepfake’ movies that tamper with media, undermine public trust, and jeopardize cybersecurity. We present a novel approach that combines big data analytics with state-of-the-art deep learning algorithms to detect altered information in both audio and visual domains. One of SecureVision’s primary innovations is the use of multi-modal analysis, which improves detection capabilities by concurrently analyzing many media forms and strengthening resistance against advanced deepfake techniques. The system’s efficacy is further enhanced by its capacity to manage large datasets and integrate self-supervised learning, which guarantees its flexibility in the ever-changing field of digital deception. In the end, this study helps to protect digital integrity by providing a proactive, scalable, and efficient defense against the ubiquitous threat of deepfakes, thereby establishing a new benchmark for privacy and security measures in the digital era.

Social robots in the home will need to solve audio identification problems to better interact with their users. This article focuses on the classification between (a) natural conversation that includes at least one co-located user and (b) media that is playing from electronic sources and does not require a social response, such as television shows. This classification can help social robots detect a user's social presence using sound. Social robots that are able to solve this problem can apply this information to assist them in making decisions, such as determining when and how to appropriately engage human users. We compiled a dataset from a variety of acoustic environments that contained either natural or media audio, including audio that we recorded in our own homes. Using this dataset, we performed an experimental evaluation on a range of traditional machine learning classifiers and assessed the classifiers' abilities to generalize to new recordings, acoustic conditions, and environments. We conclude that a C-Support Vector Classification (SVC) algorithm outperformed other classifiers. Finally, we present a classification pipeline that in-home robots can utilize, and we discuss the timing and size of the trained classifiers as well as privacy and ethics considerations. [ABSTRACT FROM AUTHOR]

Stress can be considered a mental/physiological reaction in conditions of high discomfort and challenging situations. The levels of stress can be reflected in both the physiological responses and speech signals of a person. Therefore the study of the fusion of the two modalities is of great interest. For this cause, public datasets are necessary so that the different proposed solutions can be comparable. In this work, a publicly available multimodal dataset for stress detection is introduced, including physiological signals and speech cues data. The physiological signals include electrocardiograph (ECG), respiration (RSP), and inertial measurement unit (IMU) sensors equipped in a smart vest. A data collection protocol was introduced to receive physiological and audio data based on alterations between well-known stressors and relaxation moments. Five subjects participated in the data collection, where both their physiological and audio signals were recorded by utilizing the developed smart vest and audio recording application. In addition, an analysis of the data and a decision-level fusion scheme is proposed. The analysis of physiological signals includes a massive feature extraction along with various fusion and feature selection methods. The audio analysis comprises a state-of-the-art feature extraction fed to a classifier to predict stress levels. Results from the analysis of audio and physiological signals are fused at a decision level for the final stress level detection, utilizing a machine learning algorithm. The whole framework was also tested in a real-life pilot scenario of disaster management, where users were acting as first responders while their stress was monitored in real time. [ABSTRACT FROM AUTHOR]

Forensic audio analysis is a foundation stone of many crime investigations. In forensic evidence; the audio file of the human voice is analyzed to extract much information in addition to the content of the speech, such as the speaker's identity, emotions, gender, origin, etc. The accurate determination of individuals into groups based on their age development stage and their gender are often used as early investigations to differentiate them and determine the legal rights and responsibilities associated with them. This work introduces a light CNN model with a new architecture to detect the human age-evolution being's stage (kids or adults) at the same time the gender of the adult one (male or female) based on the individual's voice characteristics, which offers a balance between computational efficiency and model accuracy. The temporal information in the audio file is prepared by scaled and normalized. Then this information is exploited to extract and track the unique and salient audio features that make up the pattern of the feature map for each target class through some convolutional layers followed by maxpooling layers. Finally, The decision is made based on these feature maps by some fully connected layers. Successful and promising results are accomplished in terms of accuracy and loss functions which realize 0.99 and 0.017 respectively over the riched Voxceleb2 dataset. The proposed model underscores the importance of leveraging Light DNNs for gender and age-evolution detection, offering a robust and ethically sound solution for real-world applications in the field of audio forensics such as span speaker identification, victim profiling, deception detection, and more, contributing to the advancement of audio forensic analysis. [ABSTRACT FROM AUTHOR]

Affected by the COVID-19 epidemic, the final examinations at many universities and the recruitment interviews of enterprises were forced to be transferred to online remote video invigilation, which undoubtedly improves the space and possibility of cheating. To solve these problems, this paper proposes an intelligent invigilation system based on the EfficientDet target detection network model combined with a centroid tracking algorithm. Experiments show that cheating behavior detection model proposed in this paper has good detection, tracking and recognition effects in remote testing scenarios. Taking the EfficientDet network as the detection target, the average detection accuracy of the network is 81%. Experiments with real online test videos show that the cheating behavior detection accuracy can reach 83.1%. In addition, to compensate for the shortage of image detection, we also design an audio detection module to carry out auxiliary detection and forensics. The audio detection module is used to continuously detect the environmental sound of the examination room, save suspicious sounds and provide evidence for judging cheating behavior. [ABSTRACT FROM AUTHOR]

Respiratory conditions have been a focal point in recent medical studies. Early detection and timely treatment are crucial factors in improving patient outcomes for any medical condition. Traditionally, doctors diagnose respiratory conditions through an investigation process that involves listening to the patient’s lungs. This study explores the potential of combining audio analysis with convolutional neural networks to detect respiratory conditions in patients. Given the significant impact of proper hyperparameter selection on network performance, contemporary optimizers are employed to enhance efficiency. Moreover, a modified algorithm is introduced that is tailored to the specific demands of this study. The proposed approach is validated using a real-world medical dataset and has demonstrated promising results. Two experiments are conducted: the first tasked models with respiratory condition detection when observing mel spectrograms of patients’ breathing patterns, while the second experiment considered the same data format for multiclass classification. Contemporary optimizers are employed to optimize the architecture selection and training parameters of models in both cases. Under identical test conditions, the best models are optimized by the introduced modified metaheuristic, with an accuracy of 0.93 demonstrated for condition detection, and a slightly reduced accuracy of 0.75 for specific condition identification.

Respiratory diseases represent a significant global burden, necessitating efficient diagnostic methods for timely intervention. Digital biomarkers based on audio, acoustics, and sound from the upper and lower respiratory system, as well as the voice, have emerged as valuable indicators of respiratory functionality. Recent advancements in machine learning (ML) algorithms offer promising avenues for the identification and diagnosis of respiratory diseases through the analysis and processing of such audio-based biomarkers. An ever-increasing number of studies employ ML techniques to extract meaningful information from audio biomarkers. Beyond disease identification, these studies explore diverse aspects such as the recognition of cough sounds amidst environmental noise, the analysis of respiratory sounds to detect respiratory symptoms like wheezes and crackles, as well as the analysis of the voice/speech for the evaluation of human voice abnormalities. To provide a more in-depth analysis, this review examines 75 relevant audio analysis studies across three distinct areas of concern based on respiratory diseases’ symptoms: (a) cough detection, (b) lower respiratory symptoms identification, and (c) diagnostics from the voice and speech. Furthermore, publicly available datasets commonly utilized in this domain are presented. It is observed that research trends are influenced by the pandemic, with a surge in studies on COVID-19 diagnosis, mobile data acquisition, and remote diagnosis systems.

Predictive Maintenance (PdM) has a main role in the Fourth Industrial Revolution; its goal is to design models that can safely detect failure in systems before they fail, aiming to reduce financial, environmental, and operational costs. A brushless DC (BLDC) electric motors have increasingly become more popular and been gaining popularity in industrial applications, so their analysis for PdM applications is only a natural progression; audio analysis proves to be a useful method to achieve this and rises as a very pragmatic case of study of the characteristics of the motors. The main goal of this paper is to showcase sound-based behavior of BLDC motors in different failure modes as result of an experiment led by researchers at Universidad del Cauca in Colombia. This dataset may provide researchers with useful information regarding signal processing and the development of Machine Learning applications that would achieve an improvement within Predictive Maintenance and I4.0.Predictive Maintenance (PdM) has a main role in the Fourth Industrial Revolution; its goal is to design models that can safely detect failure in systems before they fail, aiming to reduce financial, environmental, and operational costs. A brushless DC (BLDC) electric motors have increasingly become more popular and been gaining popularity in industrial applications, so their analysis for PdM applications is only a natural progression; audio analysis proves to be a useful method to achieve this and rises as a very pragmatic case of study of the characteristics of the motors. The main goal of this paper is to showcase sound-based behavior of BLDC motors in different failure modes as result of an experiment led by researchers at Universidad del Cauca in Colombia. This dataset may provide researchers with useful information regarding signal processing and the development of Machine Learning applications that would achieve an improvement within Predictive Maintenance and I4.0.

Several excavations at the Hemudu site in Zhejiang Province have led to the discovery of more than 100 pieces of bird bones that were likely used to create a kind of musical instrument. However, in contrast to the Jiahu bone flute, it is not clear for what the Hemudu bone whistle was used, although it was probably used for musical purposes. In this paper, the 7000-year-old Hemudu bone whistle is reproduced with 3D scanning and printing technology. Therefore, we propose a robust scan point detection and tracking method that includes three main stages. First, we conducted robust point detection and selection where noise and irrelevant points were removed. Then, we propose to track the key points and conduct 3D model optimization. With the above efforts, we obtained the 3D model of the bone whistle, and the audio of the bone whistle is replicated. The purpose of the bone whistles is inferred and evaluated through audio analyses. The experimental results show that the method is feasible.

Microphone identification is a crucial challenge in the field of digital audio forensics. The ability to accurately identify the type of microphone used to record a piece of audio can provide important information for forensic analysis and crime investigations. In recent years, transformer-based deep-learning models have been shown to be effective in many different tasks. This paper proposes a system based on a transformer for microphone identification based on recorded audio. Two types of experiments were conducted: one to identify the model of the microphones and another in which identical microphones were identified within the same model. Furthermore, extensive experiments were performed to study the effects of different input types and sub-band frequencies on system accuracy. The proposed system is evaluated on the Audio Forensic Dataset for Digital Multimedia Forensics (AF-DB). The experimental results demonstrate that our model achieves state-of-the-art accuracy for inter-model and intra-model microphone classification with 5-fold cross-validation. [ABSTRACT FROM AUTHOR]

Asthma is a common, usually long-term respiratory disease with negative impact on global society and economy. Treatment involves using medical devices (inhalers) that distribute medication to the airways and its efficiency depends on the precision of the inhalation technique. There is a clinical need for objective methods to assess the inhalation technique, during clinical consultation. Integrated health monitoring systems, equipped with sensors, enable the recognition of drug actuation, embedded with sound signal detection, analysis and identification from intelligent structures, that could provide powerful tools for reliable content management. Health monitoring systems equipped with sensors, embedded with sound signal detection, enable the recognition of drug actuation and could be used for effective audio content analysis. This paper revisits sound pattern recognition with machine learning techniques for asthma medication adherence assessment and presents the Respiratory and Drug Actuation (RDA) Suite (https://gitlab.com/vvr/monitoring-medication-adherence/rda-benchmark) for benchmarking and further research. The RDA Suite includes a set of tools for audio processing, feature extraction and classification procedures and is provided along with a dataset, consisting of respiratory and drug actuation sounds. The classification models in RDA are implemented based on conventional and advanced machine learning and deep networks’ architectures. This study provides a comparative evaluation of the implemented approaches, examines potential improvements and discusses on challenges and future tendencies.

In this paper, we advocate in favor of smartphone apps as low-cost, easy-to-deploy solutions for raising awareness among the population on the proliferation of Aedes aegypti mosquitoes. Nevertheless, devising such a smartphone app is challenging for many reasons, including the required maturity level of techniques for identifying mosquitoes based on features that can be captured using smartphone resources. In this paper, we identify a set of (non-exhaustive) requirements that smartphone apps must meet to become an effective tooling in the fight against Ae. aegypti , and advance the state-of-the-art with (i) a dataset for benchmarking solutions for detecting Ae. aegypti mosquitoes from wingbeat sound recordings collected with smartphones, and (ii) a residual convolutional neural network for classifying Ae. aegypti mosquitoes from their wingbeat sound, considering a training methodology for reducing the influence of background noise in the classification process. From the analysis of precision, accuracy, recall, and F1-score, we provide evidence that residual convolutional neural networks have potential as a cornerstone for tracking mosquito apps for smartphones. We demonstrate the proposed solution's feasibility by implementing a prototype capable of detecting mosquitoes in a controlled environment. • Residual CNN-based models running in a Smartphone App can identify Ae. aegypti. • Residual and classification blocks made classification suitable for Smartphones. • The proposed topology reached an accuracy as high as 98% and F1-measure of 90%. • These results suggest the potential of detecting Ae. aegypti using smartphone apps. • Such apps could raise awareness and help fight nearby Ae. aegypti breeding sites. [ABSTRACT FROM AUTHOR]

With the emergence of new digital technologies, a significant surge has been seen in the volume of multimedia data generated from various smart devices. Several challenges for data analysis have emerged to extract useful information from multimedia data. One such challenge is the early and accurate detection of anomalies in multimedia data. This study proposes an efficient technique for anomaly detection and classification of rare events in audio data. In this paper, we develop a vast audio dataset containing seven different rare events (anomalies) with 15 different background environmental settings (e.g., beach, restaurant, and train) to focus on both detection of anomalous audio and classification of rare sound (e.g., events—baby cry, gunshots, broken glasses, footsteps) events for audio forensics. The proposed approach uses the supreme feature extraction technique by extracting mel-frequency cepstral coefficients (MFCCs) features from the audio signals of the newly created dataset and selects the minimum number of best-performing features for optimum performance using principal component analysis (PCA). These features are input to state-of-the-art machine learning algorithms for performance analysis. We also apply machine learning algorithms to the state-of-the-art dataset and realize good results. Experimental results reveal that the proposed approach effectively detects all anomalies and superior performance to existing approaches in all environments and cases.

We introduce a method for computer-assisted analysis of orchestration. We also look into the role that texture and orchestration have in structuring musical form. The method comprises a numerical representation, a hierarchy of ‘textural situations’ and measures for heterogeneity, diversity and complexity of orchestral-textural configurations.

As the voice disorder is a typical early symptom of Parkinson, some researchers attempt to diagnose this disease based on voice data collected from suspected patients. Although existing methods can provide acceptable results, they just work in partial scenarios. In other words, they are not generable and robust enough. To this end, we present a Parkinson’s auxiliary diagnosis system based on human speech, which can adaptively build a suitable deep neural network based on sound features. The system includes two modules: hybrid features extraction and adaptive network construction. We extract kinds of information from the voice data to form a new compound feature. Furthermore, particle swarm optimization (PSO) algorithm is employed to build the corresponding 1D convolution network for features classification. Extensive experiments on two datasets consisting of English and Italian are conducted for evaluation purposes. Experimental results show that our method improves the accuracy of voice-based Parkinson’s disease detection to some extent.

Recently, attention of the emotional speech signals research has been boosted in human machine interfaces due to availability of high computation capability. Selection of suitable feature sets, design of a proper classification methods and prepare an appropriate dataset are the main key issues of speech emotion recognition systems. Detecting emotions is one of the most important marketing strategies in today’s world. You could personalize different things for an individual specifically to suit their interest. Once machines have the capability of understanding the emotions of a person, it will greatly enhance the user experience. In this report, we will try to classify the emotion of a voice (audio clip) using different algorithms of Artificial Intelligence (AI). Based on the accuracy rates, a suitable choice could be made to make such application in the future. [ABSTRACT FROM AUTHOR]

On the one hand facial expression recognition (FER) of a human to determine his emotional state is a wellstudied task. On the other hand, most of the existing approaches are focused on recognition of basic emotions or binary labels of emotional tone (positive or negative). However, facial micro-expression (FMiE) recognition (FMiER) is an understudied but growing area. The interest of researchers in this area is due to the unsolved problems of determining the subtle intentions of humans in situations with high stakes, such as lie detection, state of mind recognition, negotiation techniques recognition, human behavior patterns prediction. In contrast to facial macro-expressions FMiEs are involuntary and transient facial expressions capable of revealing the genuine feelings that humans attempt to hide. These differences are reasons why FMiE detection relies heavily on expert experience and why FMiER is a difficult task which still requires additional research. In addition to FER, the human voice also contains a lot of information about the internal state of a human and speech emotion recognition is an actively studied task as well. In this paper we consider the problem of lie detection, however we soften it to the problem of assessing the confidence of a person in a video. We hypothesized that the use of information from two modalities and the joint exploring of this information through modalities can allow to predict the subtle feelings of a human with high accuracy. We introduce a solution that extracts and analyzes person's facial and speech features on a video sequence using attention mechanisms in the task of truth determination. Our approach uses state-of-the-art methods and achieves 84.91% accuracy. The survey also includes a comparison of various audio and visual features combinations and shows that our approach shows comparable accuracy-performance ratio. [ABSTRACT FROM AUTHOR]