Search

Traditional sentiment analysis focuses on text-level sentiment mining, transforming sentiment mining into classification or regression problems, resulting in a sentiment analysis low accuracy rate. Sentiment analysis refers to the use of natural language processing, text analysis, and computational linguistics to systematically identify, extract, quantify, and study sentimental states. Therefore, more scholars have begun to focus on speech recognition and facial expression recognition research, and extracting and analysing people's sentiment tendencies can improve sentiment recognition accuracy. Traditional single-modal sentiment analysis can no longer meet people's needs. Therefore, this paper proposes a multimodal sentiment analysis method based on the multimodal sentiment analysis method that can obtain more sentimental information sources and help people make better decisions. The experimental results in this paper show that the highest recognition rates of CNN-SVM, RNN-SVM, and CRNN-SVM were 76.8%, 71.2%, and 93.5%, respectively. It can be seen that CRNN-SVM has the highest sentiment tendency recognition rate in deep learning, so it is suitable to apply CRNN-SVM to sentiment tendency analysis system design in this paper. The average accuracy rate of the system designed in this paper was 91%, and the stability was also very strong, which shows that the system designed in this paper is meaningful. The main contribution of this paper is based on the limitations of single-mode emotion analysis. It proposes a multimode emotion analysis method and introduces a convolutional neural network to help people obtain more emotional information sources to meet their needs. [ABSTRACT FROM AUTHOR]

As the listening capacity exists in humans, they are facing the critical issues of understanding the speeches even in the presence of some background or other noises in the world. To diminish the noises, Speech Enhancement (SE) is the process of improving the quality of the speech signal by applying some techniques without degrading any such information. Hence, it is used for hearing-aid people, speech recognition, etc. Recent researchers have developed some works to increase speech intelligibility. Substantial success has been achieved by executing the supervised learning methods. Nevertheless, the existing process incurs such shortcomings as attaining maximum error, computation burden, and so on. To overcome that, certain deep learning methodologies are immensely involved in SE for determining the spectrogram magnitude when reconstructing the signal source by removing the noise. Thus, it also becomes a more challenging task to acquire a clean speech signal. Though these methods aim to present the speech as more clear and intelligible, it may arise such intricacies that degrade the quality and efficiency. As it contains beneficial structure and resources, still it is in the scope of developing the novel SE model. To conquer these complexities, a successful SE task is offered utilizing deep learning in this paper. This recommended work performs the SE which employs deep learning to denoise a noisy speech to generate a quality speech. At first, the speech signal that contains noises such as cooler noise or fan noise, restaurant noise, railway station noise, factory noise, traffic in the journey, bus-stand noise, cinema theater noise, and clouding areas noise are gathered from the standard online sources. After that, this noisy speech signal is forwarded to Adaptive Residual Convolutional Neural Networks with Encoder-Decoder (ARCNNetED) architecture, where the parameters involved in this framework are optimized with the support of Random Revised Drawer Algorithm (RRDA). Thus, the noise presented in the input speech signal is completely removed by the suggested ARCNNetED and the quality of the speech also is enhanced. Finally, the performance of the suggested speech enhancement approach is evaluated over various traditional tasks with the support of several metrics. The findings of the developed model show better performance in terms of MAE, RMSE, and PSNR showing the value of 0.03, 0.19, and 62.26. This analysis significantly handles the error rate to offer accurate outcomes in the speech recognition framework. [ABSTRACT FROM AUTHOR]

Research on voice recognition for African languages is limited due to the scarcity of digital resources for training and adaptation, despite its broad usefulness. The Hausa language, spoken by almost fifty million inhabitants in West and Central Africa, is an example of a linguistic domain that has not been thoroughly studied. The Hausa language employs diacritics, which are symbols located above alphabetical characters to convey further information. By removing diacritics, the number of homographs increases, making it difficult to distinguish between similar words. This paper presents a study on speech recognition in the Hausa Language, specifically focusing on diacritized words. The study utilises the state-of-the-art wave2vec2.0 and Whisper deep learning architecture models, for transcribing audio signals into corresponding Hausa text. According to the results obtained in the study, the Whisper-large deep model emerged as the best, achieving a word error rate of 4.23% representing a considerable improvement of 43.9% when compared to the existing state-of-the-art model for Hausa language speech recognition. Additionally, the Whsiper-large model demonstrated a diacritic coverage of 92%, precision of 98.87%, with a diacritic error rate of 2.1%. [ABSTRACT FROM AUTHOR]

Deep learning is a rapidly growing research area having state of art achievement in various applications including but not limited to speech recognition, object recognition, machine translation, and image segmentation. In the current modern industrial manufacturing system, Machine Health Surveillance System (MHSS) is achieving increasing popularity because of the widespread availability of low cost sensors internet connectivity. Deep learning architecture gives useful tools to analyze and process these vast amounts of machinery data. In this paper, we review the latest deep learning techniques and their variant used for MHSS. We used Gearbox Fault Diagnosis dataset in this paper that contains the sets of vibration attributes recorded by SpectraQuest's Gearbox Fault Diagnostics Simulator. In addition, we used the variant of auto encoders for feature extraction to achieve higher accuracy in machine health surveillance. The results showed that the bagging ensemble classifier based on voting techniques achieved 99% accuracy. [ABSTRACT FROM AUTHOR]

Due to the limitations of English learning environment and teaching conditions in China, English learners in China generally have difficulties in oral English learning.With the development of computer science and technology and the progress of language teaching and learning methods, it is possible to solve this problem.Therefore, this paper applies the mature deep learning technology to English speech recognition and constructs a speech evaluation model.Through experimental research, it is found that the intonation, speech speed and intonation evaluation methods adopted in this paper are highly robust and the English pronunciation quality evaluation model is also feasible. It can effectively promote the development of foreign language teaching in China. [ABSTRACT FROM AUTHOR]

The field of speech recognition makes it simpler for humans and machines to engage with speech. Number-oriented communication, such as using a registration code, mobile number, score, or account number, can benefit from speech recognition for digits. This paper presents our Amazigh automatic speech recognition (ASR) experience based on the deep learning approach. The convolutional neural network (CNN) and Mel spectrogram are exploited to evaluate audio samples and produce spectrograms as a part of the deep learning strategy. To attempt the recognition of the Amazigh numerals, we use a database that includes digits ranging from zero to nine collected from 42 native speakers in total, men and women between the ages of 20 and 40. Our experimental results show that spoken digits in Amazigh can be identified with a maximum accuracy of 93.62%, 94% Precision, and 94% Recall. [ABSTRACT FROM AUTHOR]

The implementation of a defect speech recognition (DSR) system has the opportunity to significantly improve the lifestyle of people with speech disorders. In this paper, we developed a novel ConvGRUSpeechNet model for recognizing and understanding hyperkinetic dysarthria disorder (HDD) speech. The proposed model uniquely combines convolutional layers, recurrent layers (GRU and BiGRU), and dense layers with a LogSoftmax function to effectively recognize and translate HDD speech into text. To prevent overfitting and handling imbalances, we employed data augmentation and splitting functions during the training process. Also, the Mel-frequency cepstral coefficients (MFCC) were employed to reduce the issue of vanishing gradients. In addition, a dataset of Russian speech has been created, comprising 2000 recordings of HDD speech. The primary objective of this research is to improve speech recognition for individuals with HDD by employing the ConvGRUSpeechNet model. The proposed DSR system outperformed the recognition character error rate (CER) of 12.35% using the test dataset. Under the same conditions, the experimental findings show that the proposed solution exhibits superior performance in comparison to existing state-of-the-art CBNs and TDNN-F LF-MMI models. Furthermore, we implemented the TensorFlow model on a flask server, making it accessible for use in a web application. [ABSTRACT FROM AUTHOR]

This research paper proposes sentiment analysis of pets using deep learning technologies in the artificial intelligence of things system. Mask R-CNN is used to detect image objects and generate contour mask maps, pose analysis algorithms are used to obtain object posture information, and at the same time object sound signals are converted into spectrograms as features, using deep learning image recognition technology to obtain object emotion information. By using the fusion of object posture and emotional characteristics as the basis for pet emotion identification and analysis, the detected specific pet behaviour states will be actively notified to the owner for processing. Compared with traditional speech recognition, which uses mel-frequency cepstral coefficients for feature extraction, coupled with a Gaussian mixture model-hidden Markov model for voice recognition, the experimental method of this research paper effectively improves the accuracy by 70%. Prior work on the implementation of smart pet surveillance systems has used the pet's tail and mouth as features, and has combined these with sound features to analyse the pet's emotions. This research paper proposes a new method of sentiment analysis in pets, and our method is compared with previous related work. Experimental results show that our approach increases the accuracy rate by 70%. [ABSTRACT FROM AUTHOR]

Speech enhancement is an important method for improving speech quality and intelligibility in noisy environments. An effective speech enhancement model depends on precise modelling of the long-range dependencies of noisy speech. Several recent studies have examined ways to enhance speech by capturing the long-term contextual information. For speech enhancement, the time–frequency (T–F) distribution of speech spectral components is also important, but is usually ignored in these studies. The multi-stage learning method is an effective way to integrate various deep learning modules at the same time. The benefit of multi-stage training is that the optimization target can be iteratively updated stage by stage. In this paper, speech enhancement is investigated by multi-stage learning using a multi-stage structure in which time–frequency attention (TFA) blocks are followed by stacks of squeezed temporal convolutional networks (S-TCN) with exponentially increasing dilation rates. To reinject original information into later stages, a feature fusion block (FB) is inserted at the input of later stages to reduce the possibility of speech information being lost in the early stages. The S-TCN blocks are responsible for temporal sequence modelling tasks. The time–frequency attention (TFA) is a simple but effective network module that explicitly exploits position information to generate a 2D attention map to characterize the salient T–F distribution of speech by using two branches, time-frame attention and frequency attention in parallel. Extensive experiments have demonstrated that the proposed model consistently improves the performance over existing baselines across two widely used objective metrics such as PESQ and STOI. A significant improvement in system robustness to noise is also shown by our evaluation results using the TFA module. [ABSTRACT FROM AUTHOR]

Stock market is considered chaotic, complex, volatile and dynamic. Undoubtedly, its prediction is one of the most challenging tasks in time series forecasting. Moreover existing Artificial Neural Network (ANN) approaches fail to provide encouraging results. Meanwhile advances in machine learning have presented favourable results for speech recognition, image classification and language processing. Methods applied in digital signal processing can be applied to stock data as both are time series. Similarly, learning outcome of this paper can be applied to speech time series data. Deep learning for stock prediction has been introduced in this paper and its performance is evaluated on Google stock price multimedia data (chart) from NASDAQ. The objective of this paper is to demonstrate that deep learning can improve stock market forecasting accuracy. For this, (2D)PCA + Deep Neural Network (DNN) method is compared with state of the art method 2-Directional 2-Dimensional Principal Component Analysis (2D)PCA + Radial Basis Function Neural Network (RBFNN). It is found that the proposed method is performing better than the existing method RBFNN with an improved accuracy of 4.8% for Hit Rate with a window size of 20. Also the results of the proposed model are compared with the Recurrent Neural Network (RNN) and it is found that the accuracy for Hit Rate is improved by 15.6%. The correlation coefficient between the actual and predicted return for DNN is 17.1% more than RBFNN and it is 43.4% better than RNN. [ABSTRACT FROM AUTHOR]

The Cocktail party problem, which is tracing and identifying a specific speaker's speech while numerous speakers communicate concurrently is one of the crucial problems still to be addressed for automated speech recognition (ASR) and speaker recognition. In this study, we attempt to thoroughly explore traditional methods for speech separation in a cocktail party environment and further analyze traditional single-channel methods for example source-driven methods like Computational Auditory Scene Analysis (CASA), data-driven methods like non-negative matrix factorization (NMF), model-driven methods, customary multi-channel methods such as beamforming, blind source separation for multi-channel and the newly developed deep learning approaches such as meta-learning based methods, self-supervised learning. This paper further accentuates numerous datasets and evaluation metrics in the domain of speech processing & brings out the comparison between traditional methods and methods based on deep learning for speech separation. This study provides a basic understanding and comprehensive knowledge of state-of-the-art researches in the area of speech separation and serves as a brief overview to new researchers. [ABSTRACT FROM AUTHOR]

Social media platforms like Facebook, YouTube, and Twitter are banking on developing machine learning models to help stop the spread of hateful speech on their platforms. The idea is that machine learning models that utilize natural language processing will detect hate speech faster and better than people can. Despite numerous progress has been made for resource reach language, only a few attempts have been made for Ethiopian Languages such as Afaan Oromo. This paper examines the viability of deep learning models for Afaan Oromo hate speech recognition. Toward this, the biggest dataset of hate speech was collected and annotated by the language experts. Variations of profound deep learning models such as CNN, LSTMs, BiLSTMs, LSTM, GRU, and CNN-LSTM are examined to evaluate their viability in identifying Afaan Oromo Hate speeches. The result uncovers that the model dependent on CNN and Bi-LSTM outperforms all the other investigated models with an average F1-score of 87%. [ABSTRACT FROM AUTHOR]

Up to now, various methods are used for Automatic Speech Recognition (ASR), and among which the Hidden Markov Model (HMM) and Artificial Neural Networks (ANNs) are the most important ones. One of the existing challenges is increasing the accuracy and efficiency of these systems. One way to enhance the accuracy of them is by improving the acoustic model (AM). In this paper, for the first time, the combination of deep belief network (DBN), for extracting features of speech signals, and Deep Bidirectional Long Short-Term Memory (DBLSTM) with Connectionist Temporal Classification (CTC) output layer is used to create an AM on the Farsdat Persian speech data set. The obtained results show that the use of a deep neural network (DNN) compared to a shallow network improves the results. Also, using the bidirectional network increases the accuracy of the model in comparison with the unidirectional network, in both deep and shallow networks. Comparing obtained results with the HMM and Kaldi-DNN indicates that using DBLSTM with features extracted from the DBN increases the accuracy of Persian phoneme recognition. [ABSTRACT FROM AUTHOR]

Parkinson's disease (PD) is a condition that degenerates over time and impairs speech and pronunciation because brain cells have died. This research work aims to predict parkinson disease using the voice features extracted from speech signals recorded from PD individuals with dysphonic speech disorders by employing deep learning algorithms. PD is challenging to diagnose early on in the clinical presentation. To address the issue in machine learning methods, this paper proposes a neural network model by processing speech signals to classify PD using the University of California Irvine (UCI) machine learning repository dataset. Initially, a pre-loss reduction module is created by using pre-sampling to make the dataset balanced by reducing the dimensionality and maintaining the size of the space without influencing the learning process for data preparation. The relevant features are derived using a novel multi-agent salp swarm (MASS) algorithm, and a novel Parkinson classification neural network (PCNN) is proposed to classify Parkinson's patients with high accuracy employing these derived features. The result shows that the models that use MASS-PCNN produce higher classification accuracy of 99.1%, precision of 97.8%, recall of 94.7% and F1-score of 0.995 when paralleled to the existing models. As an outcome, the suggested model will perform superior to common convolutional neural networks. [ABSTRACT FROM AUTHOR]

With the development of deep learning techniques, the field of direction of arrival (DOA) estimation has also made significant progress. However, the accuracy of DOA estimation using end-to-end neural networks (NNs) heavily relies on the classification step of the networks, which necessitates the use of large and representative datasets. Additionally, conventional speech presence probability (SPP) estimation methods based on the ideal ratio mask (IRM) may misclassify time-frequency (T-F) bins dominated by non-speech and noise, which hinders the accurate extraction of directional information. To improve the robustness of existing DOA estimation algorithms, this paper proposes a DOA estimation method with T-F bin selection. In terms of output, instead of using IRM-based SPP, our proposed approach focuses on the a posteriori SPP, a deliberate choice aimed at circumventing potential confusion. For input optimization, we construct features that encompass spatial, temporal, and directional information concurrently, and these are coupled with a frequency bin-wise recurrent neural network (RNN) model to attain precise multi-channel SPP estimation. Subsequently, these SPP estimates are utilized to extract local information for DOA estimation. Moreover, the cascaded structure ensures that the model has the ability to complete out-of-label tasks, effectively reducing the dataset requirements by training only a subset of direction information to achieve omnidirectional DOA estimation. Besides, this contributes to the algorithm's ability to eliminate its reliance on the step size, setting it apart from other end-to-end methods. Simulation results validate that the proposed method achieves higher accuracy and lower error compared to both NN-based end-to-end approaches and traditional full-band approaches under various conditions of reverberation and signal-to-noise ratio. [ABSTRACT FROM AUTHOR]

Deep learning has been widely adopted in automatic emotion recognition and has lead to significant progress in the field. However, due to insufficient training data, pre-trained models are limited in their generalisation ability, leading to poor performance on novel test sets. To mitigate this challenge, transfer learning performed by fine-tuning pr-etrained models on novel domains has been applied. However, the fine-tuned knowledge may overwrite and/or discard important knowledge learnt in pre-trained models. In this paper, we address this issue by proposing a PathNet-based meta-transfer learning method that is able to (i) transfer emotional knowledge learnt from one visual/audio emotion domain to another domain and (ii) transfer emotional knowledge learnt from multiple audio emotion domains to one another to improve overall emotion recognition accuracy. To show the robustness of our proposed method, extensive experiments on facial expression-based emotion recognition and speech emotion recognition are carried out on three bench-marking data sets: SAVEE, EMODB, and eNTERFACE. Experimental results show that our proposed method achieves superior performance compared with existing transfer learning methods. [ABSTRACT FROM AUTHOR]

Multimodal fusion is the idea of combining information in a joint representation of multiple modalities. The goal of multimodal fusion is to improve the accuracy of results from classification or regression tasks. This paper proposes a Bimodal Variational Autoencoder (BiVAE) model for audiovisual features fusion. Reliance on audiovisual signals in a speech recognition task increases the recognition accuracy, especially when an audio signal is corrupted. The BiVAE model is trained and validated on the CUAVE dataset. Three classifiers have evaluated the fused audiovisual features: Long-short Term Memory, Deep Neural Network, and Support Vector Machine. The experiment involves the evaluation of the fused features in the case of whether two modalities are available or there is only one modality available (i.e., cross-modality). The experimental results display the superiority of the proposed model (BiVAE) of audiovisual features fusion over the state-of-the-art models by an average accuracy difference ≃ 3.28% and 13.28% for clean and noisy, respectively. Additionally, BiVAE outperforms the state-of-the-art models in the case of cross-modality by an accuracy difference ≃ 2.79% when the only audio signal is available and 1.88% when the only video signal is available. Furthermore, SVM satisfies the best recognition accuracy compared with other classifiers. [ABSTRACT FROM AUTHOR]

The rapid spread of the novel corona virus disease (COVID-19) has disrupted the traditional clinical services all over the world. Hospitals and healthcare centers have taken extreme care to minimize the risk of exposure to the virus by restricting the visitors and relatives of the patients. The dramatic changes happened in the healthcare norms have made it hard for the deaf patients to communicate and receive appropriate care. This paper reports a work on automatic sign language recognition that can mitigate the communication barrier between the deaf patients and the healthcare workers in India. Since hand gestures are the most expressive components of a sign language vocabulary, a novel dataset of dynamic hand gestures for the Indian sign language (ISL) words commonly used for emergency communication by deaf COVID-19 positive patients is proposed. A hybrid model of deep convolutional long short-term memory network has been utilized for the recognition of the proposed hand gestures and achieved an average accuracy of 83.36%. The model performance has been further validated on an alternative ISL dataset as well as a benchmarking hand gesture dataset and obtained average accuracies of 97 % and 99.34 ± 0.66 % , respectively. [ABSTRACT FROM AUTHOR]

Automatic speech recognition systems based on end-to-end models (E2E-ASRs) can achieve comparable performance to conventional ASR systems while reproducing all their essential parts automatically, from speech units to the language model. However, they hide the underlying perceptual processes modelled, if any, and they have lower adaptability to multiple application contexts, and, furthermore, they require powerful hardware and an extensive amount of training data. Model-explainability techniques can explore the internal dynamics of these ASR systems and possibly understand and explain the processes conducting to their decisions and outputs. Understanding these processes can help enhance ASR performance and reduce the required training data and hardware significantly. In this paper, we probe the internal dynamics of three E2E-ASRs pre-trained for English by building an acoustic-syllable boundary detector for Italian and Spanish based on the E2E-ASRs' internal encoding layer outputs. We demonstrate that the shallower E2E-ASR layers spontaneously form a rhythmic component correlated with prominent syllables, central in human speech processing. This finding highlights a parallel between the analysed E2E-ASRs and human speech recognition. Our results contribute to the body of knowledge by providing a human-explainable insight into behaviours encoded in popular E2E-ASR systems. [ABSTRACT FROM AUTHOR]

The aim of this paper is to exhibit two new variations of the frame-level cost function for training a deep neural network in order to achieve better word error rates in speech recognition. Optimization methods and their minimization functions are underlying aspects to consider when someone is working on neural nets, and hence their improvement is one of the salient objectives of researchers, and this paper deals in part with such a situation. The first proposed framework is based on the concept of extropy, the complementary dual function of an uncertainty measure. The conventional cross-entropy function can be mapped to a non-uniform loss function based on its corresponding extropy, enhancing the frames that have ambiguity in their belonging to specific senones. The second proposal makes a fusion of the presented mapped cross-entropy function and the idea of boosted cross-entropy, which emphasizes those frames with low target posterior probability. The proposed approaches have been performed by using a personalized mid-vocabulary speaker-independent voice corpus. This dataset is employed for recognition of digit strings and personal name lists in Spanish from the northern central part of Mexico on a connected-words phone dialing task. A relative word error rate improvement of 12.3% and 10.7% is obtained with the two proposed approaches, respectively, with regard to the conventional well-established cross-entropy objective function. [ABSTRACT FROM AUTHOR]

Information Technology has touched new vistas for a couple of decades mostly to simplify the day-to-day life of the humans. One of the key contributions of Information Technology is the application of Artificial Intelligence to achieve better results. The advent of artificial intelligence has given rise to a new branch of Natural Language Processing (NLP) called Computational Linguistics, which generates frameworks for intelligently manipulating spoken language knowledge and has brought human-machine onto a new stage. In this context, speech has arisen to be one of the imperative forms of interfaces, which is the basic mode of communication for us, and generally the most preferred one. Language identification, being the front-end for various natural language processing tasks, plays an important role in language translation. Owing to this, the focus has been given on the field of speech recognition involving the identification & recognition of languages by a machine. Spoken language identification is the identification of language present in a speech segment despite its size (duration & speed), ambiance (topic & emotion), and moderator (gender, age, demographic region). This paper has investigated various existing spoken language identification models implemented using different deep learning approaches, datasets, and performance measures utilized for their analysis. It also highlights the main features and challenges faced by these models. A comprehensive comparative study of deep learning techniques has been carried out for spoken language identification. Moreover, this review analyzes the efficiency of the spoken language models that can help the researchers to propose new language identification models for speech signals. [ABSTRACT FROM AUTHOR]

For the text content is known, the semantic information and speaker characteristics in the speech signal can be used for speech recognition and speaker verification respectively in text prompt speaker recognition, which solves the problem of forged recordings in the process of text association. In practical applications, by combining speech recognition and speaker recognition technologies, a double verification effect can be achieved, which also can effectively improve security. There are few studies on the combination of speaker recognition and speech recognition in Tibetan, mainly using non-end-to-end methods, and the performance of the model is not ideal. Based on the original research, this paper uses the mainstream end-to-end method to study the speaker verification part. The network model uses ResNet-34 and ResNet-50, and fine-tuned them. "Open set" speaker verification is essentially metric learning. The ideal embedding is to compress the frame-level features into a compact speech-level representation, thereby maximizing the inter-class distance and minimizing the intra-class distance. For the loss function, we use three classification objective loss functions and three metric learning objective loss functions to extensively evaluate the performance of the model. In order to further improve the performance of the model, we fused the two loss functions of Softmax and Angular Prototype. The experimental results show that the effect of Fast ResNet-50 is better than that of Fast ResNet-34, and the model effect of the Angular Prototype loss function is better than other single loss functions. The model with the fused loss function has the best performance, with an equal error rate of 4.25%. [ABSTRACT FROM AUTHOR]

With the global integration and the increasing level of China's internationalization, the demand of Chinese people for English learning is growing rapidly. At the same time, English has gradually become one of the most frequently used languages in the world economic exchanges and cultural information exchanges. After the material life demands of the members of the domestic society are fully satisfied, the learning and explanation of the corresponding knowledge content of the English language are gradually highly valued by the staff in various production fields. After AI proposed deep learning models, researchers also began to improve these models and use them in speech recognition. Through multi-layer nonlinear structure, the essential features are preserved and more abstract advanced features are extracted, so the recognition rate is improved. Computer-aided language learning will change the existing language teaching mode and learning environment, so that learners can learn independently anytime and anywhere. Deep learning can give learners accurate, objective and timely pronunciation evaluation and feedback guidance. It can also help learners find out the differences between their pronunciation and standard pronunciation through repeated listening and comparison, correct their pronunciation errors, and improve the efficiency of language learning. This paper constructs an English language improvement system based on deep learning. The experimental results show that the proposed method can effectively analyze the input speech signals and make corresponding feedback. [ABSTRACT FROM AUTHOR]

English is one of the widely used languages, with the shrinking of the global village, the smart home, the in-vehicle voice system and voice recognition software with English as the recognition language have gradually entered people's field of vision, and have obtained the majority of users' love by the practical accuracy. And deep learning technology in many tasks with its hierarchical feature learning ability and data modeling capabilities has achieved more than the performance of shallow learning technology. Therefore, this paper takes English speech as the research object, and proposes a deep learning speech recognition algorithm that combines speech features and speech attributes. Firstly, the deep neural network supervised learning method is used to extract the high-level features of the speech, select the output of the fixed hidden layer as the new speech feature for the newly generated network, and train the GMM–HMM acoustic model with the new speech features; secondly, the speech attribute extractor based on deep neural network is trained for multiple speech attributes, and the extracted speech attributes are classified into phoneme by deep neural network; finally, speech features and speech attribute features are merged into the same CNN framework by the neural network based on the linear feature fusion algorithm. The experimental results show that the proposed English speech recognition algorithm based on deep neural network with multiple features can directly and effectively combine the two methods by combining the speech features and the speech attributes of the speaker in the input layer of the deep neural network, and it can improve the performance of the English speech recognition system significantly. [ABSTRACT FROM AUTHOR]

Various studies have been conducted on applying intelligent recognition technology, especially speech recognition technology to improve English learning ability, mostly listening and speaking. However, few studies have touched on how image-to-text recognition technology can be used for writing. The present research was conducted to fill this gap by exploring the optimization of a deep-learning-based image captioning algorithm to facilitate English writing, so as to enable learners to break the time and space limitations and learn English writing (including sentence patterns, spelling, vocabulary, and grammar) anytime and anywhere by taking pictures. Therefore, this paper focused on image captioning based on CNN(Convolutional Neural Networks) and LSTM(Long Short-Term Memory), using DenseNet201 or Vision Transformer trained on the ImageNet-1K image classification dataset as the image encoder and LSTM as the decoder. First, pre-training was performed on the Flickr8k dataset. After selecting the best-trained model as the pre-trained weight model for the COCO dataset, fine-tuning optimization was performed on the COCO dataset, and the attention mechanism was used to design the ablation experiment. The BLEU-4, CIDEr, METEOR, and ROUGE evaluation indexes of the optimized model on the test set were 0.3437,1.121,0.2750, and 0.5117, respectively. The study results showed that the convergence of the model was accelerated and had better performance. The model was used to automatically caption 12 images that had never been used during the training process. The descriptions generated by the optimized image captioning algorithm have lexical and syntactic accuracy, and matched what the images expressed, showing that this improved algorithm could be used as a learning tool to help English learners improve lexical and syntactic acquisition to promote writing through the generated descriptions of the pictures taken anytime and anywhere in real-life situations. [ABSTRACT FROM AUTHOR]

Phonetic analysis of speech, in general, requires the alignment of audio samples to its phonetic transcription. This could be done manually for a couple of files, but as the corpus grows large, it becomes infeasibly time-consuming. This paper describes the evolution process toward creating free resources for phonetic alignment in Brazilian Portuguese (BP) using Kaldi, a toolkit that achieves state of the art for open-source speech recognition, within a toolkit we call UFPAlign. The contributions of this work are then twofold: developing resources to perform forced alignment in BP, including the release of scripts to train acoustic models via Kaldi, as well as the resources themselves under open licenses; and bringing forth a comparison to other two phonetic aligners that provide resources for BP, namely EasyAlign and Montreal Forced Aligner (MFA), the latter being also Kaldi-based. Evaluation took place in terms of phone boundary and intersection over union metrics over a dataset of 385 hand-aligned utterances, and results show that Kaldi-based aligners perform better overall, and that UFPAlign models are more accurate than MFA's. Furthermore, complex deep-learning-based approaches still do not improve performance compared to simpler models. [ABSTRACT FROM AUTHOR]

Since the contextual information has an important impact on the speaker's emotional state, how to use emotion-related context information to conduct feature learning is a key problem. The existing speech emotion recognition algorithms achieve the relatively high recognition rate; these algorithms are not very good application to the real-life speech emotion recognition systems. Therefore, in order to address the abovementioned issues, a novel speech emotion recognition algorithm based on improved stacked kernel sparse deep model is proposed in this paper, which is based on auto-encoder, denoising auto-encoder, and sparse auto-encoder to improve the Chinese speech emotion recognition. The first layer of the structure uses a denoising auto-encoder to learn a hidden feature with a larger dimension than the dimension of the input features, and the second layer employs a sparse auto-encoder to learn sparse features. Finally, a wavelet-kernel sparse SVM classifier is applied to classify the features. The proposed algorithm is evaluated on the testing dataset, which contains the speech emotion data of spontaneous, non-prototypical, and long-term. The experimental results show that the proposed algorithm outperforms the existing state-of-the-art algorithms in speech emotion recognition. [ABSTRACT FROM AUTHOR]

The last decade brought significant advances in automatic speech recognition (ASR) thanks to the evolution of deep learning methods. ASR systems evolved from pipeline-based systems, that modeled hand-crafted speech features with probabilistic frameworks and generated phone posteriors, to end-to-end (E2E) systems, that translate the raw waveform directly into words using one deep neural network (DNN). The transcription accuracy greatly increased, leading to ASR technology being integrated into many commercial applications. However, few of the existing ASR technologies are suitable for integration in embedded applications, due to their hard constrains related to computing power and memory usage. This overview paper serves as a guided tour through the recent literature on speech recognition and compares the most popular ASR implementations. The comparison emphasizes the trade-off between ASR performance and hardware requirements, to further serve decision makers in choosing the system which fits best their embedded application. To the best of our knowledge, this is the first study to provide this kind of trade-off analysis for state-of-the-art ASR systems. [ABSTRACT FROM AUTHOR]

With the rapid development of Internet technology, network assisted instruction system develops rapidly. Speech recognition is a technology that transforms the speech from human beings into words or symbols. From the acoustic feature extraction 40 years ago to the automatic speech recognition system based on deep neural network, speech recognition technology has been gradually improved. Speech recognition algorithm based on neural network has great potential and is an effective way to solve the bottleneck problem of existing speech recognition algorithm. In this paper, the deep neural network and game theory are combined to reduce the dimension of the model by singular value decomposition and reconstruction, in order to reduce the amount of data and improve the accuracy of the experiment. The proposed framework is implemented on the Android system to validate the performance. Compared with the state-of-the-art methodologies, the proposed system can effectively recognize the speech information and extract the information structure. [ABSTRACT FROM AUTHOR]

Nowadays automatic speech recognition (ASR) systems can achieve higher and higher accuracy rates depending on the methodology applied and datasets used. The rate decreases significantly when the ASR system is being used with a non-native speaker of the language to be recognized. The main reason for this is specific pronunciation and accent features related to the mother tongue of that speaker, which influence the pronunciation. At the same time, an extremely limited volume of labeled non-native speech datasets makes it difficult to train, from the ground up, sufficiently accurate ASR systems for non-native speakers.In this research, we address the problem and its influence on the accuracy of ASR systems, using the style transfer methodology. We designed a pipeline for modifying the speech of a non-native speaker so that it more closely resembles the native speech. This paper covers experiments for accent modification using different setups and different approaches, including neural style transfer and autoencoder. The experiments were conducted on English language pronounced by Japanese speakers (UME-ERJ dataset). The results show that there is a significant relative improvement in terms of the speech recognition accuracy. Our methodology reduces the necessity of training new algorithms for non-native speech (thus overcoming the obstacle related to the data scarcity) and can be used as a wrapper for any existing ASR system. The modification can be performed in real time, before a sample is passed into the speech recognition system itself. [ABSTRACT FROM AUTHOR]

Automatic speaker recognizing models consists of a foundation on building various models of speaker characterization, pattern analyzing and engineering. The effect of classification and feature selection methods for the speech emotion recognition is focused. The process of selecting the exact parameter in arrangement with the classifier is an important part of minimizing the difficulty of system computing. This process becomes essential particularly for the models which undergo deployment in real time scenario. In this paper, a new deep learning speech based recognition model is presented for automatically recognizes the speech words. The superiority of an input source, i.e. speech sound in this state has straight impact on a classifier correctness attaining process. The Berlin database consist around 500 demonstrations to media persons that is both male and female. On the applied dataset, the presented model achieves a maximum accuracy of 94.21%, 83.54%, 83.65% and 78.13% under MFCC, prosodic, LSP and LPC features. The presented model offered better recognition performance over the other methods. [ABSTRACT FROM AUTHOR]

As an emerging training model, federated deep learning has been widely applied in many fields such as speech recognition, image classification and classification of peer-to-peer (P2P) Internet traffics. However, it also entails various security and privacy concerns. In the past years, many researchers have been carried out toward elaborating solutions to alleviate the above challenges via three underlying technologies, i.e., Secure Multi-Party Computation (SMC), Homomorphic Encryption (HE) and Differential Privacy (DP). Compared with SMC and HE, differential privacy is outstanding in terms of efficiency. However, due to the involvement of noise, DP always needs to make a trade-off between security and accuracy. i.e., achieving a strong security requirement has to sacrifice certain accuracy. To seek the optimal balance, we propose APFL, an Adaptive Privacy-preserving Federated Learning framework in this paper. Specifically, in the APFL, we calculate the contribution of each attribute class to the outputs with a layer-wise relevance propagation algorithm. By injecting adaptive noise to data attributes, our APFL significantly reduces the impact of noise on the final results. Moreover, we introduce the Randomized Privacy-preserving Adjustment Technology to further improve the prediction accuracy of the model. We present a formal security analysis to demonstrate the high privacy level of APFL. Besides, extensive experiments show the superior performance of APFL in terms of accuracy, computation and communication overhead. [ABSTRACT FROM AUTHOR]

Hearing impaired persons are more expressive while speaking and expression is a salient feature in hearing impaired Visual Speech Recognition. Most Visual Speech Recognition systems focus only on the lip area for the recognition of speech or speaker. This work utilizes video data which includes information from both speech and facial expressions. As part of this study, we have developed a Malayalam audio-visual speech expression database of unimpaired people. The experiments were conducted on this newly developed Malayalam audio-visual speech database. The data has been collected from two people, 1 male, and 1 female. A combination of Convolutional Neural Network-Long Short Term Memory deep learning video processing model is applied for this system. The result demonstrate that, the classification accuracy is better for the features extracted using GoogleNet model compared to AlexNet and ResNet model. The system evaluation is carried out in both Speaker-dependent and speaker-independent domains. The recognition rate of the system for both speaker-dependent and speaker-independent experiments proves that facial expression analysis plays a crucial role in Visual Speech Recognition. [ABSTRACT FROM AUTHOR]

Training procedures of a deep neural network are still an area with ample research possibilities and constant improvement either to increase its efficiency or its time performance. One of the lesser-addressed components is its objective function, which is an underlying aspect to consider when there is the necessity to achieve better error rates in the area of automatic speech recognition. The aim of this paper is to present two new variations of the frame-level cost function for training a deep neural network with the purpose of obtaining superior word error rates in speech recognition applied to a case study in Spanish. The first proposed function is a fusion between the boosted cross-entropy and the so called cross-entropy/log-posterior-ratio. The main idea is to jointly emphasize the prediction of difficult/crucial frames provided by a boosting factor and at the same time enlarge the distance between the target senone and its closest competitor. The second proposal is a fusion between the non-uniform mapped cross-entropy and the cross-entropy/log-posterior-ratio. This function utilizes both the mapped function to enhance the frames that have ambiguity in their belonging to specific senones and the log-posterior-ratio with the purpose of separating the target senone against the most competing tied tri-phone state. The proposed approaches are compared against those frame-level cost functions discussed in the state of the art. This comparative has been made by using a personalized mid-vocabulary speaker-independent voice corpus. This dataset is employed for the recognition of digit strings and personal name lists in Spanish from the northern central part of México on a connected-words phone dialing task. A relative word error rate improvement of 15.14% and 12.30% is obtained with the two proposed approaches, respectively, against the plain well-established cross-entropy loss function. [ABSTRACT FROM AUTHOR]

Over the last decade, the increased use of social media has led to an increase in hateful activities in social networks. Hate speech is one of the most dangerous of these activities, so users have to protect themselves from these activities from YouTube, Facebook, Twitter etc. This paper introduces a method for using a hybrid of natural language processing and with machine learning technique to predict hate speech from social media websites. After hate speech is collected, steaming, token splitting, character removal and inflection elimination is performed before performing hate speech recognition process. After that collected data is examined using a killer natural language processing optimization ensemble deep learning approach (KNLPEDNN). This method detects hate speech on social media websites using an effective learning process that classifies the text into neutral, offensive and hate language. The performance of the system is then evaluated using overall accuracy, f-score, precision and recall metrics. The system attained minimum deviations mean square error − 0.019, Cross Entropy Loss − 0.015 and Logarithmic loss L-0.0238 and 98.71% accuracy. [ABSTRACT FROM AUTHOR]

Speech problems are a common issue that affects people everywhere and can affect the quality of their lives. The human speech production system involves various components. Dysfunction of any of these components can disrupt normal speech production, giving rise to speech diseases like laryngopharyngeal reflux, vocal cord paralysis, and vocal fold nodules. Early diagnosis of these disorders is very important for the patient's health. Many studies in automatic diagnosis of voice pathology have used sustained vowel sounds and read-speech as the primary source of speech data. However, it is crucial to recognize the unique value of spontaneous-speech. In addition to inheriting the characteristics of read speech, spontaneous-speech offers a more authentic glimpse into individuals' speech behavior. It captures not only linguistic features, but also subtle nuances of human emotions, such as fatigue and excitement, which may cause speech impairments, and shows their patterns in the speech signal better than in the read-speech data. Therefore, we aim to explore spontaneous speech in voice pathology detection to determine if it can help us better understand speech problems. In this research, we examine different deep learning (DL) models trained on two main features (MFCC and Mel spectrograms) for binary classification of healthy speech versus pathological speech, with a specific focus on the spontaneous speech. Extensive experimentation reveals the superiority of our proposed convolutional neural network (CNN) model trained on MFCC features. Notably, the CNN model achieves the highest accuracy, approximately 85% for test data and 92% for evaluation data. These results emphasize the potential of DL approaches in the accurate diagnosis of speech disorders through the analysis of the spontaneous-speech, offering promise for early detection and improved patient care. [ABSTRACT FROM AUTHOR]

When a person listens to natural speech, the relation between features of the speech signal and the corresponding evoked electroencephalogram (EEG) is indicative of neural processing of the speech signal. Using linguistic representations of speech, we investigate the differences in neural processing between speech in a native and foreign language that is not understood. We conducted experiments using three stimuli: a comprehensible language, an incomprehensible language, and randomly shuffled words from a comprehensible language, while recording the EEG signal of native Dutch-speaking participants. We modeled the neural tracking of linguistic features of the speech signals using a deep-learning model in a match-mismatch task that relates EEG signals to speech, while accounting for lexical segmentation features reflecting acoustic processing. The deep learning model effectively classifies coherent versus nonsense languages. We also observed significant differences in tracking patterns between comprehensible and incomprehensible speech stimuli within the same language. It demonstrates the potential of deep learning frameworks in measuring speech understanding objectively. [ABSTRACT FROM AUTHOR]

There are a wide variety of intelligence accelerators with promising performance and energy efficiency, deployed in a broad range of applications such as computer vision and speech recognition. However, programming productivity hinders the deployment of deep learning accelerators. The low-level library invoked in the high-level deep learning framework which supports the end-to-end execution with a given model, is designed to reduce the programming burden on the intelligence accelerators. Unfortunately, it is inflexible for developers to build a network model for every deep learning application, which probably brings unnecessary repetitive implementation. In this paper, a flexible and efficient programming framework for deep learning accelerators, FlexPDA, is proposed, which provides more optimization opportunities than the low-level library and realizes quick transplantation of applications to intelligence accelerators for fast upgrades. We evaluate FlexPDA by using 10 representative operators selected from deep learning algorithms and an end-to-end network. The experimental results validate the effectiveness of FlexPDA, which achieves an end-to-end performance improvement of 1.620x over the low-level library. [ABSTRACT FROM AUTHOR]

In cloud computing, resources could be provisioned in a dynamic way on demand for cloud services. Cloud providers seek to realize effective SLA execution mechanisms for avoiding SLA violations by provisioning the resources or applications and timely interacting to environmental changes and failures. Sufficient resource provisioning to cloud's services relies on the requirements of the workloads to achieve a high performance for quality of service. Therefore, deciding the suitable amount of cloud's resources for these services to achieve is one of the main works in cloud computing. During the runtime of services, the amount of cloud's resources can be specified and provisioned based on the actual workloads changes. Determining the correct amount of cloud's resources needed for running the services on clouds is not easy task, and it depends on the existing workloads of services. Consequently, it is required to predict the future workloads for dynamic provisioning of resources in order to meet the changes in workloads and demands of services in cloud computing environments. In this paper, we study the possibility of using a cognitive/intelligent approach for cloud resource provisioning which is a combination of the autonomic computing concept, deep learning technique and fuzzy logic control. Deep learning technique is a state-of-the-art in the machine learning field. It achieved promising results in many other fields like image classification and speech recognition. For these reasons, deep learning is proposed in this work to tackle the workload prediction in cloud computing. Additionally, we also propose to use a fuzzy logic-based method in order to make a decision in the case of uncertainty of the workload prediction. We study various exiting works on autonomic cloud resource provisioning and show that there is still an opportunity to improve the current methods. We also present the challenges that may exist on this domain. [ABSTRACT FROM AUTHOR]

In the emerging information retrieval world, the selection of keywords and the generation of queries are very critical for the efficient retrieval. The most new-generations database applications request keen interface to upgrade productive connections among various databases and the clients. Majority of the accessible interfaces for databases should be intelligent and also should understand natural language expressions. The paper deals with, the mapping of natural language queries which is in spoken form, and converting into words forming the foundation of SQL. We give a general framework for a smart database interface which could be linked to any database. One of the most striking features of this interface is domain-independence. The smart interface employs speech recognition techniques to convert spoken language input into text. Then a semantic matching technique is employed in converting natural language query to SQL words, complemented by using dictionary and a set of production rules. The dictionary consists of semantics sets for columns and tables. The identified type of query, query-word tuple is memorized and sent through a convolution neural network for better construction. This work tries to use natural language processing techniques, parsing and POS tagging to identify semantic structures of inputs. The natural language techniques are used to increase the production of unambiguous queries and speed in detecting and identifying the important parts in a query. [ABSTRACT FROM AUTHOR]

In order to solve the problem of instability of the traditional speech separation algorithm, a kind of reverberation speech separation model based on deep learning is proposed. The problem of speech separation in reverberation environment has been studied. The auditory scene analysis is used to simulate the human auditory perception ability. According to the ideal two value mode principle, the target speech signal can be extracted. Moreover, the deep neural network (DNN) shows great learning ability in speech recognition and artificial intelligence. In this paper, a DNN model is proposed to learn the inverse reverberation and denoising by learning the spectrum mapping between "contaminated" speech and pure speech. By extracting a series of spectrum features, the time dynamic information of adjacent frames is fused. The DNN is used to transform the coded spectrum, and restore the pure voice frequency spectrum. Finally, the time domain signal is reconstructed. In addition, the feature classification ability of DNN is also proposed to complete the separation of double sound reverberation speech. The binaural features ITD and ILD and the mono features GFCC are fused to form a long eigenvector. The DNN is pre-trained by RBM to complete the classification task. The results show that the proposed model improves the quality and intelligibility of the speech separation, and enhances the stability of the system significantly. [ABSTRACT FROM AUTHOR]

N-ary error correcting output codes (ECOC) decompose a multi-class problem into simpler multi-class problems by splitting the classes into N subsets (meta-classes) to form an ensemble of N-class classifiers and combine them to make predictions. It is one of the most accurate ensemble learning methods for traditional classification tasks. Deep learning has gained increasing attention in recent years due to its successes on various tasks such as image classification and speech recognition. However, little is known about N-ary ECOC with deep neural networks (DNNs) as base learners, probably due to the long computation time. In this paper, we show by experiments that N-ary ECOC with DNNs as base learners generally exhibits superior performance compared with several state-of-the-art ensemble learning methods. Moreover, our work contributes to a more efficient setting of the two crucial hyperparameters of N-ary ECOC: the value of N and the number of base learners to train. We also explore valuable strategies for further improving the accuracy of N-ary ECOC. [ABSTRACT FROM AUTHOR]

In human–computer interaction systems, speech emotion recognition (SER) plays a crucial role because it enables computers to understand and react to users' emotions. In the past, SER has significantly emphasised acoustic properties extracted from speech signals. The use of visual signals for enhancing SER performance, however, has been made possible by recent developments in deep learning and computer vision. This work utilizes a lightweight Vision Transformer (ViT) model to propose a novel method for improving speech emotion recognition. We leverage the ViT model's capabilities to capture spatial dependencies and high-level features in images which are adequate indicators of emotional states from mel spectrogram input fed into the model. To determine the efficiency of our proposed approach, we conduct a comprehensive experiment on two benchmark speech emotion datasets, the Toronto English Speech Set (TESS) and the Berlin Emotional Database (EMODB). The results of our extensive experiment demonstrate a considerable improvement in speech emotion recognition accuracy attesting to its generalizability as it achieved 98%, 91%, and 93% (TESS-EMODB) accuracy respectively on the datasets. The outcomes of the comparative experiment show that the non-overlapping patch-based feature extraction method substantially improves the discipline of speech emotion recognition. Our research indicates the potential for integrating vision transformer models into SER systems, opening up fresh opportunities for real-world applications requiring accurate emotion recognition from speech compared with other state-of-the-art techniques. [ABSTRACT FROM AUTHOR]

Visual speech recognition (VSR) is a challenging task that has received increasing interest during the last few decades. Current state of the art employs powerful end-to-end architectures based on deep learning which depend on large amounts of data and high computational resources for their estimation. We address the task of VSR for data scarcity scenarios with limited computational resources by using traditional approaches based on hidden Markov models. We present a novel learning strategy that employs information obtained from previous acoustic temporal alignments to improve the visual system performance. Furthermore, we studied multiple visual speech representations and how image resolution or frame rate affect its performance. All these experiments were conducted on the limited data VLRF corpus, a database which offers an audio-visual support to address continuous speech recognition in Spanish. The results show that our approach significantly outperforms the best results achieved on the task to date. [ABSTRACT FROM AUTHOR]

Due to the peculiar cell growth, apraxia is one of the common types of speech stuttering seen in youngsters. Machine learning-based architecture has been used to automatically categorize the disfluency based on the feature and its properties, however, these models use more energy, and processing time, and will have worse scalability and dependability. Deep learning architecture has been used to overcome these restrictions as it is more effective and efficient at differentiating speech characteristics. Utilizing ResNet -18 convolutional neural networks and DenseNet-121 convolution network architectures, continuous feature learning representation has been developed as a feature extraction technique. The combined extracted feature is given to the XGBoost Classifier for Apraxia identification. Utilizing Ultra Suite Repository, the suggested model's experimental findings have been assessed in Python. When compared to traditional methods, the suggested model's performance analysis produced 98 percent accuracy, which is evaluated. [ABSTRACT FROM AUTHOR]

Indonesia has a variety of languages, one of which is Sundanese. Sundanese is a regional language from Indonesia that has the potential to become extinct. One way to prevent Sundanese from potential extinction is with speech recognition. Speech recognition is one of the most advanced technologies in a machine for simulating human behavior. In this study, researchers used the existing Wav2Vec 2.0 pre-trained model for a fine-tune process so that the model could learn and predict Sundanese. The fine-tune process was carried out on the Wav2Vec 2.0 Base model and the Wav2Vec 2.0 Large model using the Sundanese language dataset of approximately 53 h duration. The fine-tune process lasted for 1 to 2 h, and the results of the fine-tune were followed by an evaluation process with the WER metric. The fine-tuned model gives good results, with a WER value of 23.5% for the Wav2Vec 2.0 Base model and 24.0% for the Wav2Vec 2.0 Large model. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs) have gained remarkable success in speech recognition, partially attributed to the flexibility of DNN models in learning complex patterns of speech signals. This flexibility, however, may lead to serious over-fitting and hence miserable performance degradation in adverse acoustic conditions such as those with high ambient noises. We propose a noisy training approach to tackle this problem: by injecting moderate noises into the training data intentionally and randomly, more generalizable DNN models can be learned. This 'noise injection' technique, although known to the neural computation community already, has not been studied with DNNs which involve a highly complex objective function. The experiments presented in this paper confirm that the noisy training approach works well for the DNN model and can provide substantial performance improvement for DNN-based speech recognition. [ABSTRACT FROM AUTHOR]

As deep learning technology matures, it's being widely deployed in fields like image classification and speech recognition. However, training a functional deep learning model requires vast computing power and a large training dataset, leading to the emergence of a new business model of selling pre-trained models. However, these models are highly susceptible to theft, which poses a threat to the interests of their creators. Moreover, the network topology and weight parameters are considered intellectual property. To address these challenges, a method that can tag trained models to claim ownership without affecting their performance is necessary. Therefore, we propose a novel neural network watermarking protocol. In this method, the trigger set is constructed differently from previous methods by using a key obtained from the authority to generate a scrambling sequence, followed by using the sequence to scramble the pixels and assign their original labels. Finally, the trigger set is put into the network training together with the original training set to complete the watermark embedding. Since Logistic chaos mapping is nonlinear, unpredictable, and sensitive to initial values, we use Logistic chaos mapping as the generation method of dislocation sequence. We involve a third-party copyright center in the embedding process to prevent forgery attacks. The third-party only needs to store the disruption key and timestamp for each owner, reducing their storage burden. Our experimental results demonstrate that the ResNet model exhibits a mere 0.05 percentage point decrease in accuracy when using fine-tuning for watermark embedding, and a mere 0.03 percentage point decrease when using the training-from-scratch method. On the other hand, when using the SENet model, embedding watermarks via fine-tuning resulted in a 1.35 percentage point decrease in classification accuracy, while embedding watermarks from training-from-scratch resulted in a 0.94 percentage point increase in classification accuracy. Furthermore, our model exhibited robustness against various attacks in the robustness experiments, including model fine-tuning, model compression, and watermark overlay. [ABSTRACT FROM AUTHOR]

In recent years, deep learning (DL) models have achieved significant progress in many domains, such as autonomous driving, facial recognition, and speech recognition. However, the vulnerability of deep learning models to adversarial attacks has raised serious concerns in the community because of their insufficient robustness and generalization. Also, transferable attacks have become a prominent method for black-box attacks. In this work, we explore the potential factors that impact adversarial examples (AEs) transferability in DL-based speech recognition. We also discuss the vulnerability of different DL systems and the irregular nature of decision boundaries. Our results show a remarkable difference in the transferability of AEs between speech and images, with the data relevance being low in images but opposite in speech recognition. Motivated by dropout-based ensemble approaches, we propose random gradient ensembles and dynamic gradient-weighted ensembles, and we evaluate the impact of ensembles on the transferability of AEs. The results show that the AEs created by both approaches are valid for transfer to the black box API. [ABSTRACT FROM AUTHOR]