Search

Voice isolation, a prominent research area in the field of speech processing, has garnered a great deal of attention due to its prospective implications in numerous domains. Deep neural networks (DNNs) have emerged as a potent instrument for addressing the challenges associated with vocal isolation. This paper presents a comprehensive study on the use of DNNs for voice isolation, focusing on speech recognition and speaker identification tasks. The proposed method uses frequency domain and time domain techniques to improve the separation of target utterances from background noise. The experimental results demonstrate the efficacy of the proposed method, revealing substantial improvements in voice isolation precision and robustness. This study's findings contribute to the increasing corpus of research on voice isolation techniques and provide valuable insights into the application of DNNs to improve speech processing tasks . [ABSTRACT FROM AUTHOR]

This project will study a fast retrieval method for music humming speech recognition based on sentence features and deep learning. The method proposed in this paper can realize the fast extraction of songs. According to the characteristics of the natural pause mode of the song, the song database and the song fragments provided by the user are divided into different sentences. The deep learning algorithm of BDTW is used to calculate the similarity of the song's pitch, and users can set matching conditions according to their preferences. It can identify the most significant differences between music fragments and the order of queries in the database. Then, a retrieval method of a music database based on DIS is proposed. It can shorten the acquisition time. Experiments show that the algorithm can recognize humming songs quickly and efficiently. [ABSTRACT FROM AUTHOR]

Recently, the machine learning research trend expands to the system performance optimization field, where it has still been proposed by researchers based on their intuitions and heuristics. Compared to conventional major machine learning research areas such as image or speech recognition, machine learning-based system performance optimization fields are at the beginning stage. However, recent papers show that this approach is promising and has significant potential. This paper reviews 11 machine learning-based system performance optimization approaches from nine recent papers based on well-known machine learning models such as perceptron, LSTM, and RNN. This survey provides a detailed design and summarizes model, input, output, and prediction method of each approach. This paper covers various system performance areas from the data structure to essential system components of a computer system such as index structure, branch predictor, sort, and cache management. The result shows that machine learning-based system performance optimization has an important potential for future research. We expect that this paper shows a wide range of applicability of machine learning technology and provides a new perspective for system performance optimization. [ABSTRACT FROM AUTHOR]

In this paper, we use machine learning algorithms to conduct in-depth research and analysis on the construction of human-computer interaction systems and propose a simple and effective method for extracting salient features based on contextual information. The method can retain the dynamic and static information of gestures intact, which results in a richer and more robust feature representation. Secondly, this paper proposes a dynamic planning algorithm based on feature matching, which uses the consistency and accuracy of feature matching to measure the similarity of two frames and then uses a dynamic planning algorithm to find the optimal matching distance between two gesture sequences. The algorithm ensures the continuity and accuracy of the gesture description and makes full use of the spatiotemporal location information of the features. The features and limitations of common motion target detection methods in motion gesture detection and common machine learning tracking methods in gesture tracking are first analyzed, and then, the kernel correlation filter method is improved by designing a confidence model and introducing a scale filter, and finally, comparison experiments are conducted on a self-built gesture dataset to verify the effectiveness of the improved method. During the training and validation of the model by the corpus, the complementary feature extraction methods are ablated and learned, and the corresponding results obtained are compared with the three baseline methods. But due to this feature, GMMs are not suitable when users want to model the time structure. It has been widely used in classification tasks. By using the kernel function, the support vector machine can transform the original input set into a high-dimensional feature space. After experiments, the speech emotion recognition method proposed in this paper outperforms the baseline methods, proving the effectiveness of complementary feature extraction and the superiority of the deep learning model. The speech is used as the input of the system, and the emotion recognition is performed on the input speech, and the corresponding emotion obtained is successfully applied to the human-computer dialogue system in combination with the online speech recognition method, which proves that the speech emotion recognition applied to the human-computer dialogue system has application research value. [ABSTRACT FROM AUTHOR]

Finding ways to enable seamless communication between deaf and able-bodied individuals has been a challenging and pressing issue. This paper proposes a solution to this problem by designing a low-cost data glove that utilizes multiple inertial sensors with the purpose of achieving efficient and accurate sign language recognition. In this study, four machine learning models—decision tree (DT), support vector machine (SVM), K-nearest neighbor method (KNN), and random forest (RF)—were employed to recognize 20 different types of dynamic sign language data used by deaf individuals. Additionally, a proposed attention-based mechanism of long and short-term memory neural networks (Attention-BiLSTM) was utilized in the process. Furthermore, this study verifies the impact of the number and position of data glove nodes on the accuracy of recognizing complex dynamic sign language. Finally, the proposed method is compared with existing state-of-the-art algorithms using nine public datasets. The results indicate that both the Attention-BiLSTM and RF algorithms have the highest performance in recognizing the twenty dynamic sign language gestures, with an accuracy of 98.85% and 97.58%, respectively. This provides evidence for the feasibility of our proposed data glove and recognition methods. This study may serve as a valuable reference for the development of wearable sign language recognition devices and promote easier communication between deaf and able-bodied individuals. [ABSTRACT FROM AUTHOR]

AI has been used to process the data for decision-making, problem-solving, interaction with humans and to understand human's feelings, emotions and their behavior. In today's world, communication between humans takes place digitally, so human's emotions play a very important role for communication as well as detection and analysis. Although there are many surveys related to emotions from speech already done, selecting appropriate datasets and methods are challenging tasks. This survey will primarily concentrate on efficient techniques, including Machine Learning, Deep Learning, and transformer-based approaches, while also providing brief descriptions of existing challenges and outlining future prospects. Additionally, this paper provides a comparative analysis of various datasets and techniques employed by researchers. After conducting the survey, we discovered that deep learning and transformer-based techniques are more effective and yield superior performance results. [ABSTRACT FROM AUTHOR]

Automatic Speech Recognition (ASR) systems have witnessed significant advancements in recent years, thanks to the emergence of deep learning techniques and the availability of large speech datasets in various languages. With the increasing demand for Arabic voice-enabled technologies, the availability of a high-quality and representative dataset for the Arabic language becomes crucial. This paper presents the development of a new dataset called ArabAlg, specifically designed for Arabic Speech Command Recognition (ASCR), to support the integration of Arabic voice recognition systems into smart devices in the Internet of Things (IoT). This research focuses on collecting and annotating a diverse range of Arabic speech commands, encompassing various domains and applications. The dataset construction process involves recording and preprocessing several utterances from native Arabic speakers. To ensure precision and reliability, quality control measures are implemented during data collection and annotation. The resulting dataset provides a valuable resource for training and evaluating ASCR systems tailored for Arabic speakers using Machine Learning and Deep Learning. [ABSTRACT FROM AUTHOR]

A growing focus among scientists has been on researching the techniques of automatic detection of dementia that can be applied to the speech samples of individuals with dementia. Leveraging the rapid advancements in Deep Learning (DL) and Natural Language Processing (NLP), these techniques have shown great potential in dementia detection. In this context, this paper proposes a method for dementia detection from the transcribed speech of subjects. Unlike conventional methods that rely on advanced language models to address the ability of the subject to make coherent and meaningful sentences, our approach relies on the center of focus of the subjects and how it changes over time as the subject describes the content of the cookie theft image, a commonly used image for evaluating one's cognitive abilities. To do so, we divide the cookie theft image into regions of interest, and identify, in each sentence spoken by the subject, which regions are being talked about. We employed a Long Short-Term Memory (LSTM) neural network to learn different patterns of dementia subjects and control ones and used it to perform a 10-fold cross validation-based classification. Our experimental results on the Pitt corpus from the DementiaBank resulted in a 82.9% accuracy at the subject level and 81.0% at the sample level. By employing data-augmentation techniques, the accuracy at both levels was increased to 83.6% and 82.1%, respectively. The performance of our proposed method outperforms most of the conventional methods, which reach, at best, an accuracy equal to 81.5% at the subject level. [ABSTRACT FROM AUTHOR]

The deep learning advancements have greatly improved the performance of speech recognition systems, and most recent systems are based on the Recurrent Neural Network (RNN). Overall, the RNN works fine with the small sequence data, but suffers from the gradient vanishing problem in case of large sequence. The transformer networks have neutralized this issue and have shown state-of-the-art results on sequential or speech-related data. Generally, in speech recognition, the input audio is converted into an image using Mel-spectrogram to illustrate frequencies and intensities. The image is classified by the machine learning mechanism to generate a classification transcript. However, the audio frequency in the image has low resolution and causing inaccurate predictions. This paper presents a novel end-to-end binary view transformer-based architecture for speech recognition to cope with the frequency resolution problem. Firstly, the input audio signal is transformed into a 2D image using Mel-spectrogram. Secondly, the modified universal transformers utilize the multi-head attention to derive contextual information and derive different speech-related features. Moreover, a feed forward neural network is also deployed for classification. The proposed system has generated robust results on Google's speech command dataset with an accuracy of 95.16% and with minimal loss. The binary-view transformer eradicates the eventuality of the over-fitting problem by deploying a multi view mechanism to diversify the input data, and multi-head attention captures multiple contexts from the data's feature map. [ABSTRACT FROM AUTHOR]

This paper develops an approach for household appliance identification and classification of household activities of daily living (ADLs) using residential smart meter data. The process of household appliance identification, i.e., decomposing a mains electricity measurement into each of its constituent individual appliances, is a very challenging classification problem. Recent advances have made deep learning a dominant approach for classification in fields, such as image processing and speech recognition. This paper presents a deep learning approach based on multilayer, feedforward neural networks that can identify common household electrical appliances from a typical household smart meter measurement. The performance of this approach is tested and validated using publicly available smart meter data sets. The identified appliances are then mapped to household activities, or ADLs. The resulting ADL classifier can provide insights into the behavior of the household occupants, which has a number of applications in the energy domain and in other fields. [ABSTRACT FROM AUTHOR]

Underwater creatures play a vital role in maintaining the delicate balance of the ocean ecosystem. In recent years, machine learning methods have been developed to identify underwater biologicals in the complex underwater environment. However, the scarcity and poor quality of underwater biological images present significant challenges to the recognition of underwater biological targets, especially multi-target recognition. To solve these problems, this paper proposed an ensemble method for underwater biological multi-target recognition. First, the CutMix method was improved for underwater biological image augmentation. Second, the white balance, multiscale retinal, and dark channel prior algorithms were combined to enhance the underwater biological image quality, which could largely improve the performance of underwater biological target recognition. Finally, an improved model was proposed for underwater biological multi-target recognition by using a mask region-based convolutional neural network (Mask-RCNN), which was optimized by the soft non-maximum suppression and attention-guided context feature pyramid network algorithms. We achieved 4.97 FPS, the mAP was 0.828, and the proposed methods could adapt well to underwater biological multi-target recognition. The recognition effectiveness of the proposed method was verified on the URPC2018 dataset by comparing it with current state-of-the-art recognition methods including you-only-look-once version 5 (YOLOv5) and the original Mask-RCNN model, where the mAP of the YOLOv5 model was lower. Compared with the original Mask-RCNN model, the mAP of the improved model increased by 3.2% to 82.8% when the FPS was reduced by only 0.38. [ABSTRACT FROM AUTHOR]

The recognition of human activities using vision-based techniques has become a crucial research field in video analytics. Over the last decade, there have been numerous advancements in deep learning algorithms aimed at accurately detecting complex human actions in video streams. While these algorithms have demonstrated impressive performance in activity recognition, they often exhibit a bias towards either model performance or computational efficiency. This biased trade-off between robustness and efficiency poses challenges when addressing complex human activity recognition problems. To address this issue, this paper presents a computationally efficient yet robust approach, exploiting saliency-aware spatial and temporal features for human action recognition in videos. To achieve effective representation of human actions, we propose an efficient approach called the dual-attentional Residual 3D Convolutional Neural Network (DA-R3DCNN). Our proposed method utilizes a unified channel-spatial attention mechanism, allowing it to efficiently extract significant human-centric features from video frames. By combining dual channel-spatial attention layers with residual 3D convolution layers, the network becomes more discerning in capturing spatial receptive fields containing objects within the feature maps. To assess the effectiveness and robustness of our proposed method, we have conducted extensive experiments on four well-established benchmark datasets for human action recognition. The quantitative results obtained validate the efficiency of our method, showcasing significant improvements in accuracy of up to 11% as compared to state-of-the-art human action recognition methods. Additionally, our evaluation of inference time reveals that the proposed method achieves up to a 74× improvement in frames per second (FPS) compared to existing approaches, thus showing the suitability and effectiveness of the proposed DA-R3DCNN for real-time human activity recognition. [ABSTRACT FROM AUTHOR]

Speech synthesis using machine learning has received much attention in the past decades. Recent development in speech synthesis applications make use of deep learning techniques as it yields far better results when compared to others in variety of applications. The perception of speech signal is dependent on both acoustic and visual information. This paper is review of the current available databases, video and audio features for speech synthesis. In this paper, we describe speech reconstruction from lip movements based on deep neural network model. This paper also highlights the performance and limitations of available methods. [ABSTRACT FROM AUTHOR]

The demand for talents extends beyond theoretical knowledge to include students' IPE (Ideological and Political Education). We can only continue to promote the rapid development of university teaching in China by putting in place a solid management system. It is the foundation of college students' IPE in this era of rapid information technology development to accurately adapt to and understand the actual problems and requirements of IPE proposed by the development of the times and innovate and develop accordingly. Deep learning (DL) is a sophisticated machine learning algorithm that outperforms previous technologies in speech and image recognition. In this paper, DLNN (deep learning neural network) technology is applied to IPE evaluation and a set of university-specific IPE evaluation index systems is developed. The BPNN (BP Neural Network) evaluation model is used to train and study a specific amount of teaching quality data in the MATLAB simulation tool. On the prediction of IPE evaluation, the BPNN algorithm has a prediction error of about 0.050.4 according to the findings. The validity and accuracy of using the BPNN algorithm to model IPE quality are demonstrated. [ABSTRACT FROM AUTHOR]

Social media platforms like Facebook, YouTube, and Twitter are banking on developing machine learning models to help stop the spread of hateful speech on their platforms. The idea is that machine learning models that utilize natural language processing will detect hate speech faster and better than people can. Despite numerous progress has been made for resource reach language, only a few attempts have been made for Ethiopian Languages such as Afaan Oromo. This paper examines the viability of deep learning models for Afaan Oromo hate speech recognition. Toward this, the biggest dataset of hate speech was collected and annotated by the language experts. Variations of profound deep learning models such as CNN, LSTMs, BiLSTMs, LSTM, GRU, and CNN-LSTM are examined to evaluate their viability in identifying Afaan Oromo Hate speeches. The result uncovers that the model dependent on CNN and Bi-LSTM outperforms all the other investigated models with an average F1-score of 87%. [ABSTRACT FROM AUTHOR]

Artificial intelligence speech recognition mostly judges the accuracy of grammar or sentence in the detection of pronunciation error, but has little research on pronunciation judgment, so it cannot effectively correct the pronunciation. This study analyzes the application of image target recognition in English learning task. Task-based approach emphasizes the process of English learning, not the result, the purposeful communication and meaning expression, encourages learners to open their mouths, and emphasizes that English language learning activities and their tasks are realistic in life. In addition, this paper introduces the DNN adaptive technique based on KL divergence regularization to adapt the acoustic model. Finally, this paper uses the experimental contrast method to compare and analyze the algorithm of this research with the traditional algorithm. The research shows that the recognition ability of the algorithm for confusing phonemes is improved than that of traditional algorithms, and this conclusion provides a powerful result for the introduction of error correction algorithms into education networks. By using the platform of autonomous learning center, students can improve their English level by completing the tasks chosen by teachers or by themselves and through training. [ABSTRACT FROM AUTHOR]

As a result of the fast rise of globalization, people in China are learning English at a rapid pace. However, there is a severe shortage of English teachers in the region, which is a major hindrance. To address these concerns, a deep learning-based algorithm is proposed that can not only check English pronunciation but also help learners distinguish between phonemic and quality phonemic while listening and differentiating, as well as correct phonemic errors, thereby increasing their language learning capacity. In order to study the application of nonlinear network identification technology in English learning, this paper evaluates the English pronunciation quality through the deep learning algorithm of deep learning combined with the related contents of neural network data model, and the experimental results of speech recognition structure are analyzed and discussed in detail. The concordance between machine and manual intonation evaluation is 80%, the concordance rate of adjacent intonation evaluation is 98.33%, and the Pearson correlation coefficient is 0.627 that shows the technique is reliable. The method of English pronunciation and speech identification model is sensible and dependable, which can give beginners a punctual, exact and impartial judgment and response guidance, assist learners to get on the differences between their phonemic and standard phonemic, and correct phonemic mistakes, in order to enhance the ability of oral English learning. [ABSTRACT FROM AUTHOR]

Parkinson's disease (PD) is a condition that degenerates over time and impairs speech and pronunciation because brain cells have died. This research work aims to predict parkinson disease using the voice features extracted from speech signals recorded from PD individuals with dysphonic speech disorders by employing deep learning algorithms. PD is challenging to diagnose early on in the clinical presentation. To address the issue in machine learning methods, this paper proposes a neural network model by processing speech signals to classify PD using the University of California Irvine (UCI) machine learning repository dataset. Initially, a pre-loss reduction module is created by using pre-sampling to make the dataset balanced by reducing the dimensionality and maintaining the size of the space without influencing the learning process for data preparation. The relevant features are derived using a novel multi-agent salp swarm (MASS) algorithm, and a novel Parkinson classification neural network (PCNN) is proposed to classify Parkinson's patients with high accuracy employing these derived features. The result shows that the models that use MASS-PCNN produce higher classification accuracy of 99.1%, precision of 97.8%, recall of 94.7% and F1-score of 0.995 when paralleled to the existing models. As an outcome, the suggested model will perform superior to common convolutional neural networks. [ABSTRACT FROM AUTHOR]

Data normalization is essential in many fields, such as speech recognition, deep learning, machine learning, and optimization. Many researchers focus on developing various normalization techniques, such as min-max, score-based, maximum, vector, or sum, on processing data efficiently. However, classical normalization methods may need to be improved in Multi-Criteria Decision-Analysis (MCDA), where criteria are often non-monotonic. This paper proposes a new approach for identifying and normalizing nonmonotonic criteria in MCDA. The method combines a stochastic optimization technique - a genetic algorithm - with normalization based on triangular fuzzy numbers. This paper compares the proposed approach with classical normalization techniques, such as min-max, maximum, vector, or non-linear, using the classical MCDA method - the Technique of Order Preference Similarity to the Ideal Solution (TOPSIS). The study results showed that the proposed approach to normalizing non-monotonic criteria yields a better match between decision preferences and the actual reference model than traditional normalization methods. Optimization using the genetic algorithm helps identify the means of triangular fuzzy numbers, which generate more accurate results. Analysis of the R2 coefficient confirms a very good fit of the proposed model. Therefore, the new non-linear normalization approach in MCDA presented in this work has great potential in adjusting decision preferences to non-monotonic criteria. Future research should continue on the accuracy of this approach in identifying non-linear decision models and apply it to other MCDA methods and real-world decision problems. [ABSTRACT FROM AUTHOR]

As a key field in music information retrieval, music emotion recognition is indeed a challenging task. To enhance the accuracy of music emotion classification and recognition, this paper uses the idea of inception structure to use different receptive fields to extract features of different dimensions and perform compression, expansion, and recompression operations to mine more effective features and connect the timing signals in the residual network to the GRU module to extract timing features. A one-dimensional (1D) residual Convolutional Neural Network (CNN) with an improved Inception module and Gate Recurrent Unit (GRU) was presented and tested on the Soundtrack dataset. Fast Fourier Transform (FFT) was used to process the samples experimentally and determine their spectral characteristics. Compared with the shallow learning methods such as support vector machine and random forest and the deep learning method based on Visual Geometry Group (VGG) CNN proposed by Sarkar et al., the proposed deep learning method of the 1D CNN with the Inception-GRU residual structure demonstrated better performance in music emotion recognition and classification tasks, achieving an accuracy of 84%. [ABSTRACT FROM AUTHOR]

SYNOPSIS: This paper aims to promote the application of deep learning to audit procedures by illustrating how the capabilities of deep learning for text understanding, speech recognition, visual recognition, and structured data analysis fit into the audit environment. Based on these four capabilities, deep learning serves two major functions in supporting audit decision making: information identification and judgment support. The paper proposes a framework for applying these two deep learning functions to a variety of audit procedures in different audit phases. An audit data warehouse of historical data can be used to construct prediction models, providing suggested actions for various audit procedures. The data warehouse will be updated and enriched with new data instances through the application of deep learning and a human auditor's corrections. Finally, the paper discusses the challenges faced by the accounting profession, regulators, and educators when it comes to applying deep learning. [ABSTRACT FROM AUTHOR]

The rehabilitation of aphasics is fundamentally based on the assessment of speech impairment. Developing methods for assessing speech impairment automatically is important due to the growing number of stroke cases each year. Traditionally, aphasia is assessed manually using one of the well-known assessment batteries, such as the Western Aphasia Battery (WAB), the Chinese Rehabilitation Research Center Aphasia Examination (CRRCAE), and the Boston Diagnostic Aphasia Examination (BDAE). In aphasia testing, a speech-language pathologist (SLP) administers multiple subtests to assess people with aphasia (PWA). The traditional assessment is a resource-intensive process that requires the presence of an SLP. Thus, automating the assessment of aphasia is essential. This paper evaluated and compared custom machine learning (ML) speech recognition algorithms against off-the-shelf platforms using healthy and aphasic speech datasets on the naming and repetition subtests of the aphasia battery. Convolutional neural networks (CNN) and linear discriminant analysis (LDA) are the customized ML algorithms, while Microsoft Azure and Google speech recognition are off-the-shelf platforms. The results of this study demonstrated that CNN-based speech recognition algorithms outperform LDA and off-the-shelf platforms. The ResNet-50 architecture of CNN yielded an accuracy of 99.64 ± 0.26% on the healthy dataset. Even though Microsoft Azure was not trained on the same healthy dataset, it still generated comparable results to the LDA and superior results to Google's speech recognition platform. [ABSTRACT FROM AUTHOR]

Robust Lombard speech-in-noise detecting is challenging. This study proposes a strategy to detect Lombard speech using a machine learning approach for applications such as public address systems that work in near real time. The paper starts with the background concerning the Lombard effect. Then, assumptions of the work performed for Lombard speech detection are outlined. The framework proposed combines convolutional neural networks (CNNs) and various two-dimensional (2D) speech signal representations. To reduce the computational cost and not resign from the 2D representation-based approach, a strategy for threshold-based averaging of the Lombard effect detection results is introduced. The pseudocode of the averaging process is also included. A series of experiments are performed to determine the most effective network structure and the 2D speech signal representation. Investigations are carried out on German and Polish recordings containing Lombard speech. All 2D signal speech representations are tested with and without augmentation. Augmentation means using the alpha channel to store additional data: gender of the speaker, F0 frequency, and first two MFCCs. The experimental results show that Lombard and neutral speech recordings can clearly be discerned, which is done with high detection accuracy. It is also demonstrated that the proposed speech detection process is capable of working in near real-time. These are the key contributions of this work. [ABSTRACT FROM AUTHOR]

Lip-reading technologies are rapidly progressing following the breakthrough of deep learning. It plays a vital role in its many applications, such as: human-machine communication practices or security applications. In this paper, we propose to develop an effective lip-reading recognition model for Arabic visual speech recognition by implementing deep learning algorithms. The Arabic visual datasets that have been collected contains 2400 records of Arabic digits and 960 records of Arabic phrases from 24 native speakers. The primary purpose is to provide a high-performance model in terms of enhancing the preprocessing phase. Firstly, we extract keyframes from our dataset. Secondly, we produce a Concatenated Frame Images (CFIs) that represent the utterance sequence in one single image. Finally, the VGG-19 is employed for visual features extraction in our proposed model. We have examined different keyframes: 10, 15, and 20 for comparing two types of approaches in the proposed model: (1) the VGG-19 base model and (2) VGG-19 base model with batch normalization. The results show that the second approach achieves greater accuracy: 94% for digit recognition, 97% for phrase recognition, and 93% for digits and phrases recognition in the test dataset. Therefore, our proposed model is superior to models based on CFIs input. [ABSTRACT FROM AUTHOR]

Ensuring the best quality and performance of modern speech technologies, today, is possible based on the widespread use of machine learning methods. The idea of this project is to study and implement an end-to-end system of automatic speech recognition using machine learning methods, as well as to develop new mathematical models and algorithms for solving the problem of automatic speech recognition for agglutinative (Turkic) languages. Many research papers have shown that deep learning methods make it easier to train automatic speech recognition systems that use an end-to-end approach. This method can also train an automatic speech recognition system directly, that is, without manual work with raw signals. Despite the good recognition quality, this model has some drawbacks. These disadvantages are based on the need for a large amount of data for training. This is a serious problem for low-data languages, especially Turkic languages such as Kazakh and Azerbaijani. To solve this problem, various methods are needed to apply. Some methods are used for end-to-end speech recognition of languages belonging to the group of languages of the same family (agglutinative languages). Method for low-resource languages is transfer learning, and for large resources – multi-task learning. To increase efficiency and quickly solve the problem associated with a limited resource, transfer learning was used for the end-to-end model. The transfer learning method helped to fit a model trained on the Kazakh dataset to the Azerbaijani dataset. Thereby, two language corpora were trained simultaneously. Conducted experiments with two corpora show that transfer learning can reduce the symbol error rate, phoneme error rate (PER), by 14.23 % compared to baseline models (DNN+HMM, WaveNet, and CNC+LM). Therefore, the realized model with the transfer method can be used to recognize other low-resource languages. [ABSTRACT FROM AUTHOR]

Direct word discovery from audio speech signals is a very difficult and challenging problem for a developmental robot. Human infants are able to discover words directly from speech signals, and, to understand human infants' developmental capability using a constructive approach, it is very important to build a machine learning system that can acquire knowledge about words and phonemes, i.e. a language model and an acoustic model, autonomously in an unsupervised manner. To achieve this, the nonparametric Bayesian double articulation analyzer (NPB-DAA) with the deep sparse autoencoder (DSAE) is proposed in this paper. The NPB-DAA has been proposed to achieve totally unsupervised direct word discovery from speech signals. However, the performance was still unsatisfactory, although it outperformed pre-existing unsupervised learning methods. In this paper, we integrate the NPB-DAA with the DSAE, which is a neural network model that can be trained in an unsupervised manner, and demonstrate its performance through an experiment about direct word discovery from auditory speech signals. The experiment shows that the combinedmethod, the NPB-DAAwith the DSAE, outperforms pre-existing unsupervised learningmethods, and shows stateof- the-art performance. It is also shown that the proposed method outperforms several standard speech recognizer-based methods with true word dictionaries. [ABSTRACT FROM AUTHOR]

With the rapid development of network technology and entertainment creation, the types of movies have become more and more diverse, which makes users wonder how to choose the type of movies. In order to improve the selection efficiency, recommend Algorithm came into being. Deep learning is a research field that has received extensive attention from scholars in recent years. Due to the characteristics of its deep architecture, deep learning models can learn more complex structures. Therefore, deep learning algorithms in speech recognition, machine translation, image recognition, and other fields have achieved impressive results. This article mainly introduces the research of personalized movie recommendation methods based on deep learning and intends to provide ideas and directions for the research of personalized movie recommendation under deep learning. This paper proposes a research method of personalized movie recommendation methods based on deep learning, including an overview of personalized recommendation and collaborative filtering recommendation algorithms, which are used to conduct research experiments on personalized movie recommendation methods based on deep learning. The experimental results in this paper show that the accuracy of the training set of the Seq2Seq model based on the LSTM recurrent neural network reaches 96.27% and the accuracy of the test set reaches 95.89%, which can be better for personalized movie recommendation. [ABSTRACT FROM AUTHOR]

Nowadays automatic speech recognition (ASR) systems can achieve higher and higher accuracy rates depending on the methodology applied and datasets used. The rate decreases significantly when the ASR system is being used with a non-native speaker of the language to be recognized. The main reason for this is specific pronunciation and accent features related to the mother tongue of that speaker, which influence the pronunciation. At the same time, an extremely limited volume of labeled non-native speech datasets makes it difficult to train, from the ground up, sufficiently accurate ASR systems for non-native speakers.In this research, we address the problem and its influence on the accuracy of ASR systems, using the style transfer methodology. We designed a pipeline for modifying the speech of a non-native speaker so that it more closely resembles the native speech. This paper covers experiments for accent modification using different setups and different approaches, including neural style transfer and autoencoder. The experiments were conducted on English language pronounced by Japanese speakers (UME-ERJ dataset). The results show that there is a significant relative improvement in terms of the speech recognition accuracy. Our methodology reduces the necessity of training new algorithms for non-native speech (thus overcoming the obstacle related to the data scarcity) and can be used as a wrapper for any existing ASR system. The modification can be performed in real time, before a sample is passed into the speech recognition system itself. [ABSTRACT FROM AUTHOR]

Deep Learning has revolutionized Machine Learning, enhancing our ability to solve complex computational problems. From image classification to speech recognition, the technology can be beneficial in a broad range of scenarios. However, the barrier to entry is quite high, especially when programming skills are missing. In this paper, we present the development of a learning application that is easy to use, yet powerful enough to solve practical Deep Learning problems. We followed the human-centered design approach and conducted a technical evaluation to identify solvable classification problems. Afterwards, we conducted an online user evaluation to gain insights on users' experience with the app, and to understand positive as well as negative aspects of our implemented concept. Our results show that participants liked using the app and found it useful, especially for beginners. Nonetheless, future iterations of the learning app should step-wise include more features to support advancing users. [ABSTRACT FROM AUTHOR]

Training procedures of a deep neural network are still an area with ample research possibilities and constant improvement either to increase its efficiency or its time performance. One of the lesser-addressed components is its objective function, which is an underlying aspect to consider when there is the necessity to achieve better error rates in the area of automatic speech recognition. The aim of this paper is to present two new variations of the frame-level cost function for training a deep neural network with the purpose of obtaining superior word error rates in speech recognition applied to a case study in Spanish. The first proposed function is a fusion between the boosted cross-entropy and the so called cross-entropy/log-posterior-ratio. The main idea is to jointly emphasize the prediction of difficult/crucial frames provided by a boosting factor and at the same time enlarge the distance between the target senone and its closest competitor. The second proposal is a fusion between the non-uniform mapped cross-entropy and the cross-entropy/log-posterior-ratio. This function utilizes both the mapped function to enhance the frames that have ambiguity in their belonging to specific senones and the log-posterior-ratio with the purpose of separating the target senone against the most competing tied tri-phone state. The proposed approaches are compared against those frame-level cost functions discussed in the state of the art. This comparative has been made by using a personalized mid-vocabulary speaker-independent voice corpus. This dataset is employed for the recognition of digit strings and personal name lists in Spanish from the northern central part of México on a connected-words phone dialing task. A relative word error rate improvement of 15.14% and 12.30% is obtained with the two proposed approaches, respectively, against the plain well-established cross-entropy loss function. [ABSTRACT FROM AUTHOR]

• Reviews more than 45 recent solutions papers, and more than 40 different privacy-preserving deep learning techniques. • Proposes a multi-level taxonomy that classifies the privacy-preserving deep learning techniques. • Summarizes evaluation results of the reviewed solutions with respect to performance metrics. • Discusses and outline a number of learned lessons of each privacy-preserving task. • Presents solutions comparison, highlights open research challenges and provides some recommendations for future research. Deep learning is one of the advanced approaches of machine learning, and has attracted a growing attention in the recent years. It is used nowadays in different domains and applications such as pattern recognition, medical prediction, and speech recognition. Differently from traditional learning algorithms, deep learning can overcome the dependency on hand-designed features. Deep learning experience is particularly improved by leveraging powerful infrastructures such as clouds and adopting collaborative learning for model training. However, this comes at the expense of privacy, especially when sensitive data are processed during the training and the prediction phases, as well as when training model is shared. In this paper, we provide a review of the existing privacy-preserving deep learning techniques, and propose a novel multi-level taxonomy, which categorizes the current state-of-the-art privacy-preserving deep learning techniques on the basis of privacy-preserving tasks at the top level, and key technological concepts at the base level. This survey further summarizes evaluation results of the reviewed solutions with respect to defined performance metrics. In addition, it derives a set of learned lessons from each privacy-preserving task. Finally, it highlights open research challenges and provides some recommendations as future research directions. [ABSTRACT FROM AUTHOR]

Deep learning has been proved to be an advanced technology for big data analysis with a large number of successful cases in image processing, speech recognition, object detection, and so on. Recently, it has also been introduced in food science and engineering. To our knowledge, this review is the first in the food domain. In this paper, we provided a brief introduction of deep learning and detailedly described the structure of some popular architectures of deep neural networks and the approaches for training a model. We surveyed dozens of articles that used deep learning as the data analysis tool to solve the problems and challenges in food domain, including food recognition, calories estimation, quality detection of fruits, vegetables, meat and aquatic products, food supply chain, and food contamination. The specific problems, the datasets, the preprocessing methods, the networks and frameworks used, the performance achieved, and the comparison with other popular solutions of each research were investigated. We also analyzed the potential of deep learning to be used as an advanced data mining tool in food sensory and consume researches. The result of our survey indicates that deep learning outperforms other methods such as manual feature extractors, conventional machine learning algorithms, and deep learning as a promising tool in food quality and safety inspection. The encouraging results in classification and regression problems achieved by deep learning will attract more research efforts to apply deep learning into the field of food in the future. [ABSTRACT FROM AUTHOR]

There are a wide variety of intelligence accelerators with promising performance and energy efficiency, deployed in a broad range of applications such as computer vision and speech recognition. However, programming productivity hinders the deployment of deep learning accelerators. The low-level library invoked in the high-level deep learning framework which supports the end-to-end execution with a given model, is designed to reduce the programming burden on the intelligence accelerators. Unfortunately, it is inflexible for developers to build a network model for every deep learning application, which probably brings unnecessary repetitive implementation. In this paper, a flexible and efficient programming framework for deep learning accelerators, FlexPDA, is proposed, which provides more optimization opportunities than the low-level library and realizes quick transplantation of applications to intelligence accelerators for fast upgrades. We evaluate FlexPDA by using 10 representative operators selected from deep learning algorithms and an end-to-end network. The experimental results validate the effectiveness of FlexPDA, which achieves an end-to-end performance improvement of 1.620x over the low-level library. [ABSTRACT FROM AUTHOR]

In the emerging information retrieval world, the selection of keywords and the generation of queries are very critical for the efficient retrieval. The most new-generations database applications request keen interface to upgrade productive connections among various databases and the clients. Majority of the accessible interfaces for databases should be intelligent and also should understand natural language expressions. The paper deals with, the mapping of natural language queries which is in spoken form, and converting into words forming the foundation of SQL. We give a general framework for a smart database interface which could be linked to any database. One of the most striking features of this interface is domain-independence. The smart interface employs speech recognition techniques to convert spoken language input into text. Then a semantic matching technique is employed in converting natural language query to SQL words, complemented by using dictionary and a set of production rules. The dictionary consists of semantics sets for columns and tables. The identified type of query, query-word tuple is memorized and sent through a convolution neural network for better construction. This work tries to use natural language processing techniques, parsing and POS tagging to identify semantic structures of inputs. The natural language techniques are used to increase the production of unambiguous queries and speed in detecting and identifying the important parts in a query. [ABSTRACT FROM AUTHOR]

In recent years, we have witnessed the developments that deep learning has brought to machine learning. It has solved many problems in the areas of computer vision, speech recognition, natural language processing, and various other tasks with stateof- the-art performance. However, the data in these tasks is typically represented in Euclidean space. As technology develops, more and more applications are generating data from non-Euclidean domains and representing them as graphs with complex relationships and interdependencies between objects. This poses a significant challenge to deep learning algorithms. This is because, due to the uniqueness of graphs, applying deep learning to the ubiquitous graph data is not an easy task. To solve the problem in non-Euclidean domains, Graph Neural Networks (GNNs) have emerged. A Graph Neural Network (GNN) is a neural model that captures dependencies between graphs by passing messages between graph nodes. This paper introduces commonly used graph neural networks, their learning methods, and common datasets for graph neural networks. It also provides an outlook on the future of Graph Neural Networks. [ABSTRACT FROM AUTHOR]

Conventionally, the extraction of hand-crafted acoustic features has been separated from the task of establishing robust machine-learning models in speech processing. The manual approach of feature engineering is both time-consuming and necessitates specialist knowledge, posing significant hindrances. Moreover, the resulting features may not be ideal for the desired application. The speech community has adopted raw waveform modeling to enhance performance. These techniques learn an optimized representation of the input automatically. With deep learning (DL) advancements, raw waveform modeling has become valuable for tasks like classification and prediction. The primary aim of this survey is to offer valuable insights and fills a gap in the existing literature by providing a comprehensive review of the state-of-the-art in speech and speaker recognition using raw waveform modeling for both adult and children's speech. The article covers papers from 2013–2023 and is the first to review both adult and children's databases. The article focuses on the advantages of raw waveform models. It presents essential concepts and techniques while discussing the challenges and limitations of using raw waveforms for speech and speaker recognition in both adult and children's speech. The article also evaluates recent progress in DL architectures such as SincNet, ResNet, and RawNet, and outlines future research directions in the field. [ABSTRACT FROM AUTHOR]

Deep learning has recently received a lot of attention as a feasible solution to a variety of artificial intelligence difficulties. Convolutional neural networks (CNNs) outperform other deep learning architectures in the application of object identification and recognition when compared to other machine learning methods. Speech recognition, pattern analysis, and image identification, all benefit from deep neural networks. When performing image operations on noisy images, such as fog removal or low light enhancement, image processing methods such as filtering or image enhancement are required. The study shows the effect of using Multi-scale deep learning Context Aggregation Network CAN on Bilateral Filtering Approximation (BFA) for de-noising noisy CCTV images. Data-store is used tomanage our dataset, which is an object or collection of data that are huge to enter in memory, it allows to read, manage, and process data located in multiple files as a single entity. The CAN architecture provides integral deep learning layers such as input, convolution, back normalization, and Leaky ReLu layers to construct multi-scale. It is also possible to add custom layers like adaptor normalization (µ) and adaptive normalization (Lambda) to the network. The performance of the developed CAN approximation operator on the bilateral filtering noisy image is proven when improving both the noisy reference image and a CCTV foggy image. The three image evaluation metrics (SSIM, NIQE, and PSNR) evaluate the developed CAN approximation visually and quantitatively when comparing the created de-noised image over the reference image.Compared with the input noisy image, these evaluation metrics for the developed CAN de-noised image were (0.92673/0.76253, 6.18105/12.1865, and 26.786/20.3254) respectively [ABSTRACT FROM AUTHOR]

Social media has become a constant in our everyday life. However, its steady growth has increased the hate speech and hostile content problem. To curb this, hate speech detection and recognition is required, but it is faced to two major challenges - laws and enforcement, and automatic computerized hate speech detection. Although many studies are already implemented in detecting hate content, many of these are done in a single setting showing a single dataset in comparison to machine learning or deep learning models. Thus, there is no comparison between previous approaches and recent inventions such as transformer model. Therefore, in this work we explored and compared recent advanced approaches in automatic hate speech detection. Our aim is to analyze the influence different approaches in detecting hate content and its applicability in the real world. Several experiments were conducted on eight real hate speech datasets from recent studies. We present the results of each comparison which shows that the recent transformer model approach is able to outmatch many of the previous hate speech recognition models by significant G-Means and F1 scores. To the author's knowledge, this paper is the first attempt to present a large comparative study of approaches in hate speech detection. [ABSTRACT FROM AUTHOR]

Deep learning is a timely research direction in machine learning, where breakthrough progress has been made in both academe and industries, bringing promising results in speech recognition, computer vision, industrial control and automation, etc. The motivation of deep learning is primarily to establish a model to simulate the neural connection structure of the human brain. While dealing with complex tasks, deep learning adopts a number of transformation stages to deliver the in-depth description and interpretation of the data. Deep learning achieves exceptional power and flexibility by learning to represent the task through a nested hierarchy of layers, with more abstract representations formed successively in terms of less abstract ones. One of the key issues of existing deep learning approaches is that the meaningful representations can be learned only when their hyperparameter settings are properly specified beforehand, and general parameters are learned during the training process. Until now, not much research has been dedicated to automatically set the hyperparameters, and accurately find the globally optimal general parameters. However, this problem can be formulated as optimization problems, including discrete optimization, constrained optimization, large-scale global optimization, and multiobjective optimization, by engaging mechanisms of evolutionary computation. [ABSTRACT FROM AUTHOR]

Artificial Neural Networks (ANNs) were created inspired by the neural networks in the human brain and have been widely applied in speech processing. The application areas of ANN include: Speech recognition, speech emotion recognition, language identification, speech enhancement, and speech separation, amongst others. Likewise, given that speech processing performed by humans involves complex cognitive processes known as auditory attention, there has been a growing amount of papers proposing ANNs supported by deep learning algorithms in conjunction with some mechanism to achieve symmetry with the human attention process. However, while these ANN approaches include attention, there is no categorization of attention integrated into the deep learning algorithms and their relation with human auditory attention. Therefore, we consider it necessary to have a review of the different ANN approaches inspired in attention to show both academic and industry experts the available models for a wide variety of applications. Based on the PRISMA methodology, we present a systematic review of the literature published since 2000, in which deep learning algorithms are applied to diverse problems related to speech processing. In this paper 133 research works are selected and the following aspects are described: (i) Most relevant features, (ii) ways in which attention has been implemented, (iii) their hypothetical relationship with human attention, and (iv) the evaluation metrics used. Additionally, the four publications most related with human attention were analyzed and their strengths and weaknesses were determined. [ABSTRACT FROM AUTHOR]

To improve the performance of multi-accent Mandarin speech recognition task,this paper presented a method for hybrid end-to-end automatic speech recognition(ASR) by combining CTC and multi-head attention by using a multiobjective training and joint decoding.The analysis shows that hybrid model with lower CTC weight and deeper encoder layers performance better learning capacity.And it trained a very deep models with up to 48 layers for encode-decoder architecture,which outperforms all previous end-to-end ASR approaches on Aidatatang 200 h multi-accent dataset,achieves 5.6% character error rate(CER) and 26.2% sentence error rate(SER).The experiment proves that the recognition rate of the end-to-end model proposed exceeds the general end-to-end model,and it has certain advancedness in solving the Mandarin recognition with accents. [ABSTRACT FROM AUTHOR]

Deep learning is a branch of neural network which has been intensively developed in the last decade. Due to the high-accuracy classification ability, the deep learning algorithms have been widely used in many fields, such as speech recognition, image recognition, and natural speech processing. However, they also show some shortcomings especially on the selection of some parameters in the network, including hyper-parameters, which is still treated as a time consuming task. In this paper, a modified ABC (ABC-ISB) optimization algorithm is proposed to automatically train the parameters of Feed-Forward Artificial Neural Networks, which is a typical a neural network. In the proposed ABC algorithm, we utilize the information of neighbors with better performance to accelerate the convergence of employed and onlooker bees respectively. In addition, a new selection strategy and a gbest-guided strategy are introduced to enhance the global search capability and balance the exploration and exploitation of the algorithm separately. The experimental results show our ABC-ISB is generally leading and competitive. [ABSTRACT FROM AUTHOR]

Classification task is one of the most common tasks in machine learning. This supervised learning problem consists in assigning each input to one of a finite number of discrete categories. Classification task appears naturally in numerous applications, such as medical image processing, speech recognition, maintenance systems, accident detection, autonomous driving etc. In the last decade methods of deep learning have proven to be extremely efficient in multiple machine learning problems, including classification. Whereas the neural network architecture might depend a lot on data type and restrictions posed by the nature of the problem (for example, real-time applications), the process of its training (i.e. finding model's parameters) is almost always presented as loss function optimization problem. Cross-entropy is a loss function often used for multiclass classification problems, as it allows to achieve high accuracy results. Here we propose to use a generalized version of this loss based on Renyi divergence and entropy. We remark that in case of binary labels proposed generalization is reduced to cross-entropy, thus we work in the context of soft labels. Specifically, we consider a problem of image classification being solved by application of convolution neural networks with mixup regularizer. The latter expands the training set by taking convex combination of pairs of data samples and corresponding labels. Consequently, labels are no longer binary (corresponding to single class), but have a form of vector of probabilities. In such settings cross-entropy and proposed generalization with Renyi divergence and entropy are distinct, and their comparison makes sense. To measure effectiveness of the proposed loss function we consider image classification problem on benchmark CIFAR-10 dataset. This dataset consists of 60000 images belonging to 10 classes, where images are color and have the size of 32×32. Training set consists of 50000 images, and the test set contains 10000 images. For the convolution neural network, we follow [1] where the same classification task was studied with respect to different loss functions and consider the same neural network architecture in order to obtain comparable results. Experiments demonstrate superiority of the proposed method over cross-entropy for loss function parameter value α < 1. For parameter value α > 1 proposed method shows worse results than crossentropy loss function. Finally, parameter value α = 1 corresponds to cross-entropy. [ABSTRACT FROM AUTHOR]

Fundamental speech recognition technologies for high-resourced languages are currently successful to build high-quality applications with the use of deep learning models. However, the problem of "borrowing" these speech recognition technologies for under-resourced languages like Vietnamese still has challenges. This study reviews fundamental studies on speech recognition in general as well as speech recognition in Vietnamese, an under-resourced language in particular. Then, it specifies the urgent issues that need current research attention to build Vietnamese speech recognition applications in practice, especially the need to build an open large sentence-labeled speech corpus and open platform for related research, which mostly benefits small individuals/organizations who do not have enough resources. [ABSTRACT FROM AUTHOR]

The Zhuang ethnic minority in China possesses its own ethnic language and no ethnic script. Cultural exchange and transmission encounter hurdles as the Zhuang rely exclusively on oral communication. An online cloud-based platform was required to enhance linguistic communication. First, a database of 200 h of annotated Zhuang speech was created by collecting standard Zhuang speeches and improving database quality by removing transcription inconsistencies and text normalization. Second, SAformerNet, a more efficient and accurate transformer-based automatic speech recognition (ASR) network, is achieved by inserting additional downsampling modules. Subsequently, a Neural Machine Translation (NMT) model for translating Zhuang into other languages is constructed by fine-tuning the BART model and corpus filtering strategy. Finally, for the network's responsiveness to real-world needs, edge-computing techniques are applied to relieve network bandwidth pressure. An edge-computing private cloud system based on FPGA acceleration is proposed to improve model operation efficiency. Experiments show that the most critical metric of the system, model accuracy, is above 93%, and inference time is reduced by 29%. The computational delay for multi-head self-attention (MHSA) and feed-forward network (FFN) modules has been reduced by 7.1 and 1.9 times, respectively, and terminal response time is accelerated by 20% on average. Generally, the scheme provides a prototype tool for small-scale Zhuang remote natural language tasks in mountainous areas. [ABSTRACT FROM AUTHOR]