Search

Voice isolation, a prominent research area in the field of speech processing, has garnered a great deal of attention due to its prospective implications in numerous domains. Deep neural networks (DNNs) have emerged as a potent instrument for addressing the challenges associated with vocal isolation. This paper presents a comprehensive study on the use of DNNs for voice isolation, focusing on speech recognition and speaker identification tasks. The proposed method uses frequency domain and time domain techniques to improve the separation of target utterances from background noise. The experimental results demonstrate the efficacy of the proposed method, revealing substantial improvements in voice isolation precision and robustness. This study's findings contribute to the increasing corpus of research on voice isolation techniques and provide valuable insights into the application of DNNs to improve speech processing tasks . [ABSTRACT FROM AUTHOR]

A lack of labeled examples is a problem in different domains, such as text and image processing, medicine, and static reservoir characterization, because supervised learning relies on vast volumes of these data to perform successfully, but this is quite expensive. However, large amounts of unlabeled data exist in these domains. The deep semi-supervised learning (DSSL) approach leverages unlabeled data to improve supervised learning performance using deep neural networks. This approach has succeeded in image recognition, text classification, and speech recognition. Nevertheless, there have been few works on pre-stack seismic reservoir characterization, in which knowledge of rock and fluid properties is fundamental for oil exploration. This paper proposes a methodology to estimate acoustic impedance using pre-stack seismic data and DSSL with a recurrent neural network. The few labeled datasets for training were pre-processed from raw seismic and acoustic impedance data from five borehole logs. The results showed that the acoustic impedance estimation at the well location and outside it was better predicted by the DSSL compared to the supervised version of the same neural network. Therefore, employing a large amount of unlabeled data can be helpful in the development of seismic data interpretation systems. [ABSTRACT FROM AUTHOR]

Dysarthric speech has several pathological characteristics, such as discontinuous pronunciation, uncontrolled volume, slow speech, explosive pronunciation, improper pauses, excessive nasal sounds, and air-flow noise during pronunciation, which differ from healthy speech. Automatic speech recognition (ASR) can be very helpful for speakers with dysarthria. Our research aims to provide a scoping review of ASR for dysarthric speech, covering papers in this field from 1990 to 2022. Our survey found that the development of research studies about the acoustic features and acoustic models of dysarthric speech is nearly synchronous. During the 2010s, deep learning technologies were widely applied to improve the performance of ASR systems. In the era of deep learning, many advanced methods (such as convolutional neural networks, deep neural networks, and recurrent neural networks) are being applied to design acoustic models and lexical and language models for dysarthric-speech-recognition tasks. Deep learning methods are also used to extract acoustic features from dysarthric speech. Additionally, this scoping review found that speaker-dependent problems seriously limit the generalization applicability of the acoustic model. The scarce available speech data cannot satisfy the amount required to train models using big data. [ABSTRACT FROM AUTHOR]

Acoustic signal analysis is an important component of transformer online monitoring. Currently, traditional methods have problems such as low spectral resolution, imbalanced sample distribution, and unsatisfactory classification performance. This article first introduces the matrix pencil algorithm for time-frequency spectrum analysis of acoustic signals, and then uses the SMOTE algorithm to expand the imbalanced samples. Then, an ACmix hybrid deep neural network model is constructed to classify 11 types of transformer operation and environmental acoustic signals. Finally, detailed experiments were conducted on the method proposed in this paper, and the experimental results showed that the matrix pencil algorithm has high time-frequency resolution and good noise resistance performance. The SMOTE sample expansion method can significantly improve the recognition accuracy by more than 2%. Overall accuracy of the proposed method in acoustic signal classification tasks reaches 91.81%. [ABSTRACT FROM AUTHOR]

This paper analyzes the simulation of special effects animation through fluid equations and data-driven methods. This paper also considers the needs of computer fluid animation simulation in terms of computational accuracy and simulation efficiency, takes high real-time, high interactivity, and high physical accuracy of simulation algorithm as the research focus and target, and proposes a solution algorithm and acceleration scheme based on deep neural network framework for the key problems of simulation of natural phenomena including smoke and liquid. With the deep development of artificial intelligence technology, deep neural network models are widely used in research fields such as computer image classification, speech recognition, and fluid detail synthesis with their powerful data learning capability. Its stable and efficient computational model provides a new problem-solving approach for computerized fluid animation simulation. In terms of time series reconstruction, this paper adopts a tracking-based reconstruction method, including target tracking, 2D trajectory fitting and repair, and 3D trajectory reconstruction. For continuous image sequences, a linear dynamic model algorithm based on pyramidal optical flow is used to track the feature centers of the objects, and the spatial coordinates and motion parameters of the feature points are obtained by reconstructing the motion trajectories. The experimental results show that in terms of spatial reconstruction, the matching method proposed in this paper is more accurate compared with the traditional stereo matching algorithm; in terms of time series reconstruction, the error of target tracking reduced. Finally, the 3D motion trajectory of the point feature object and the motion pattern at a certain moment are shown, and the method in this paper obtains more ideal results, which proves the effectiveness of the method. [ABSTRACT FROM AUTHOR]

Stock market is considered chaotic, complex, volatile and dynamic. Undoubtedly, its prediction is one of the most challenging tasks in time series forecasting. Moreover existing Artificial Neural Network (ANN) approaches fail to provide encouraging results. Meanwhile advances in machine learning have presented favourable results for speech recognition, image classification and language processing. Methods applied in digital signal processing can be applied to stock data as both are time series. Similarly, learning outcome of this paper can be applied to speech time series data. Deep learning for stock prediction has been introduced in this paper and its performance is evaluated on Google stock price multimedia data (chart) from NASDAQ. The objective of this paper is to demonstrate that deep learning can improve stock market forecasting accuracy. For this, (2D)PCA + Deep Neural Network (DNN) method is compared with state of the art method 2-Directional 2-Dimensional Principal Component Analysis (2D)PCA + Radial Basis Function Neural Network (RBFNN). It is found that the proposed method is performing better than the existing method RBFNN with an improved accuracy of 4.8% for Hit Rate with a window size of 20. Also the results of the proposed model are compared with the Recurrent Neural Network (RNN) and it is found that the accuracy for Hit Rate is improved by 15.6%. The correlation coefficient between the actual and predicted return for DNN is 17.1% more than RBFNN and it is 43.4% better than RNN. [ABSTRACT FROM AUTHOR]

In this study, multidimensional feature extraction is performed on the U-language recordings of the test takers, and these features are evaluated separately, with five categories of features: pronunciation, fluency, vocabulary, grammar, and semantics. A deep neural network model is constructed to model the feature values to obtain the final score. Based on the previous research, this study uses a deep neural network training model instead of linear regression to improve the correlation between model score and expert score. The method of using word frequency for semantic scoring is replaced by the LDA topic model for semantic analysis, which eliminates the need for experts to manually label keywords before scoring and truly automates the critique. Also, this paper introduces text cleaning after speech recognition and deep learning-based speech noise reduction technology in the scoring model, which improves the accuracy of speech recognition and the overall accuracy of the scoring model. Also, innovative applications and improvements are made to key technologies, and the latest technical solutions are integrated and improved. A new open oral grading model is proposed and implemented, and innovations are made in the method of speech feature extraction to improve the dimensionality of open oral grading. [ABSTRACT FROM AUTHOR]

Speech synthesis using machine learning has received much attention in the past decades. Recent development in speech synthesis applications make use of deep learning techniques as it yields far better results when compared to others in variety of applications. The perception of speech signal is dependent on both acoustic and visual information. This paper is review of the current available databases, video and audio features for speech synthesis. In this paper, we describe speech reconstruction from lip movements based on deep neural network model. This paper also highlights the performance and limitations of available methods. [ABSTRACT FROM AUTHOR]

The demand for talents extends beyond theoretical knowledge to include students' IPE (Ideological and Political Education). We can only continue to promote the rapid development of university teaching in China by putting in place a solid management system. It is the foundation of college students' IPE in this era of rapid information technology development to accurately adapt to and understand the actual problems and requirements of IPE proposed by the development of the times and innovate and develop accordingly. Deep learning (DL) is a sophisticated machine learning algorithm that outperforms previous technologies in speech and image recognition. In this paper, DLNN (deep learning neural network) technology is applied to IPE evaluation and a set of university-specific IPE evaluation index systems is developed. The BPNN (BP Neural Network) evaluation model is used to train and study a specific amount of teaching quality data in the MATLAB simulation tool. On the prediction of IPE evaluation, the BPNN algorithm has a prediction error of about 0.050.4 according to the findings. The validity and accuracy of using the BPNN algorithm to model IPE quality are demonstrated. [ABSTRACT FROM AUTHOR]

Up to now, various methods are used for Automatic Speech Recognition (ASR), and among which the Hidden Markov Model (HMM) and Artificial Neural Networks (ANNs) are the most important ones. One of the existing challenges is increasing the accuracy and efficiency of these systems. One way to enhance the accuracy of them is by improving the acoustic model (AM). In this paper, for the first time, the combination of deep belief network (DBN), for extracting features of speech signals, and Deep Bidirectional Long Short-Term Memory (DBLSTM) with Connectionist Temporal Classification (CTC) output layer is used to create an AM on the Farsdat Persian speech data set. The obtained results show that the use of a deep neural network (DNN) compared to a shallow network improves the results. Also, using the bidirectional network increases the accuracy of the model in comparison with the unidirectional network, in both deep and shallow networks. Comparing obtained results with the HMM and Kaldi-DNN indicates that using DBLSTM with features extracted from the DBN increases the accuracy of Persian phoneme recognition. [ABSTRACT FROM AUTHOR]

As a result of the fast rise of globalization, people in China are learning English at a rapid pace. However, there is a severe shortage of English teachers in the region, which is a major hindrance. To address these concerns, a deep learning-based algorithm is proposed that can not only check English pronunciation but also help learners distinguish between phonemic and quality phonemic while listening and differentiating, as well as correct phonemic errors, thereby increasing their language learning capacity. In order to study the application of nonlinear network identification technology in English learning, this paper evaluates the English pronunciation quality through the deep learning algorithm of deep learning combined with the related contents of neural network data model, and the experimental results of speech recognition structure are analyzed and discussed in detail. The concordance between machine and manual intonation evaluation is 80%, the concordance rate of adjacent intonation evaluation is 98.33%, and the Pearson correlation coefficient is 0.627 that shows the technique is reliable. The method of English pronunciation and speech identification model is sensible and dependable, which can give beginners a punctual, exact and impartial judgment and response guidance, assist learners to get on the differences between their phonemic and standard phonemic, and correct phonemic mistakes, in order to enhance the ability of oral English learning. [ABSTRACT FROM AUTHOR]

Parkinson's disease (PD) is a condition that degenerates over time and impairs speech and pronunciation because brain cells have died. This research work aims to predict parkinson disease using the voice features extracted from speech signals recorded from PD individuals with dysphonic speech disorders by employing deep learning algorithms. PD is challenging to diagnose early on in the clinical presentation. To address the issue in machine learning methods, this paper proposes a neural network model by processing speech signals to classify PD using the University of California Irvine (UCI) machine learning repository dataset. Initially, a pre-loss reduction module is created by using pre-sampling to make the dataset balanced by reducing the dimensionality and maintaining the size of the space without influencing the learning process for data preparation. The relevant features are derived using a novel multi-agent salp swarm (MASS) algorithm, and a novel Parkinson classification neural network (PCNN) is proposed to classify Parkinson's patients with high accuracy employing these derived features. The result shows that the models that use MASS-PCNN produce higher classification accuracy of 99.1%, precision of 97.8%, recall of 94.7% and F1-score of 0.995 when paralleled to the existing models. As an outcome, the suggested model will perform superior to common convolutional neural networks. [ABSTRACT FROM AUTHOR]

While deep neural networks have shown impressive results in automatic speaker recognition and related tasks, it is dissatisfactory how little is understood about what exactly is responsible for these results. Part of the success has been attributed in prior work to their capability to model supra-segmental temporal information (SST), i.e., learn rhythmic-prosodic characteristics of speech in addition to spectral features. In this paper, we (i) present and apply a novel test to quantify to what extent the performance of state-of-the-art neural networks for speaker recognition can be explained by modeling SST; and (ii) present several means to force respective nets to focus more on SST and evaluate their merits. We find that a variety of CNN- and RNN-based neural network architectures for speaker recognition do not model SST to any sufficient degree, even when forced. The results provide a highly relevant basis for impactful future research into better exploitation of the full speech signal and give insights into the inner workings of such networks, enhancing explainability of deep learning for speech technologies. • Literature explains speaker recognition in neural nets by modeling of voice dynmaics. • Diagnostic: We quantify how well deep learning models actually capture dynamics. • Observation: State-of-the-art deep nets do not model speaker prosody but ignore it. • Interpretation as "cheating": Achieving high without putting in due effort. • Outlook: Increasing task difficulty biases models towards prosody, but not enough. [ABSTRACT FROM AUTHOR]

In recent years, speech processing resides a major application in the domain of signal processing. Due to the audibility loss of some speech signals, people with hearing impairment have difficulty in understanding speech, which reintroduces a crucial role in speech recognition. Automatic Speech Recognition (ASR) development is a major challenge in research in the case of noise, domain, vocabulary size, and language and speaker variability. Speech recognition system design needs careful attention to challenges or issues like performance and database evaluation, feature extraction methods, speech representations and speech classes. In this paper, HDF-DNN model has been proposed with the hybridization of discriminant fuzzy function and deep neural network for speech recognition. Initially, the speech signals are pre-processed to eliminate the unwanted noise and the features are extracted using Mel Frequency Cepstral Coefficient (MFCC). A hybrid Deep Neural Network and Discriminant Fuzzy Logic is used for assisting hearing-impaired listeners with enhanced speech intelligibility. Both DNN and DF have some problems with parameters to address this problem, Enhanced Modularity function-based Bat Algorithm (EMBA) is used as a powerful optimization tool. The experimental results show that the proposed automatic speech recognition-based hybrid deep learning model is effectively-identifies speech recognition more than the MFCC-CNN, CSVM and Deep auto encoder techniques. The proposed method improves the overall accuracy of 8.31%, 9.71% and 10.25% better than, MFCC-CNN, CSVM and Deep auto encoder respectively. [ABSTRACT FROM AUTHOR]

With the emergence of the information age, computers have entered the homes of ordinary people and have become essential daily appliances for people. The integration of people and computers has become more popular and in-depth. Based on this situation, how to make computers and humans communicate intelligently and make human-computer interaction more convenient and practical is the main issue of scientific research and discussion on computers. Language recognition is widely used in industry, the economy, commerce, tourism, style, and sports. The BP neural network based on deep learning has supercomputing functions and is very suitable for speech recognition, which promotes the promotion and application of speech recognition technology in many fields. Different types of applications and diverse scenarios enable users to experience different personal experiences. This paper examines the use of application software for ideological and political theory courses in colleges and universities and the application of application software for ideological and political theory courses in colleges and universities in the management of ideological and political theory courses. Analyzed the use of apps in the management of ideological and political theory courses in colleges and universities, analyzed the possibilities and challenges that this brought to the management of ideological and political theory courses in colleges and universities based on the app, and put forward the necessary countermeasures. Including the development and design of ideological and political theory applications, combining deep learning and CTC algorithms to build acoustic models, using server-client interaction, designing an ideological and political theory course app based on speech recognition and deep learning, and forming an offline speech recognition system software platform, the app provides a certain reference for improving the skills of teaching managers. [ABSTRACT FROM AUTHOR]

The location attention mechanism has been widely applied in deep neural networks. However, as the mechanism entails heavy computing workload, significant memories consumed for weights storage, and shows poor parallelism in some calculations, it is hard to achieve high efficiency deployment. In this paper, the field-programmable gate array (FPGA) is employed to implement the location attention mechanism in hardware, and a novel fusion approach is proposed to connect the convolutional layer with the fully connected layer, which not only improves the parallelism of both the algorithm and the hardware pipeline, but also reduces the computation cost for such operations as multiplication and addition. Meanwhile, the shared computing architecture is used to reduce the demand for hardware resources. Parallel computing arrays are utilized to time-multiplex a single computing array, which can speed up the pipeline parallel computing of the attention mechanism. Experimental results show that for the location attention mechanism, the FPGA's inference speed is 0.010 ms, which is around a quarter of the speed achieved by running it with GPU, and its power consumption is 1.73 W, which is about 2.89% of the power consumed by running it with CPU. Compared with other FPGA implementation methods of attention mechanism, it has less hardware resource consumption and less inference time. When applied to speech recognition tasks, the trained attention model is symmetrically quantized and deployed on the FPGA. The result shows that the word error rate is only 0.79% higher than that before quantization, which proves the effectiveness and correctness of the hardware circuit. [ABSTRACT FROM AUTHOR]

Information Technology has touched new vistas for a couple of decades mostly to simplify the day-to-day life of the humans. One of the key contributions of Information Technology is the application of Artificial Intelligence to achieve better results. The advent of artificial intelligence has given rise to a new branch of Natural Language Processing (NLP) called Computational Linguistics, which generates frameworks for intelligently manipulating spoken language knowledge and has brought human-machine onto a new stage. In this context, speech has arisen to be one of the imperative forms of interfaces, which is the basic mode of communication for us, and generally the most preferred one. Language identification, being the front-end for various natural language processing tasks, plays an important role in language translation. Owing to this, the focus has been given on the field of speech recognition involving the identification & recognition of languages by a machine. Spoken language identification is the identification of language present in a speech segment despite its size (duration & speed), ambiance (topic & emotion), and moderator (gender, age, demographic region). This paper has investigated various existing spoken language identification models implemented using different deep learning approaches, datasets, and performance measures utilized for their analysis. It also highlights the main features and challenges faced by these models. A comprehensive comparative study of deep learning techniques has been carried out for spoken language identification. Moreover, this review analyzes the efficiency of the spoken language models that can help the researchers to propose new language identification models for speech signals. [ABSTRACT FROM AUTHOR]

Parkinson's disease is a degenerative disease that leads to brain disorder and nonfunctioning of different body parts. Deep learning tools like artificial neural network (ANN), convolution neural network (CNN), regression Analysis (RA), and so on, has been considered to a great extent in recent days. Several data sets based on the motor and nonmotor symptoms are applied to different classifier for correct identification of Parkinson's patient from healthy people. In this paper, hybridization of two deep learning tools such as, RA and ANN are done for effective diagnosis of the disease by probability estimation. The communal merits of individual approaches of the existing approaches are realized in this context for accurate probability estimation. Data preprocessing and probability estimation of preprocessed data is done in RA. The second existing approach is used to identify the PD patient by comparing with a predefined threshold value of a neuron. The estimation is performed on the data set of speech recognition, iron content, and pulse rate among a group of people. The proposed approach is compared with the existing approaches like, SVM and k‐NN classifier. The computed result reveals the superiority of the proposed algorithm with 93.46% accuracy. [ABSTRACT FROM AUTHOR]

Recently, machine learning, especially deep learning, has been a core algorithm to be widely used in many fields such as natural language processing, speech recognition, object recognition, and so on. At the same time, another trend is that more and more applications are moved to wearable and mobile devices. However, traditional deep learning methods such as convolutional neural network (CNN) and its variants consume a lot of memory resources. In this case, these powerful deep learning methods are difficult to apply on mobile memory-limited platforms. In order to solve this problem, we present a novel memory-management strategy called mmCNN in this paper. With the help of this method, we can easily deploy a trained large-size CNN on any memory size platform such as GPU, FPGA, or memory-limited mobile devices. In our experiments, we run a feed-forward CNN process in some extremely small memory sizes (as low as 5 MB) on a GPU platform. The result shows that our method saves more than 98% memory compared to a traditional CNN algorithm and further saves more than 90% compared to the state-of-the-art related work "vDNNs" (virtualized deep neural networks). Our work in this paper improves the computing scalability of lightweight applications and breaks the memory bottleneck of using deep learning method on memory-limited devices. [ABSTRACT FROM AUTHOR]

Speech Translation consists in translating speech in one language into text or speech in a different language. These systems have numerous applications, particularly in multilingual communities such as the European Union. The standard approach in the field involves the chaining of separate components for speech recognition, machine translation and speech synthesis. With the advances made possible by artificial neural networks and Deep Learning, training end-to-end speech translation systems has given rise to intense research and development activities in recent times. In this paper, we review the state of the art and describe project MINTZAI, which is being carried out in this field. [ABSTRACT FROM AUTHOR]

In this paper, a ternary neural network with complementary binary arrays is proposed for representing the signed synaptic weights. The proposed ternary neural network is deployed on a low-cost Raspberry Pi board embedded system for the application of speech and image recognition. In conventional neural networks, the signed synaptic weights of –1, 0, and 1 are represented by 8-bit integers. To reduce the amount of required memory for signed synaptic weights, the signed values were represented by a complementary binary array. For the binary inputs, the multiplication of two binary numbers is replaced by the bit-wise AND operation to speed up the performance of the neural network. Regarding image recognition, the MINST dataset was used for training and testing of the proposed neural network. The recognition rate was as high as 94%. The proposed ternary neural network was applied to real-time object recognition. The recognition rate for recognizing 10 simple objects captured from the camera was 89%. The proposed ternary neural network with the complementary binary array for representing the signed synaptic weights can reduce the required memory for storing the model’s parameters and internal parameters by 75%. The proposed ternary neural network is 4.2, 2.7, and 2.4 times faster than the conventional ternary neural network for MNIST image recognition, speech commands recognition, and real-time object recognition respectively. [ABSTRACT FROM AUTHOR]

Deep Neural Networks are widely used for solving complex problems in several scientific areas, such as speech recognition, machine translation, image analysis. The strategies employed to investigate their theoretical properties mainly rely on Euclidean geometry, but in the last years new approaches based on Riemannian geometry have been developed. Motivated by some open problems, we study a particular sequence of maps between manifolds, with the last manifold of the sequence equipped with a Riemannian metric. We investigate the structures induced through pullbacks on the other manifolds of the sequence and on some related quotients. In particular, we show that the pullbacks of the final Riemannian metric to any manifolds of the sequence is a degenerate Riemannian metric inducing a structure of pseudometric space. We prove that the Kolmogorov quotient of this pseudometric space yields a smooth manifold, which is the base space of a particular vertical bundle. We investigate the theoretical properties of the maps of such sequence, eventually we focus on the case of maps between manifolds implementing neural networks of practical interest and we present some applications of the geometric framework we introduced in the first part of the paper. • We consider Neural Networks as sequences of smooth maps between manifolds. • We investigate the structures induced through pull-backs on the manifolds. • The pull-backs of the final Riemannian metric to any manifolds is a degenerate metric. • The theoretical study leads to construct equivalence classes in the input manifold. [ABSTRACT FROM AUTHOR]

Lipreading is the ability to recognize words or sentences from the mouth movements of a speaking person. This process is also known as Visual Speech Recognition (VSR). Lipreading has two main advantages: facilitate communication for people with hearing or speaking problems and aid speech recognition in noisy environments. In this paper, we propose a lipreading computing system capable of recognizing ten common Arabic words by performing word extraction from the mouth movements. The system receives a video of a person uttering an Arabic word as an input and outputs the text of the predicted word. During the implementation stage of the proposed system, three deep learning and neural network architectures are alternatively used to train, validate, and test the system using a locally collected and preprocessed dataset. The dataset contains 1051 videos and will be made available upon request. Moreover, a voting model that combines the three architectures is proposed. The highest testing accuracy (i.e. 82.84%) is achieved by leveraging the voting model. [ABSTRACT FROM AUTHOR]

A wavelet packet based adaptive filter-bank construction combined with Deep Belief Network(DBN) feature learning method is proposed for speech signal processing in this paper. On this basis, a set of acoustic features are extracted for speech emotion recognition, namely Coiflet Wavelet Packet Cepstral Coefficients (CWPCC). CWPCC extends the conventional Mel- Frequency Cepstral Coefficients (MFCC) by adapting the filter-bank structure according to the decision task. And Deep Belief Networks (DBNs) are artificial neural networks having more than one hidden layer, which are first pre-trained layer by layer and then fine-tuned using back propagation algorithm. The well-trained deep neural networks are capable of modeling complex and non-linear features of input training data and can better predict the probability distribution over classification labels. Speech emotion recognition system is constructed with the feature set, DBNs feature learning structure and Support Vector Machine as classifier. Experimental results on Berlin emotional speech database show that the Coiflet Wavelet Packet is more suitable in speech emotion recognition than other acoustics features and proposed DBNs feature learning structure combined with CWPCC improve emotion recognition performance over the conventional emotion recognition method. [ABSTRACT FROM AUTHOR]

Automatic Speech Recognition, (ASR) has achieved the best results for English, with end-to-end neural network based supervised models. These supervised models need huge amounts of labeled speech data for good generalization, which can be quite a challenge to obtain for low-resource languages like Urdu. Most models proposed for Urdu ASR are based on Hidden Markov Models (HMMs). This paper proposes an end-to-end neural network model, for Urdu ASR, regularized with dropout, ensemble averaging and Maxout units. Dropout and ensembles are averaging techniques over multiple neural network models while Maxout are units in a neural network which adapt their activation functions. Due to limited labeled data, Semi Supervised Learning (SSL) techniques are also incorporated to improve model generalization. Speech features are transformed into a lower dimensional manifold using an unsupervised dimensionality-reduction technique called Locally Linear Embedding (LLE). Transformed data along with higher dimensional features is used to train neural networks. The proposed model also utilizes label propagation-based self-training of initially trained models and achieves a Word Error Rate (WER) of 4% less than that reported as the benchmark on the same Urdu corpus using HMM. The decrease in WER after incorporating SSL is more significant with an increased validation data size. [ABSTRACT FROM AUTHOR]

N-ary error correcting output codes (ECOC) decompose a multi-class problem into simpler multi-class problems by splitting the classes into N subsets (meta-classes) to form an ensemble of N-class classifiers and combine them to make predictions. It is one of the most accurate ensemble learning methods for traditional classification tasks. Deep learning has gained increasing attention in recent years due to its successes on various tasks such as image classification and speech recognition. However, little is known about N-ary ECOC with deep neural networks (DNNs) as base learners, probably due to the long computation time. In this paper, we show by experiments that N-ary ECOC with DNNs as base learners generally exhibits superior performance compared with several state-of-the-art ensemble learning methods. Moreover, our work contributes to a more efficient setting of the two crucial hyperparameters of N-ary ECOC: the value of N and the number of base learners to train. We also explore valuable strategies for further improving the accuracy of N-ary ECOC. [ABSTRACT FROM AUTHOR]

Ship recognition based on ship-radiated noise is one of the most important and challenging subjects in underwater acoustic signal processing. The recognition methods for ship-radiated noise recognition include traditional methods and deep learning (DL) methods. Developing from the DL methods and inspired by audio–video speech recognition (AVSR), the paper further introduces multimodal deep learning (multimodal-DL) methods for the recognition of ship-radiated noise. In this paper, ship-radiated noise (acoustics modality) and visual observation of the ships (visual modality) are two different modalities that the multimodal-DL methods model on. The paper specially designs a multimodal-DL framework, the multimodal convolutional neural networks (multimodal-CNNs) for the recognition of ship-radiated noise. Then the paper proposes a strategy based on canonical correlation analysis (CCA-based strategy) to build a joint representation and recognition on the two different single-modality (acoustics modality and visual modality). The multimodal-CNNs and the CCA-based strategy are tested on real ship-radiated noise data recorded. Experimental results show that, using the CCA-based strategy, strong-discriminative information can be built from weak-discriminative information provided from a single-modality. Experimental results also show that as long as any one of the single-modalities can provide information for the recognition, the multimodal-DL methods can have a much better multiclass recognition performance than the DL methods. The paper also discusses the advantages and superiorities of the multimodal-Dl methods over the traditional methods for ship-radiated noise recognition. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs) are widely used in various application domains such as image processing, speech recognition, and natural language processing. However, testing DNN models may be challenging due to the complexity and size of their input domain. In particular, testing DNN models often requires generating or exploring large unlabeled datasets. In practice, DNN test oracles, which identify the correct outputs for inputs, often require expensive manual effort to label test data, possibly involving multiple experts to ensure labeling correctness. In this article, we propose DeepGD, a black-box multi-objective test selection approach for DNN models. It reduces the cost of labeling by prioritizing the selection of test inputs with high fault-revealing power from large unlabeled datasets. DeepGD not only selects test inputs with high uncertainty scores to trigger as many mispredicted inputs as possible but also maximizes the probability of revealing distinct faults in the DNN model by selecting diverse mispredicted inputs. The experimental results conducted on four widely used datasets and five DNN models show that in terms of fault-revealing ability, (1) white-box, coverage-based approaches fare poorly, (2) DeepGD outperforms existing black-box test selection approaches in terms of fault detection, and (3) DeepGD also leads to better guidance for DNN model retraining when using selected inputs to augment the training set. [ABSTRACT FROM AUTHOR]

• As the steep switching device has double-gate, the threshold voltage can be adaptively controlled. • The adjustment of the threshold can be employed to mitigate the accuracy deterioration. • Off current of proposed integrate and fire neuron circuit decreases significantly. • Adjustable threshold of the neuron circuit can achieve a high inference accuracy. Deep learning has shown impressive capabilities in tasks like speech recognition and image classification. However, modern deep neural networks often demand a significant number of weights and extensive computational resources, creating efficiency challenges for applications on edge devices. To address these issues, researchers have introduced deep spiking neural networks (DSNNs) that leverage specialized hardware for synapses and neurons. DSNNs offer a potential solution by improving efficiency in edge-device applications. In this paper, the hardware based DSNN with integrate and fire neuron using steep switching device was investigated. We propose integrate and fire neuron using steep switching device to implement rate coding as input encoding method. Because the steep switching device has double-gate, the threshold voltage of the neuron circuits can be adaptively controlled, which changes the rates of input pulse. Hence, the adjustment of the threshold of neuron can be employed to mitigate the accuracy deterioration resulting from the transformation from deep neural networks (DNNs) to DSNNs. In addition, the off-current of proposed integrate and fire neuron circuit decreases significantly as the steep switching device has steep subthreshold swing. A system simulation of a hardware based DSNN shows that the adjustable threshold of the neuron circuit can achieve a high inference accuracy of 98.36 % which is comparable to that obtained with software based DNN. [ABSTRACT FROM AUTHOR]

Artificial Neural Networks (ANNs) were created inspired by the neural networks in the human brain and have been widely applied in speech processing. The application areas of ANN include: Speech recognition, speech emotion recognition, language identification, speech enhancement, and speech separation, amongst others. Likewise, given that speech processing performed by humans involves complex cognitive processes known as auditory attention, there has been a growing amount of papers proposing ANNs supported by deep learning algorithms in conjunction with some mechanism to achieve symmetry with the human attention process. However, while these ANN approaches include attention, there is no categorization of attention integrated into the deep learning algorithms and their relation with human auditory attention. Therefore, we consider it necessary to have a review of the different ANN approaches inspired in attention to show both academic and industry experts the available models for a wide variety of applications. Based on the PRISMA methodology, we present a systematic review of the literature published since 2000, in which deep learning algorithms are applied to diverse problems related to speech processing. In this paper 133 research works are selected and the following aspects are described: (i) Most relevant features, (ii) ways in which attention has been implemented, (iii) their hypothetical relationship with human attention, and (iv) the evaluation metrics used. Additionally, the four publications most related with human attention were analyzed and their strengths and weaknesses were determined. [ABSTRACT FROM AUTHOR]

Deep learning is a branch of neural network which has been intensively developed in the last decade. Due to the high-accuracy classification ability, the deep learning algorithms have been widely used in many fields, such as speech recognition, image recognition, and natural speech processing. However, they also show some shortcomings especially on the selection of some parameters in the network, including hyper-parameters, which is still treated as a time consuming task. In this paper, a modified ABC (ABC-ISB) optimization algorithm is proposed to automatically train the parameters of Feed-Forward Artificial Neural Networks, which is a typical a neural network. In the proposed ABC algorithm, we utilize the information of neighbors with better performance to accelerate the convergence of employed and onlooker bees respectively. In addition, a new selection strategy and a gbest-guided strategy are introduced to enhance the global search capability and balance the exploration and exploitation of the algorithm separately. The experimental results show our ABC-ISB is generally leading and competitive. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs) have gained remarkable success in speech recognition, partially attributed to the flexibility of DNN models in learning complex patterns of speech signals. This flexibility, however, may lead to serious over-fitting and hence miserable performance degradation in adverse acoustic conditions such as those with high ambient noises. We propose a noisy training approach to tackle this problem: by injecting moderate noises into the training data intentionally and randomly, more generalizable DNN models can be learned. This 'noise injection' technique, although known to the neural computation community already, has not been studied with DNNs which involve a highly complex objective function. The experiments presented in this paper confirm that the noisy training approach works well for the DNN model and can provide substantial performance improvement for DNN-based speech recognition. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel annealed gradient descent (AGD) algorithm for deep learning. AGD optimizes a sequence of gradually improving smoother mosaic functions that approximate the original non-convex objective function according to an annealing schedule during optimization process. We present a theoretical analysis on AGD's convergence properties and learning speed, as well as use some visualization methods to show its advantages. The proposed AGD algorithm is applied to learn both deep neural networks (DNNs) and Convolutional Neural Networks (CNNs) for variety of tasks includes image recognition and speech recognition. Experimental results on several widely-used databases, such as Switchboard, CIFAR-10 and Pascal VOC 2012, show that AGD yields better classification accuracy than SGD, and obviously accelerates the training speed of DNNs and CNNs. [ABSTRACT FROM AUTHOR]

As deep learning technology matures, it's being widely deployed in fields like image classification and speech recognition. However, training a functional deep learning model requires vast computing power and a large training dataset, leading to the emergence of a new business model of selling pre-trained models. However, these models are highly susceptible to theft, which poses a threat to the interests of their creators. Moreover, the network topology and weight parameters are considered intellectual property. To address these challenges, a method that can tag trained models to claim ownership without affecting their performance is necessary. Therefore, we propose a novel neural network watermarking protocol. In this method, the trigger set is constructed differently from previous methods by using a key obtained from the authority to generate a scrambling sequence, followed by using the sequence to scramble the pixels and assign their original labels. Finally, the trigger set is put into the network training together with the original training set to complete the watermark embedding. Since Logistic chaos mapping is nonlinear, unpredictable, and sensitive to initial values, we use Logistic chaos mapping as the generation method of dislocation sequence. We involve a third-party copyright center in the embedding process to prevent forgery attacks. The third-party only needs to store the disruption key and timestamp for each owner, reducing their storage burden. Our experimental results demonstrate that the ResNet model exhibits a mere 0.05 percentage point decrease in accuracy when using fine-tuning for watermark embedding, and a mere 0.03 percentage point decrease when using the training-from-scratch method. On the other hand, when using the SENet model, embedding watermarks via fine-tuning resulted in a 1.35 percentage point decrease in classification accuracy, while embedding watermarks from training-from-scratch resulted in a 0.94 percentage point increase in classification accuracy. Furthermore, our model exhibited robustness against various attacks in the robustness experiments, including model fine-tuning, model compression, and watermark overlay. [ABSTRACT FROM AUTHOR]

Here, a new layered spiking neural network (SNN) learning framework is proposed using optimization algorithms for rapid and efficient pattern recognition and classification. In connection with the problem of learning deep SNN layers and with the help of different algorithms of gradient-based optimization and wild horse optimization, the two main parameters of spike neurons (threshold voltage and input weights) for different layers are calculated. SNN has been utilized to model several prominent datasets under machine learning systems, including IRIS and Trip datasets and digital speech recognition systems under MATLAB. Then, their performance is compared and evaluated in different scenarios with other deep learning methods such as artificial neural network and adaptive network-based fuzzy inference system. The results indicated an upsurge in identification and estimation accuracy. Integrating the algorithmic power of deep SNNs with adequate neuromorphic hardware creates an opportunity for speech recognition applications running locally on mobile and other applications. [ABSTRACT FROM AUTHOR]

Since 2006, deep learning (DL) has become a rapidly growing research direction, redefining state-of-the-art performances in a wide range of areas such as object recognition, image segmentation, speech recognition and machine translation. In modern manufacturing systems, data-driven machine health monitoring is gaining in popularity due to the widespread deployment of low-cost sensors and their connection to the Internet. Meanwhile, deep learning provides useful tools for processing and analyzing these big machinery data. The main purpose of this paper is to review and summarize the emerging research work of deep learning on machine health monitoring. After the brief introduction of deep learning techniques, the applications of deep learning in machine health monitoring systems are reviewed mainly from the following aspects: Auto-encoder (AE) and its variants, Restricted Boltzmann Machines and its variants including Deep Belief Network (DBN) and Deep Boltzmann Machines (DBM), Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN). In addition, an experimental study on the performances of these approaches has been conducted, in which the data and code have been online. Finally, some new trends of DL-based machine health monitoring methods are discussed. [ABSTRACT FROM AUTHOR]

This study is devoted to the transcription of human speech in the Kazakh language in dynamically changing conditions. It discusses key aspects related to the phonetic structure of the Kazakh language, technical considerations in collecting the transcribed audio corpus, and the use of deep neural networks for speech modeling. A high-quality decoded audio corpus was collected, containing 554 h of data, giving an idea of the frequencies of letters and syllables, as well as demographic parameters such as the gender, age, and region of residence of native speakers. The corpus contains a universal vocabulary and serves as a valuable resource for the development of modules related to speech. Machine learning experiments were conducted using the DeepSpeech2 model, which includes a sequence-to-sequence architecture with an encoder, decoder, and attention mechanism. To increase the reliability of the model, filters initialized with symbol-level embeddings were introduced to reduce the dependence on accurate positioning on object maps. The training process included simultaneous preparation of convolutional filters for spectrograms and symbolic objects. The proposed approach, using a combination of supervised and unsupervised learning methods, resulted in a 66.7% reduction in the weight of the model while maintaining relative accuracy. The evaluation on the test sample showed a 7.6% lower character error rate (CER) compared to existing models, demonstrating its most modern characteristics. The proposed architecture provides deployment on platforms with limited resources. Overall, this study presents a high-quality audio corpus, an improved speech recognition model, and promising results applicable to speech-related applications and languages beyond Kazakh. [ABSTRACT FROM AUTHOR]

Distant speech recognition is being revolutionized by deep learning, that has contributed to significantly outperform previous HMM-GMM systems. A key aspect behind the rapid rise and success of DNNs is their ability to better manage large time contexts. With this regard, asymmetric context windows that embed more past than future frames have been recently used with feed-forward neural networks. This context configuration turns out to be useful not only to address low-latency speech recognition, but also to boost the recognition performance under reverberant conditions. This paper investigates on the mechanisms occurring inside DNNs, which lead to an effective application of asymmetric contexts. In particular, we propose a novel method for automatic context window composition based on a gradient analysis. The experiments, performed with different acoustic environments, features, DNN architectures, microphone settings, and recognition tasks show that our simple and efficient strategy leads to a less redundant frame configuration, which makes DNN training more effective in reverberant scenarios. [ABSTRACT FROM AUTHOR]

In the last few years, deep learning has led to very good performance on a variety of problems, such as visual recognition, speech recognition and natural language processing. Among different types of deep neural networks, convolutional neural networks have been most extensively studied. Leveraging on the rapid growth in the amount of the annotated data and the great improvements in the strengths of graphics processor units, the research on convolutional neural networks has been emerged swiftly and achieved state-of-the-art results on various tasks. In this paper, we provide a broad survey of the recent advances in convolutional neural networks. We detailize the improvements of CNN on different aspects, including layer design, activation function, loss function, regularization, optimization and fast computation. Besides, we also introduce various applications of convolutional neural networks in computer vision, speech and natural language processing. [ABSTRACT FROM AUTHOR]

Mobile devices such as smartphones and autonomous vehicles increasingly rely on deep neural networks (DNNs) to execute complex inference tasks such as image classification and speech recognition, among others. However, continuously executing the entire DNN on mobile devices can quickly deplete their battery. Although task offloading to cloud/edge servers may decrease the mobile device’s computational burden, erratic patterns in channel quality, network, and edge server load can lead to a significant delay in task execution. Recently, approaches based on split computing (SC) have been proposed, where the DNN is split into a head and a tail model, executed respectively on the mobile device and on the edge server. Ultimately, this may reduce bandwidth usage as well as energy consumption. Another approach, called early exiting (EE), trains models to embed multiple “exits” earlier in the architecture, each providing increasingly higher target accuracy. Therefore, the tradeoff between accuracy and delay can be tuned according to the current conditions or application demands. In this article, we provide a comprehensive survey of the state of the art in SC and EE strategies by presenting a comparison of the most relevant approaches.We conclude the article by providing a set of compelling research challenges. [ABSTRACT FROM AUTHOR]

This paper addresses the robust speech recognition problem as a domain adaptation task. Specifically, we introduce an unsupervised deep domain adaptation (DDA) approach to acoustic modeling in order to eliminate the training–testing mismatch that is common in real-world use of speech recognition. Under a multi-task learning framework, the approach jointly learns two discriminative classifiers using one deep neural network (DNN). As the main task, a label predictor predicts phoneme labels and is used during training and at test time. As the second task, a domain classifier discriminates between the source and the target domains during training. The network is optimized by minimizing the loss of the label classifier and to maximize the loss of the domain classifier at the same time. The proposed approach is easy to implement by modifying a common feed-forward network. Moreover, this unsupervised approach only needs labeled training data from the source domain and some unlabeled raw data of the new domain. Speech recognition experiments on noise/channel distortion and domain shift confirm the effectiveness of the proposed approach. For instance, on the Aurora-4 corpus, compared with the acoustic model trained only using clean data, the DDA approach achieves relative 37.8% word error rate (WER) reduction. [ABSTRACT FROM AUTHOR]

Deep neural networks have achieved state-of-the-art performance in many artificial intelligence areas, such as object recognition, speech recognition, and machine translation. While deep neural networks have high expression capabilities, they are prone to over fitting due to the high dimensionalities of the networks. In recent applications, deep neural networks have been found to be unstable in adversarial perturbations, which are small but can increase the network's prediction errors. This paper proposes a novel training algorithm to improve the robustness of the neural networks in adversarial examples. [ABSTRACT FROM AUTHOR]

The benefits of deep neural networks (DNNs) and other big-data algorithms have led to their use in almost every modern application. The rising use of DNNs in diverse domains including computer vision, speech recognition, image classification, and prediction has increased the demand for energy-efficient hardware architectures. Massive amounts of parallel processing in large-scale DNN algorithms have made communication and storage a strong wall in front of a DNN's power and performance. Nowadays, DNNs have gained a great deal of success by utilizing the inherent parallelism of GPU architectures. However, recent research shows that the integration of CPUs and GPUs presents a more efficient solution for running the next generation of machine learning (ML) chips. Designing interconnection networks for a heterogenous CPU-GPU platform are a challenge (especially for the execution of DNN workloads) as it must be scalable and efficient. A study in this work shows that the majority of traffic in DNN workloads is associated with last level caches (LLCs). Therefore, there is a need to design a low-overhead interconnect fabric to minimize the energy and access time to the LLC banks. To address this issue, a low-overhead on-chip interconnection, named Godiva, for running DNNs energy-efficiently has been proposed. Godiva interconnection affords low LLCs accesses delay using a low-overhead and small cost hardware in a heterogenous CPU-GPU platform. An experimental evaluation targeting a 16CPU-48GPU system and a set of popular DNN workloads reveals that the proposed heterogenous architecture improves system energy by about 21.7 × and reduces interconnection network area by about 51% when compared to a mesh-based CPU design. [ABSTRACT FROM AUTHOR]

Automatic speech recognition (ASR) has been one of the biggest and hardest challenges in the field. A large majority of research in this area focuses on widely spoken languages such as English. The problems of automatic Lithuanian speech recognition have attracted little attention so far. Due to complicated language structure and scarcity of data, models proposed for other languages such as English cannot be directly adopted for Lithuanian. In this paper we propose an ASR system for the Lithuanian language, which is based on deep learning methods and can identify spoken words purely from their phoneme sequences. Two encoder-decoder models are used to solve the ASR task: a traditional encoder-decoder model and a model with attention mechanism. The performance of these models is evaluated in isolated speech recognition task (with an accuracy of 0.993) and long phrase recognition task (with an accuracy of 0.992). [ABSTRACT FROM AUTHOR]

Listening in a noisy environment is challenging, but many previous studies have demonstrated that comprehension of speech can be substantially improved by looking at the talker's face. We recently developed a deep neural network (DNN) based system that generates movies of a talking face from speech audio and a single face image. In this study, we aimed to quantify the benefits that such a system can bring to speech comprehension, especially in noise. The target speech audio was masked with signal to noise ratios of −9, −6, −3, and 0 dB and was presented to subjects in three audio-visual (AV) stimulus conditions: (1) synthesized AV: audio with the synthesized talking face movie; (2) natural AV: audio with the original movie from the corpus; and (3) audio-only: audio with a static image of the talker. Subjects were asked to type the sentences they heard in each trial and keyword recognition was quantified for each condition. Overall, performance in the synthesized AV condition fell approximately halfway between the other two conditions, showing a marked improvement over the audio-only control but still falling short of the natural AV condition. Every subject showed some benefit from the synthetic AV stimulus. The results of this study support the idea that a DNN-based model that generates a talking face from speech audio can meaningfully enhance comprehension in noisy environments, and has the potential to be used as a visual hearing aid. [ABSTRACT FROM AUTHOR]

While artificial neural networks have been in existence for over half a century, it was not until year 2010 that they had made a significant impact on speech recognition with a deep form of such networks. This invited paper, based on my keynote talk given at Interspeech conference in Singapore in September 2014, will first reflect on the historical path to this transformative success, after providing brief reviews of earlier studies on (shallow) neural networks and on (deep) generative models relevant to the introduction of deep neural networks (DNN) to speech recognition several years ago. The role of well-timed academic-industrial collaboration is highlighted, so are the advances of big data, big compute, and the seamless integration between the application-domain knowledge of speech and general principles of deep learning. Then, an overview is given on sweeping achievements of deep learning in speech recognition since its initial success. Such achievements, summarized into six major areas in this article, have resulted in across-the-board, industry-wide deployment of deep learning in speech recognition systems. Next, more challenging applications of deep learning, natural language and multimodal processing, are selectively reviewed and analyzed. Examples include machine translation, knowledgebase completion, information retrieval, and automatic image captioning, where fresh ideas from deep learning, continuous-space embedding in particular, are shown to be revolutionizing these application areas albeit with less rapid pace than for speech and image recognition. Finally, a number of key issues in deep learning are discussed, and future directions are analyzed for perceptual tasks such as speech, image, and video, as well as for cognitive tasks involving natural language. [ABSTRACT FROM AUTHOR]