Search

In recent years, the research pertaining to speech recognition technology in the Kazakh language has gained significant importance. This is due to the increasing demand for natural language processing applications in the region where Kazakh is predominantly spoken. Thus, there exists an urgent requirement for precise and dependable speech recognition systems. The research study examines the application of sophisticated deep learning methodologies, such as Natural Language Processing (NLP) and Hidden Markov Model (HMM), in facilitating speech recognition for the Kazakh language. Additionally, the investigation delves into how various techniques, including data preprocessing, acoustic modeling, and language modeling, can aid in devising effective speech recognition systems. The article deliberates on the feasible uses of speech recognition technology in the geographic area where Kazakh language is spoken and outlines its future research prospects. The investigation underscores the significance of persistent inquiry in this realm to confront distinctive obstacles encountered in creating speech recognition systems for languages with restricted resources. [ABSTRACT FROM AUTHOR]

In indoor environments, reverberation often distorts clean speech. Although deep learning-based speech dereverberation approaches have shown much better performance than traditional ones, the inferior speech quality of the dereverberated speech caused by magnitude distortion and limited phase recovery is still a serious problem for practical applications. This paper improves the performance of deep learning-based speech dereverberation from the perspectives of both network design and mapping target optimization. Specifically, on the one hand, a bifurcated-and-fusion network and its guidance loss functions were designed to help reduce the magnitude distortion while enhancing the phase recovery. On the other hand, the time boundary between the early and late reflections in the mapped speech was investigated, so as to make a balance between the reverberation tailing effect and the difficulty of magnitude/phase recovery. Mathematical derivations were provided to show the rationality of the specially designed loss functions. Geometric illustrations were given to explain the importance of preserving early reflections in reducing the difficulty of phase recovery. Ablation study results confirmed the validity of the proposed network topology and the importance of preserving 20 ms early reflections in the mapped speech. Objective and subjective test results showed that the proposed system outperformed other baselines in the speech dereverberation task., (© 2024 Acoustical Society of America.)

This paper investigates multimodal sensor architectures with deep learning for audio-visual speech recognition, focusing on in-the-wild scenarios. The term "in the wild" is used to describe AVSR for unconstrained natural-language audio streams and video-stream modalities. Audio-visual speech recognition (AVSR) is a speech-recognition task that leverages both an audio input of a human voice and an aligned visual input of lip motions. However, since in-the-wild scenarios can include more noise, AVSR's performance is affected. Here, we propose new improvements for AVSR models by incorporating data-augmentation techniques to generate more data samples for building the classification models. For the data-augmentation techniques, we utilized a combination of conventional approaches (e.g., flips and rotations), as well as newer approaches, such as generative adversarial networks (GANs). To validate the approaches, we used augmented data from well-known datasets (LRS2-Lip Reading Sentences 2 and LRS3) in the training process and testing was performed using the original data. The study and experimental results indicated that the proposed AVSR model and framework, combined with the augmentation approach, enhanced the performance of the AVSR framework in the wild for noisy datasets. Furthermore, in this study, we discuss the domains of automatic speech recognition (ASR) architectures and audio-visual speech recognition (AVSR) architectures and give a concise summary of the AVSR models that have been proposed.

Voice isolation, a prominent research area in the field of speech processing, has garnered a great deal of attention due to its prospective implications in numerous domains. Deep neural networks (DNNs) have emerged as a potent instrument for addressing the challenges associated with vocal isolation. This paper presents a comprehensive study on the use of DNNs for voice isolation, focusing on speech recognition and speaker identification tasks. The proposed method uses frequency domain and time domain techniques to improve the separation of target utterances from background noise. The experimental results demonstrate the efficacy of the proposed method, revealing substantial improvements in voice isolation precision and robustness. This study's findings contribute to the increasing corpus of research on voice isolation techniques and provide valuable insights into the application of DNNs to improve speech processing tasks . [ABSTRACT FROM AUTHOR]

Frequency-domain monaural speech enhancement has been extensively studied for over 60 years, and a great number of methods have been proposed and applied to many devices. In the last decade, monaural speech enhancement has made tremendous progress with the advent and development of deep learning, and performance using such methods has been greatly improved relative to traditional methods. This survey paper first provides a comprehensive overview of traditional and deep-learning methods for monaural speech enhancement in the frequency domain. The fundamental assumptions of each approach are then summarized and analyzed to clarify their limitations and advantages. A comprehensive evaluation of some typical methods was conducted using the WSJ + Deep Noise Suppression (DNS) challenge and Voice Bank + DEMAND datasets to give an intuitive and unified comparison. The benefits of monaural speech enhancement methods using objective metrics relevant for normal-hearing and hearing-impaired listeners were evaluated. The objective test results showed that compression of the input features was important for simulated normal-hearing listeners but not for simulated hearing-impaired listeners. Potential future research and development topics in monaural speech enhancement are suggested., Competing Interests: Declaration of Conflicting InterestsThe authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Robust Lombard speech-in-noise detecting is challenging. This study proposes a strategy to detect Lombard speech using a machine learning approach for applications such as public address systems that work in near real time. The paper starts with the background concerning the Lombard effect. Then, assumptions of the work performed for Lombard speech detection are outlined. The framework proposed combines convolutional neural networks (CNNs) and various two-dimensional (2D) speech signal representations. To reduce the computational cost and not resign from the 2D representation-based approach, a strategy for threshold-based averaging of the Lombard effect detection results is introduced. The pseudocode of the averaging process is also included. A series of experiments are performed to determine the most effective network structure and the 2D speech signal representation. Investigations are carried out on German and Polish recordings containing Lombard speech. All 2D signal speech representations are tested with and without augmentation. Augmentation means using the alpha channel to store additional data: gender of the speaker, F0 frequency, and first two MFCCs. The experimental results show that Lombard and neutral speech recordings can clearly be discerned, which is done with high detection accuracy. It is also demonstrated that the proposed speech detection process is capable of working in near real-time. These are the key contributions of this work.

With the emergence of the information age, computers have entered the homes of ordinary people and have become essential daily appliances for people. The integration of people and computers has become more popular and in-depth. Based on this situation, how to make computers and humans communicate intelligently and make human-computer interaction more convenient and practical is the main issue of scientific research and discussion on computers. Language recognition is widely used in industry, the economy, commerce, tourism, style, and sports. The BP neural network based on deep learning has supercomputing functions and is very suitable for speech recognition, which promotes the promotion and application of speech recognition technology in many fields. Different types of applications and diverse scenarios enable users to experience different personal experiences. This paper examines the use of application software for ideological and political theory courses in colleges and universities and the application of application software for ideological and political theory courses in colleges and universities in the management of ideological and political theory courses. Analyzed the use of apps in the management of ideological and political theory courses in colleges and universities, analyzed the possibilities and challenges that this brought to the management of ideological and political theory courses in colleges and universities based on the app, and put forward the necessary countermeasures. Including the development and design of ideological and political theory applications, combining deep learning and CTC algorithms to build acoustic models, using server-client interaction, designing an ideological and political theory course app based on speech recognition and deep learning, and forming an offline speech recognition system software platform, the app provides a certain reference for improving the skills of teaching managers., Competing Interests: The author(s) declare(s) that they have no conflicts of interest., (Copyright © 2022 Xiajin Chen.)

Bone-conducted microphone (BCM) senses vibrations from bones in the skull during speech to electrical audio signal. When transmitting speech signals, bone-conduction microphones (BCMs) capture speech signals based on the vibrations of the speaker's skull and have better noise-resistance capabilities than standard air-conduction microphones (ACMs). BCMs have a different frequency response than ACMs because they only capture the low-frequency portion of speech signals. When we replace an ACM with a BCM, we may get satisfactory noise suppression results, but the speech quality and intelligibility may suffer due to the nature of the solid vibration. Mismatched BCM and ACM characteristics can also have an impact on ASR performance, and it is impossible to recreate a new ASR system using voice data from BCMs. The speech intelligibility of a BCM-conducted speech signal is determined by the location of the bone used to acquire the signal and accurately model phonemes of words. Deep learning techniques such as neural network have traditionally been used for speech recognition. However, neural networks have a high computational cost and are unable to model phonemes in signals. In this paper, the intelligibility of BCM signal speech was evaluated for different bone locations, namely the right ramus, larynx, and right mastoid. Listener and deep learning architectures such as CapsuleNet, UNet, and S-Net were used to acquire the BCM signal for Tamil words and evaluate speech intelligibility. As validated by the listener and deep learning architectures, the Larynx bone location improves speech intelligibility., Competing Interests: The authors declare that they have no conflicts of interest., (Copyright © 2022 Venkata Subbaiah Putta et al.)

The main goal of speech recognition technology is to use computers to convert human analog speech signals into computer-generated signals, such as behavior patterns or binary codes. Different from speaker identification and speaker confirmation, the latter attempts to identify or confirm the speaker who uttered the speech rather than the lexical content contained in it. The short-term idea is that it should be able to record the musical sound played by the user with a certain musical instrument, then extract the note and duration information from it, and finally generate the corresponding MID file according to the MIDI standard, which can set the type of musical instrument in advance to complete the function of musical sound transformation, such as playing with a harmonica, and playing the MID at the end is the piano sound. With the rapid development of the mobile Internet, fields such as machine learning, electronic communication, and navigation have placed high demands on real-time and standard text recognition technology. This paper merges the sound of visual music into text-based data set training, uses the exported scanner features for model training, uses the model to extract features, then uses the features for prior training, and then uses pretraining. DNN results show that the combined training of target prevention and expansion plans, by replacing long-term and short-term memory networks, end-to-end speech recognition programs, and behavioral tests organized by mobile devices, can provide a larger receptive field combined with expanded convolution instead of long and short periods. The experimental results show that when the input sampling point is 2400, it can be seen that the convergence speed of the model becomes slower with more than 90 iterations and the loss of the model on the verification set increases with the increase in the number of iterations. This shows that the model in this paper can fully meet the needs of speech recognition in piano music scenes., Competing Interests: The author declares no conflicts of interest regarding the publication of this article., (Copyright © 2022 Jie Bai.)

Communication has been an important aspect of human life, civilization, and globalization for thousands of years. Biometric analysis, education, security, healthcare, and smart cities are only a few examples of speech recognition applications. Most studies have mainly concentrated on English, Spanish, Japanese, or Chinese, disregarding other low-resource languages, such as Uzbek, leaving their analysis open. In this paper, we propose an End-To-End Deep Neural Network-Hidden Markov Model speech recognition model and a hybrid Connectionist Temporal Classification (CTC)-attention network for the Uzbek language and its dialects. The proposed approach reduces training time and improves speech recognition accuracy by effectively using CTC objective function in attention model training. We evaluated the linguistic and lay-native speaker performances on the Uzbek language dataset, which was collected as a part of this study. Experimental results show that the proposed model achieved a word error rate of 14.3% using 207 h of recordings as an Uzbek language training dataset.

As a result of the fast rise of globalization, people in China are learning English at a rapid pace. However, there is a severe shortage of English teachers in the region, which is a major hindrance. To address these concerns, a deep learning-based algorithm is proposed that can not only check English pronunciation but also help learners distinguish between phonemic and quality phonemic while listening and differentiating, as well as correct phonemic errors, thereby increasing their language learning capacity. In order to study the application of nonlinear network identification technology in English learning, this paper evaluates the English pronunciation quality through the deep learning algorithm of deep learning combined with the related contents of neural network data model, and the experimental results of speech recognition structure are analyzed and discussed in detail. The concordance between machine and manual intonation evaluation is 80%, the concordance rate of adjacent intonation evaluation is 98.33%, and the Pearson correlation coefficient is 0.627 that shows the technique is reliable. The method of English pronunciation and speech identification model is sensible and dependable, which can give beginners a punctual, exact and impartial judgment and response guidance, assist learners to get on the differences between their phonemic and standard phonemic, and correct phonemic mistakes, in order to enhance the ability of oral English learning., Competing Interests: The authors declare that there are no conflicts of interest regarding the publication of this paper., (Copyright © 2022 Liang Meng et al.)

The cocktail party effect refers to the human sense of hearing's ability to pay attention to a single conversation while filtering out all other background noise. To mimic this human hearing ability for people with hearing loss, scientists integrate beamforming algorithms into the signal processing path of hearing aids or implants' audio processors. Although these algorithms' performance strongly depends on the number and spatial arrangement of the microphones, most devices are equipped with a small number of microphones mounted close to each other on the audio processor housing. We measured and evaluated the impact of the number and spatial arrangement of hearing aid or head-mounted microphones on the performance of the established Minimum Variance Distortionless Response beamformer in cocktail party scenarios. The measurements revealed that the optimal microphone placement exploits monaural cues (pinna-effect), is close to the target signal, and creates a large distance spread due to its spatial arrangement. However, this microphone placement is impractical for hearing aid or implant users, as it includes microphone positions such as on the forehead. To overcome microphones' placement at impractical positions, we propose a deep virtual sensing estimation of the corresponding audio signals. The results of objective measures and a subjective listening test with 20 participants showed that the virtually sensed microphone signals significantly improved the speech quality, especially in cocktail party scenarios with low signal-to-noise ratios. Subjective speech quality was assessed using a 3-alternative forced choice procedure to determine which of the presented speech mixtures was most pleasant to understand. Hearing aid and cochlear implant (CI) users might benefit from the presented approach using virtually sensed microphone signals, especially in noisy environments., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2021 The Author(s). Published by Elsevier B.V. All rights reserved.)

The goal of monaural speech enhancement is to separate clean speech from noisy speech. Recently, many studies have employed generative adversarial networks (GAN) to deal with monaural speech enhancement tasks. When using generative adversarial networks for this task, the output of the generator is a speech waveform or a spectrum, such as a magnitude spectrum, a mel-spectrum or a complex-valued spectrum. The spectra generated by current speech enhancement methods in the time-frequency domain usually lack details, such as consonants and harmonics with low energy. In this paper, we propose a new type of adversarial training framework for spectrum generation, named μ-law spectrum generative adversarial networks (μ-law SGAN). We introduce a trainable μ-law spectrum compression layer (USCL) into the proposed discriminator to compress the dynamic range of the spectrum. As a result, the compressed spectrum can display more detailed information. In addition, we use the spectrum transformed by USCL to regularize the generator's training, so that the generator can pay more attention to the details of the spectrum. Experimental results on the open dataset Voice Bank + DEMAND show that μ-law SGAN is an effective generative adversarial architecture for speech enhancement. Moreover, visual spectrogram analysis suggests that μ-law SGAN pays more attention to the enhancement of low energy harmonics and consonants., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2020 Elsevier Ltd. All rights reserved.)

Speaker separation is a special case of speech separation, in which the mixture signal comprises two or more speakers. Many talker-independent speaker separation methods have been introduced in recent years to address this problem in anechoic conditions. To consider more realistic environments, this paper investigates talker-independent speaker separation in reverberant conditions. To effectively deal with speaker separation and speech dereverberation, extending the deep computational auditory scene analysis (CASA) approach to a two-stage system is proposed. In this method, reverberant utterances are first separated and separated utterances are then dereverberated. The proposed two-stage deep CASA system significantly outperforms a baseline one-stage deep CASA method in real reverberant conditions. The proposed system has superior separation performance at the frame level and higher accuracy in assigning separated frames to individual speakers. The proposed system successfully generalizes to an unseen speech corpus and exhibits similar performance to a talker-dependent system.

Traditional sentiment analysis focuses on text-level sentiment mining, transforming sentiment mining into classification or regression problems, resulting in a sentiment analysis low accuracy rate. Sentiment analysis refers to the use of natural language processing, text analysis, and computational linguistics to systematically identify, extract, quantify, and study sentimental states. Therefore, more scholars have begun to focus on speech recognition and facial expression recognition research, and extracting and analysing people's sentiment tendencies can improve sentiment recognition accuracy. Traditional single-modal sentiment analysis can no longer meet people's needs. Therefore, this paper proposes a multimodal sentiment analysis method based on the multimodal sentiment analysis method that can obtain more sentimental information sources and help people make better decisions. The experimental results in this paper show that the highest recognition rates of CNN-SVM, RNN-SVM, and CRNN-SVM were 76.8%, 71.2%, and 93.5%, respectively. It can be seen that CRNN-SVM has the highest sentiment tendency recognition rate in deep learning, so it is suitable to apply CRNN-SVM to sentiment tendency analysis system design in this paper. The average accuracy rate of the system designed in this paper was 91%, and the stability was also very strong, which shows that the system designed in this paper is meaningful. The main contribution of this paper is based on the limitations of single-mode emotion analysis. It proposes a multimode emotion analysis method and introduces a convolutional neural network to help people obtain more emotional information sources to meet their needs. [ABSTRACT FROM AUTHOR]

The effort required to listen to and understand noisy speech is an important factor in the evaluation of noise reduction schemes. This paper introduces a model for Listening Effort prediction from Acoustic Parameters (LEAP). The model is based on methods from automatic speech recognition, specifically on performance measures that quantify the degradation of phoneme posteriorgrams produced by a deep neural net: Noise or artifacts introduced by speech enhancement often result in a temporal smearing of phoneme representations, which is measured by comparison of phoneme vectors. This procedure does not require a priori knowledge about the processed speech, and is therefore single-ended. The proposed model was evaluated using three datasets of noisy speech signals with listening effort ratings obtained from normal hearing and hearing impaired subjects. The prediction quality was compared to several baseline models such as the ITU-T standard P.563 for single-ended speech quality assessment, the American National Standard ANIQUE+ for single-ended speech quality assessment, and a single-ended SNR estimator. In all three datasets, the proposed new model achieved clearly better prediction accuracies than the baseline models; correlations with subjective ratings were above 0.9. So far, the model is trained on the specific noise types used in the evaluation. Future work will be concerned with overcoming this limitation by training the model on a variety of different noise types in a multi-condition way in order to make it generalize to unknown noise types., (Copyright © 2018 Elsevier B.V. All rights reserved.)

Goodness of pronunciation (GOP) is the most widely used method for automatic mispronunciation detection. In this paper, a transfer learning approach to GOP based mispronunciation detection when applying maximum F1-score criterion (MFC) training to deep neural network (DNN)-hidden Markov model based acoustic models is proposed. Rather than train the whole network using MFC, a DNN is used, whose hidden layers are borrowed from native speech recognition with only the softmax layer trained according to the MFC objective function. As a result, significant mispronunciation detection improvement is obtained. In light of this, the two-stage transfer learning based GOP is investigated in depth. The first stage exploits the hidden layer(s) to extract phonetic-discriminating features. The second stage uses a trainable softmax layer to learn the human standard for judgment. The validation is carried out by experimenting with different mispronunciation detection architectures using acoustic models trained by different criteria. It is found that it is preferable to use frame-level cross-entropy to train the hidden layer parameters. Classifier based mispronunciation detection is further experimented with using features computed by transfer learning based GOP and it is shown that it also helps to achieve better results.

This project will study a fast retrieval method for music humming speech recognition based on sentence features and deep learning. The method proposed in this paper can realize the fast extraction of songs. According to the characteristics of the natural pause mode of the song, the song database and the song fragments provided by the user are divided into different sentences. The deep learning algorithm of BDTW is used to calculate the similarity of the song's pitch, and users can set matching conditions according to their preferences. It can identify the most significant differences between music fragments and the order of queries in the database. Then, a retrieval method of a music database based on DIS is proposed. It can shorten the acquisition time. Experiments show that the algorithm can recognize humming songs quickly and efficiently. [ABSTRACT FROM AUTHOR]

Multimodal brain signal analysis has shown great potential in decoding complex cognitive processes, particularly in the challenging task of inner speech recognition. This paper introduces an innovative I nner Speech Recognition via Cross-Perception (ISRCP) approach that significantly enhances accuracy by fusing electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) data. Our approach comprises three core components: (1) multigranularity encoders that separately process EEG time series, EEG Markov Transition Fields, and fMRI spatial data; (2) a cross-perception expert structure that learns both modality-specific and shared representations; and (3) an attention-based adaptive fusion strategy that dynamically adjusts the contributions of different modalities based on task relevance. Extensive experiments on the Bimodal Dataset on Inner Speech demonstrate that our model outperforms existing methods across accuracy and F1 score. [ABSTRACT FROM AUTHOR]

As the listening capacity exists in humans, they are facing the critical issues of understanding the speeches even in the presence of some background or other noises in the world. To diminish the noises, Speech Enhancement (SE) is the process of improving the quality of the speech signal by applying some techniques without degrading any such information. Hence, it is used for hearing-aid people, speech recognition, etc. Recent researchers have developed some works to increase speech intelligibility. Substantial success has been achieved by executing the supervised learning methods. Nevertheless, the existing process incurs such shortcomings as attaining maximum error, computation burden, and so on. To overcome that, certain deep learning methodologies are immensely involved in SE for determining the spectrogram magnitude when reconstructing the signal source by removing the noise. Thus, it also becomes a more challenging task to acquire a clean speech signal. Though these methods aim to present the speech as more clear and intelligible, it may arise such intricacies that degrade the quality and efficiency. As it contains beneficial structure and resources, still it is in the scope of developing the novel SE model. To conquer these complexities, a successful SE task is offered utilizing deep learning in this paper. This recommended work performs the SE which employs deep learning to denoise a noisy speech to generate a quality speech. At first, the speech signal that contains noises such as cooler noise or fan noise, restaurant noise, railway station noise, factory noise, traffic in the journey, bus-stand noise, cinema theater noise, and clouding areas noise are gathered from the standard online sources. After that, this noisy speech signal is forwarded to Adaptive Residual Convolutional Neural Networks with Encoder-Decoder (ARCNNetED) architecture, where the parameters involved in this framework are optimized with the support of Random Revised Drawer Algorithm (RRDA). Thus, the noise presented in the input speech signal is completely removed by the suggested ARCNNetED and the quality of the speech also is enhanced. Finally, the performance of the suggested speech enhancement approach is evaluated over various traditional tasks with the support of several metrics. The findings of the developed model show better performance in terms of MAE, RMSE, and PSNR showing the value of 0.03, 0.19, and 62.26. This analysis significantly handles the error rate to offer accurate outcomes in the speech recognition framework. [ABSTRACT FROM AUTHOR]

Research on voice recognition for African languages is limited due to the scarcity of digital resources for training and adaptation, despite its broad usefulness. The Hausa language, spoken by almost fifty million inhabitants in West and Central Africa, is an example of a linguistic domain that has not been thoroughly studied. The Hausa language employs diacritics, which are symbols located above alphabetical characters to convey further information. By removing diacritics, the number of homographs increases, making it difficult to distinguish between similar words. This paper presents a study on speech recognition in the Hausa Language, specifically focusing on diacritized words. The study utilises the state-of-the-art wave2vec2.0 and Whisper deep learning architecture models, for transcribing audio signals into corresponding Hausa text. According to the results obtained in the study, the Whisper-large deep model emerged as the best, achieving a word error rate of 4.23% representing a considerable improvement of 43.9% when compared to the existing state-of-the-art model for Hausa language speech recognition. Additionally, the Whsiper-large model demonstrated a diacritic coverage of 92%, precision of 98.87%, with a diacritic error rate of 2.1%. [ABSTRACT FROM AUTHOR]

Recently, the machine learning research trend expands to the system performance optimization field, where it has still been proposed by researchers based on their intuitions and heuristics. Compared to conventional major machine learning research areas such as image or speech recognition, machine learning-based system performance optimization fields are at the beginning stage. However, recent papers show that this approach is promising and has significant potential. This paper reviews 11 machine learning-based system performance optimization approaches from nine recent papers based on well-known machine learning models such as perceptron, LSTM, and RNN. This survey provides a detailed design and summarizes model, input, output, and prediction method of each approach. This paper covers various system performance areas from the data structure to essential system components of a computer system such as index structure, branch predictor, sort, and cache management. The result shows that machine learning-based system performance optimization has an important potential for future research. We expect that this paper shows a wide range of applicability of machine learning technology and provides a new perspective for system performance optimization. [ABSTRACT FROM AUTHOR]

Formosan languages, spoken by the indigenous peoples of Taiwan, have unique roles in the reconstruction of Proto-Austronesian Languages. This paper presents a real-world Formosan language speech dataset, including 144 h of news footage for 16 Formosan languages, and uses self-supervised models to obtain and analyze their speech representations. Among the news footage, 13 h of the validated speech data of Formosan languages are selected, and a language classifier, based on XLSR-53, is trained to classify the 16 Formosan languages with an accuracy of 86%. We extracted and analyzed the speech vector representations learned from the model and compared them with 152 manually coded linguistic typological features. The comparison shows that the speech vectors reflect Formosan languages' phonological and morphological aspects. Furthermore, the speech vectors and linguistic features are used to construct a linguistic phylogeny, and the resulting genealogical grouping corresponds with previous literature. These results suggest that we can investigate the current real-world language usages through the speech model, and the dataset opens a window to look into the Formosan languages in vivo. [ABSTRACT FROM AUTHOR]

Articulation disorder, as a medical difficulty, currently mainstream speech recognition technologies are not well adapted to the needs of this field. At the same time, speech recognition technology for dysarthria utilizes a combination of pre training and personalized training to further improve algorithm performance and reduce recognition word error rate through data-driven methods. However, currently, speech recognition technology for dysarthria still has a certain distance from practical commercial use, and its development is limited by data scale and technology. So far, there have been no comprehensive articles on speech recognition for dysarthria. It is urgent to compare and analyze the construction methods and advanced technologies of various datasets in this field, in order to facilitate researchers entering the field to quickly acquire knowledge in this field. This paper conducts a survey on existing datasets, mainstream algorithms, and evaluation methods, and summarizes the scale, form, and characteristics of mainstream speech impairment datasets at home and abroad. It analyzes the mainstream algorithms for speech recognition with dysarthria, and provides the performance and characteristics of different algorithms. Finally, the performance evaluation indicators of the algorithm model based on the severity level of patients with dysarthria are studied, and future research directions are discussed, in order to provide help for the researchers engaged in speech recognition with dysarthria and assist in the rapid development of this field. [ABSTRACT FROM AUTHOR]

A lack of labeled examples is a problem in different domains, such as text and image processing, medicine, and static reservoir characterization, because supervised learning relies on vast volumes of these data to perform successfully, but this is quite expensive. However, large amounts of unlabeled data exist in these domains. The deep semi-supervised learning (DSSL) approach leverages unlabeled data to improve supervised learning performance using deep neural networks. This approach has succeeded in image recognition, text classification, and speech recognition. Nevertheless, there have been few works on pre-stack seismic reservoir characterization, in which knowledge of rock and fluid properties is fundamental for oil exploration. This paper proposes a methodology to estimate acoustic impedance using pre-stack seismic data and DSSL with a recurrent neural network. The few labeled datasets for training were pre-processed from raw seismic and acoustic impedance data from five borehole logs. The results showed that the acoustic impedance estimation at the well location and outside it was better predicted by the DSSL compared to the supervised version of the same neural network. Therefore, employing a large amount of unlabeled data can be helpful in the development of seismic data interpretation systems. [ABSTRACT FROM AUTHOR]

This document is an editorial titled "Physiological signal processing for wellness" published in the journal Frontiers in Signal Processing. The editorial discusses the field of physiological signal processing and its potential to transform the way we approach wellness. By analyzing signals like heart rate, brain activity, and skin temperature, researchers can gain insights into our internal state and develop personalized wellbeing solutions. The editorial also acknowledges the challenges of data privacy, algorithm accuracy, and generalizability across diverse populations. The document includes summaries of three research papers on topics related to physiological signal processing, including predicting mental states using heart rate variability, using AI cough analysis for early detection of COVID-19, and decoding silent thoughts through imagined speech recognition. Each paper highlights the potential of advanced signal processing techniques for improving mental state assessment, COVID-19 detection, and communication technology. Another paper explores the use of deep learning for more accurate and automated ECG analysis, addressing challenges in signal transformation and fine-tuning pre-trained models. The document concludes by emphasizing the importance of collaboration and responsible development of this technology to ensure equitable access to personalized wellbeing solutions. [Extracted from the article]

In this paper, we use machine learning algorithms to conduct in-depth research and analysis on the construction of human-computer interaction systems and propose a simple and effective method for extracting salient features based on contextual information. The method can retain the dynamic and static information of gestures intact, which results in a richer and more robust feature representation. Secondly, this paper proposes a dynamic planning algorithm based on feature matching, which uses the consistency and accuracy of feature matching to measure the similarity of two frames and then uses a dynamic planning algorithm to find the optimal matching distance between two gesture sequences. The algorithm ensures the continuity and accuracy of the gesture description and makes full use of the spatiotemporal location information of the features. The features and limitations of common motion target detection methods in motion gesture detection and common machine learning tracking methods in gesture tracking are first analyzed, and then, the kernel correlation filter method is improved by designing a confidence model and introducing a scale filter, and finally, comparison experiments are conducted on a self-built gesture dataset to verify the effectiveness of the improved method. During the training and validation of the model by the corpus, the complementary feature extraction methods are ablated and learned, and the corresponding results obtained are compared with the three baseline methods. But due to this feature, GMMs are not suitable when users want to model the time structure. It has been widely used in classification tasks. By using the kernel function, the support vector machine can transform the original input set into a high-dimensional feature space. After experiments, the speech emotion recognition method proposed in this paper outperforms the baseline methods, proving the effectiveness of complementary feature extraction and the superiority of the deep learning model. The speech is used as the input of the system, and the emotion recognition is performed on the input speech, and the corresponding emotion obtained is successfully applied to the human-computer dialogue system in combination with the online speech recognition method, which proves that the speech emotion recognition applied to the human-computer dialogue system has application research value. [ABSTRACT FROM AUTHOR]

The characteristics of acoustic emission signals generated in the process of rock deformation and fission contain rich information on internal rock damage. The use of acoustic emissions monitoring technology can analyze and identify the precursor information of rock failure. At present, in the field of acoustic emissions monitoring and the early warning of rock fracture disasters, there is no real-time identification method for a disaster precursor characteristic signal. It is easy to lose information by analyzing the characteristic parameters of traditional acoustic emissions to find signals that serve as precursors to disasters, and analysis has mostly been based on post-analysis, which leads to poor real-time recognition of disaster precursor characteristics and low application levels in the engineering field. Based on this, this paper regards the acoustic emissions signal of rock fracture as a kind of speech signal generated by rock fracture uses this idea of speech recognition for reference alongside spectral analysis (STFT) and Mel frequency analysis to realize the feature extraction of acoustic emissions from rock fracture. In deep learning, based on the VGG16 convolutional neural network and AlexNet convolutional neural network, six intelligent real-time recognition models of rock fracture and key acoustic emission signals were constructed, and the network structure and loss function of traditional VGG16 were optimized. The experimental results show that these six deep-learning models can achieve the real-time intelligent recognition of key signals, and Mel, combined with the improved VGG16, achieved the best performance with 87.68% accuracy and 81.05% recall. Then, by comparing multiple groups of signal recognition models, Mel+VGG-FL proposed in this paper was verified as having a high recognition accuracy and certain recognition efficiency, performing the intelligent real-time recognition of key acoustic emission signals in the process of rock fracture more accurately, which can provide new ideas and methods for related research and the real-time intelligent recognition of rock fracture precursor characteristics. [ABSTRACT FROM AUTHOR]

Dysarthric speech has several pathological characteristics, such as discontinuous pronunciation, uncontrolled volume, slow speech, explosive pronunciation, improper pauses, excessive nasal sounds, and air-flow noise during pronunciation, which differ from healthy speech. Automatic speech recognition (ASR) can be very helpful for speakers with dysarthria. Our research aims to provide a scoping review of ASR for dysarthric speech, covering papers in this field from 1990 to 2022. Our survey found that the development of research studies about the acoustic features and acoustic models of dysarthric speech is nearly synchronous. During the 2010s, deep learning technologies were widely applied to improve the performance of ASR systems. In the era of deep learning, many advanced methods (such as convolutional neural networks, deep neural networks, and recurrent neural networks) are being applied to design acoustic models and lexical and language models for dysarthric-speech-recognition tasks. Deep learning methods are also used to extract acoustic features from dysarthric speech. Additionally, this scoping review found that speaker-dependent problems seriously limit the generalization applicability of the acoustic model. The scarce available speech data cannot satisfy the amount required to train models using big data. [ABSTRACT FROM AUTHOR]

Finding ways to enable seamless communication between deaf and able-bodied individuals has been a challenging and pressing issue. This paper proposes a solution to this problem by designing a low-cost data glove that utilizes multiple inertial sensors with the purpose of achieving efficient and accurate sign language recognition. In this study, four machine learning models—decision tree (DT), support vector machine (SVM), K-nearest neighbor method (KNN), and random forest (RF)—were employed to recognize 20 different types of dynamic sign language data used by deaf individuals. Additionally, a proposed attention-based mechanism of long and short-term memory neural networks (Attention-BiLSTM) was utilized in the process. Furthermore, this study verifies the impact of the number and position of data glove nodes on the accuracy of recognizing complex dynamic sign language. Finally, the proposed method is compared with existing state-of-the-art algorithms using nine public datasets. The results indicate that both the Attention-BiLSTM and RF algorithms have the highest performance in recognizing the twenty dynamic sign language gestures, with an accuracy of 98.85% and 97.58%, respectively. This provides evidence for the feasibility of our proposed data glove and recognition methods. This study may serve as a valuable reference for the development of wearable sign language recognition devices and promote easier communication between deaf and able-bodied individuals. [ABSTRACT FROM AUTHOR]

Deep learning is a rapidly growing research area having state of art achievement in various applications including but not limited to speech recognition, object recognition, machine translation, and image segmentation. In the current modern industrial manufacturing system, Machine Health Surveillance System (MHSS) is achieving increasing popularity because of the widespread availability of low cost sensors internet connectivity. Deep learning architecture gives useful tools to analyze and process these vast amounts of machinery data. In this paper, we review the latest deep learning techniques and their variant used for MHSS. We used Gearbox Fault Diagnosis dataset in this paper that contains the sets of vibration attributes recorded by SpectraQuest's Gearbox Fault Diagnostics Simulator. In addition, we used the variant of auto encoders for feature extraction to achieve higher accuracy in machine health surveillance. The results showed that the bagging ensemble classifier based on voting techniques achieved 99% accuracy. [ABSTRACT FROM AUTHOR]

Due to the limitations of English learning environment and teaching conditions in China, English learners in China generally have difficulties in oral English learning.With the development of computer science and technology and the progress of language teaching and learning methods, it is possible to solve this problem.Therefore, this paper applies the mature deep learning technology to English speech recognition and constructs a speech evaluation model.Through experimental research, it is found that the intonation, speech speed and intonation evaluation methods adopted in this paper are highly robust and the English pronunciation quality evaluation model is also feasible. It can effectively promote the development of foreign language teaching in China. [ABSTRACT FROM AUTHOR]

AI has been used to process the data for decision-making, problem-solving, interaction with humans and to understand human's feelings, emotions and their behavior. In today's world, communication between humans takes place digitally, so human's emotions play a very important role for communication as well as detection and analysis. Although there are many surveys related to emotions from speech already done, selecting appropriate datasets and methods are challenging tasks. This survey will primarily concentrate on efficient techniques, including Machine Learning, Deep Learning, and transformer-based approaches, while also providing brief descriptions of existing challenges and outlining future prospects. Additionally, this paper provides a comparative analysis of various datasets and techniques employed by researchers. After conducting the survey, we discovered that deep learning and transformer-based techniques are more effective and yield superior performance results. [ABSTRACT FROM AUTHOR]

Acoustic echo cancellation (AEC) methods aim to suppress the acoustic coupling for hands‐free speech communication. Traditional AEC works by identifying the acoustic impulse response using adaptive algorithms. With recent research advances, deep learning has become an attractive choice for AEC. This paper introduces a two‐stage bidirectional long short term memory (TS‐BLSTM) framework, incorporating multi‐head self‐attention mechanisms after each BLSTM block. This is aimed at better capturing contextual information and further enhancing ability of the model to handle complex acoustic scenarios. The BLSTM blocks are utilized to aggregate magnitude spectrum information, modelling both time and frequency dependencies. Additionally, dilation convolution is introduced to broaden the range of information in each convolution output. The magnitude decoder estimates a mask for the input, resulting in the generation of an estimated magnitude spectrum for near‐end speech. Experimental results indicate that the proposed method achieves promising outcomes. [ABSTRACT FROM AUTHOR]

Acoustic signal analysis is an important component of transformer online monitoring. Currently, traditional methods have problems such as low spectral resolution, imbalanced sample distribution, and unsatisfactory classification performance. This article first introduces the matrix pencil algorithm for time-frequency spectrum analysis of acoustic signals, and then uses the SMOTE algorithm to expand the imbalanced samples. Then, an ACmix hybrid deep neural network model is constructed to classify 11 types of transformer operation and environmental acoustic signals. Finally, detailed experiments were conducted on the method proposed in this paper, and the experimental results showed that the matrix pencil algorithm has high time-frequency resolution and good noise resistance performance. The SMOTE sample expansion method can significantly improve the recognition accuracy by more than 2%. Overall accuracy of the proposed method in acoustic signal classification tasks reaches 91.81%. [ABSTRACT FROM AUTHOR]

The field of speech recognition makes it simpler for humans and machines to engage with speech. Number-oriented communication, such as using a registration code, mobile number, score, or account number, can benefit from speech recognition for digits. This paper presents our Amazigh automatic speech recognition (ASR) experience based on the deep learning approach. The convolutional neural network (CNN) and Mel spectrogram are exploited to evaluate audio samples and produce spectrograms as a part of the deep learning strategy. To attempt the recognition of the Amazigh numerals, we use a database that includes digits ranging from zero to nine collected from 42 native speakers in total, men and women between the ages of 20 and 40. Our experimental results show that spoken digits in Amazigh can be identified with a maximum accuracy of 93.62%, 94% Precision, and 94% Recall. [ABSTRACT FROM AUTHOR]

The implementation of a defect speech recognition (DSR) system has the opportunity to significantly improve the lifestyle of people with speech disorders. In this paper, we developed a novel ConvGRUSpeechNet model for recognizing and understanding hyperkinetic dysarthria disorder (HDD) speech. The proposed model uniquely combines convolutional layers, recurrent layers (GRU and BiGRU), and dense layers with a LogSoftmax function to effectively recognize and translate HDD speech into text. To prevent overfitting and handling imbalances, we employed data augmentation and splitting functions during the training process. Also, the Mel-frequency cepstral coefficients (MFCC) were employed to reduce the issue of vanishing gradients. In addition, a dataset of Russian speech has been created, comprising 2000 recordings of HDD speech. The primary objective of this research is to improve speech recognition for individuals with HDD by employing the ConvGRUSpeechNet model. The proposed DSR system outperformed the recognition character error rate (CER) of 12.35% using the test dataset. Under the same conditions, the experimental findings show that the proposed solution exhibits superior performance in comparison to existing state-of-the-art CBNs and TDNN-F LF-MMI models. Furthermore, we implemented the TensorFlow model on a flask server, making it accessible for use in a web application. [ABSTRACT FROM AUTHOR]

Automatic Speech Recognition (ASR) systems have witnessed significant advancements in recent years, thanks to the emergence of deep learning techniques and the availability of large speech datasets in various languages. With the increasing demand for Arabic voice-enabled technologies, the availability of a high-quality and representative dataset for the Arabic language becomes crucial. This paper presents the development of a new dataset called ArabAlg, specifically designed for Arabic Speech Command Recognition (ASCR), to support the integration of Arabic voice recognition systems into smart devices in the Internet of Things (IoT). This research focuses on collecting and annotating a diverse range of Arabic speech commands, encompassing various domains and applications. The dataset construction process involves recording and preprocessing several utterances from native Arabic speakers. To ensure precision and reliability, quality control measures are implemented during data collection and annotation. The resulting dataset provides a valuable resource for training and evaluating ASCR systems tailored for Arabic speakers using Machine Learning and Deep Learning. [ABSTRACT FROM AUTHOR]

Simple Summary: This paper introduces a new method for cleaning impaired speech by combining Pareto-optimized deep learning with Non-negative Matrix Factorization (NMF). The approach effectively reduces noise in impaired speech while preserving the desired speech quality. The method involves calculating the spectrogram of a noisy voice clip, determining a noise threshold, computing a noise-to-signal mask, and smoothing it to avoid abrupt transitions. Using a Pareto-optimized NMF, the modified spectrogram is decomposed into basis functions and weights, allowing for reconstruction of the clean speech spectrogram. The final result is a noise-reduced waveform achieved by inverting the clean speech spectrogram. Experimental results validate the method's effectiveness in cleaning alaryngeal speech signals, indicating its potential for real-world applications. The problem of cleaning impaired speech is crucial for various applications such as speech recognition, telecommunication, and assistive technologies. In this paper, we propose a novel approach that combines Pareto-optimized deep learning with non-negative matrix factorization (NMF) to effectively reduce noise in impaired speech signals while preserving the quality of the desired speech. Our method begins by calculating the spectrogram of a noisy voice clip and extracting frequency statistics. A threshold is then determined based on the desired noise sensitivity, and a noise-to-signal mask is computed. This mask is smoothed to avoid abrupt transitions in noise levels, and the modified spectrogram is obtained by applying the smoothed mask to the signal spectrogram. We then employ a Pareto-optimized NMF to decompose the modified spectrogram into basis functions and corresponding weights, which are used to reconstruct the clean speech spectrogram. The final noise-reduced waveform is obtained by inverting the clean speech spectrogram. Our proposed method achieves a balance between various objectives, such as noise suppression, speech quality preservation, and computational efficiency, by leveraging Pareto optimization in the deep learning model. The experimental results demonstrate the effectiveness of our approach in cleaning alaryngeal speech signals, making it a promising solution for various real-world applications. [ABSTRACT FROM AUTHOR]

Text recognition is an important research topic in computer vision. Scene text, which refers to the text in real scenes, sometimes needs to meet the requirement of attracting attention, and there is the situation such as deformation. At the same time, the image acquisition process is affected by factors such as occlusion, noise, and obstruction, making scene text recognition tasks more challenging. In this paper, we improve the CRNN model for text recognition, which has relatively low accuracy, poor performance in recognizing irregular text, and only considers obtaining text sequence information from a single aspect, resulting in incomplete information acquisition. Firstly, to address the problems of low text recognition accuracy and poor recognition of irregular text, we add label smoothing to ensure the model's generalization ability. Then, we introduce the smoothing loss function from speech recognition into the field of text recognition, and add a language model to increase information acquisition channels, ultimately achieving the goal of improving text recognition accuracy. This method was experimentally verified on six public datasets and compared with other advanced methods. The experimental results show that this method performs well in most benchmark tests, and the improved model outperforms the original model in recognition performance. [ABSTRACT FROM AUTHOR]

English is now one of the most important languages for economic exchange in various countries around the world, and it is also the most widely used language for cultural and information exchange. Like other countries, China likewise attaches highest significance to English learning, and people's demand for applied learning is also increasing rapidly these days. However, there are significant differences between Chinese pronunciation and English pronunciation, and China lacks an English environment while teaching English language. Furthermore, the traditional education is limited by the place and time of classes, due to which it cannot meet people's needs for learning English. With the fast progress of computer knowledge, the emergence of deep learning technology can better identify English pronunciation and evaluate the quality of English pronunciation. Additionally, deep learning can provide learners with precise, objective, and rapid pronunciation information. It can also assist learners in determining the differences between their pronunciation and conventional pronunciation through frequent listening and comparison, as well as correcting their pronunciation faults and increasing language learning efficacy. This study looks into the difficulty of using deep learning to evaluate the quality of English speech recognition and pronunciation. To evaluate English pronunciation quality, this paper selects intonation, speed, and rhythm, as the distinguishing indicators. The comparison between the results of manual evaluation and our evaluation clearly shows that English speech recognition and pronunciation quality model using deep learning established in this paper has much higher reliability. Among the 240 samples tested, only 32 samples differ by one grade, and the rest are similar. [ABSTRACT FROM AUTHOR]

This paper analyzes the simulation of special effects animation through fluid equations and data-driven methods. This paper also considers the needs of computer fluid animation simulation in terms of computational accuracy and simulation efficiency, takes high real-time, high interactivity, and high physical accuracy of simulation algorithm as the research focus and target, and proposes a solution algorithm and acceleration scheme based on deep neural network framework for the key problems of simulation of natural phenomena including smoke and liquid. With the deep development of artificial intelligence technology, deep neural network models are widely used in research fields such as computer image classification, speech recognition, and fluid detail synthesis with their powerful data learning capability. Its stable and efficient computational model provides a new problem-solving approach for computerized fluid animation simulation. In terms of time series reconstruction, this paper adopts a tracking-based reconstruction method, including target tracking, 2D trajectory fitting and repair, and 3D trajectory reconstruction. For continuous image sequences, a linear dynamic model algorithm based on pyramidal optical flow is used to track the feature centers of the objects, and the spatial coordinates and motion parameters of the feature points are obtained by reconstructing the motion trajectories. The experimental results show that in terms of spatial reconstruction, the matching method proposed in this paper is more accurate compared with the traditional stereo matching algorithm; in terms of time series reconstruction, the error of target tracking reduced. Finally, the 3D motion trajectory of the point feature object and the motion pattern at a certain moment are shown, and the method in this paper obtains more ideal results, which proves the effectiveness of the method. [ABSTRACT FROM AUTHOR]

This research paper proposes sentiment analysis of pets using deep learning technologies in the artificial intelligence of things system. Mask R-CNN is used to detect image objects and generate contour mask maps, pose analysis algorithms are used to obtain object posture information, and at the same time object sound signals are converted into spectrograms as features, using deep learning image recognition technology to obtain object emotion information. By using the fusion of object posture and emotional characteristics as the basis for pet emotion identification and analysis, the detected specific pet behaviour states will be actively notified to the owner for processing. Compared with traditional speech recognition, which uses mel-frequency cepstral coefficients for feature extraction, coupled with a Gaussian mixture model-hidden Markov model for voice recognition, the experimental method of this research paper effectively improves the accuracy by 70%. Prior work on the implementation of smart pet surveillance systems has used the pet's tail and mouth as features, and has combined these with sound features to analyse the pet's emotions. This research paper proposes a new method of sentiment analysis in pets, and our method is compared with previous related work. Experimental results show that our approach increases the accuracy rate by 70%. [ABSTRACT FROM AUTHOR]

A growing focus among scientists has been on researching the techniques of automatic detection of dementia that can be applied to the speech samples of individuals with dementia. Leveraging the rapid advancements in Deep Learning (DL) and Natural Language Processing (NLP), these techniques have shown great potential in dementia detection. In this context, this paper proposes a method for dementia detection from the transcribed speech of subjects. Unlike conventional methods that rely on advanced language models to address the ability of the subject to make coherent and meaningful sentences, our approach relies on the center of focus of the subjects and how it changes over time as the subject describes the content of the cookie theft image, a commonly used image for evaluating one's cognitive abilities. To do so, we divide the cookie theft image into regions of interest, and identify, in each sentence spoken by the subject, which regions are being talked about. We employed a Long Short-Term Memory (LSTM) neural network to learn different patterns of dementia subjects and control ones and used it to perform a 10-fold cross validation-based classification. Our experimental results on the Pitt corpus from the DementiaBank resulted in a 82.9% accuracy at the subject level and 81.0% at the sample level. By employing data-augmentation techniques, the accuracy at both levels was increased to 83.6% and 82.1%, respectively. The performance of our proposed method outperforms most of the conventional methods, which reach, at best, an accuracy equal to 81.5% at the subject level. [ABSTRACT FROM AUTHOR]

General video search and recommendation systems primarily rely on metadata and personal information. Metadata includes file names, keywords, tags, and genres, among others, and is used to describe the video's content. The video platform assesses the relevance of user search queries to the video metadata and presents search results in order of highest relevance. Recommendations are based on videos with metadata judged to be similar to the one the user is currently watching. Most platforms offer search and recommendation services by employing separate algorithms for metadata and personal information. Therefore, metadata plays a vital role in video search. Video service platforms develop various algorithms to provide users with more accurate search results and recommendations. Quantifying video similarity is essential to enhance the accuracy of search results and recommendations. Since content producers primarily provide basic metadata, it can be abused. Additionally, the resemblance between similar video segments may diminish depending on its duration. This paper proposes a metadata expansion model that utilizes object recognition and Speechto-Text (STT) technology. The model selects key objects by analyzing the frequency of their appearance in the video, extracts audio separately, transcribes it into text, and extracts the script. Scripts are quantified by tokenizing them into words using text-mining techniques. By augmenting metadata with key objects and script tokens, various video content search and recommendation platforms are expected to deliver results closer to user search terms and recommend related content. [ABSTRACT FROM AUTHOR]

Speech enhancement is an important method for improving speech quality and intelligibility in noisy environments. An effective speech enhancement model depends on precise modelling of the long-range dependencies of noisy speech. Several recent studies have examined ways to enhance speech by capturing the long-term contextual information. For speech enhancement, the time–frequency (T–F) distribution of speech spectral components is also important, but is usually ignored in these studies. The multi-stage learning method is an effective way to integrate various deep learning modules at the same time. The benefit of multi-stage training is that the optimization target can be iteratively updated stage by stage. In this paper, speech enhancement is investigated by multi-stage learning using a multi-stage structure in which time–frequency attention (TFA) blocks are followed by stacks of squeezed temporal convolutional networks (S-TCN) with exponentially increasing dilation rates. To reinject original information into later stages, a feature fusion block (FB) is inserted at the input of later stages to reduce the possibility of speech information being lost in the early stages. The S-TCN blocks are responsible for temporal sequence modelling tasks. The time–frequency attention (TFA) is a simple but effective network module that explicitly exploits position information to generate a 2D attention map to characterize the salient T–F distribution of speech by using two branches, time-frame attention and frequency attention in parallel. Extensive experiments have demonstrated that the proposed model consistently improves the performance over existing baselines across two widely used objective metrics such as PESQ and STOI. A significant improvement in system robustness to noise is also shown by our evaluation results using the TFA module. [ABSTRACT FROM AUTHOR]

Automatic Speech Recognition (ASR) systems usually have difficulty accurately transcribing dialectal variations, resulting in subpar performance in areas where dialectal variants are common. The pitch-based Dialect ASR method we described in this paper aims to improve voice recognition for dialectal differences of Punjabi language. We use the pitch information that was taken out of the voice signal as a feature to enhance the dialectal nuance recognition. The suggested system includes a cutting-edge pitch-based feature extraction module that records minute differences in pitch patterns linked to various dialects. This module gives the ASR system the ability to distinguish between phonetic units more effectively and faithfully depict the distinguishing features of dialectal speech. To develop reliable representations from the pitch-based data, we also use deep learning approaches, speaker adaptive training, and vocal-tract length normalization (VTLN). The experimental results show the significant reduction in the WER of 6.63% and 4.98% for Malwa and Majha dialects. Language learning applications could benefit from the developed Punjabi dialectal speech recognition system by offering learners exposure to various dialects and accents. This can help learners develop a well-rounded understanding of the language and better adapt to different regional variations. [ABSTRACT FROM AUTHOR]