Search

The classification of underwater acoustic signals has garnered a great deal of attention in recent years due to its potential applications in military and civilian contexts. While deep neural networks have emerged as the preferred method for this task, the representation of the signals plays a crucial role in determining the performance of the classification. However, the representation of underwater acoustic signals remains an under-explored area. In addition, the annotation of large-scale datasets for the training of deep networks is a challenging and expensive task. To tackle these challenges, we propose a novel self-supervised representation learning method for underwater acoustic signal classification. Our approach consists of two stages: a pretext learning stage using unlabeled data and a downstream fine-tuning stage using a small amount of labeled data. The pretext learning stage involves randomly masking the log Mel spectrogram and reconstructing the masked part using the Swin Transformer architecture. This allows us to learn a general representation of the acoustic signal. Our method achieves a classification accuracy of 80.22% on the DeepShip dataset, outperforming or matching previous competitive methods. Furthermore, our classification method demonstrates good performance in low signal-to-noise ratio or few-shot settings. [ABSTRACT FROM AUTHOR]

Understanding the dynamic system that produces speech is essential to advancing speech science, and several simultaneous sensory streams can be leveraged to describe the process. As the tongue functional deformation correlates with the lip's shapes of the speaker, this paper aims to explore the association between them. The problem is formulated as a sequence to sequence learning task and a deep neural network is trained using unlabeled lip videos to predict an upcoming ultrasound tongue image sequence. Experimental results show that the machine learning model can predict the tongue's motion with satisfactory performance, which demonstrates that the learned neural network can build the association between two imaging modalities. [ABSTRACT FROM AUTHOR]

Audio tagging aims to infer descriptive labels from audio clips and it is challenging due to the limited size of data and noisy labels. The solution to the tagging task is described in this paper. The main contributions include the following: an ensemble learning framework is applied to ensemble statistical features and the outputs from the deep classifiers, with the goal to utilize complementary information. Moreover, a sample re-weight strategy is employed to address the noisy label problem within the framework. The approach achieves a mean average precision of 0.958, outperforming the baseline system with a large margin. [ABSTRACT FROM AUTHOR]

The feasibility of an automatic re-initialization of contour tracking is explored by using an image similarity-based method in the ultrasound tongue sequences. To this end, the re-initialization method was incorporated into current state-of-art tongue tracking algorithms, and a quantitative comparison was made between different algorithms by computing the mean sum of distances errors. The results demonstrate that with automatic re-initialization, the tracking error can be reduced from an average of 5-6 to about 4 pixels, a result obtained by using a large number of hand-labeled frames and similarity measurements to extract the contours, which results in improved performance. [ABSTRACT FROM AUTHOR]

B-mode ultrasound has emerged as a prevalent tool for observing tongue motion in speech production, gaining traction in speech therapy applications. However, the effective analysis of ultrasound tongue image frame sequences (UTIFs) encounters many challenges, such as the presence of high levels of speckle noise and obscured views. Recently, the application of machine learning, especially deep learning techniques, to UTIF interpretation has shown promise in overcoming these hurdles. This paper presents a thorough examination of the existing literature, focusing on UTIF analysis. The scope of our work encompasses four key areas: a foundational introduction to deep learning principles, an exploration of motion tracking methodologies, a discussion of feature extraction techniques, and an examination of cross-modality mapping. The paper concludes with a detailed discussion of insights gleaned from the comprehensive literature review, outlining potential trends and challenges that lie ahead in the field., (© 2024 Acoustical Society of America.)

Tongue gestural target classification is of great interest to researchers in the speech production field. Recently, deep convolutional neural networks (CNN) have shown superiority to standard feature extraction techniques in a variety of domains. In this letter, both CNN-based speaker-dependent and speaker-independent tongue gestural target classification experiments are conducted to classify tongue gestures during natural speech production. The CNN-based method achieves state-of-the-art performance, even though no pre-training of the CNN (with the exception of a data augmentation preprocessing) was carried out.