Search

This paper introduces Easy One-Step Text-to-Speech (E1 TTS), an efficient non-autoregressive zero-shot text-to-speech system based on denoising diffusion pretraining and distribution matching distillation. The training of E1 TTS is straightforward; it does not require explicit monotonic alignment between the text and audio pairs. The inference of E1 TTS is efficient, requiring only one neural network evaluation for each utterance. Despite its sampling efficiency, E1 TTS achieves naturalness and speaker similarity comparable to various strong baseline models. Audio samples are available at http://e1tts.github.io/ .

Digital Volume Correlation (DVC) is widely used for the analysis of three-dimensional displacement and strain fields based on CT scans. However, the applicability of DVC methods is limited when it comes to geomaterials: CT speckles are directly correlated with the material's microstructure, and the speckle structure cannot be artificially altered, with generally poor speckle quality. Additionally, most geomaterials exhibit elastoplastic properties and will undergo complex-large deformations under external loading, sometimes leading to strain localization phenomena. These factors contribute to inaccuracies in the displacement field obtained through DVC, and at present, there is a shortage of correction methods and accuracy assessment techniques for the displacement field. If the accuracy of the DVC displacement field is sufficiently high, the gray residue of the two volume images before and after deformation should be minimal, utilizing this characteristic to develop a correction method for the displacement field is feasible. The proposed self-correcting strategy of the DVC displacement field based on image matching, which from the experimental measurement error. We demonstrated the effectiveness of the proposed method by CT triaxial tests of granite residual soil. Without adding other parameters or adjusting the original parameters of DVC, the gray residue showed that the proposed method can effectively improve the accuracy of the displacement field. Additionally, the accuracy evaluation method can reasonably estimate the accuracy of the displacement field. The proposed method can effectively improve the accuracy of DVC three-dimensional displacement field for the state of speckles with poor quality and complex-large deformation.

Audio language models have recently emerged as a promising approach for various audio generation tasks, relying on audio tokenizers to encode waveforms into sequences of discrete symbols. Audio tokenization often poses a necessary compromise between code bitrate and reconstruction accuracy. When dealing with low-bitrate audio codes, language models are constrained to process only a subset of the information embedded in the audio, which in turn restricts their generative capabilities. To circumvent these issues, we propose encoding audio as vector sequences in continuous space $\mathbb R^d$ and autoregressively generating these sequences using a decoder-only diffusion transformer (ARDiT). Our findings indicate that ARDiT excels in zero-shot text-to-speech and exhibits performance that compares to or even surpasses that of state-of-the-art models. High-bitrate continuous speech representation enables almost flawless reconstruction, allowing our model to achieve nearly perfect speech editing. Our experiments reveal that employing Integral Kullback-Leibler (IKL) divergence for distillation at each autoregressive step significantly boosts the perceived quality of the samples. Simultaneously, it condenses the iterative sampling process of the diffusion model into a single step. Furthermore, ARDiT can be trained to predict several continuous vectors in one step, significantly reducing latency during sampling. Impressively, one of our models can generate $170$ ms of $24$ kHz speech per evaluation step with minimal degradation in performance. Audio samples are available at http://ardit-tts.github.io/ .

Good health and well-being is among key issues in the United Nations 2030 Sustainable Development Goals. The rising prevalence of large-scale infectious diseases and the accelerated aging of the global population are driving the transformation of healthcare technologies. In this context, establishing large-scale public health datasets, developing medical models, and creating decision-making systems with a human-centric approach are of strategic significance. Recently, by leveraging the extraordinary number of accessible cameras, groundbreaking advancements have emerged in AI methods for physiological signal monitoring and disease diagnosis using camera sensors. These approaches, requiring no specialized medical equipment, offer convenient manners of collecting large-scale medical data in response to public health events. Therefore, we outline a prospective framework and heuristic vision for a camera-based public health (CBPH) framework utilizing visual physiological monitoring technology. The CBPH can be considered as a convenient and universal framework for public health, advancing the United Nations Sustainable Development Goals, particularly in promoting the universality, sustainability, and equity of healthcare in low- and middle-income countries or regions. Furthermore, CBPH provides a comprehensive solution for building a large-scale and human-centric medical database, and a multi-task large medical model for public health and medical scientific discoveries. It has a significant potential to revolutionize personal monitoring technologies, digital medicine, telemedicine, and primary health care in public health. Therefore, it can be deemed that the outcomes of this paper will contribute to the establishment of a sustainable and fair framework for public health, which serves as a crucial bridge for advancing scientific discoveries in the realm of AI for medicine (AI4Medicine)., Comment: 16 pages, 5 figures

The utilization of discrete speech tokens, divided into semantic tokens and acoustic tokens, has been proven superior to traditional acoustic feature mel-spectrograms in terms of naturalness and robustness for text-to-speech (TTS) synthesis. Recent popular models, such as VALL-E and SPEAR-TTS, allow zero-shot speaker adaptation through auto-regressive (AR) continuation of acoustic tokens extracted from a short speech prompt. However, these AR models are restricted to generate speech only in a left-to-right direction, making them unsuitable for speech editing where both preceding and following contexts are provided. Furthermore, these models rely on acoustic tokens, which have audio quality limitations imposed by the performance of audio codec models. In this study, we propose a unified context-aware TTS framework called UniCATS, which is capable of both speech continuation and editing. UniCATS comprises two components, an acoustic model CTX-txt2vec and a vocoder CTX-vec2wav. CTX-txt2vec employs contextual VQ-diffusion to predict semantic tokens from the input text, enabling it to incorporate the semantic context and maintain seamless concatenation with the surrounding context. Following that, CTX-vec2wav utilizes contextual vocoding to convert these semantic tokens into waveforms, taking into consideration the acoustic context. Our experimental results demonstrate that CTX-vec2wav outperforms HifiGAN and AudioLM in terms of speech resynthesis from semantic tokens. Moreover, we show that UniCATS achieves state-of-the-art performance in both speech continuation and editing., Comment: Accepted to AAAI 2024

In this work, we present DiffVoice, a novel text-to-speech model based on latent diffusion. We propose to first encode speech signals into a phoneme-rate latent representation with a variational autoencoder enhanced by adversarial training, and then jointly model the duration and the latent representation with a diffusion model. Subjective evaluations on LJSpeech and LibriTTS datasets demonstrate that our method beats the best publicly available systems in naturalness. By adopting recent generative inverse problem solving algorithms for diffusion models, DiffVoice achieves the state-of-the-art performance in text-based speech editing, and zero-shot adaptation., Comment: Accepted to ICASSP2023

In this paper, we establish the central limit theorem (CLT) for linear spectral statistics (LSSs) of a large-dimensional sample covariance matrix when the population covariance matrices are involved with diverging spikes. This constitutes a nontrivial extension of the Bai-Silverstein theorem (BST) (Ann Probab 32(1):553--605, 2004), a theorem that has strongly influenced the development of high-dimensional statistics, especially in the applications of random matrix theory to statistics. Recently, there has been a growing realization that the assumption of uniform boundedness of the population covariance matrices in the BST is not satisfied in some fields, such as economics, where the variances of principal components may diverge as the dimension tends to infinity. Therefore, in this paper, we aim to eliminate this obstacle to applications of the BST. Our new CLT accommodates spiked eigenvalues, which may either be bounded or tend to infinity. A distinguishing feature of our result is that the variance in the new CLT is related to both spiked eigenvalues and bulk eigenvalues, with dominance being determined by the divergence rate of the largest spiked eigenvalues. The new CLT for LSS is then applied to test the hypothesis that the population covariance matrix is the identity matrix or a generalized spiked model. The asymptotic distributions of the corrected likelihood ratio test statistic and the corrected Nagao's trace test statistic are derived under the alternative hypothesis. Moreover, we present power comparisons between these two LSSs and Roy's largest root test. In particular, we demonstrate that except for the case in which the number of spikes is equal to one, the LSSs could exhibit higher asymptotic power than Roy's largest root test., Comment: It is an update version of arXiv:2212.05896

With the development and progress of science and technology, the Internet of Things(IoT) has gradually entered people's lives, bringing great convenience to our lives and improving people's work efficiency. Specifically, the IoT can replace humans in jobs that they cannot perform. As a new type of IoT vehicle, the current status and trend of research on Unmanned Aerial Vehicle(UAV) is gratifying, and the development prospect is very promising. However, privacy and communication are still very serious issues in drone applications. This is because most drones still use centralized cloud-based data processing, which may lead to leakage of data collected by drones. At the same time, the large amount of data collected by drones may incur greater communication overhead when transferred to the cloud. Federated learning as a means of privacy protection can effectively solve the above two problems. However, federated learning when applied to UAV networks also needs to consider the heterogeneity of data, which is caused by regional differences in UAV regulation. In response, this paper proposes a new algorithm FedBA to optimize the global model and solves the data heterogeneity problem. In addition, we apply the algorithm to some real datasets, and the experimental results show that the algorithm outperforms other algorithms and improves the accuracy of the local model for UAVs.

In this paper, we establish the central limit theorem (CLT) for linear spectral statistics (LSS) of large-dimensional sample covariance matrix when the population covariance matrices are not uniformly bounded, which is a nontrivial extension of the Bai-Silverstein theorem (BST) (2004). The latter has strongly stimulated the development of high-dimensional statistics, especially the application of random matrix theory to statistics. However, the assumption of uniform boundedness of the population covariance matrices is found strongly limited to the applications of BST. The aim of this paper is to remove the blockages to the applications of BST. The new CLT, allows the spiked eigenvalues to exist and tend to infinity. It is interesting to note that the roles of either spiked eigenvalues or the bulk eigenvalues or both of the two are dominating in the CLT. Moreover, the results are checked by simulation studies with various population settings. The CLT for LSS is then applied for testing the hypothesis that a covariance matrix $ \bSi $ is equal to an identity matrix. For this, the asymptotic distributions for the corrected likelihood ratio test (LRT) and Nagao's trace test (NT) under alternative are derived, and we also propose the asymptotic power of LRT and NT under certain alternatives.

Vast tunable optical components are realized based on dynamic reconfigurations of the incident wavefronts, such as beam steering and tunable lens. However, the dominant paradigm of current wavefront reconfiguration technologies relies on complex control systems with degrees of freedom much larger than output wavefronts, e.g. beam steering based on spatial light modulator or phased array antennas. Here, we propose a new paradigm for dynamic reconfiguration of arbitrary output wavefronts using control systems with the same degrees of freedom. As an example, a wavefront transformer is demonstrated using an in-plane two-degree-of-freedom (2DOF) mechanical actuation system of metasurface doublet for semi-omnidirectional beam steering, which measured a 6-kHz modulation speed and a "+-65.6 degree" field of view. This paradigm can be applied to metasurface transformers for dynamic wavefront reconfiguration with any control system for vast applications, such as tunable lens, beam steering, and dynamic beam profiler, just to name a few.