Search

The release of WaveNet and Tacotron has forever transformed the speech synthesis landscape. Thanks to these game-changing innovations, the quality of synthetic speech has reached unprecedented levels. However, to measure this leap in quality, an overwhelming majority of studies still rely on the Absolute Category Rating (ACR) protocol and compare systems using its output; the Mean Opinion Score (MOS). This protocol is not without controversy, and as the current state-of-the-art synthesis systems now produce outputs remarkably close to human speech, it is now vital to determine how reliable this score is. To do so, we conducted a series of four experiments replicating and following the 2013 edition of the Blizzard Challenge. With these experiments, we asked four questions about the MOS: How stable is the MOS of a system across time? How do the scores of lower quality systems influence the MOS of higher quality systems? How does the introduction of modern technologies influence the scores of past systems? How does the MOS of modern technologies evolve in isolation? The results of our experiments are manyfold. Firstly, we verify the superiority of modern technologies in comparison to historical synthesis. Then, we show that despite its origin as an absolute category rating, MOS is a relative score. While minimal variations are observed during the replication of the 2013-EH2 task, these variations can still lead to different conclusions for the intermediate systems. Our experiments also illustrate the sensitivity of MOS to the presence/absence of lower and higher anchors. Overall, our experiments suggest that we may have reached the end of a cul-de-sac by only evaluating the overall quality with MOS. We must embark on a new road and develop different evaluation protocols better suited to the analysis of modern speech synthesis technologies. [ABSTRACT FROM AUTHOR]

Traditionally, random sensing matrices such as Gaussian are widely used in Compressed Sensing (CS). The higher computational complexity and non-structure of matrix make it inappropriate for practical use. Hence, we have projected the application of structural sensing matrices in CS. Since these matrices impose structure and hence easy to implement in hardware. In this paper, we have proposed optimization method based on structural sensing matrices and Grassmannian frame theory. These proposed sensing matrices are DCT–Grassmannian sensing matrix, Hadamard–Grassmannian sensing matrix, and Walsh–Hadamard Grassmannian sensing matrix, respectively. These sensing matrices are optimized for minimal mutual coherence based on shrinkage method; which further improves the signal recovery performance of speech signal. The result demonstrates that the performance of the proposed optimization method based on structural sensing matrices is improved compared to conventional Elad's method which employs the Gaussian sensing matrix. Furthermore, the result reveals that the proposed optimization method with Hadamard–Grassmannian sensing matrix yields better performance compared to conventional Elad's method. Finally, speech quality is evaluated using subjective MOS and objective measures such as LLR and WSS. The result exhibited that the proposed optimization method based on Hadamard–Grassmannian sensing matrix and DCT–Grassmannian sensing matrix exhibits the better MOS, LLR, and WSS performance indicating the good quality of speech. [ABSTRACT FROM AUTHOR]