Search

Emotion recognition from speech and music shares similarities due to their acoustic overlap, which has led to interest in transferring knowledge between these domains. However, the shared acoustic cues between speech and music, particularly those encoded by Self-Supervised Learning (SSL) models, remain largely unexplored, given the fact that SSL models for speech and music have rarely been applied in cross-domain research. In this work, we revisit the acoustic similarity between emotion speech and music, starting with an analysis of the layerwise behavior of SSL models for Speech Emotion Recognition (SER) and Music Emotion Recognition (MER). Furthermore, we perform cross-domain adaptation by comparing several approaches in a two-stage fine-tuning process, examining effective ways to utilize music for SER and speech for MER. Lastly, we explore the acoustic similarities between emotional speech and music using Frechet audio distance for individual emotions, uncovering the issue of emotion bias in both speech and music SSL models. Our findings reveal that while speech and music SSL models do capture shared acoustic features, their behaviors can vary depending on different emotions due to their training strategies and domain-specificities. Additionally, parameter-efficient fine-tuning can enhance SER and MER performance by leveraging knowledge from each other. This study provides new insights into the acoustic similarity between emotional speech and music, and highlights the potential for cross-domain generalization to improve SER and MER systems.

Utilizing Self-Supervised Learning (SSL) models for Speech Emotion Recognition (SER) has proven effective, yet limited research has explored cross-lingual scenarios. This study presents a comparative analysis between human performance and SSL models, beginning with a layer-wise analysis and an exploration of parameter-efficient fine-tuning strategies in monolingual, cross-lingual, and transfer learning contexts. We further compare the SER ability of models and humans at both utterance- and segment-levels. Additionally, we investigate the impact of dialect on cross-lingual SER through human evaluation. Our findings reveal that models, with appropriate knowledge transfer, can adapt to the target language and achieve performance comparable to native speakers. We also demonstrate the significant effect of dialect on SER for individuals without prior linguistic and paralinguistic background. Moreover, both humans and models exhibit distinct behaviors across different emotions. These results offer new insights into the cross-lingual SER capabilities of SSL models, underscoring both their similarities to and differences from human emotion perception.

Recent work has shown the feasibility and benefit of bootstrapping an integrated sequence-to-sequence (Seq2Seq) linguistic frontend from a traditional pipeline-based frontend for text-to-speech (TTS). To overcome the fixed lexical coverage of bootstrapping training data, previous work has proposed to leverage easily accessible transcribed speech audio as an additional training source for acquiring novel pronunciation knowledge for uncovered words, which relies on an auxiliary ASR model as part of a cumbersome implementation flow. In this work, we propose an alternative method to leverage transcribed speech audio as an additional training source, based on multi-task learning (MTL). Experiments show that, compared to a baseline Seq2Seq frontend, the proposed MTL-based method reduces PER from 2.5% to 1.6% for those word types covered exclusively in transcribed speech audio, achieving a similar performance to the previous method but with a much simpler implementation flow., Comment: 5 pages

While recent Zero-Shot Text-to-Speech (ZS-TTS) models have achieved high naturalness and speaker similarity, they fall short in accent fidelity and control. To address this issue, we propose zero-shot accent generation that unifies Foreign Accent Conversion (FAC), accented TTS, and ZS-TTS, with a novel two-stage pipeline. In the first stage, we achieve state-of-the-art (SOTA) on Accent Identification (AID) with 0.56 f1 score on unseen speakers. In the second stage, we condition a ZS-TTS system on the pretrained speaker-agnostic accent embeddings extracted by the AID model. The proposed system achieves higher accent fidelity on inherent/cross accent generation, and enables unseen accent generation., Comment: Accepted by ICASSP 2025

Self-supervised learning (SSL) representations from massively multilingual models offer a promising solution for low-resource language speech tasks. Despite advancements, language adaptation in TTS systems remains an open problem. This paper explores the language adaptation capability of ZMM-TTS, a recent SSL-based multilingual TTS system proposed in our previous work. We conducted experiments on 12 languages using limited data with various fine-tuning configurations. We demonstrate that the similarity in phonetics between the pre-training and target languages, as well as the language category, affects the target language's adaptation performance. Additionally, we find that the fine-tuning dataset size and number of speakers influence adaptability. Surprisingly, we also observed that using paired data for fine-tuning is not always optimal compared to audio-only data. Beyond speech intelligibility, our analysis covers speaker similarity, language identification, and predicted MOS., Comment: Accepted to Interspeech 2024

Neural text-to-speech (TTS) has achieved human-like synthetic speech for single-speaker, single-language synthesis. Multilingual TTS systems are limited to resource-rich languages due to the lack of large paired text and studio-quality audio data. TTS systems are typically built using a single speaker's voices, but there is growing interest in developing systems that can synthesize voices for new speakers using only a few seconds of their speech. This paper presents ZMM-TTS, a multilingual and multispeaker framework utilizing quantized latent speech representations from a large-scale, pre-trained, self-supervised model. Our paper combines text-based and speech-based self-supervised learning models for multilingual speech synthesis. Our proposed model has zero-shot generalization ability not only for unseen speakers but also for unseen languages. We have conducted comprehensive subjective and objective evaluations through a series of experiments. Our model has proven effective in terms of speech naturalness and similarity for both seen and unseen speakers in six high-resource languages. We also tested the efficiency of our method on two hypothetically low-resource languages. The results are promising, indicating that our proposed approach can synthesize audio that is intelligible and has a high degree of similarity to the target speaker's voice, even without any training data for the new, unseen language., Comment: Accepted by IEEE/ACM TASLP, 16 pages plus 1 page of bio and photos

The blood protein Von Willebrand Factor (VWF) is critical in facilitating arterial thrombosis. At pathologically high shear rates the protein unfolds and binds to the arterial wall, enabling the rapid deposition of platelets from the blood. We present a novel continuum model for VWF dynamics in flow based on a modified viscoelastic fluid model that incorporates a single constitutive relation to describe the propensity of VWF to unfold as a function of the scalar shear rate. Using experimental data of VWF unfolding in pure shear flow, we fix the parameters for VWF's unfolding propensity and the maximum VWF length, so that the protein is half unfolded at a shear rate of approximately 5,000 s$^{-1}$. We then use the theoretical model to predict VWF's behaviour in two complex flows where experimental data is challenging to obtain: pure elongational flow and stenotic arterial flow. In pure elongational flow, our model predicts that VWF is 50% unfolded at approximately 2,000 s$^{-1}$, matching the established hypothesis that VWF unfolds at lower shear rates in elongational flow than in shear flow. We demonstrate the sensitivity of this elongational flow prediction to the value of maximum VWF length used in the model, which varies significantly across experimental studies, predicting that VWF can unfold between 600 - 3,200 s$^{-1}$ depending on the selected value. Finally, we examine VWF dynamics in a range of idealised arterial stenoses, predicting the relative extension of VWF in elongational flow structures in the centre of the artery compared to high-shear regions near the arterial walls.

For an undirected unweighted graph G = (V, E) with n vertices and m edges, let d(u, v) denote the distance from u in V to v in V in G. An (alpha, beta)-stretch approximate distance oracle (ADO) for G is a data structure that, given u, v in V, returns in constant time a value d-hat (u, v) such that d(u, v) <= d-hat (u, v) <= alpha * d(u, v) + beta, for some reals alpha > 1, beta. If beta = 0, we say that the ADO has stretch alpha. Thorup and Zwick (2005) showed that one cannot beat stretch 3 with subquadratic space (in terms of n) for general graphs. Patrascu and Roditty (2010) showed that one can obtain stretch 2 using O(m^(1/3)n^(4/3)) space, and so if m is subquadratic in n, then the space usage is also subquadratic. Moreover, Patrascu and Roditty (2010) showed that one cannot beat stretch 2 with subquadratic space even for graphs where m = O-tilde(n), based on the set-intersection hypothesis. In this paper, we investigate the minimum possible stretch achievable by an ADO as a function of the graph's maximum degree, a study motivated by the question of identifying the conditions under which an ADO can be stored with subquadratic space while still ensuring a sub-2 stretch. In particular, we show that if the maximum degree in G is Delta_G <= O(n^(1/k - epsilon)) for some 0 < epsilon <= 1/k, then there exists a (2, 1 - k)-stretch ADO for G that uses O-tilde(n^(2 - (k * epsilon) / 3)) space. For k = 2, this result implies a subquadratic sub-2 stretch ADO for graphs with Delta_G <= O(n^(1/2 - epsilon)). We provide tight lower bounds for the upper bound under the same set intersection hypothesis, showing that if Delta_G = Theta(n^(1/k)), a (2, 1 - k)-stretch ADO requires Omega-tilde(n^2) space. Moreover, we show that for constants epsilon, c > 0, a (2 - epsilon, c)-stretch ADO requires Omega-tilde(n^2) space even for graphs with Delta_G = Theta-tilde(1).

In expressive and controllable Text-to-Speech (TTS), explicit prosodic features significantly improve the naturalness and controllability of synthesised speech. However, manual prosody annotation is labor-intensive and inconsistent. To address this issue, a two-stage automatic annotation pipeline is novelly proposed in this paper. In the first stage, we use contrastive pretraining of Speech-Silence and Word-Punctuation (SSWP) pairs to enhance prosodic information in latent representations. In the second stage, we build a multi-modal prosody annotator, comprising pretrained encoders, a text-speech fusing scheme, and a sequence classifier. Experiments on English prosodic boundaries demonstrate that our method achieves state-of-the-art (SOTA) performance with 0.72 and 0.93 f1 score for Prosodic Word and Prosodic Phrase boundary respectively, while bearing remarkable robustness to data scarcity.

This study examines the impact of COVID-19 and associated restrictions on the non-state school sector in low- and middle-income countries (LMICs). Drawing on a survey of Global Schools Forum (GSF) member organisations across 17 countries, and surveys and in-depth remote interviews with schools in India, Nigeria and Kenya, the authors explore how schools were impacted by the pandemic and the ways in which they coped with the disruption. Areas for policy intervention and areas for potential peer learning and scaling up of best practices are highlighted. [This report was co-produced by Global Schools Forum.]

During the height of the COVID-19 pandemic in September 2020, the Rwanda Basic Education Board (REB) -- in its efforts to improve the country's school leaders -- formed a learning partnership with the Education Commission, Education Development Trust, and the World Innovation Summit for Education (WISE) to undertake rapid research on school and system leadership during school closures and as they started to reopen for the first time. This research used an innovative 'learning partnership' approach with Rwandan education stakeholders engaged as a critical 'co-creation' partner in each phase of the process -- from designing the research questions to interpreting the findings. This approach ensured the issues investigated were priorities for local stakeholders and promoted ongoing, dynamic dialogue around emerging findings to influence decision-making which could be acted on to create impact. Key findings from the learning partnership highlighted three main strategies used by leaders to provide continuity of learning and support during school closures and reopening: (1) engaging with caregivers, communities, and other sectors; (2) collaborating with teachers and other school leaders; and (3) tracking, sharing information, and providing targeted support. [This report was produced in partnership with the Education Commission, the Rwanda Basic Education Board (REB), and the World Innovation Summit for Education (WISE). For the accompanying brief, "Bridging the Evidence to Policy Gap: A Learning Partnership Approach to Research," see ED615148.]

Automatically predicting the outcome of subjective listening tests is a challenging task. Ratings may vary from person to person even if preferences are consistent across listeners. While previous work has focused on predicting listeners' ratings (mean opinion scores) of individual stimuli, we focus on the simpler task of predicting subjective preference given two speech stimuli for the same text. We propose a model based on anti-symmetric twin neural networks, trained on pairs of waveforms and their corresponding preference scores. We explore both attention and recurrent neural nets to account for the fact that stimuli in a pair are not time aligned. To obtain a large training set we convert listeners' ratings from MUSHRA tests to values that reflect how often one stimulus in the pair was rated higher than the other. Specifically, we evaluate performance on data obtained from twelve MUSHRA evaluations conducted over five years, containing different TTS systems, built from data of different speakers. Our results compare favourably to a state-of-the-art model trained to predict MOS scores.

The health emergency has stress-tested the education systems. What has been learned from the experience? And how can that be applied to lessons while seeking to ensure that 'building back better' is an evidence-informed undertaking? In this report, the authors identify ten lessons from the crisis of 2020-2021 that should be used to inform planning for the reconstruction of education in the long term: (1) The crisis has reinforced the need for adaptive, agile policymaking; (2) Meeting the equity challenge depends on data, detail and deliberate action; (3) The best external support for teachers comes from other teachers; (4) School leadership matters; (5) Without effective assessment for learning, it is impossible to meet the needs of individual students; (6) Great teaching and learning are not enough: schools need to address the wellbeing of students and teachers; (7) Access to technology is necessary but not sufficient: many teachers urgently need training in digital pedagogy; (8) Technology solutions must be accessible -- and include a no-tech safety net; (9) Parental and community resources must be harnessed to support learning; and (10) Effective support for girls must be prioritised in plans for reopening and learning recovery. The authors arrived at these ten lessons through a thorough global review of a range of sources including policy documents and research findings and through a series of interviews with senior leaders in some of the organisations and programmes featured. [This report was written with Susy Ndaruhutse and Ruth Naylor.]

In Fall 2020, at the height of the pandemic, the World Innovation Summit for Education (WISE), the Education Commission, and Education Development Trust came together with the aim of generating new evidence on effective leadership during COVID-19. A learning partnership was developed with the Rwanda Basic Education Board (REB), which was already focused on improving the country's school leaders, to undertake rapid research on school and system leadership during COVID-19. The aim of the project was threefold: to shed light on the role that effective school and system leaders played during school closures and reopening in mitigating inequalities for the most vulnerable; to ascertain what essential leadership skills, competencies and behaviors leaders demonstrated to promote equity in delivery during and after the crisis; and to contribute to the emerging body of evidence of case studies in school leadership in the Global South. As the first in a series of case studies, Rwanda was chosen because of its commitment to improving school leadership and management, and its work embedding collaborative structures to implement change. The learning partnership approach consists of three stages: Co-designing, data gathering and reflection, and influencing decision-making. A critical element of the learning partnership approach was immediate feedback and discussion of findings as they emerged at each stage of the research (survey, interviews, case study schools) to influence policy and decision-making so they could be acted on in real time and create live impact. The findings suggest three overarching ways in which leaders were able to provide learning continuity and other support during closures and reopening, especially for the most marginalized. These are: (1) engagement with caregivers, communities and other sectors; (2) school leader and teacher collaboration; and (3) tracking, sharing information, and providing targeted support for vulnerable learners. From the findings, a set of skills, competencies, and behaviors can be inferred that could be useful for policies aiming to support leadership now and in the future. [This report was produced in partnership with the Education Commission, the Rwanda Basic Education Board (REB), and the World Innovation Summit for Education (WISE). For the accompanying report, "Rwanda Learning Partnership: Insights on School and System Leadership during COVID-19," see ED615112.]

This report is based on a rapid survey of recently published materials, guidance documents and media commentary. It summarises what is known and understood about the impacts of the prolonged school closures that followed the spread of COVID-19 and the context of school reopening and plans for learning recovery. In March 2020, schools around the world began closing their doors and governments, jurisdictions, educators and a range of connected stakeholders moved rapidly to support educational continuity through remote models of provision. Education Development Trust has been following these developments from the beginning, mapping the response, the challenges that have arisen and the concerns expressed by many educators across the globe. The first chapter of this report updates what is known about student and teacher wellbeing, the time spent on learning for pupils during the period of school closure and the experience of lockdown for teachers. In the second chapter, the status of school reopening planning around the world is considered, alongside the measures being taken to ensure that students and teachers can return to school safely. Emerging plans to tackle learning loss and to structure and support recovery are discussed. Finally, the report assesses the conditions for long-term educational recovery with a focus on funding and financial support for governments in low-income contexts, where fragile and unequal learning environments had already persisted prior to COVID-19 school closures. Throughout the report, the key findings are illustrated with country examples of both challenges and promising solutions. The report concludes with a list of recommendations for policymakers.