Search

Automated speaking assessment in conversation tests (ASAC) aims to evaluate the overall speaking proficiency of an L2 (second-language) speaker in a setting where an interlocutor interacts with one or more candidates. Although prior ASAC approaches have shown promising performance on their respective datasets, there is still a dearth of research specifically focused on incorporating the coherence of the logical flow within a conversation into the grading model. To address this critical challenge, we propose a hierarchical graph model that aptly incorporates both broad inter-response interactions (e.g., discourse relations) and nuanced semantic information (e.g., semantic words and speaker intents), which is subsequently fused with contextual information for the final prediction. Extensive experimental results on the NICT-JLE benchmark dataset suggest that our proposed modeling approach can yield considerable improvements in prediction accuracy with respect to various assessment metrics, as compared to some strong baselines. This also sheds light on the importance of investigating coherence-related facets of spoken responses in ASAC., Comment: Accepted by IEEE SLT 2024

End-to-end (E2E) automatic speech recognition (ASR) models have become standard practice for various commercial applications. However, in real-world scenarios, the long-tailed nature of word distribution often leads E2E ASR models to perform well on common words but fall short in recognizing uncommon ones. Recently, the notion of a contextual adapter (CA) was proposed to infuse external knowledge represented by a context word list into E2E ASR models. Although CA can improve recognition performance on rare words, two crucial data imbalance problems remain. First, when using low-frequency words as context words during training, since these words rarely occur in the utterance, CA becomes prone to overfit on attending to the token due to higher-frequency words not being present in the context list. Second, the long-tailed distribution within the context list itself still causes the model to perform poorly on low-frequency context words. In light of this, we explore in-depth the impact of altering the context list to have words with different frequency distributions on model performance, and meanwhile extend CA with a simple yet effective context-balanced learning objective. A series of experiments conducted on the AISHELL-1 benchmark dataset suggests that using all vocabulary words from the training corpus as the context list and pairing them with our balanced objective yields the best performance, demonstrating a significant reduction in character error rate (CER) by up to 1.21% and a more pronounced 9.44% reduction in the error rate of zero-shot words., Comment: Accepted by SLT 2024

Automatic pronunciation assessment (APA) manages to evaluate the pronunciation proficiency of a second language (L2) learner in a target language. Existing efforts typically draw on regression models for proficiency score prediction, where the models are trained to estimate target values without explicitly accounting for phoneme-awareness in the feature space. In this paper, we propose a contrastive phonemic ordinal regularizer (ConPCO) tailored for regression-based APA models to generate more phoneme-discriminative features while considering the ordinal relationships among the regression targets. The proposed ConPCO first aligns the phoneme representations of an APA model and textual embeddings of phonetic transcriptions via contrastive learning. Afterward, the phoneme characteristics are retained by regulating the distances between inter- and intra-phoneme categories in the feature space while allowing for the ordinal relationships among the output targets. We further design and develop a hierarchical APA model to evaluate the effectiveness of our method. Extensive experiments conducted on the speechocean762 benchmark dataset suggest the feasibility and efficacy of our approach in relation to some cutting-edge baselines., Comment: This paper has been withdrawn because the authors aim to achieve better organization in writing and more detailed experimental analysis

End-to-end automatic speech recognition (E2E ASR) systems often suffer from mistranscription of domain-specific phrases, such as named entities, sometimes leading to catastrophic failures in downstream tasks. A family of fast and lightweight named entity correction (NEC) models for ASR have recently been proposed, which normally build on phonetic-level edit distance algorithms and have shown impressive NEC performance. However, as the named entity (NE) list grows, the problems of phonetic confusion in the NE list are exacerbated; for example, homophone ambiguities increase substantially. In view of this, we proposed a novel Description Augmented Named entity CorrEctoR (dubbed DANCER), which leverages entity descriptions to provide additional information to facilitate mitigation of phonetic confusion for NEC on ASR transcription. To this end, an efficient entity description augmented masked language model (EDA-MLM) comprised of a dense retrieval model is introduced, enabling MLM to adapt swiftly to domain-specific entities for the NEC task. A series of experiments conducted on the AISHELL-1 and Homophone datasets confirm the effectiveness of our modeling approach. DANCER outperforms a strong baseline, the phonetic edit-distance-based NEC model (PED-NEC), by a character error rate (CER) reduction of about 7% relatively on AISHELL-1 for named entities. More notably, when tested on Homophone that contain named entities of high phonetic confusion, DANCER offers a more pronounced CER reduction of 46% relatively over PED-NEC for named entities., Comment: Accepted by LREC-COLING 2024

With the massive developments of end-to-end (E2E) neural networks, recent years have witnessed unprecedented breakthroughs in automatic speech recognition (ASR). However, the codeswitching phenomenon remains a major obstacle that hinders ASR from perfection, as the lack of labeled data and the variations between languages often lead to degradation of ASR performance. In this paper, we focus exclusively on improving the acoustic encoder of E2E ASR to tackle the challenge caused by the codeswitching phenomenon. Our main contributions are threefold: First, we introduce a novel disentanglement loss to enable the lower-layer of the encoder to capture inter-lingual acoustic information while mitigating linguistic confusion at the higher-layer of the encoder. Second, through comprehensive experiments, we verify that our proposed method outperforms the prior-art methods using pretrained dual-encoders, meanwhile having access only to the codeswitching corpus and consuming half of the parameterization. Third, the apparent differentiation of the encoders' output features also corroborates the complementarity between the disentanglement loss and the mixture-of-experts (MoE) architecture., Comment: ICASSP 2024

The non-Hermitian one-photon and two-photon quantum Rabi models (QRMs) with imaginary couplings are respectively solved through the Bogoliubov operators approach. Transcendental functions responsible for exact solutions are derived, whose zeros produce the complete spectra. Exceptional points (EPs) can be identified in terms of the transcendental function. The EP is formed in the two nearest-neighboring excited energy levels, and shifts towards lower coupling strength at higher energy levels. Interestingly, under the resonant condition in the non-Hermitian two-photon QRM, the lowest two excited states within the same parity in the even photonic number subspace coalesce at an EP, but take always the purely real energy, in sharp contrast to the conventional EP in the non-Hermitian systems. For both non-Hermitian QRMs, the fidelity susceptibility goes to negative infinity at the EPs, consistent with the recent observations in non-Hermitian systems. All eigenstates can be labeled by the conserved energy and the QRM parity, we argue that the non-Hermitian QRMs are also integrable, similar to their Hermitian counterparts., Comment: 8 pages, 9 figures

In the study of optical properties of large atomic system, a weak laser driving is often assumed to simplify the system dynamics by linearly coupled equations. Here, we investigate the light scattering properties of atomic ensembles beyond weak-field excitation through the cumulant expansion method. By progressively incorporating higher-order correlations into the steady-state equations, an enhanced accuracy can be achieved in comparison to the exact solutions from solving a full density matrix. Our analysis reveals that, in the regime of weak dipole-dipole interaction (DDI), the first-order expansion yields satisfactory predictions for optical depth, while denser atomic configurations necessitate consideration of higher-order correlations. As the intensity of incident light increases, atom saturation effects become noticeable, giving rise to significant changes in light transparency, energy shift, and decay rate. This saturation phenomenon extends to subradiant atom arrays even under weak driving conditions, leading to substantial deviations from the linear model. Our findings demonstrate the mean-field models as good extensions to linear models as it balances both accuracy and computational complexity. However, the crucial role of higher-order cumulants in large and dense atom systems remains unclear, since it is challenging theoretically owing to the exponentially increasing Hilbert space in such light-matter interacting systems., Comment: 5 figures

Automatic pronunciation assessment (APA) manages to quantify the pronunciation proficiency of a second language (L2) learner in a language. Prevailing approaches to APA normally leverage neural models trained with a regression loss function, such as the mean-squared error (MSE) loss, for proficiency level prediction. Despite most regression models can effectively capture the ordinality of proficiency levels in the feature space, they are confronted with a primary obstacle that different phoneme categories with the same proficiency level are inevitably forced to be close to each other, retaining less phoneme-discriminative information. On account of this, we devise a phonemic contrast ordinal (PCO) loss for training regression-based APA models, which aims to preserve better phonemic distinctions between phoneme categories meanwhile considering ordinal relationships of the regression target output. Specifically, we introduce a phoneme-distinct regularizer into the MSE loss, which encourages feature representations of different phoneme categories to be far apart while simultaneously pulling closer the representations belonging to the same phoneme category by means of weighted distances. An extensive set of experiments carried out on the speechocean762 benchmark dataset suggest the feasibility and effectiveness of our model in relation to some existing state-of-the-art models., Comment: Accepted by ASRU 2023

Voice, as input, has progressively become popular on mobiles and seems to transcend almost entirely text input. Through voice, the voice search (VS) system can provide a more natural way to meet user's information needs. However, errors from the automatic speech recognition (ASR) system can be catastrophic to the VS system. Building on the recent advanced lightweight autoregressive retrieval model, which has the potential to be deployed on mobiles, leading to a more secure and personal VS assistant. This paper presents a novel study of VS leveraging autoregressive retrieval and tackles the crucial problems facing VS, viz. the performance drop caused by ASR noise, via data augmentations and contrastive learning, showing how explicit and implicit modeling the noise patterns can alleviate the problems. A series of experiments conducted on the Open-Domain Question Answering (ODSQA) confirm our approach's effectiveness and robustness in relation to some strong baseline systems.

Recent years have witnessed a surge of research on the non-Hermitian skin effect (NHSE) in one-dimensional lattices with finite-range couplings. In this work, we show that the long-range couplings that decay as $1/l^{\alpha}$ at distance $l$ can fundamentally modify the behavior of NHSE and the scaling of quantum entanglement in the presence of nonreciprocity. At $\alpha=0$, the nonlocality of couplings gives rise to the scale-free skin modes, whose localization length is proportional to the system size. Increasing the exponent $\alpha$ drives a complex-to-real spectral transition and a crossover from a scale-free to constant localization length. Furthermore, the scaling of nonequilibrium steady-state entanglement entropy exhibits a subextensive law due to the nonlocality and the complex spectrum, in contrast to an area law arising from NHSE. Our results provide a theoretical understanding on the interplay between long-range couplings and non-Hermiticity., Comment: 6 pages, 4 figures

Localization in one-dimensional interacting systems can be caused by disorder potentials or non-Hermiticity. The former phenomenon is the many-body localization (MBL), and the latter is the many-body non-Hermitian skin effect (NHSE). In this work, we numerically investigate the interplay between these two kinds of localization, where the energy-resolved MBL arises from a deterministic quasiperiodic potential in a fermionic chain. We propose a set of eigenstate properties and long-time dynamics that can collectively distinguish the two localization mechanisms in the presence of non-Hermiticity. By computing the proposed diagnostics, we show that the thermal states are vulnerable to the many-body NHSE while the MBL states remain resilient up to a strong non-Hermiticity. Finally, we discuss experimental observables that can probe the difference between the two localizations in a non-Hermitian quasiperiodic fermionic chain. Our results pave the way toward experimental observations on the interplay of interaction, quasiperiodic potential, and non-Hermiticity., Comment: 7 pages, 4 figures

Resolved sideband cooling provides a crucial step in subrecoil cooling the trapped atoms toward their motional ground state, which is essential in atom-based quantum technologies. Here we present an enhanced dark-state sideband cooling in trapped atoms utilizing photon-mediated dipole-dipole interactions among them. By placing the atoms at the magic interparticle distances, we manifest an outperformed cooling behavior in the target atom, which surpasses the limit that a single atom permits. We further investigate various atomic configurations in a multiatom setup with a laser detuning and different light polarization angles, where multiple magic spacings can be identified and a moderate improvement in cooling performance is predicted as the number of atoms increases. Our results provide insights to subrecoil cooling of atoms with collective and light-induced long-range dipole-dipole interactions, and pave the way toward implementing genuine quantum operations in multiple quantum registers., Comment: 5 figures

Cooling the trapped atoms toward their motional ground states is key to applications of quantum simulation and quantum computation. By utilizing nonreciprocal couplings between constituent atoms, we present an intriguing dark-state cooling scheme in $\Lambda$-type three-level structure, which is shown superior than the conventional electromagnetically-induced-transparency cooling in a single atom. The effective nonreciprocal couplings can be facilitated either by an atom-waveguide interface or a free-space photonic quantum link. By tailoring system parameters allowed in dark-state cooling, we identify the parameter regions of better cooling performance with an enhanced cooling rate. We further demonstrate a mapping to the dark-state sideband cooling under asymmetric laser driving fields, which shows a distinct heat transfer and promises an outperforming dark-state sideband cooling assisted by collective spin-exchange interactions., Comment: 4 figures

The tapped ions can be cooled close to their motional ground state, which is imperative in implementing quantum computation and quantum simulation. Here we demonstrate the capability of light-mediated chiral couplings between ions, which enables a superior cooling scheme exceeding the single-ion limit of sideband cooling. We present the chiral-coupling-assisted refrigeration in the target ion at the price of heating the others under asymmetric drivings, where its steady-state phonon occupation outperforms the lower bound set by a single ion. We further locate the optimal operation condition of the refrigeration and identify the parameter region where a faster rate of cooling emerges. Under an additional nonguided decay channel, the heating effect in the reciprocal coupling regime becomes suppressed and turns into cooling instead. Our results present a resource of collective chiral couplings which help surpass the bottleneck of cooling procedure in applications of trapped-ion-based quantum computer and simulator., Comment: 7 figures in the main text

The explorations of non-Hermiticity have been devoted to investigate the disorder-induced many-body localization (MBL). However, the sensitivity of the spatial boundary conditions and the interplay of the non-Hermitian skin effect with many-body phenomena are not yet clear. For a MBL system in the presence of non-reciprocal tunnelings and random disorder potential, we identify two different complex-real spectral transitions, one is present for both open and periodic boundaries while the other is present only for open boundaries of a coupled non-Hermitian chains. The later is driven due to the inter-chain coupling at weak disorder where the level statistics of the real eigenenergy phase follows Gaussian orthogonal ensemble. We further characterize wavefunctions through the (biorthogonal) inverse participation ratio and fractal dimension, which reveal the suppression of skin effect in the non-Hermitian MBL phase. Finally, we demonstrate that the quench dynamics of the local particle density, spin imbalance, and entanglement entropy also signify the hallmark of the boundary effects and non-ergodic character of many-body localization., Comment: 11 pages, 10 figures; version accepted for publication in Phys. Rev. B

Explorations of symmetry and topology have led to important breakthroughs in quantum optics, but much richer behaviors arise from the non-Hermitian nature of light-matter interactions. A high-reflectivity, non-Hermitian optical mirror can be realized by a two-dimensional subwavelength array of neutral atoms near the cooperative resonance associated with the collective dipole modes. Here we show that exceptional points develop from a nondefective degeneracy by lowering the crystal symmetry of a square atomic lattice, and dispersive bulk Fermi arcs that originate from exceptional points are truncated by the light cone. We also find, although the dipole-dipole interaction is reciprocal, the geometry-dependent non-Hermitian skin effect emerges. Furthermore, skin modes localized at a boundary show a scale-free behavior that stems from the long-range interaction and whose mechanism goes beyond the framework of non-Bloch band theory. Our work opens the door to the study of the interplay among non-Hermiticity, topology, and long-range interaction., Comment: 8 + 20 pages, 4 + 7 figures