Search

Hallucinations pose a significant challenge to the reliability of large language models (LLMs) in critical domains. Recent benchmarks designed to assess LLM hallucinations within conventional NLP tasks, such as knowledge-intensive question answering (QA) and summarization, are insufficient for capturing the complexities of user-LLM interactions in dynamic, real-world settings. To address this gap, we introduce HaluEval-Wild, the first benchmark specifically designed to evaluate LLM hallucinations in the wild. We meticulously collect challenging (adversarially filtered by Alpaca) user queries from ShareGPT, an existing real-world user-LLM interaction datasets, to evaluate the hallucination rates of various LLMs. Upon analyzing the collected queries, we categorize them into five distinct types, which enables a fine-grained analysis of the types of hallucinations LLMs exhibit, and synthesize the reference answers with the powerful GPT-4 model and retrieval-augmented generation (RAG). Our benchmark offers a novel approach towards enhancing our comprehension of and improving LLM reliability in scenarios reflective of real-world interactions. Our benchmark is available at https://github.com/HaluEval-Wild/HaluEval-Wild.

Helping end users comprehend the abstract distribution shifts can greatly facilitate AI deployment. Motivated by this, we propose a novel task, dataset explanation. Given two image data sets, dataset explanation aims to automatically point out their dataset-level distribution shifts with natural language. Current techniques for monitoring distribution shifts provide inadequate information to understand datasets with the goal of improving data quality. Therefore, we introduce GSCLIP, a training-free framework to solve the dataset explanation task. In GSCLIP, we propose the selector as the first quantitative evaluation method to identify explanations that are proper to summarize dataset shifts. Furthermore, we leverage this selector to demonstrate the superiority of a generator based on language model generation. Systematic evaluation on natural data shift verifies that GSCLIP, a combined system of a hybrid generator group and an efficient selector is not only easy-to-use but also powerful for dataset explanation at scale., Comment: Accepted by ICML 2022 DataPerf

We are interested in a novel task, singing voice beautifying (SVB). Given the singing voice of an amateur singer, SVB aims to improve the intonation and vocal tone of the voice, while keeping the content and vocal timbre. Current automatic pitch correction techniques are immature, and most of them are restricted to intonation but ignore the overall aesthetic quality. Hence, we introduce Neural Singing Voice Beautifier (NSVB), the first generative model to solve the SVB task, which adopts a conditional variational autoencoder as the backbone and learns the latent representations of vocal tone. In NSVB, we propose a novel time-warping approach for pitch correction: Shape-Aware Dynamic Time Warping (SADTW), which ameliorates the robustness of existing time-warping approaches, to synchronize the amateur recording with the template pitch curve. Furthermore, we propose a latent-mapping algorithm in the latent space to convert the amateur vocal tone to the professional one. To achieve this, we also propose a new dataset containing parallel singing recordings of both amateur and professional versions. Extensive experiments on both Chinese and English songs demonstrate the effectiveness of our methods in terms of both objective and subjective metrics. Audio samples are available at~\url{https://neuralsvb.github.io}. Codes: \url{https://github.com/MoonInTheRiver/NeuralSVB}., Comment: Accepted by ACL 2022 Main conference; Code: https://github.com/MoonInTheRiver/NeuralSVB

As a key component of talking face generation, lip movements generation determines the naturalness and coherence of the generated talking face video. Prior literature mainly focuses on speech-to-lip generation while there is a paucity in text-to-lip (T2L) generation. T2L is a challenging task and existing end-to-end works depend on the attention mechanism and autoregressive (AR) decoding manner. However, the AR decoding manner generates current lip frame conditioned on frames generated previously, which inherently hinders the inference speed, and also has a detrimental effect on the quality of generated lip frames due to error propagation. This encourages the research of parallel T2L generation. In this work, we propose a parallel decoding model for fast and high-fidelity text-to-lip generation (ParaLip). Specifically, we predict the duration of the encoded linguistic features and model the target lip frames conditioned on the encoded linguistic features with their duration in a non-autoregressive manner. Furthermore, we incorporate the structural similarity index loss and adversarial learning to improve perceptual quality of generated lip frames and alleviate the blurry prediction problem. Extensive experiments conducted on GRID and TCD-TIMIT datasets demonstrate the superiority of proposed methods. Video samples are available via \url{https://paralip.github.io/}., Comment: Author draft

We find a new instability in the four-dimensional Reissner-Nordstrom-de Sitter black holes against charged scalar perturbations with vanishing angular momentum, $l=0$. We show that such an instability is caused by superradiance. The instability does not occur for an larger angular index, as explicitely proven for $l=1$. Our results are obtained from a numerical investigation of the time domain profiles of the perturbations., Comment: 15pages, 13figures, final version for journal publication

We demonstrate that the thermal Casimir-Polder forces on molecules near a conducting surface whose transition wavelengths are comparable to the molecule-surface separation are dependent on the ambient temperature and molecular polarization and they can even be changed from attractive to repulsive via varying the temperature across a threshold value for anisotropically polarizable molecules. Remarkably, this attractive-to-repulsive transition may be realized at room temperature. Let us note that the predicted repulsion is essentially a nonequilibrium effect since the force we calculated on a ground-state (or an excited-stated) molecule actually contains the contribution of the absorption (or emission) of thermal photons., Comment: 10 pages, 2 figures

We consider a uniformly accelerated atom interacting with a vacuum electromagnetic field in the presence of an infinite conducting plane boundary and calculate separately the contributions of vacuum fluctuations and radiation reaction to the atomic energy level shift. We analyze in detail the behavior of the total energy shift in three different regimes of the distance in both the low acceleration and high acceleration limits. Our results show that, in general, an accelerated atom does not behave as if immersed in a thermal bath at the Unruh temperature in terms of the atomic energy level shifts, and the effect of the acceleration on the atomic energy level shifts may in principle become appreciable in certain circumstances, although it may not be realistic for actual experimental measurements. We also examine the effects of the acceleration on the level shifts when the acceleration is of the order of the transition frequency of the atom and we find some features differ from what was obtained in the existing literature., Comment: 26 pages, 6 figures, version published in PRA

We study the modification by the presence of a plane wall of energy level shifts of two-level atoms which are in multipolar coupling with quantized electromagnetic fields in a thermal bath in a formalism which separates the contributions of thermal fluctuations and radiation reaction and allows a distinct treatment to atoms in the ground and excited states. The position dependent energy shifts give rise to an induced force acting on the atoms. We are able to identify three different regimes where the force shows distinct features and examine, in all regimes, the behaviors of this force in both the low temperature limit and the high temperature limit for both the ground state and excited state atoms, thus providing some physical insights into the atom-wall interaction at finite temperature. In particular, we show that both the magnitude and the direction of the force acting on an atom may have a clear dependence on atomic the polarization directions. In certain cases, a change of relative ratio of polarizations in different directions may result in a change of direction of the force., Comment: 29 pages, 3 figures

We consider, in de Sitter spacetime, both freely falling and static two-level atoms in interaction with a conformally coupled massless scalar field in the de Sitter-invariant vacuum, and separately calculate the contributions of vacuum fluctuations and radiation reaction to the atom's spontaneous excitation rate. We find that spontaneous excitations occur even for the freely falling atom as if there is a thermal bath of radiation at the Gibbons-Hawking temperature and we thus recover, in a different physical context, the results of Gibbons and Hawking that reveals the thermal nature of de Sitter spacetime. Similarly, for the case of the static atom, our results show that the atom also perceives a thermal bath which now arises as a result of the intrinsic thermal nature of de Sitter spacetime and the Unruh effect associated with the inherent acceleration of the atom., Comment: 11 pages

We investigate, from the point of view of a coaccelerated frame, the spontaneous excitation of a uniformly accelerated two-level atom interacting with a scalar field in a thermal state at a finite temperature $T$ and show that the same spontaneous excitation rate for the uniformly accelerated atom in the Minkowski vacuum obtained in the inertial frame can only be recovered in the coaccelerated frame assuming a thermal bath at the Fulling-Davies-Unruh temperature $T_{FDU}=a/2\pi$ for what appears to be the Minkowski vacuum to the inertial observer. Our discussion provides another example of a physical process different from those examined before in the literature to better understand the Fulling-Davies-Unruh effect., Comment: 12 pages, no figures, accepted for publication in PLB

We study, in the multipolar coupling scheme, a uniformly accelerated multilevel hydrogen atom in interaction with the quantum electromagnetic field near a conducting boundary and separately calculate the contributions of the vacuum fluctuation and radiation reaction to the rate of change of the mean atomic energy. It is found that the perfect balance between the contributions of vacuum fluctuations and radiation reaction that ensures the stability of ground-state atoms is disturbed, making spontaneous transition of ground-state atoms to excited states possible in vacuum with a conducting boundary. The boundary-induced contribution is effectively a nonthermal correction, which enhances or weakens the nonthermal effect already present in the unbounded case, thus possibly making the effect easier to observe. An interesting feature worth being noted is that the nonthermal corrections may vanish for atoms on some particular trajectories., Comment: 19 pages, no figures, Revtex4

We consider a multilevel hydrogen atom in interaction with the quantum electromagnetic field and separately calculate the contributions of the vacuum fluctuation and radiation reaction to the rate of change of the mean atomic energy of the atom for uniform acceleration. It is found that the acceleration disturbs the vacuum fluctuations in such a way that the delicate balance between the contributions of vacuum fluctuation and radiation reaction that exists for inertial atoms is broken, so that the transitions to higher-lying states from ground state are possible even in vacuum. In contrast to the case of an atom interacting with a scalar field, the contributions of both electromagnetic vacuum fluctuations and radiation reaction to the spontaneous emission rate are affected by the acceleration, and furthermore the contribution of the vacuum fluctuations contains a non-thermal acceleration-dependent correction, which is possibly observable., Comment: 8 pages, Revtex4, accepted for publication in PRD