Search

Synthesizing realistic videos of talking faces under custom lighting conditions and viewing angles benefits various downstream applications like video conferencing. However, most existing relighting methods are either time-consuming or unable to adjust the viewpoints. In this paper, we present the first real-time 3D-aware method for relighting in-the-wild videos of talking faces based on Neural Radiance Fields (NeRF). Given an input portrait video, our method can synthesize talking faces under both novel views and novel lighting conditions with a photo-realistic and disentangled 3D representation. Specifically, we infer an albedo tri-plane, as well as a shading tri-plane based on a desired lighting condition for each video frame with fast dual-encoders. We also leverage a temporal consistency network to ensure smooth transitions and reduce flickering artifacts. Our method runs at 32.98 fps on consumer-level hardware and achieves state-of-the-art results in terms of reconstruction quality, lighting error, lighting instability, temporal consistency and inference speed. We demonstrate the effectiveness and interactivity of our method on various portrait videos with diverse lighting and viewing conditions., Comment: Accepted to CVPR 2024 (Highlight). Project page: http://geometrylearning.com/VideoRelighting

Let $\Omega\subset\mathbb{R}^n$, $n\ge 2$, be a bounded connected $C^2$ domain. For any unit vector $\nu\in\mathbb{R}^n$, let $T_{\lambda}^{\nu}=\{x\in\mathbb{R}^n:x\cdot\nu=\lambda\}$, $\Sigma_{\lambda}^{\nu}=\{x\in\Omega:x\cdot\nu<\lambda\}$ and $x^{\ast}=x-2(x\cdot\nu-\lambda)\nu$ be the reflection of a point $x\in\mathbb{R}^n$ about the plane $T_{\lambda}^{\nu}$. Let $\widetilde{\Sigma}_{\lambda}^{\nu}=\{x\in\Omega:x^{\ast}\in\Sigma_{\lambda}^{\nu}\}$ and $u\in C^2(\overline{\Omega})$. Suppose for any unit vector $\nu\in\mathbb{R}^n$, there exists a constant $\lambda_{\nu}\in\mathbb{R}$ such that $\Omega$ is symmetric about the plane $T_{\lambda_{\nu}}^{\nu}$ and $u$ is symmetric about the plane $T_{\lambda_{\nu}}^{\nu}$ and satisfies (i)$\,\frac{\partial u}{\partial\nu}(x)>0\quad\forall x\in \Sigma_{\lambda_{\nu}}^{\nu}$ and (ii)$\,\frac{\partial u}{\partial\nu}(x)<0\quad\forall x\in \widetilde{\Sigma}_{\lambda_{\nu}}^{\nu}$. We will give a simple proof that $u$ is radially symmetric about some point $x_0\in\Omega$ and $\Omega$ is a ball with center at $x_0$. Similar result holds for the domain $\mathbb{R}^n$ and function $u\in C^2(\mathbb{R}^n)$ satisfying similar monotonicity and symmetry conditions. We also extend this result under weaker hypothesis on the function $u$., Comment: 6 pages

In this work, under the hypothesis that quark matter may not be strange [Phys. Rev. Lett. 120, 222001 (2018)], we adopt a modification of the coupling constant of the four-quark scalar interaction $G\rightarrow G_1+G_2\langle\bar{\psi}\psi\rangle$ in the 2-flavor Nambu-Jona-Lasinio model to study nonstrange hybrid stars. According to lattice QCD simulation results of the critical temperature at zero chemical potential, $G_1$ and $G_2$ are constrained as $G_1\in(1.935, 1.972)$ GeV$^{-2}$, and $G_2\in(-1.582, -0.743)$ GeV$^{-5}$, respectively. To obtain hybrid equation of states, the Maxwell construction is used to describe the first-order confinement-deconfinement phase transition in hybrid stars. With recent measurements on neutron star mass, radius, and tidal deformability, the hybrid equation of states are constrained. The result suggests that pure nonstrange quark matter cores can exist in hybrid stars, possessing 0.014-0.026 solar mass with the bag constant $B^{1/4}$ in a range of 148-161 MeV. It is argued that the binary neutron stars in GW170817 should be hadron stars., Comment: 11 pages, 10 figures

We study the imaginary-time relaxation critical dynamics of the Neel-paramagnetic quantum phase transition in the two-dimensional (2D) dimerized S = 1/2 Heisenberg model. We focus on the scaling correction in the short-time region. A unified scaling form including both short-time and finite-size corrections is proposed. According to this full scaling form, improved short-imaginary-time scaling relations are obtained. We numerically verify the scaling form and the improved short-time scaling relations for different initial states using projector quantum Monte Carlo algorithm., Comment: 10 pages, 8 figures

We conduct an experimental study of high-mobility two-dimensional electron gas (2DEG) in GaAs/AlGaAs quantum wells modulated by strong magnetism at an in-plane magnetic field ($B$). The modulated $B$-fields are performed via the single stripe and gratings which are made of the heavy rare earth metal Terbium (Tb) thin films on the sample surface. The robust ferromagnetic resonances (FMRs) persist to the temperature of 50-70 K in both the stripe and grating samples, for the ferromagnetism (FM) phase of Tb exists at above 100 K. The high-order (with integer numbers of $j \equiv \hbar\omega/g\mu_{B}B = 1, 2,$...) magnetic resonances can also be observed in the stripe structure via the microwave (MW) photovoltaic detection and magnetoresistance under microwave irradiation. In addition, the resonance features around $j = 1/2$ are robust in the single-stripe modulated sample, which suggests the spinons with spin-1/2 collective excitations in a 1D Heisenberg model.

We investigate the possibility of detecting the leptophilic gauge boson $Z_x$ predicted by the $U(1)_{L_e-L_\mu}$ model via the processes $e^+e^-\rightarrow\ell^+\ell^-Z_x(Z_x\rightarrow \ell^+\ell^-)$ and $e^+e^-\rightarrow \ell^+\ell^-Z_x(Z_x\rightarrow \nu_\ell\bar{\nu_\ell})$ at the Circular Electron Positron Collider (CEPC) with the center of mass energy $\sqrt s=240$ GeV and the luminosity $\mathcal{L}=5.6$ $\mathrm{ab^{-1}}$. We give the expected sensitivities of the CEPC to the parameter space at $1\sigma$, $2\sigma$, $3\sigma$ and $5\sigma$ levels., Comment: 20 pages, 22 figures

The Belle II collaboration recently announced that they observed the $B^+ \!\to K^+\nu\bar{\nu}$ decay process for the first time. This dineutrino mode of $B^+ \!\to K^+\nu\bar{\nu}$ has been theoretically identified as a very clean channel. However, their result encounters a $2.7{}^{}\sigma$ deviation from the Standard Model (SM) calculation. On the other hand, last year, Fermilab released new data on muon $g-2$ away from the SM expectation with $5{}^{}\sigma$. In this letter, we study the simplest UV-complete $\text{U}(1)_{\textsf{L}_\mu - \textsf{L}_\tau}^{}$-charged complex scalar Dark Matter (DM) model. Thanks to the existence of light dark Higgs boson and light dark photon, we can explain the observed relic density of DM and resolve the results reported by both Belle II and Fermilab experiments simultaneously. As a byproduct, the Hubble tension is alleviated by taking $\Delta N_\textsf{eff}^{} \simeq 0.3$ induced by the light dark photon., Comment: 7 pages, 3 figures, updated figures, updated reference list, minor modification and conclusions unchanged

We study single-photon scattering spectra of a giant atom chirally coupled to a one-dimensional waveguide at multiple connection points, and examine chirality induced effects in the scattering spectra. We show that the transmission spectra typically possess an anti-Lorentzian lineshape with a nonzero minimum, but by engineering the chirality of the multi-point coupling, the transmission spectrum of an incident photon can undergo a transition from complete transmission to total reflection at multiple frequency ``windows'', where the width of the anti-Lorentzian lineshape for each of the window can be flexibly tuned at a fixed frequency detuning. Moreover, we show that a perfect nonreciprocal photon scattering can be achieved due to the interplay between internal atomic spontaneous emission and the chirally external decay to the waveguide, in contrast to that induced by the non-Markovian retardation effect. We also consider the non-Markovian retardation effect on the scattering spectra, which allows for a photonic band gap even with only two chiral coupling points. The giant-atom-waveguide system with chiral coupling is a promising candidate for realizing single-photon routers with multiple channels., Comment: 15 pages, 8 figures

Surgical tool segmentation and action recognition are fundamental building blocks in many computer-assisted intervention applications, ranging from surgical skills assessment to decision support systems. Nowadays, learning-based action recognition and segmentation approaches outperform classical methods, relying, however, on large, annotated datasets. Furthermore, action recognition and tool segmentation algorithms are often trained and make predictions in isolation from each other, without exploiting potential cross-task relationships. With the EndoVis 2022 SAR-RARP50 challenge, we release the first multimodal, publicly available, in-vivo, dataset for surgical action recognition and semantic instrumentation segmentation, containing 50 suturing video segments of Robotic Assisted Radical Prostatectomy (RARP). The aim of the challenge is twofold. First, to enable researchers to leverage the scale of the provided dataset and develop robust and highly accurate single-task action recognition and tool segmentation approaches in the surgical domain. Second, to further explore the potential of multitask-based learning approaches and determine their comparative advantage against their single-task counterparts. A total of 12 teams participated in the challenge, contributing 7 action recognition methods, 9 instrument segmentation techniques, and 4 multitask approaches that integrated both action recognition and instrument segmentation. The complete SAR-RARP50 dataset is available at: https://rdr.ucl.ac.uk/projects/SARRARP50_Segmentation_of_surgical_instrumentation_and_Action_Recognition_on_Robot-Assisted_Radical_Prostatectomy_Challenge/191091

Universal critical properties can manifest themselves not only in spatial but also in temporal directions. It has been found that critical point with emergent symmetry exhibits intriguing spatial critical properties characterized by two divergent length scales, attracting long-term investigations. However, how the temporal critical properties are affected by emergent symmetry is largely unknown. Here we study the nonequilibrium critical dynamics in the three-dimensional ($3$D) clock model, whose critical point has emergent $U(1)$ symmetry. We find that in contrast to the magnetization $M$, whose relaxation process is described by the usual dynamic exponent $z$ of the $3$D XY universality class, the angular order parameter $\phi_q$ shows a remarkable two-stage evolution characterized by different dynamic critical exponents. While in the short-time stage the relaxation dynamics is governed by $z$, in the long-time stage the dynamics is controlled by a new dynamic exponent $z'$. Further scaling analyses confirm that $z'$ is an indispensable dynamic critical exponent. Our results may be detected in the hexagonal RMnO$_3$ (R$=$rare earth) materials experimentally., Comment: 12 pages, 6 figures

We present a novel method for tuning the regularization hyper-parameter, $\lambda$, of a ridge regression that is faster to compute than leave-one-out cross-validation (LOOCV) while yielding estimates of the regression parameters of equal, or particularly in the setting of sparse covariates, superior quality to those obtained by minimising the LOOCV risk. The LOOCV risk can suffer from multiple and bad local minima for finite $n$ and thus requires the specification of a set of candidate $\lambda$, which can fail to provide good solutions. In contrast, we show that the proposed method is guaranteed to find a unique optimal solution for large enough $n$, under relatively mild conditions, without requiring the specification of any difficult to determine hyper-parameters. This is based on a Bayesian formulation of ridge regression that we prove to have a unimodal posterior for large enough $n$, allowing for both the optimal $\lambda$ and the regression coefficients to be jointly learned within an iterative expectation maximization (EM) procedure. Importantly, we show that by utilizing an appropriate preprocessing step, a single iteration of the main EM loop can be implemented in $O(\min(n, p))$ operations, for input data with $n$ rows and $p$ columns. In contrast, evaluating a single value of $\lambda$ using fast LOOCV costs $O(n \min(n, p))$ operations when using the same preprocessing. This advantage amounts to an asymptotic improvement of a factor of $l$ for $l$ candidate values for $\lambda$ (in the regime $q, p \in O(\sqrt{n})$ where $q$ is the number of regression targets).

Multimodal large language models (MLLMs) have shown great potential in perception and interpretation tasks, but their capabilities in predictive reasoning remain under-explored. To address this gap, we introduce a novel benchmark that assesses the predictive reasoning capabilities of MLLMs across diverse scenarios. Our benchmark targets three important domains: abstract pattern reasoning, human activity prediction, and physical interaction prediction. We further develop three evaluation methods powered by large language model to robustly quantify a model's performance in predicting and reasoning the future based on multi-visual context. Empirical experiments confirm the soundness of the proposed benchmark and evaluation methods via rigorous testing and reveal pros and cons of current popular MLLMs in the task of predictive reasoning. Lastly, our proposed benchmark provides a standardized evaluation framework for MLLMs and can facilitate the development of more advanced models that can reason and predict over complex long sequence of multimodal input.

Quantum criticality within Dirac fermions harbors a plethora of exotic phenomena, attracting sustained attention in the past decades. Nevertheless, the nonequilibrium dynamics therein has rarely been studied. To fill in the gap, we explore the imaginary-time relaxation dynamics in a typical Dirac quantum criticality belonging to chiral Heisenberg universality class. Performing large-scale quantum Monte Carlo simulation, we unveil rich nonequilibrium critical phenomena from different initial states. Particularly, a new dynamic exponent characterizing the non-stationary evolution in the short-time state is determined as $\theta=-0.84(4)$, in sharp contrast with the prevalent belief that $\theta$ is positive as demonstrated in classical cases. Furthermore, we propose a universal dynamic scaling theory governing the fruitful nonequilibrium properties in Dirac quantum criticality. Armed with the scaling theory, we develop a new framework to investigate fermionic quantum criticality based on short-time dynamics, paving a promising avenue to fathoming quantum criticality in diverse fermionic systems with high efficiency., Comment: 7+5 pages,5+4 figures

In this paper, we construct a viable model for a GeV scale self-interacting dark matter (DM), where the DM was thermally produced in the early universe. Here, a new vector-like fermion with a dark charge under the $U(1)_{D}$ gauge symmetry serves as a secluded WIMP DM and it can dominantly annihilate into the light dark gauge boson and singlet scalar through the dark gauge interaction. Also, the self-interaction of DM is induced by the light dark gauge boson via the same gauge interaction. In addition to these particles, we further introduce two Weyl fermions and a doublet scalar, by which the dark gauge boson produced from s-wave DM annihilations can mostly decay into active neutrinos after the dark symmetry breaking such that the CMB bound on the DM with low masses can be eluded. In order to have a common parameter region to explain the observed relic abundance and self-interaction of DM, we also study this model in a non-standard cosmological evolution, where the cosmic expansion driven by a new field species is faster than the standard radiation-dominated universe during the frozen time of DM. Reversely, one can also use the self-interacting nature of light thermal DM to examine the non-standard cosmological history of the universe., Comment: 18 pages, 5 figures, 1 table, matches the version accepted for publication in JCAP

Large language models (LLMs) have enabled remarkable advances in automated task-solving with multi-agent systems. However, most existing LLM-based multi-agent approaches rely on predefined agents to handle simple tasks, limiting the adaptability of multi-agent collaboration to different scenarios. Therefore, we introduce AutoAgents, an innovative framework that adaptively generates and coordinates multiple specialized agents to build an AI team according to different tasks. Specifically, AutoAgents couples the relationship between tasks and roles by dynamically generating multiple required agents based on task content and planning solutions for the current task based on the generated expert agents. Multiple specialized agents collaborate with each other to efficiently accomplish tasks. Concurrently, an observer role is incorporated into the framework to reflect on the designated plans and agents' responses and improve upon them. Our experiments on various benchmarks demonstrate that AutoAgents generates more coherent and accurate solutions than the existing multi-agent methods. This underscores the significance of assigning different roles to different tasks and of team cooperation, offering new perspectives for tackling complex tasks. The repository of this project is available at https://github.com/Link-AGI/AutoAgents., Comment: IJCAI 2024

Multi-modal large language models have garnered significant interest recently. Though, most of the works focus on vision-language multi-modal models providing strong capabilities in following vision-and-language instructions. However, we claim that speech is also an important modality through which humans interact with the world. Hence, it is crucial for a general-purpose assistant to be able to follow multi-modal speech-and-language instructions. In this work, we propose Large Language and Speech Model (LLaSM). LLaSM is an end-to-end trained large multi-modal speech-language model with cross-modal conversational abilities, capable of following speech-and-language instructions. Our early experiments show that LLaSM demonstrates a more convenient and natural way for humans to interact with artificial intelligence. Specifically, we also release a large Speech Instruction Following dataset LLaSM-Audio-Instructions. Code and demo are available at https://github.com/LinkSoul-AI/LLaSM and https://huggingface.co/spaces/LinkSoul/LLaSM. The LLaSM-Audio-Instructions dataset is available at https://huggingface.co/datasets/LinkSoul/LLaSM-Audio-Instructions.

Given a graph $G$ on $n$ vertices, its adjacency matrix and degree diagonal matrix are represented by $A(G)$ and $D(G)$, respectively. The $Q$-spectrum of $G$ consists of all the eigenvalues of its signless Laplacian matrix $Q(G)=A(G)+D(G)$ (including the multiplicities). A graph $G$ is known as being determined by its generalized $Q$-spectrum ($DGQS$ for short) if, for any graph $H$, $H$ and $G$ have the same $Q$-spectrum and so do their complements, then $H$ is isomorphic to $G$. In this paper, we present a method to construct $DGQS$ graphs. More specifically, let the matrix $W_{Q}(G)=\left [e,Qe,\dots ,Q^{n-1}e \right ]$ ($e$ denotes the all-one column vector ) be the $Q$-walk matrix of $G$. It is shown that $G\circ P_{k}$ ($k=2,3$) is $DGQS$ if and only if $G$ is $DGQS$ for some specific graphs. This also provides a way to construct $DGQS$ graphs with more vertices by using $DGQS$ graphs with fewer vertices. At the same time, we also prove that $G\circ P_{2}$ is still $DGQS$ under specific circumstances. In particular, on the basis of the above results, we obtain an infinite sequences of $DGQS$ graphs $G\circ P_{k}^{t}$ ($k=2,3;t\ge 1$) for some specific $DGQS$ graph $G$., Comment: 15 pages, 1 figures

Deconfined quantum criticality (DQC) arises from fractionalization of quasi-particles and leads to fascinating behaviors beyond the Landau-Ginzburg-Wilson (LGW) description of phase transitions. The unusual aspects of DQC in equilibrium also suggest exotic phenomena out of equilibrium, which are still largely unexplored. Here we study manifestations of DQC when driving (quantum annealing) a two-dimensional quantum magnet through a critical point separating antiferromagnetic and spontaneously dimerized ground states. Numerical simulations show that the celebrated Kibble-Zurek scaling (KZS) mechanism is inadequate for describing the annealing process. To explain our results, we introduce the concept of dual asymmetric KZS, where a deconfinement time enters in addition to the conventional relaxation time and the scaling also depends on the direction in which the system is driven through the critical point according to a duality principle connecting the topological defects in the two phases. These defects -- spinons in the dimerized phase and space-time hedgehogs in the antiferromagnetic phase -- require a much longer time scale for equilibration than the amplitude of the order parameter. Beyond the new insights into DQC, our scaling approach provides a new window into out-of-equilibrium criticality with multiple length and time scales., Comment: 15 pages, 9 figures

Instruction tuning is widely recognized as a key technique for building generalist language models, which has attracted the attention of researchers and the public with the release of InstructGPT~\citep{ouyang2022training} and ChatGPT\footnote{\url{https://chat.openai.com/}}. Despite impressive progress in English-oriented large-scale language models (LLMs), it is still under-explored whether English-based foundation LLMs can perform similarly on multilingual tasks compared to English tasks with well-designed instruction tuning and how we can construct the corpora needed for the tuning. To remedy this gap, we propose the project as an attempt to create a Chinese instruction dataset by various methods adapted to the intrinsic characteristics of 4 sub-tasks. We collect around 200k Chinese instruction tuning samples, which have been manually checked to guarantee high quality. We also summarize the existing English and Chinese instruction corpora and briefly describe some potential applications of the newly constructed Chinese instruction corpora. The resulting \textbf{C}hinese \textbf{O}pen \textbf{I}nstruction \textbf{G}eneralist (\textbf{COIG}) corpora are available in Huggingface\footnote{\url{https://huggingface.co/datasets/BAAI/COIG}} and Github\footnote{\url{https://github.com/BAAI-Zlab/COIG}}, and will be continuously updated.