Search

We present a novel approach for synthesizing 3D facial motions from audio sequences using key motion embeddings. Despite recent advancements in data-driven techniques, accurately mapping between audio signals and 3D facial meshes remains challenging. Direct regression of the entire sequence often leads to over-smoothed results due to the ill-posed nature of the problem. To this end, we propose a progressive learning mechanism that generates 3D facial animations by introducing key motion capture to decrease cross-modal mapping uncertainty and learning complexity. Concretely, our method integrates linguistic and data-driven priors through two modules: the linguistic-based key motion acquisition and the cross-modal motion completion. The former identifies key motions and learns the associated 3D facial expressions, ensuring accurate lip-speech synchronization. The latter extends key motions into a full sequence of 3D talking faces guided by audio features, improving temporal coherence and audio-visual consistency. Extensive experimental comparisons against existing state-of-the-art methods demonstrate the superiority of our approach in generating more vivid and consistent talking face animations. Consistent enhancements in results through the integration of our proposed learning scheme with existing methods underscore the efficacy of our approach. Our code and weights will be at the project website: \url{https://github.com/ffxzh/KMTalk}., Comment: Accepted by ECCV 2024

This article compares two style transfer methods in image processing: the traditional method, which synthesizes new images by stitching together small patches from existing images, and a modern machine learning-based approach that uses a segmentation network to isolate foreground objects and apply style transfer solely to the background. The traditional method excels in creating artistic abstractions but can struggle with seamlessness, whereas the machine learning method preserves the integrity of foreground elements while enhancing the background, offering improved aesthetic quality and computational efficiency. Our study indicates that machine learning-based methods are more suited for real-world applications where detail preservation in foreground elements is essential.

Field Programmable Gate Array (FPGA) logic synthesis compilers (e.g., Vivado, Iverilog, Yosys, and Quartus) are widely applied in Electronic Design Automation (EDA), such as the development of FPGA programs.However, defects (i.e., incorrect synthesis) in logic synthesis compilers may lead to unexpected behaviors in target applications, posing security risks. Therefore, it is crucial to thoroughly test logic synthesis compilers to eliminate such defects.Despite several Hardware Design Language (HDL) code generators (e.g., Verismith) have been proposed to find defects in logic synthesis compilers, the effectiveness of these generators is still limited by the simple code generation strategy and the monogeneity of the generated HDL code.This paper proposes LegoHDL, a novel method to generate syntax valid HDL code for comprehensively testing FPGA logic synthesis compilers.LegoHDL can generate more complex and diverse defect-trigger HDL code (e.g., Verilog, VHDL, and SystemVerilog) by leveraging the guidance of abstract syntax tree and the extensive function block libraries of cyber-physical systems. Extensive experiments show that the diversity and defect-trigger capability of HDL code generated by LegoHDL are significantly better than the state-of-the-art method (i.e., Verismith).In three months, LegoHDL has reported 20 new defects--many of which are deep and important; 16 of them have been confirmed.

Quantum algorithms are emerging tools in the design of functional materials due to their powerful solution space search capability. How to balance the high price of quantum computing resources and the growing computing needs has become an urgent problem to be solved. We propose a novel optimization strategy based on an active learning scheme that combines the improved Quantum Genetic Algorithm (QGA) with machine learning surrogate model regression. Using Random Forests as the surrogate model circumvents the time-consuming physical modeling or experiments, thereby improving the optimization efficiency. QGA, a genetic algorithm embedded with quantum mechanics, combines the advantages of quantum computing and genetic algorithms, enabling faster and more robust convergence to the optimum. Using the design of planar multilayer photonic structures for transparent radiative cooling as a testbed, we show superiority of our algorithm over the classical genetic algorithm (CGA). Additionally, we show the precision advantage of the RF model as a flexible surrogate model, which relaxes the constraints on the type of surrogate model that can be used in other quantum computing optimization algorithms (e.g., quantum annealing needs Ising model as a surrogate)., Comment: 19 pages, 8 figures

Despite the surprisingly high intelligence exhibited by Large Language Models (LLMs), we are somehow intimidated to fully deploy them into real-life applications considering their black-box nature. Concept-based explanations arise as a promising avenue for explaining what the LLMs have learned, making them more transparent to humans. However, current evaluations for concepts tend to be heuristic and non-deterministic, e.g. case study or human evaluation, hindering the development of the field. To bridge the gap, we approach concept-based explanation evaluation via faithfulness and readability. We first introduce a formal definition of concept generalizable to diverse concept-based explanations. Based on this, we quantify faithfulness via the difference in the output upon perturbation. We then provide an automatic measure for readability, by measuring the coherence of patterns that maximally activate a concept. This measure serves as a cost-effective and reliable substitute for human evaluation. Finally, based on measurement theory, we describe a meta-evaluation method for evaluating the above measures via reliability and validity, which can be generalized to other tasks as well. Extensive experimental analysis has been conducted to validate and inform the selection of concept evaluation measures.

Despite careful safety alignment, current large language models (LLMs) remain vulnerable to various attacks. To further unveil the safety risks of LLMs, we introduce a Safety Concept Activation Vector (SCAV) framework, which effectively guides the attacks by accurately interpreting LLMs' safety mechanisms. We then develop an SCAV-guided attack method that can generate both attack prompts and embedding-level attacks with automatically selected perturbation hyperparameters. Both automatic and human evaluations demonstrate that our attack method significantly improves the attack success rate and response quality while requiring less training data. Additionally, we find that our generated attack prompts may be transferable to GPT-4, and the embedding-level attacks may also be transferred to other white-box LLMs whose parameters are known. Our experiments further uncover the safety risks present in current LLMs. For example, we find that six out of seven open-source LLMs that we attack consistently provide relevant answers to more than 85\% malicious instructions. Finally, we provide insights into the safety mechanism of LLMs.

In this paper, we investigate the influence of quasiperiodic perturbations on one-dimensional non-Hermitian diamond lattices with an artificial magnetic flux $\theta$ that possess flat bands. Our study shows that the symmetry of these perturbations and the magnetic flux $\theta$ play a pivotal role in shaping the localization properties of the system. When $\theta=0$, the non-Hermitian lattice exhibits a single flat band in the crystalline case, and symmetric as well as antisymmetric perturbations can induce accurate mobility edges. In contrast, when $\theta=\pi$, the clean diamond lattice manifests three dispersionless bands referred to as an "all-band-flat" (ABF) structure, irrespective of the non-Hermitian parameter. The ABF structure restricts the transition from delocalized to localized states, as all states remain localized for any finite symmetric perturbation. Our numerical calculations further unveil that the ABF system subjected to antisymmetric perturbations exhibits multifractal-to-localized edges. Multifractal states are predominantly concentrated in the internal region of the spectrum. Additionally, we explore the case where $\theta$ lies within the range of $(0, \pi)$, revealing a diverse array of complex localization features., Comment: 7 pages, 7 figures

In this paper, we explore the localization features of wave functions in a family of mosaic quasiperiodic chains obtained by continuously interpolating between two limits: the mosaic Aubry-Andr\'{e} (AA) model, known for its exact mobility edges with extended states in the band-center region, and localized ones in the band-edge regions for a large enough modulation amplitude, and the mosaic Fibonacci chain, which exhibits its multifractal nature for all the states except for the extended one with $E=0$ for an arbitrary finite modulation amplitude. We discover that the mosaic AA limit for the states in the band-edge regions evolves into multifractal ones through a cascade of delocalization transitions. This cascade shows lobes of lower fractal dimension values separated by maxima of fractal dimension. In contrast, the states in the band-center region (except for the $E=0$ state) display an anomalous cascading process, where it emerges lobes of higher fractal dimension values are separated by the regions with lower fractal dimensions. Our findings offer insight into understanding the multifractality of quasiperiodic chains., Comment: 12 pages, 11 figures

This paper addresses influence pathway discovery, a key emerging problem in today's online media. We propose a discovery algorithm that leverages recently published work on unsupervised interpretable ideological embedding, a mapping of ideological beliefs (done in a self-supervised fashion) into interpretable low-dimensional spaces. Computing the ideological embedding at scale allows one to analyze correlations between the ideological positions of leaders, influencers, news portals, or population segments, deriving potential influence pathways. The work is motivated by the importance of social media as the preeminent means for global interactions and collaborations on today's Internet, as well as their frequent (mis-)use to wield influence that targets social beliefs and attitudes of selected populations. Tools that enable the understanding and mapping of influence propagation through population segments on social media are therefore increasingly important. In this paper, influence is measured by the perceived ideological shift over time that is correlated with influencers' activity. Correlated shifts in ideological embeddings indicate changes, such as swings/switching (among competing ideologies), polarization (depletion of neutral ideological positions), escalation/radicalization (shifts to more extreme versions of the ideology), or unification/cooldown (shifts towards more neutral stances). Case-studies are presented to explore selected influence pathways (i) in a recent French election, (ii) during political discussions in the Philippines, and (iii) for some Russian messaging during the Russia/Ukraine conflict., Comment: This paper is accepted by IEEE CIC as an invited vision paper

We study the topology and localization properties of a generalized Su-Schrieffer-Heeger (SSH) model with a quasi-periodic modulated hopping. It is found that the interplay of off-diagonal quasi-periodic modulations can induce topological Anderson insulator (TAI) phases and reentrant topological Anderson insulator (RTAI), and the topological phase boundaries can be uncovered by the divergence of the localization length of the zero-energy mode. In contrast to the conventional case that the TAI regime emerges in a finite range with the increase of disorder, the TAI and RTAI are robust against arbitrary modulation amplitude for our system. Furthermore, we find that the TAI and RTAI can induce the emergence of reentrant localization transitions. Such an interesting connection between the reentrant localization transition and the TAI/RTAI can be detected from the wave-packet dynamics in cold atom systems by adopting the technique of momentum-lattice engineering., Comment: 11 pages, 10 figures

Recent studies on disorder-induced many-body localization (MBL) in non-Hermitian quantum systems have attracted great interest. However, the non-Hermitian disorder-free MBL still needs to be clarified. We consider a one-dimensional interacting Stark model with nonreciprocal hoppings having time-reversal symmetry, the properties of which are boundary dependent. Under periodic boundary conditions (PBCs), such a model exhibits three types of phase transitions: the real-complex transition of eigenenergies, the topological phase transition, and the non-Hermitian Stark MBL transition. The real-complex and topological phase transitions occur at the same point in the thermodynamic limit but do not coincide with the non-Hermitian Stark MBL transition, which is quite different from the non-Hermitian disordered cases. By the level statistics, the system transitions from the Ginibre ensemble (GE) to the Gaussian orthogonal ensemble (GOE) to the Possion ensemble with the increase of the linear tilt potential's strength. The real-complex transition of the eigenvalues is accompanied by the GE-to-GOE transition in the ergodic regime. Moreover, the second transition of the level statistics corresponds to the occurrence of non-Hermitian Stark MBL. We demonstrate that the non-Hermitian Stark MBL is robust and shares many similarities with disorder-induced MBL, which several existing characteristic quantities of the spectral statistics and eigenstate properties can confirm. The dynamical evolutions of the entanglement entropy and the density imbalance can distinguish the real-complex and Stark MBL transitions. Finally, we find that our system under open boundary conditions lacks a real-complex transition, and the transition of non-Hermitian Stark MBL is the same as that under PBCs., Comment: 11 pages, 10 figures

The integration of two-dimensional (2D) materials with photonic structures has catalyzed a wide spectrum of optical and optoelectronic applications. Conventional nanophotonic structures generally lack efficient reconfigurability and multifunctionality. The atomically thin 2D van der Waals materials can thus infuse new functionality and reconfigurability to the well-established library of photonic structures such as integrated waveguides, optical fibers, photonic crystals, micro-cavities, and metasurface, to name a few. Thanks to the handiness of van der Waals interfaces, the 2D materials can be easily transferred and mixed with other prefabricated photonic templates with high degrees of freedom, and can act as the optical gain, modulation, sensing, or plasmonic media for diverse applications. Here we review recent advents on combining 2D materials to nanophotonic structures for new functionality development or performance enhancements. Challenges and emerging opportunities in integrating van der Waals building blocks beyond 2D materials are also discussed., Comment: 23 pages

In this paper, we introduce a class of homeomorphisms between metric spaces, which are locally biH\"{o}lder continuous mappings. Then an embedding result between Besov spaces induced by locally biH\"{o}lder continuous mappings between Ahlfors regular spaces is established, which extends the corresponding result of Bj\"{o}rn-Bj\"{o}rn-Gill-Shanmugalingam (J. Reine Angew. Math. 725: 63-114, 2017). Furthermore, an example is constructed to show that our embedding result is more general. We also introduce a geometric condition, named as uniform boundedness, to characterize when a quasisymmetric mapping between uniformly perfect spaces is locally biH\"{o}lder continuous.

In this paper, we first prove that any power quasi-symmetry of two metric spaces induces a rough quasi-isometry between their infinite hyperbolic cones. Second, we prove that for a complete metric space $Z$, there exists a point $\omega$ in the Gromov boundary of its infinite hyperbolic cone such that $Z$ can be seen as the Gromov boundary relative to $\omega$ of its infinite hyperbolic cone. Third, we prove that for a visual Gromov hyperbolic metric space $X$ and a Gromov boundary point $\omega$, $X$ is roughly similar to the infinite hyperbolic cone of its Gromov boundary relative to $\omega$. These are the generalizations of Theorem 7.4, Theorem 8.1 and Theorem 8.2 in [3] since the underlying spaces are not assumed to be bounded and the hyperbolic cones are infinite., Comment: 37 pages

In this paper, we study the existence of traces for Sobolev spaces on the hyperbolic filling $X$ of a compact metric space $Z$ equipped with a doubling measure. Given a suitable metric on $X$, we can regard $Z$ as the boundary of $X$. After equipping $X$ with a weighted measure $\mu$ via the measure on $Z$ and the Euclidean arc length, we give characterizations for the existence of traces for first-order Sobolev spaces.

We uncover the relationship of topology and disorder in a one-dimensional Su-Schrieffer-Heeger chain subjected to a slowly varying quasi-periodic modulation. By numerically calculating the disorder-averaged winding number and analytically studying the localization length of the zero modes, we obtain the topological phase diagram, which implies that the topological Anderson insulator (TAI) can be induced by a slowly varying quasi-periodic modulation. Moreover, unlike the localization properties in the TAI phase caused by random disorder, mobility edges can enter into the TAI region identified by the fractal dimension, the inverse participation ratio, and the spatial distributions of the wave functions, the boundaries of which coincide with our analytical results., Comment: 9 pages, 8 figures

In this paper, we study the traces and the extensions for weighted Sobolev spaces on upper half spaces when the weights reach to the borderline cases. We first give a full characterization of the existence of trace spaces for these weighted Sobolev spaces, and then study the trace parts and the extension parts between the weighted Sobolev spaces and a new kind of Besov-type spaces (on hyperplanes) which are defined by using integral averages over selected layers of dyadic cubes., Comment: 22 pages, 1 figure