Search

Decoding neurophysiological signals into language is of great research interest within brain-computer interface (BCI) applications. Electroencephalography (EEG), known for its non-invasiveness, ease of use, and cost-effectiveness, has been a popular method in this field. However, current EEG-to-Text decoding approaches face challenges due to the huge domain gap between EEG recordings and raw texts, inherent data bias, and small closed vocabularies. In this paper, we propose SEE: Semantically Aligned EEG-to-Text Translation, a novel method aimed at improving EEG-to-Text decoding by seamlessly integrating two modules into a pre-trained BART language model. These two modules include (1) a Cross-Modal Codebook that learns cross-modal representations to enhance feature consolidation and mitigate domain gap, and (2) a Semantic Matching Module that fully utilizes pre-trained text representations to align multi-modal features extracted from EEG-Text pairs while considering noise caused by false negatives, i.e., data from different EEG-Text pairs that have similar semantic meanings. Experimental results on the Zurich Cognitive Language Processing Corpus (ZuCo) demonstrate the effectiveness of SEE, which enhances the feasibility of accurate EEG-to-Text decoding., Comment: 4 pages

Complex 3D scene understanding has gained increasing attention, with scene encoding strategies playing a crucial role in this success. However, the optimal scene encoding strategies for various scenarios remain unclear, particularly compared to their image-based counterparts. To address this issue, we present a comprehensive study that probes various visual encoding models for 3D scene understanding, identifying the strengths and limitations of each model across different scenarios. Our evaluation spans seven vision foundation encoders, including image-based, video-based, and 3D foundation models. We evaluate these models in four tasks: Vision-Language Scene Reasoning, Visual Grounding, Segmentation, and Registration, each focusing on different aspects of scene understanding. Our evaluations yield key findings: DINOv2 demonstrates superior performance, video models excel in object-level tasks, diffusion models benefit geometric tasks, and language-pretrained models show unexpected limitations in language-related tasks. These insights challenge some conventional understandings, provide novel perspectives on leveraging visual foundation models, and highlight the need for more flexible encoder selection in future vision-language and scene-understanding tasks., Comment: Project page: https://yunzeman.github.io/lexicon3d , Github: https://github.com/YunzeMan/Lexicon3D

Based on our previous study [S. Wang $\textit{et al}$. J. Chem. Phys. $\textbf{153}$, 184102 (2020)], we generalize the theory of molecular emission power spectra from single molecules to multichromophoric systems in the framework of macroscopic quantum electrodynamics. This generalized theory is applicable to ensembles of molecules in arbitrary inhomogeneous, dispersive, and absorbing media. In the far-field region, the emission power spectra can be decomposed into products of \textit{electromagnetic environment factors} and \textit{lineshape functions}. To demonstrate the polaritonic effect on multichromophoric systems, we numerically simulate the emission power spectra of one to three molecules above a silver surface. The peak position redshifts as the number of the molecules increases, consistent with Kasha's model. However, surprisingly, the peak height does not increase with the number of molecules. To understand this unusual phenomenon, we derive analytical expressions for lineshape functions and find that the peak heights are associated to the imaginary part of dipole-dipole interactions, which stem from the absorption of the silver surface. It is worth noting that our theory not only covers the key results derived from Kasha's model, but also can predict the lineshape functions of molecules in complex dielectric environments. This study offers one alternative approach to directly analyze the hybrid-state dynamics of multichromophoric systems coupled with polariton., Comment: 16 pages, 4 figures, submitted to The Journal of Chemical Physics

Recent advancements in 3D object reconstruction from single images have primarily focused on improving the accuracy of object shapes. Yet, these techniques often fail to accurately capture the inter-relation between the object, ground, and camera. As a result, the reconstructed objects often appear floating or tilted when placed on flat surfaces. This limitation significantly affects 3D-aware image editing applications like shadow rendering and object pose manipulation. To address this issue, we introduce ORG (Object Reconstruction with Ground), a novel task aimed at reconstructing 3D object geometry in conjunction with the ground surface. Our method uses two compact pixel-level representations to depict the relationship between camera, object, and ground. Experiments show that the proposed ORG model can effectively reconstruct object-ground geometry on unseen data, significantly enhancing the quality of shadow generation and pose manipulation compared to conventional single-image 3D reconstruction techniques., Comment: Project Page: https://yunzeman.github.io/ORG/

The unsourced random access (URA) has emerged as a viable scheme for supporting the massive machine-type communications (mMTC) in the sixth generation (6G) wireless networks. Notably, the tensor-based URA (TURA), with its inherent tensor structure, stands out by simultaneously enhancing performance and reducing computational complexity for the multi-user separation, especially in mMTC networks with a large numer of active devices. However, current TURA scheme lacks the soft decoder, thus precluding the incorporation of existing advanced coding techniques. In order to fully explore the potential of the TURA, this paper investigates the Polarcoded TURA (PTURA) scheme and develops the corresponding iterative Bayesian receiver with feedback (IBR-FB). Specifically, in the IBR-FB, we propose the Grassmannian modulation-aided Bayesian tensor decomposition (GM-BTD) algorithm under the variational Bayesian learning (VBL) framework, which leverages the property of the Grassmannian modulation to facilitate the convergence of the VBL process, and has the ability to generate the required soft information without the knowledge of the number of active devices. Furthermore, based on the soft information produced by the GM-BTD, we design the soft Grassmannian demodulator in the IBR-FB. Extensive simulation results demonstrate that the proposed PTURA in conjunction with the IBR-FB surpasses the existing state-of-the-art unsourced random access scheme in terms of accuracy and computational complexity., Comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

Through integrating the evolutionary correlations across global states in the bidirectional recursion, an explainable Bayesian recurrent neural smoother (EBRNS) is proposed for offline data-assisted fixed-interval state smoothing. At first, the proposed model, containing global states in the evolutionary interval, is transformed into an equivalent model with bidirectional memory. This transformation incorporates crucial global state information with support for bi-directional recursive computation. For the transformed model, the joint state-memory-trend Bayesian filtering and smoothing frameworks are derived by introducing the bidirectional memory iteration mechanism and offline data into Bayesian estimation theory. The derived frameworks are implemented using the Gaussian approximation to ensure analytical properties and computational efficiency. Finally, the neural network modules within EBRNS and its two-stage training scheme are designed. Unlike most existing approaches that artificially combine deep learning and model-based estimation, the bidirectional recursion and internal gated structures of EBRNS are naturally derived from Bayesian estimation theory, explainably integrating prior model knowledge, online measurement, and offline data. Experiments on representative real-world datasets demonstrate that the high smoothing accuracy of EBRNS is accompanied by data efficiency and a lightweight parameter scale.

Being able to carry out complicated vision language reasoning tasks in 3D space represents a significant milestone in developing household robots and human-centered embodied AI. In this work, we demonstrate that a critical and distinct challenge in 3D vision language reasoning is situational awareness, which incorporates two key components: (1) The autonomous agent grounds its self-location based on a language prompt. (2) The agent answers open-ended questions from the perspective of its calculated position. To address this challenge, we introduce SIG3D, an end-to-end Situation-Grounded model for 3D vision language reasoning. We tokenize the 3D scene into sparse voxel representation and propose a language-grounded situation estimator, followed by a situated question answering module. Experiments on the SQA3D and ScanQA datasets show that SIG3D outperforms state-of-the-art models in situation estimation and question answering by a large margin (e.g., an enhancement of over 30% on situation estimation accuracy). Subsequent analysis corroborates our architectural design choices, explores the distinct functions of visual and textual tokens, and highlights the importance of situational awareness in the domain of 3D question answering., Comment: CVPR 2024. Project Page: https://yunzeman.github.io/situation3d

Open-world entity segmentation, as an emerging computer vision task, aims at segmenting entities in images without being restricted by pre-defined classes, offering impressive generalization capabilities on unseen images and concepts. Despite its promise, existing entity segmentation methods like Segment Anything Model (SAM) rely heavily on costly expert annotators. This work presents Self-supervised Open-world Hierarchical Entity Segmentation (SOHES), a novel approach that eliminates the need for human annotations. SOHES operates in three phases: self-exploration, self-instruction, and self-correction. Given a pre-trained self-supervised representation, we produce abundant high-quality pseudo-labels through visual feature clustering. Then, we train a segmentation model on the pseudo-labels, and rectify the noises in pseudo-labels via a teacher-student mutual-learning procedure. Beyond segmenting entities, SOHES also captures their constituent parts, providing a hierarchical understanding of visual entities. Using raw images as the sole training data, our method achieves unprecedented performance in self-supervised open-world segmentation, marking a significant milestone towards high-quality open-world entity segmentation in the absence of human-annotated masks. Project page: https://SOHES.github.io., Comment: ICLR 2024

In the presence of high magnetic field, quantum Hall systems usually host both even- and odd-integer quantized states because of lifted band degeneracies. Selective control of these quantized states is challenging but essential to understand the exotic ground states and manipulate the spin textures. Here, we study the quantum Hall effect in Bi2O2Se thin films. In magnetic fields as high as 50 T, we observe only even-integer quantum Hall states, but no sign of odd-integer states. However, when reducing the thickness of the epitaxial Bi2O2Se film to one unit cell, we observe both odd- and even-integer states in this Janus (asymmetric) film grown on SrTiO3. By means of a Rashba bilayer model based on ab initio band structures of Bi2O2Se thin films, we can ascribe the absence of odd-integer states in thicker films to the hidden Rasbha effect, where the local inversion symmetry breaking in two sectors of the [Bi2O2]2+ layer yields opposite Rashba spin polarizations, which compensate with each other. In the one unit cell Bi2O2Se film grown on SrTiO3, the asymmetry introduced by top surface and bottom interface induces a net polar field. The resulting global Rashba effect lifts the band degeneracies present in the symmetric case of thicker films., Comment: 6 Figures, 23 pages

Continual learning requires the model to learn multiple tasks sequentially. In continual learning, the model should possess the ability to maintain its performance on old tasks (stability) and the ability to adapt to new tasks continuously (plasticity). Recently, parameter-efficient fine-tuning (PEFT), which involves freezing a pre-trained model and injecting a small number of learnable parameters to adapt to downstream tasks, has gained increasing popularity in continual learning. Although existing continual learning methods based on PEFT have demonstrated superior performance compared to those not based on PEFT, most of them do not consider how to eliminate the interference of the new task on the old tasks, which inhibits the model from making a good trade-off between stability and plasticity. In this work, we propose a new PEFT method, called interference-free low-rank adaptation (InfLoRA), for continual learning. InfLoRA injects a small number of parameters to reparameterize the pre-trained weights and shows that fine-tuning these injected parameters is equivalent to fine-tuning the pre-trained weights within a subspace. Furthermore, InfLoRA designs this subspace to eliminate the interference of the new task on the old tasks, making a good trade-off between stability and plasticity. Experimental results show that InfLoRA outperforms existing state-of-the-art continual learning methods on multiple datasets., Comment: Accepted by the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2024)

Simultaneous functional PET/MR (sf-PET/MR) presents a cutting-edge multimodal neuroimaging technique. It provides an unprecedented opportunity for concurrently monitoring and integrating multifaceted brain networks built by spatiotemporally covaried metabolic activity, neural activity, and cerebral blood flow (perfusion). Albeit high scientific/clinical values, short in hardware accessibility of PET/MR hinders its applications, let alone modern AI-based PET/MR fusion models. Our objective is to develop a clinically feasible AI-based disease diagnosis model trained on comprehensive sf-PET/MR data with the power of, during inferencing, allowing single modality input (e.g., PET only) as well as enforcing multimodal-based accuracy. To this end, we propose MX-ARM, a multimodal MiXture-of-experts Alignment and Reconstruction Model. It is modality detachable and exchangeable, allocating different multi-layer perceptrons dynamically ("mixture of experts") through learnable weights to learn respective representations from different modalities. Such design will not sacrifice model performance in uni-modal situation. To fully exploit the inherent complex and nonlinear relation among modalities while producing fine-grained representations for uni-modal inference, we subsequently add a modal alignment module to line up a dominant modality (e.g., PET) with representations of auxiliary modalities (MR). We further adopt multimodal reconstruction to promote the quality of learned features. Experiments on precious multimodal sf-PET/MR data for Mild Cognitive Impairment diagnosis showcase the efficacy of our model toward clinically feasible precision medicine., Comment: 11 pages

Text-conditioned human motion generation has experienced significant advancements with diffusion models trained on extensive motion capture data and corresponding textual annotations. However, extending such success to 3D dynamic human-object interaction (HOI) generation faces notable challenges, primarily due to the lack of large-scale interaction data and comprehensive descriptions that align with these interactions. This paper takes the initiative and showcases the potential of generating human-object interactions without direct training on text-interaction pair data. Our key insight in achieving this is that interaction semantics and dynamics can be decoupled. Being unable to learn interaction semantics through supervised training, we instead leverage pre-trained large models, synergizing knowledge from a large language model and a text-to-motion model. While such knowledge offers high-level control over interaction semantics, it cannot grasp the intricacies of low-level interaction dynamics. To overcome this issue, we further introduce a world model designed to comprehend simple physics, modeling how human actions influence object motion. By integrating these components, our novel framework, InterDreamer, is able to generate text-aligned 3D HOI sequences in a zero-shot manner. We apply InterDreamer to the BEHAVE and CHAIRS datasets, and our comprehensive experimental analysis demonstrates its capability to generate realistic and coherent interaction sequences that seamlessly align with the text directives., Comment: Project Page: https://sirui-xu.github.io/InterDreamer/

In this study, we unveil an eccentric superradiance phenomenon in molecular aggregates coupled to surface plasmon polaritons. Through the quantization of electromagnetic fields in media, we demonstrate that superradiance can be significantly enhanced by polaritons and its behavior distinguishably surpasses the Dick's $N$ scaling law. To understand the mechanism of this anomalous phenomenon, we derive an analytical expression of the superradiance rate, which is general for molecular aggregates in arbitrary dispersive and absorbing media. Furthermore, we show the importance of intermolecular distance for this extraordinary superradiance.

Early infancy is a rapid and dynamic neurodevelopmental period for behavior and neurocognition. Longitudinal magnetic resonance imaging (MRI) is an effective tool to investigate such a crucial stage by capturing the developmental trajectories of the brain structures. However, longitudinal MRI acquisition always meets a serious data-missing problem due to participant dropout and failed scans, making longitudinal infant brain atlas construction and developmental trajectory delineation quite challenging. Thanks to the development of an AI-based generative model, neuroimage completion has become a powerful technique to retain as much available data as possible. However, current image completion methods usually suffer from inconsistency within each individual subject in the time dimension, compromising the overall quality. To solve this problem, our paper proposed a two-stage cascaded diffusion model, Cas-DiffCom, for dense and longitudinal 3D infant brain MRI completion and super-resolution. We applied our proposed method to the Baby Connectome Project (BCP) dataset. The experiment results validate that Cas-DiffCom achieves both individual consistency and high fidelity in longitudinal infant brain image completion. We further applied the generated infant brain images to two downstream tasks, brain tissue segmentation and developmental trajectory delineation, to declare its task-oriented potential in the neuroscience field.

Quantum Image Processing (QIP) is a field that aims to utilize the benefits of quantum computing for manipulating and analyzing images. However, QIP faces two challenges: the limitation of qubits and the presence of noise in a quantum machine. In this research, we propose a novel approach to address the issue of noise in QIP. By training and employing a machine learning model that identifies and corrects the noise in quantum-processed images, we can compensate for the noisiness caused by the machine and retrieve a processing result similar to that performed by a classical computer with higher efficiency. The model is trained by learning a dataset consisting of both existing processed images and quantum-processed images from open-access datasets. This model will be capable of providing us with the confidence level for each pixel and its potential original value. To assess the model's accuracy in compensating for loss and decoherence in QIP, we evaluate it using three metrics: Peak Signal to Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Mean Opinion Score (MOS). Additionally, we discuss the applicability of our model across domains well as its cost effectiveness compared to alternative methods., Comment: 9 pages, 3 figures

With the advent of Web 3.0, the swift advancement of technology confronts an imminent threat from quantum computing. Security protocols safeguarding the integrity of Web 2.0 and Web 3.0 are growing more susceptible to both quantum attacks and sophisticated classical threats. The article introduces our novel long-distance free-space quantum secure direct communication (LF QSDC) as a method to safeguard against security breaches in both quantum and classical contexts. Differing from techniques like quantum key distribution (QKD), LF QSDC surpasses constraints by facilitating encrypted data transmission sans key exchanges, thus diminishing the inherent weaknesses of key-based systems. The distinctiveness of this attribute, coupled with its quantum mechanics base, protects against quantum computer assaults and advanced non-quantum dangers, harmonizing seamlessly with the untrustworthy tenets of the Web 3.0 age. The focus of our study is the technical design and incorporation of LF QSDC into web 3.0 network infrastructures, highlighting its efficacy for extended-range communication. LF QSDC is based on the memory DL04 protocol and enhanced with our novel Quantum-Aware Low-Density Parity Check (LDPC), Pointing, Acquisition, and Tracking (PAT) technologies, and Atmospheric Quantum Correction Algorithm (AQCA). Utilizing this method not only bolsters the security of worldwide Web 3.0 networks but also guarantees their endurance in a time when quantum and sophisticated classical threats exist simultaneously. Consequently, LF QSDC stands out as a robust security solution, well-suited for Web 3.0 systems amidst the constantly evolving digital environment., Comment: 17 pages, 6 figures

The human visual perception system demonstrates exceptional capabilities in learning without explicit supervision and understanding the part-to-whole composition of objects. Drawing inspiration from these two abilities, we propose Hierarchical Adaptive Self-Supervised Object Detection (HASSOD), a novel approach that learns to detect objects and understand their compositions without human supervision. HASSOD employs a hierarchical adaptive clustering strategy to group regions into object masks based on self-supervised visual representations, adaptively determining the number of objects per image. Furthermore, HASSOD identifies the hierarchical levels of objects in terms of composition, by analyzing coverage relations between masks and constructing tree structures. This additional self-supervised learning task leads to improved detection performance and enhanced interpretability. Lastly, we abandon the inefficient multi-round self-training process utilized in prior methods and instead adapt the Mean Teacher framework from semi-supervised learning, which leads to a smoother and more efficient training process. Through extensive experiments on prevalent image datasets, we demonstrate the superiority of HASSOD over existing methods, thereby advancing the state of the art in self-supervised object detection. Notably, we improve Mask AR from 20.2 to 22.5 on LVIS, and from 17.0 to 26.0 on SA-1B. Project page: https://HASSOD-NeurIPS23.github.io., Comment: NeurIPS 2023

Classical data with graph structure always exists when dealing with many real-world problems. In parallel, quantum data with graph structure also need to be investigated since they are always produced by common quantum data sources.In this paper, we build a hybrid quantum-classical neural network with deep residual learning (Res-HQCNN) with graph-structured quantum data. Specifically, based on this special graph-structured quantum data, we first find suitable cost functions for Res-HQCNN model to learn semisupervised quantum data with graphs. Then, we present the training algorithm of Res-HQCNN for graph-structured training data in detail. Next, in order to show the learning ability of Res-HQCNN,we perform extensive experiments to show that the using of information about graph structures in quantum data can lead to better learning efficiency compared with the state-of-the-art model. At the same time, we also design comparable experiments to explain that the using of residual block structure can help deeper quantum neural networks learn graph-structured quantum data faster and better., Comment: 30 pages,9 figures

High-fidelity quantum gates are an essential prerequisite for large-scale quantum computation. When manipulating practical quantum systems, environmentally and operationally induced errors are inevitable, and thus, in addition to being fast, it is preferable that operations should be intrinsically robust against different errors. Here, we propose a general protocol for constructing geometric quantum gates with on-demand trajectories by modulating the applied pulse shapes that define the system's evolution trajectory. Our scheme adopts reverse engineering of the target Hamiltonian using smooth pulses, which also eliminates the difficulty of calculating geometric phases for an arbitrary trajectory. Furthermore, because a particular geometric gate can be induced by various different trajectories, we can further optimize the gate performance under different scenarios; the results of numerical simulations indicate that this optimization can greatly enhance the quality of the gate. In addition, we present an implementation of our proposal using superconducting circuits, showcasing substantial enhancements in gate performance compared with conventional schemes. Our protocol thus presents a promising approach for high-fidelity and strong-robust geometric quantum gates for large-scale quantum computation.

Cavity quantum electrodynamics (CQED) and its extensions are widely used for the description of exciton-polariton systems. However, the exciton-polariton models based on CQED vary greatly within different contexts. One of the most significant discrepancies among these CQED models is whether one should include direct intermolecular interactions in the CQED Hamiltonian. To answer this question, in this article, we derive an effective dissipative CQED model including free-space dipole-dipole interactions (CQED-DDI) from a microscopic Hamiltonian based on macroscopic quantum electrodynamics. Dissipative CQED-DDI successfully captures the nature of vacuum fluctuations in dielectric media and separates it into the free-space effects and the dielectric-induced effects. The former include spontaneous emissions, dephasings and dipole-dipole interactions in free space; the latter include exciton-polariton interactions and photonic losses due to dielectric media. We apply dissipative CQED-DDI to investigate the exciton-polariton dynamics (the population dynamics of molecules above a plasmonic surface) and compare the results with those based on the methods proposed by several previous studies. We find that direct intermolecular interactions are a crucial element when employing CQED-like models to study exciton-polariton systems involving multiple molecules.

A constant stellar mass-to-light ratio $M_\star/L$ has been widely-used in studies of galaxy dynamics and strong lensing, which aim at disentangling the mass distributions of dark matter and baryons. However, systematic biases arising from constant $M_\star/L$ assumption have not been fully quantified. In this work, we take massive early-type galaxies from the TNG100 simulation to investigate possible systematic biases in the inferences due to a constant $M_\star/L$ assumption. We construct two-component matter density models, where one component describes the dark matter, the other for the stars, which is made to follow the light profile by assuming a constant $M_\star/L$. We fit the two-component model directly to the {\it total} matter density distributions of simulated galaxies to eliminate systematics coming from other model assumptions. We find that galaxies generally have more centrally-concentrated stellar mass profile than their light distribution. Given the light profiles adopted (i.e., single- and double-S{\'e}rsic profiles), the assumption of a constant $M_\star/L$ would artificially break the model degeneracy {\it for non-constant} $M_\star/L$ systems. For such systems, without knowing the true $M_{\star}/L$ but assuming a constant ratio, the two-component modeling procedure tend to generally overestimate $M_{\star}/L$ by $30\%-50\%$, and underestimate the central dark matter fraction $f_{\rm DM}$ by $\sim 20\%$ on average., Comment: 18 pages, 10 figures, submitted to MNRAS

The optimality of Bayesian filtering relies on the completeness of prior models, while deep learning holds a distinct advantage in learning models from offline data. Nevertheless, the current fusion of these two methodologies remains largely ad hoc, lacking a theoretical foundation. This paper presents a novel solution, namely an explainable gated Bayesian recurrent neural network specifically designed to state estimation under model mismatches. Firstly, we transform the non-Markov state-space model into an equivalent first-order Markov model with memory. It is a generalized transformation that overcomes the limitations of the first-order Markov property and enables recursive filtering. Secondly, by deriving a data-assisted joint state-memory-mismatch Bayesian filtering, we design a Bayesian gated framework that includes a memory update gate for capturing the temporal regularities in state evolution, a state prediction gate with the evolution mismatch compensation, and a state update gate with the observation mismatch compensation. The Gaussian approximation implementation of the filtering process within the gated framework is derived, taking into account the computational efficiency. Finally, the corresponding internal neural network structures and end-to-end training methods are designed. The Bayesian filtering theory enhances the interpretability of the proposed gated network, enabling the effective integration of offline data and prior models within functionally explicit gated units. In comprehensive experiments, including simulations and real-world datasets, the proposed gated network demonstrates superior estimation performance compared to benchmark filters and state-of-the-art deep learning filtering methods.

Catalyzing reactions effectively by vacuum fluctuations of electromagnetic fields is a significant challenge within the realm of chemistry. Different from most studies based on vibrational strong coupling, we introduce an innovative catalytic mechanism driven by weakly coupled polaritonic fields. Through the amalgamation of macroscopic quantum electrodynamics (QED) principles with Marcus electron transfer (ET) theory, our results reveal that ET reaction rates can be precisely modulated across a wide dynamic range by controlling the size and structure of nanocavities. Comparing to QED-driven radiative ET rates in free space, plasmonic cavities induce substantial rate enhancements spanning from orders of magnitude ranging from 10^3-fold to 10^1-fold. By contrast, Fabry-Perot cavities engender rate suppression spanning from 10^{-2}-fold to 10^{-1}-fold. This work overcomes the necessity of using strong light-matter interactions in QED chemistry, opening up a new era of manipulating QED-based chemical reactions in a wide dynamic range.

Ferroelectric quantum spin Hall insulator (FEQSHI) exhibits coexisting ferroelectricity and time-reversal symmetry protected edge states, holding fascinating prospects for inviting both scientific and application advances, especially in two dimensions. However, all of the previously demonstrated FEQSHIs consist two or more constituent elements. We herein propose the $\psi$-bismuthene, an uncharted allotrope of bilayer Bi (110), to be the first example of 2D elemental FEQSHI. It is demonstrated that $\psi$-bismuthene harbors measurable ferroelectric polarization and nontrivial band gap with moderate switching barrier, which are highly beneficial for the detection and observation of the ferroelectric topologically insulating states. In addition, all-angle auxetic behavior with giant negative Poisson's ratio and ferroelectric controllable persistent spin helix in $\psi$-bismuthene are also discussed. The emergent elemental FEQSHI represents a novel domain for both fundamental physics and technological innovation.

Large Multimodal Models (LMM) are built across modalities and the misalignment between two modalities can result in "hallucination", generating textual outputs that are not grounded by the multimodal information in context. To address the multimodal misalignment issue, we adapt the Reinforcement Learning from Human Feedback (RLHF) from the text domain to the task of vision-language alignment, where human annotators are asked to compare two responses and pinpoint the more hallucinated one, and the vision-language model is trained to maximize the simulated human rewards. We propose a new alignment algorithm called Factually Augmented RLHF that augments the reward model with additional factual information such as image captions and ground-truth multi-choice options, which alleviates the reward hacking phenomenon in RLHF and further improves the performance. We also enhance the GPT-4-generated training data (for vision instruction tuning) with previously available human-written image-text pairs to improve the general capabilities of our model. To evaluate the proposed approach in real-world scenarios, we develop a new evaluation benchmark MMHAL-BENCH with a special focus on penalizing hallucinations. As the first LMM trained with RLHF, our approach achieves remarkable improvement on the LLaVA-Bench dataset with the 94% performance level of the text-only GPT-4 (while previous best methods can only achieve the 87% level), and an improvement by 60% on MMHAL-BENCH over other baselines. We opensource our code, model, data at https://llava-rlhf.github.io., Comment: Preprint

Geometric phases are robust to local noises and the nonadiabatic ones can reduce the evolution time, thus nonadiabatic geometric gates have strong robustness and can approach high fidelity. However, the advantage of geometric phase has not being fully explored in previous investigations. Here, we propose a scheme for universal quantum gates with pure nonadiabatic and noncyclic geometric phases from smooth evolution paths. In our scheme, only geometric phase can be accumulated in a fast way, and thus it not only fully utilizes the local noise resistant property of geometric phase but also reduces the difficulty in experimental realization. Numerical results show that the implemented geometric gates have stronger robustness than dynamical gates and the geometric scheme with cyclic path. Furthermore, we propose to construct universal quantum gate on superconducting circuits, with the fidelities of single-qubit gate and nontrivial two-qubit gate can achieve $99.97\%$ and $99.87\%$, respectively. Therefore, these high-fidelity quantum gates are promising for large-scale fault-tolerant quantum computation.

This paper addresses a novel task of anticipating 3D human-object interactions (HOIs). Most existing research on HOI synthesis lacks comprehensive whole-body interactions with dynamic objects, e.g., often limited to manipulating small or static objects. Our task is significantly more challenging, as it requires modeling dynamic objects with various shapes, capturing whole-body motion, and ensuring physically valid interactions. To this end, we propose InterDiff, a framework comprising two key steps: (i) interaction diffusion, where we leverage a diffusion model to encode the distribution of future human-object interactions; (ii) interaction correction, where we introduce a physics-informed predictor to correct denoised HOIs in a diffusion step. Our key insight is to inject prior knowledge that the interactions under reference with respect to contact points follow a simple pattern and are easily predictable. Experiments on multiple human-object interaction datasets demonstrate the effectiveness of our method for this task, capable of producing realistic, vivid, and remarkably long-term 3D HOI predictions., Comment: ICCV 2023; Project Page: https://sirui-xu.github.io/InterDiff/

This study proposes a coupled velocity model (CVM) that establishes the relation between the orientation and velocity using their correlation, avoiding that the existing extended object tracking (EOT) models treat them as two independent quantities. As a result, CVM detects the mismatch between the prior dynamic model and actual motion pattern to correct the filtering gain, and simultaneously becomes a nonlinear and state-coupled model with multiplicative noise. The study considers CVM to design a feasible distributed weighted least squares (WLS) filter. The WLS criterion requires a linear state-space model containing only additive noise about the estimated state. To meet the requirement, we derive such two separate pseudo-linearized models by using the first-order Taylor series expansion. The separation is merely in form, and the estimates of interested states are embedded as parameters into each other's model, which implies that their interdependency is still preserved in the iterative operation of two linear filters. With the two models, we first propose a centralized WLS filter by converting the measurements from all nodes into a summation form. Then, a distributed consensus scheme, which directly performs an inner iteration on the priors across different nodes, is proposed to incorporate the cross-covariances between nodes. Under the consensus scheme, a distributed WLS filter over a realistic network with ``naive'' node is developed by proper weighting of the priors and measurements. Finally, the performance of proposed filters in terms of accuracy, robustness, and consistency is testified under different prior situations., Comment: Corrected Version

Resource-constrained perception systems such as edge computing and vision-for-robotics require vision models to be both accurate and lightweight in computation and memory usage. While knowledge distillation is a proven strategy to enhance the performance of lightweight classification models, its application to structured outputs like object detection and instance segmentation remains a complicated task, due to the variability in outputs and complex internal network modules involved in the distillation process. In this paper, we propose a simple yet surprisingly effective sequential approach to knowledge distillation that progressively transfers the knowledge of a set of teacher detectors to a given lightweight student. To distill knowledge from a highly accurate but complex teacher model, we construct a sequence of teachers to help the student gradually adapt. Our progressive strategy can be easily combined with existing detection distillation mechanisms to consistently maximize student performance in various settings. To the best of our knowledge, we are the first to successfully distill knowledge from Transformer-based teacher detectors to convolution-based students, and unprecedentedly boost the performance of ResNet-50 based RetinaNet from 36.5% to 42.0% AP and Mask R-CNN from 38.2% to 42.5% AP on the MS COCO benchmark., Comment: ICML 2023

The Research & Development (R&D) phase of drug development is a lengthy and costly process. To revolutionize this process, we introduce our new concept QMLS to shorten the whole R&D phase to three to six months and decrease the cost to merely fifty to eighty thousand USD. For Hit Generation, Machine Learning Molecule Generation (MLMG) generates possible hits according to the molecular structure of the target protein while the Quantum Simulation (QS) filters molecules from the primary essay based on the reaction and binding effectiveness with the target protein. Then, For Lead Optimization, the resultant molecules generated and filtered from MLMG and QS are compared, and molecules that appear as a result of both processes will be made into dozens of molecular variations through Machine Learning Molecule Variation (MLMV), while others will only be made into a few variations. Lastly, all optimized molecules would undergo multiple rounds of QS filtering with a high standard for reaction effectiveness and safety, creating a few dozen pre-clinical-trail-ready drugs. This paper is based on our first paper, where we pitched the concept of machine learning combined with quantum simulations. In this paper we will go over the detailed design and framework of QMLS, including MLMG, MLMV, and QS., Comment: 13 pages, 6 figures

The rotating-wave approximation to light-matter interactions is widely used in the quantum electrodynamics Hamiltonian; however, its validity has long been a matter of debate. In this article, we explore the impact of the rotating-wave approximation on the quantum dynamics of multiple molecules in complex dielectric environments within the framework of macroscopic quantum electrodynamics. In general, we find that the energy shifts of the molecules and the inter-molecule dipole-dipole interaction obtained in the weak coupling regime are correct only when the counter-rotating interactions are considered. Moreover, under the rotating-wave approximation, the energy shifts of the ground-state molecules and a portion of the inter-molecule interaction are discarded. Notably, in the near-field zone (short inter-molecular distance), the reduction of inter-molecule interaction can reach up to 50 percent. We also conduct a case study on the population dynamics of a pair of identical molecules above a plasmonic surface. Through analytical and numerical analysis, it is revealed that the rotating-wave approximation can profoundly affect the dynamics of the molecules in both strong and weak coupling regimes, emphasizing the need for careful consideration when making the rotating-wave approximation in a multiple-molecule system coupled with quantum light.

We present an unsupervised search for outliers in the Bright Galaxy Survey (BGS) dataset from the DESI Early Data Release. This analysis utilizes an autoencoder to compress galaxy spectra into a compact, redshift-invariant latent space, and a normalizing flow to identify low-probability objects. The most prominent outliers show distinctive spectral features such as irregular or double-peaked emission lines, or originate from galaxy mergers, blended sources, and rare quasar types, including one previously unknown Broad Absorption Line system. A significant portion of the BGS outliers are stars spectroscopically misclassified as galaxies. By building our own star model trained on spectra from the DESI Milky Way Survey, we have determined that the misclassification likely stems from the Principle Component Analysis of stars in the DESI pipeline. To aid follow-up studies, we make the full probability catalog of all BGS objects and our pre-trained models publicly available., Comment: 9 pages, 5 figures

High-fidelity universal quantum gates are widely acknowledged as essential for scalable quantum computation. However, in solid-state quantum systems, which hold promise as physical implementation platforms for quantum computation, the inevitable $ZZ$ crosstalk resulting from interqubit interactions significantly impairs quantum operation performance. Here we propose a scalable protocol to achieve $ZZ$-crosstalk mitigation in universal quantum gates. This method converts the noisy Hamiltonian with $ZZ$ crosstalk into a framework that efficiently suppresses all $ZZ$-crosstalk effects, leading to ideal target quantum operations. Specifically, we first analytically derive the $ZZ$-crosstalk mitigation conditions and then apply them to enhance the performance of target universal quantum gates. Moreover, numerical simulations validate the effectiveness of $ZZ$-crosstalk mitigation when multiple qubit gates operate concurrently. As a result, our protocol presents a promising approach for implementing practical parallel quantum gates in large-scale quantum computation scenarios.

Knowledge graph embeddings (KGE) have been extensively studied to embed large-scale relational data for many real-world applications. Existing methods have long ignored the fact many KGs contain two fundamentally different views: high-level ontology-view concepts and fine-grained instance-view entities. They usually embed all nodes as vectors in one latent space. However, a single geometric representation fails to capture the structural differences between two views and lacks probabilistic semantics towards concepts' granularity. We propose Concept2Box, a novel approach that jointly embeds the two views of a KG using dual geometric representations. We model concepts with box embeddings, which learn the hierarchy structure and complex relations such as overlap and disjoint among them. Box volumes can be interpreted as concepts' granularity. Different from concepts, we model entities as vectors. To bridge the gap between concept box embeddings and entity vector embeddings, we propose a novel vector-to-box distance metric and learn both embeddings jointly. Experiments on both the public DBpedia KG and a newly-created industrial KG showed the effectiveness of Concept2Box.

The realization of high-performance two-dimensional (2D) solar photovoltaic systems are both fundamentally intriguing and practically appealing to meet the fast-growing energy requirements. Since the limited application of single 2D crystals in photovoltaic, here we propose a family of 2D KAgSe/KAgX(X=S,Te) van der Waals heterostructures (vdWHs), which are constructed by combining two different KAgX layers through interlayer vdW interaction. After a systematic study and further regulatory research of these vdWHs based on the first-principles, numerous fascinating characteristics and physical mechanisms are obtained. Firstly, favorable potential applications of these vdWHs in photovoltaics are confirmed in virtue of their desirable optoelectronic properties, such as the robust stabilitis, moderate direct band gaps, type-II band alignments together with superior carrier mobilities, visible optical absorptions, power conversion efficiencys (PCEs) and photocurrents in their based photovoltaic devices. More importantly, when under varying vertical electric field Ez, a phase transition of band alignment from type-II to type-I of these vdWHs can be induced by the opposite band shifts between layers, which may enrich their applications in light-emitting diodes and lasers. Meanwhile, the PCE can be expanded up to 23%, and an obvious red-shift peak of the photocurrent in the visible light range are also obtained at different Ez. These fascinating tunable properties of KAgSe/KAgX vdWHs under varying Ez not only promote the improvement of their photoelectric performances, but the underlying mechanisms can also be applied to next experimental design and practical application of other 2D photovoltaic systems. Especially for the red-shift peak of the photocurrent, which is rarely found but highly desirable in practical visible photoelectric conversion., Comment: 11 pages, 7figures

Geometric phase has the intrinsic property of being resistant to some types of local noises as it only depends on global properties of the evolution path. Meanwhile, the non-Abelian geometric phase is in the matrix form, and thus can naturally be used to implement high performance quantum gates, i.e., the so-called holonomic quantum computation. This article reviews recent advances in nonadiabatic holonomic quantum computation, and focuses on various optimal control approaches that can improve the gate performance, in terms of the gate fidelity and robustness. Besides, we also pay special attention to its possible physical realizations and some concrete examples of experimental realizations. Finally, with all these efforts, within state-of-the-art technology, the performance of the implemented holonomic quantum gates can outperform the conventional dynamical ones, under certain conditions.

This paper aims to deal with the ignored real-world complexities in prior work on human motion forecasting, emphasizing the social properties of multi-person motion, the diversity of motion and social interactions, and the complexity of articulated motion. To this end, we introduce a novel task of stochastic multi-person 3D motion forecasting. We propose a dual-level generative modeling framework that separately models independent individual motion at the local level and social interactions at the global level. Notably, this dual-level modeling mechanism can be achieved within a shared generative model, through introducing learnable latent codes that represent intents of future motion and switching the codes' modes of operation at different levels. Our framework is general; we instantiate it with different generative models, including generative adversarial networks and diffusion models, and various multi-person forecasting models. Extensive experiments on CMU-Mocap, MuPoTS-3D, and SoMoF benchmarks show that our approach produces diverse and accurate multi-person predictions, significantly outperforming the state of the art., Comment: ICLR 2023 (Top 25% Paper); Project Page: https://sirui-xu.github.io/DuMMF

Closing the domain gap between training and deployment and incorporating multiple sensor modalities are two challenging yet critical topics for self-driving. Existing work only focuses on single one of the above topics, overlooking the simultaneous domain and modality shift which pervasively exists in real-world scenarios. A model trained with multi-sensor data collected in Europe may need to run in Asia with a subset of input sensors available. In this work, we propose DualCross, a cross-modality cross-domain adaptation framework to facilitate the learning of a more robust monocular bird's-eye-view (BEV) perception model, which transfers the point cloud knowledge from a LiDAR sensor in one domain during the training phase to the camera-only testing scenario in a different domain. This work results in the first open analysis of cross-domain cross-sensor perception and adaptation for monocular 3D tasks in the wild. We benchmark our approach on large-scale datasets under a wide range of domain shifts and show state-of-the-art results against various baselines., Comment: IROS 2023. Project website: https://yunzeman.github.io/DualCross

Object detectors often suffer from the domain gap between training (source domain) and real-world applications (target domain). Mean-teacher self-training is a powerful paradigm in unsupervised domain adaptation for object detection, but it struggles with low-quality pseudo-labels. In this work, we identify the intriguing alignment and synergy between mean-teacher self-training and contrastive learning. Motivated by this, we propose Contrastive Mean Teacher (CMT) -- a unified, general-purpose framework with the two paradigms naturally integrated to maximize beneficial learning signals. Instead of using pseudo-labels solely for final predictions, our strategy extracts object-level features using pseudo-labels and optimizes them via contrastive learning, without requiring labels in the target domain. When combined with recent mean-teacher self-training methods, CMT leads to new state-of-the-art target-domain performance: 51.9% mAP on Foggy Cityscapes, outperforming the previously best by 2.1% mAP. Notably, CMT can stabilize performance and provide more significant gains as pseudo-label noise increases., Comment: CVPR 2023

In this study, we propose the concept of harnessing quantum coherence to control electron transport in a many-body system. Combining an open quantum system technique based on Hubbard operators, we show that many-body coherence can eliminate the well-known Coulomb staircase and cause strong negative differential resistance. To explore the mechanism, we analytically derive the current-coherence relationship in the zero electron-phonon coupling limit. Furthermore, by incorporating a gate field, we demonstrate the possibility of constructing a coherence-controlled transistor. This development opens up a new direction for exploring quantum electronic devices based on many-body coherence., Comment: 15 pages, 5 figures