Search

Nonlinear partial differential equations (PDEs) are crucial for modeling complex fluid dynamics and are foundational to many computational fluid dynamics (CFD) applications. However, solving these nonlinear PDEs is challenging due to the vast computational resources they demand, highlighting the pressing need for more efficient computational methods. Quantum computing offers a promising but technically challenging approach to solving nonlinear PDEs. Recently, Liao proposed a framework that leverages quantum computing to accelerate the solution of nonlinear PDEs based on the homotopy analysis method (HAM), a semi-analytical technique that transforms nonlinear PDEs into a series of linear PDEs. However, the no-cloning theorem in quantum computing poses a major limitation, where directly applying quantum simulation to each HAM step results in exponential complexity growth with the HAM truncation order. This study introduces a "secondary linearization" approach that maps the whole HAM process into a system of linear PDEs, allowing for a one-time solution using established quantum PDE solvers. Our method preserves the exponential speedup of quantum linear PDE solvers while ensuring that computational complexity increases only polynomially with the HAM truncation order. We demonstrate the efficacy of our approach by applying it to the Burgers' equation and the Korteweg-de Vries (KdV) equation. Our approach provides a novel pathway for transforming nonlinear PDEs into linear PDEs, with potential applications to fluid dynamics. This work thus lays the foundation for developing quantum algorithms capable of solving the Navier-Stokes equations, ultimately offering a promising route to accelerate their solutions using quantum computing., Comment: 22 pages, 4 figures

We propose one-to-one correspondence reconstruction for electron-positron Higgs factories. For each visible particle, one-to-one correspondence aims to associate relevant detector hits with only one reconstructed particle and accurately identify its species. To achieve this goal, we develop a novel detector concept featuring 5-dimensional calorimetry that provides spatial, energy, and time measurements for each hit, and a reconstruction framework that combines state-of-the-art particle flow and artificial intelligence algorithms. In the benchmark process of Higgs to di-jets, over 90% of visible energy can be successfully mapped into well-reconstructed particles that not only maintain a one-to-one correspondence relationship but also associate with the correct combination of cluster and track, improving the invariant mass resolution of hadronically decayed Higgs bosons by 25%. Performing simultaneous identification on these well-reconstructed particles, we observe efficiencies of 97% to nearly 100% for charged particles ($e^{\pm}$, $\mu^{\pm}$, $\pi^{\pm}$, $K^{\pm}$, $p/\bar{p}$) and photons ($\gamma$), and 75% to 80% for neutral hadrons ($K_L^0$, $n$, $\bar{n}$). For physics measurements of Higgs to invisible and exotic decays, golden channels to probe new physics, one-to-one correspondence could enhance discovery power by 10% to up to a factor of two. This study demonstrates the necessity and feasibility of one-to-one correspondence reconstruction at electron-positron Higgs factories.

Solving partial differential equations (PDEs) is a required step in the simulation of natural and engineering systems. The associated computational costs significantly increase when exploring various scenarios, such as changes in initial or boundary conditions or different input configurations. This study proposes the Variational Physics-Informed Neural Operator (VINO), a deep learning method designed for solving PDEs by minimizing the energy formulation of PDEs. This framework can be trained without any labeled data, resulting in improved performance and accuracy compared to existing deep learning methods and conventional PDE solvers. By discretizing the domain into elements, the variational format allows VINO to overcome the key challenge in physics-informed neural operators, namely the efficient evaluation of the governing equations for computing the loss. Comparative results demonstrate VINO's superior performance, especially as the mesh resolution increases. As a result, this study suggests a better way to incorporate physical laws into neural operators, opening a new approach for modeling and simulating nonlinear and complex processes in science and engineering.

We present the Early Data Release of the Multi-Cycle JWST-NEXUS Treasury program (2024-2028), which includes NIRCam imaging and WFSS observations from the first (partial) NEXUS-Wide epoch covering the central 100 ${\rm arcmin^2}$ of the NEXUS field, located near the North Ecliptic Pole and within the Euclid Ultra-Deep Field. We release reduced NIRCam mosaics (F090W, F115W, F150W, F200W, F356W, F444W), photometric source catalogs, as well as preliminary WFSS spectra (in F322W2 and F444W) for the subset of bright sources (F356W$<$21 mag or F444W$<$21 mag). These observations fully cover the NEXUS-Deep area, and anchor the long-term baseline of the program. These data will be used for initial target selection for the NIRSpec/MSA spectroscopy starting from June 2025. The NIRCam imaging reaches depths of 27.4--28.2 (AB) mags in F090W--F444W. Upcoming NEXUS-Wide epochs will expand the area to the full $\sim 400\,{\rm arcmin^2}$, and improve the NIRCam exposure depths in the Wide tier by a factor of three. In addition, this central region will be repeatedly covered by the NEXUS-Deep observations (NIRCam imaging and NIRSpec/MSA PRISM spectroscopy) over 18 epochs with a $\sim 2$-month cadence. We demonstrate the data quality of the first NEXUS observations, and showcase some example science cases enabled by these data., Comment: 14 pages, 9 figures, 2 tables. Data products are publicly accessible at https://ariel.astro.illinois.edu/nexus/edr/ Online interactive map for quick visualization of released images and WFSS spectra can be found at https://ariel.astro.illinois.edu/nexus/map/

Over the past year, Speculative Decoding has gained popularity as a technique for accelerating Large Language Model inference. While several methods have been introduced, most struggle to deliver satisfactory performance at batch sizes typical for data centers ($\geq 8$) and often involve significant deployment complexities. In this work, we offer a theoretical explanation of how Speculative Decoding can be effectively utilized with larger batch sizes. We also introduce a method that integrates seamlessly into existing systems without additional training or the complexity of deploying a small LLM. In a continuous batching setting, we achieve a 4x increase in throughput without any latency impact for short context generation, and a 1.7-2x improvement in both latency and throughput for longer contexts., Comment: 14 pages, 7 figures

We investigate the host galaxy properties of eight little red dots (LRDs) selected from the JWST UNCOVER survey, applying a new technique ({\tt\string GalfitS}) to simultaneously fit the morphology and spectral energy distribution using multi-band NIRCam images covering $\sim 1-4\,\mu {\rm m}$. We detect the host galaxy in only one LRD, MSAID38108 at $z = 4.96$, which has a stellar mass of $\log (M_*/M_{\odot}) = 8.66^{+0.24}_{-0.23}$, an effective radius $R_e=0.66^{+0.08}_{-0.05}$ kpc, and a S\'ersic index $n=0.71^{+0.07}_{-0.08}$. No host emission centered on the central point source is found in the other seven LRDs. We derive stringent upper limits for the stellar mass of a hypothetical host galaxy by conducting realistic mock simulations that place high-redshift galaxy images under the LRDs. Based on the black hole masses estimated from the broad H$\alpha$ emission line, the derived stellar mass limits are at least a factor of 10 lower than expected from the $z \approx 0$ scaling relation between black hole mass and host galaxy stellar mass. Intriguingly, four of the LRDs (50\% of the sample) show extended, off-centered emission, which is particularly prominent in the bluer bands. The asymmetric emission of two sources can be modeled as stellar emission, but the nature of the other two is unclear., Comment: 29 pages, 15 figures, submitted to ApJ

We introduce MVSplat360, a feed-forward approach for 360{\deg} novel view synthesis (NVS) of diverse real-world scenes, using only sparse observations. This setting is inherently ill-posed due to minimal overlap among input views and insufficient visual information provided, making it challenging for conventional methods to achieve high-quality results. Our MVSplat360 addresses this by effectively combining geometry-aware 3D reconstruction with temporally consistent video generation. Specifically, it refactors a feed-forward 3D Gaussian Splatting (3DGS) model to render features directly into the latent space of a pre-trained Stable Video Diffusion (SVD) model, where these features then act as pose and visual cues to guide the denoising process and produce photorealistic 3D-consistent views. Our model is end-to-end trainable and supports rendering arbitrary views with as few as 5 sparse input views. To evaluate MVSplat360's performance, we introduce a new benchmark using the challenging DL3DV-10K dataset, where MVSplat360 achieves superior visual quality compared to state-of-the-art methods on wide-sweeping or even 360{\deg} NVS tasks. Experiments on the existing benchmark RealEstate10K also confirm the effectiveness of our model. The video results are available on our project page: https://donydchen.github.io/mvsplat360., Comment: NeurIPS 2024, Project page: https://donydchen.github.io/mvsplat360, Code: https://github.com/donydchen/mvsplat360

Recently, an intriguing research trend for automatic target recognition (ATR) from synthetic aperture radar (SAR) imagery has arisen: using simulated data to train ATR models is a feasible solution to the issue of inadequate measured data. To close the domain gap that exists between the real and simulated data, the unsupervised domain adaptation (UDA) techniques are frequently exploited to construct ATR models. However, for UDA, the target domain lacks labeled data to direct the model training, posing a great challenge to ATR performance. To address the above problem, a semi-supervised domain adaptation (SSDA) framework has been proposed adopting progressive multi-level alignments for simulated data-aided SAR ATR. First, a progressive wavelet transform data augmentation (PWTDA) is presented by analyzing the discrepancies of wavelet decomposition sub-bands of two domain images, obtaining the domain-level alignment. Specifically, the domain gap is narrowed by mixing the wavelet transform high-frequency sub-band components. Second, we develop an asymptotic instance-prototype alignment (AIPA) strategy to push the source domain instances close to the corresponding target prototypes, aiming to achieve category-level alignment. Moreover, the consistency alignment is implemented by excavating the strong-weak augmentation consistency of both individual samples and the multi-sample relationship, enhancing the generalization capability of the model. Extensive experiments on the Synthetic and Measured Paired Labeled Experiment (SAMPLE) dataset, indicate that our approach obtains recognition accuracies of 99.63% and 98.91% in two common experimental settings with only one labeled sample per class of the target domain, outperforming the most advanced SSDA techniques.

We present Robot-centric Pooling (RcP), a novel pooling method designed to enhance end-to-end visuomotor policies by enabling differentiation between the robots and similar entities or their surroundings. Given an image-proprioception pair, RcP guides the aggregation of image features by highlighting image regions correlating with the robot's proprioceptive states, thereby extracting robot-centric image representations for policy learning. Leveraging contrastive learning techniques, RcP integrates seamlessly with existing visuomotor policy learning frameworks and is trained jointly with the policy using the same dataset, requiring no extra data collection involving self-distractors. We evaluate the proposed method with reaching tasks in both simulated and real-world settings. The results demonstrate that RcP significantly enhances the policies' robustness against various unseen distractors, including self-distractors, positioned at different locations. Additionally, the inherent robot-centric characteristic of RcP enables the learnt policy to be far more resilient to aggressive pixel shifts compared to the baselines., Comment: Accepted at IROS 2024

How can we test AI performance? This question seems trivial, but it isn't. Standard benchmarks often have problems such as in-distribution and small-size test sets, oversimplified metrics, unfair comparisons, and short-term outcome pressure. As a consequence, good performance on standard benchmarks does not guarantee success in real-world scenarios. To address these problems, we present Touchstone, a large-scale collaborative segmentation benchmark of 9 types of abdominal organs. This benchmark is based on 5,195 training CT scans from 76 hospitals around the world and 5,903 testing CT scans from 11 additional hospitals. This diverse test set enhances the statistical significance of benchmark results and rigorously evaluates AI algorithms across various out-of-distribution scenarios. We invited 14 inventors of 19 AI algorithms to train their algorithms, while our team, as a third party, independently evaluated these algorithms on three test sets. In addition, we also evaluated pre-existing AI frameworks--which, differing from algorithms, are more flexible and can support different algorithms--including MONAI from NVIDIA, nnU-Net from DKFZ, and numerous other open-source frameworks. We are committed to expanding this benchmark to encourage more innovation of AI algorithms for the medical domain., Comment: Accepted to NeurIPS-2024

Random quantum states play a critical role in quantum information processing. While random quantum circuits typically provide pseudo-random states, deep thermalization introduces quantum measurement to generate genuinely random states. However, the requirement of large ancillae in conventional deep thermalization poses a challenge to scale up the system size. We introduce holographic deep thermalization to substantially reduce the required ancillae to a system-size independent constant. Our circuit design trades space with time, via adopting a sequential application of an scrambling-measure-reset process on a small number of ancillae. Via tuning the ancilla size and number of time steps, holographic deep thermalization allows a continuous trade-off between the total quantum circuit size and the ancilla size. In the case of finite-size systems, we further enhance the performance of holographic deep thermalization via generative quantum machine learning, which leads to constant-factor advantages in the convergence towards Haar random. The theoretical predictions are verified with IBM Quantum noisy simulations., Comment: 6+10 pages, 12 figures

We propose ShadowGPT, a novel approach for solving quantum many-body problems by learning from randomized measurement data collected from quantum experiments. The model is a generative pretrained transformer (GPT) trained on simulated classical shadow data of ground states of quantum Hamiltonians, obtained through randomized Pauli measurements. Once trained, the model can predict a range of ground state properties across the Hamiltonian parameter space. We demonstrate its effectiveness on the transverse-field Ising model and the $\mathbb{Z}_2 \times \mathbb{Z}_2$ cluster-Ising model, accurately predicting ground state energy, correlation functions, and entanglement entropy. This approach highlights the potential of combining quantum data with classical machine learning to address complex quantum many-body challenges.

With fast developments in computational power and algorithms, deep learning has made breakthroughs and been applied in many fields. However, generalization remains to be a critical challenge, and the limited generalization capability severely constrains its practical applications. Hallucination issue is another unresolved conundrum haunting deep learning and large models. By leveraging a physical model of imaging through scattering media, we studied the lack of generalization to system response functions in deep learning, identified its cause, and proposed a universal solution. The research also elucidates the creation process of a hallucination in image prediction and reveals its cause, and the common relationship between generalization and hallucination is discovered and clarified. Generally speaking, it enhances the interpretability of deep learning from a physics-based perspective, and builds a universal physical framework for deep learning in various fields. It may pave a way for direct interaction between deep learning and the real world, facilitating the transition of deep learning from a demo model to a practical tool in diverse applications.

Resource slicing in low Earth orbit satellite networks (LSN) is essential to support diversified services. In this paper, we investigate a resource slicing problem in LSN to reserve resources in satellites to achieve efficient resource provisioning. To address the challenges of non-stationary service demands, inaccurate prediction, and satellite mobility, we propose an adaptive digital twin (DT)-assisted resource slicing scheme for robust and adaptive resource management in LSN. Specifically, a slice DT, being able to capture the service demand prediction uncertainty through collected service demand data, is constructed to enhance the robustness of resource slicing decisions for dynamic service demands. In addition, the constructed DT can emulate resource slicing decisions for evaluating their performance, enabling adaptive slicing decision updates to efficiently reserve resources in LSN. Simulation results demonstrate that the proposed scheme outperforms benchmark methods, achieving low service demand violations with efficient resource consumption., Comment: Accepted by IEEE GLOBECOM 2024

Time series anomaly detection (TSAD) is becoming increasingly vital due to the rapid growth of time series data across various sectors. Anomalies in web service data, for example, can signal critical incidents such as system failures or server malfunctions, necessitating timely detection and response. However, most existing TSAD methodologies rely heavily on manual feature engineering or require extensive labeled training data, while also offering limited interpretability. To address these challenges, we introduce a pioneering framework called the Time Series Anomaly Multimodal Analyzer (TAMA), which leverages the power of Large Multimodal Models (LMMs) to enhance both the detection and interpretation of anomalies in time series data. By converting time series into visual formats that LMMs can efficiently process, TAMA leverages few-shot in-context learning capabilities to reduce dependence on extensive labeled datasets. Our methodology is validated through rigorous experimentation on multiple real-world datasets, where TAMA consistently outperforms state-of-the-art methods in TSAD tasks. Additionally, TAMA provides rich, natural language-based semantic analysis, offering deeper insights into the nature of detected anomalies. Furthermore, we contribute one of the first open-source datasets that includes anomaly detection labels, anomaly type labels, and contextual description, facilitating broader exploration and advancement within this critical field. Ultimately, TAMA not only excels in anomaly detection but also provides a comprehensive approach for understanding the underlying causes of anomalies, pushing TSAD forward through innovative methodologies and insights., Comment: Under review

Hong-Ou-Mandel effect is an important demonstration of particle indistinguishability, when identical single photons interfere at a beamsplitter to generate the two-photon entangled NOON state. On the other hand, NOON states with $N\ge3$ photons have long been conjectured beyond the deterministic generation of photon interference. To characterize the separation, we introduce the notion of genuine non-Gaussian entanglement (NGE), which cannot be generated via a generalized Hong-Ou-Mandel experiment, with Gaussian protocols extending the beamsplitter and separable input states replacing the single photons. We establish a resource theory to characterize such quantum correlations beyond Hong-Ou-Mandel and prove that NOON states with $N\ge 3$ are indeed among the NGE class. With the generalized Hong-Ou-Mandel protocol as free operations, we introduce two monotones to characterize genuine non-Gaussian entanglement: one derived from the entanglement entropy and the other from the minimal extension size required to convert a state into a free state. Finally, we demonstrate that the tomography process of pure free states can be performed efficiently, while all learning protocols of states with genuine non-Gaussian entanglement require exponential overheads connected to the monotone. This implies that states generated in Boson sampling are efficiently learnable despite its measurement statistics being hard to sample from.

To meet the growing demand for accurate, rapid, and cost-effective at-home clinical testing, we developed a radical-mediated enzyme assay (REEA) integrated with a paper fluidic system and electrically read by a handheld field-effect transistor (FET) device. The REEA utilizes horseradish peroxidase (HRP) to catalyze the conversion of aromatic substrates into radical forms, producing protons detected by an ion-sensitive FET for biomarker quantification. Through screening 14 phenolic compounds, halogenated phenols emerged as optimal substrates for the REEA. Encased in an affordable cartridge ($0.55 per test), the system achieved a detection limit of 146 fg/mL for estradiol (E2), with a coefficient of variation (CV) below 9.2% in E2-spiked samples and an r2 of 0.963 across a measuring range of 19 to 4,551 pg/mL in clinical plasma samples, providing results in under 10 minutes. This adaptable system not only promises to offer a fast and reliable platform, but also holds significant potential for expansion to a wide array of biomarkers, paving the way for broader clinical and home-based applications.

Neural contextual biasing allows speech recognition models to leverage contextually relevant information, leading to improved transcription accuracy. However, the biasing mechanism is typically based on a cross-attention module between the audio and a catalogue of biasing entries, which means computational complexity can pose severe practical limitations on the size of the biasing catalogue and consequently on accuracy improvements. This work proposes an approximation to cross-attention scoring based on vector quantization and enables compute- and memory-efficient use of large biasing catalogues. We propose to use this technique jointly with a retrieval based contextual biasing approach. First, we use an efficient quantized retrieval module to shortlist biasing entries by grounding them on audio. Then we use retrieved entries for biasing. Since the proposed approach is agnostic to the biasing method, we investigate using full cross-attention, LLM prompting, and a combination of the two. We show that retrieval based shortlisting allows the system to efficiently leverage biasing catalogues of several thousands of entries, resulting in up to 71% relative error rate reduction in personal entity recognition. At the same time, the proposed approximation algorithm reduces compute time by 20% and memory usage by 85-95%, for lists of up to one million entries, when compared to standard dot-product cross-attention., Comment: 13 pages, 7 figures, submitted to IEEE/ACM Transactions on Audio, Speech, and Language Processing

In the era of foundation models, we revisit continual learning~(CL), which aims to enable vision transformers (ViTs) to learn new tasks over time. However, as the scale of these models increases, catastrophic forgetting remains a persistent challenge, particularly in the presence of significant domain shifts across tasks. Recent studies highlight a crossover between CL techniques and parameter-efficient fine-tuning (PEFT), which focuses on fine-tuning only a small set of trainable parameters to adapt to downstream tasks, such as low-rank adaptation (LoRA). While LoRA achieves faster convergence and requires fewer trainable parameters, it has seldom been explored in the context of continual learning. To address this gap, we propose a novel PEFT-CL method called Dual Low-Rank Adaptation (DualLoRA), which introduces both an orthogonal LoRA adapter and a residual LoRA adapter parallel to pre-trained weights in each layer. These components are orchestrated by a dynamic memory mechanism to strike a balance between stability and plasticity. The orthogonal LoRA adapter's parameters are updated in an orthogonal subspace of previous tasks to mitigate catastrophic forgetting, while the residual LoRA adapter's parameters are updated in the residual subspace spanned by task-specific bases without interaction across tasks, offering complementary capabilities for fine-tuning new tasks. On ViT-based models, we demonstrate that DualLoRA offers significant advantages in accuracy, inference speed, and memory efficiency over existing CL methods across multiple benchmarks.

Tracking dense 3D motion from monocular videos remains challenging, particularly when aiming for pixel-level precision over long sequences. We introduce DELTA, a novel method that efficiently tracks every pixel in 3D space, enabling accurate motion estimation across entire videos. Our approach leverages a joint global-local attention mechanism for reduced-resolution tracking, followed by a transformer-based upsampler to achieve high-resolution predictions. Unlike existing methods, which are limited by computational inefficiency or sparse tracking, DELTA delivers dense 3D tracking at scale, running over 8x faster than previous methods while achieving state-of-the-art accuracy. Furthermore, we explore the impact of depth representation on tracking performance and identify log-depth as the optimal choice. Extensive experiments demonstrate the superiority of DELTA on multiple benchmarks, achieving new state-of-the-art results in both 2D and 3D dense tracking tasks. Our method provides a robust solution for applications requiring fine-grained, long-term motion tracking in 3D space., Comment: Project Page: https://snap-research.github.io/DELTA/

Cross-modal text-molecule retrieval model aims to learn a shared feature space of the text and molecule modalities for accurate similarity calculation, which facilitates the rapid screening of molecules with specific properties and activities in drug design. However, previous works have two main defects. First, they are inadequate in capturing modality-shared features considering the significant gap between text sequences and molecule graphs. Second, they mainly rely on contrastive learning and adversarial training for cross-modality alignment, both of which mainly focus on the first-order similarity, ignoring the second-order similarity that can capture more structural information in the embedding space. To address these issues, we propose a novel cross-modal text-molecule retrieval model with two-fold improvements. Specifically, on the top of two modality-specific encoders, we stack a memory bank based feature projector that contain learnable memory vectors to extract modality-shared features better. More importantly, during the model training, we calculate four kinds of similarity distributions (text-to-text, text-to-molecule, molecule-to-molecule, and molecule-to-text similarity distributions) for each instance, and then minimize the distance between these similarity distributions (namely second-order similarity losses) to enhance cross-modal alignment. Experimental results and analysis strongly demonstrate the effectiveness of our model. Particularly, our model achieves SOTA performance, outperforming the previously-reported best result by 6.4%., Comment: BIBM 2024 regular paper

LiFePO4 batteries are widely used in electric vehicles and energy storage systems due to long cycle life and high safety performance. However, the OCV-SOC curve (OSC) of these batteries features a long plateau region, making state of charge (SOC) estimation highly sensitive to OSC error, which arises due to aging and temperature. To address this, we propose an SOC estimation method that accounts for error in OSC. First, we establish battery equivalent circuit model (ECM) and introduce a parameters identification algorithm based on adaptive recursive least squares. Next, we derive the relationship between the innovation's cross-correlation matrix (CCM)/ auto-correlation matrix (ACM) of the Kalman filter and the OSC error. We then develop an adaptive multi-model Kalman filter (AMMKF), which dynamically adjusts the measurement model parameters of each filter based on the sign of the OSC error. By assigning a probability to each filter according to its predicted voltage distribution function, the optimal filter is selected. The proposed method is tested under various OSC error types and operating conditions. Results demonstrate that the proposed method achieves high accuracy and robustness, with an RMSE of less than 3\%, which is more than 10\% lower than the estimation error of traditional method., Comment: 10 pages,9 figures

The 3-hinge gyrus (3HG) is a newly defined folding pattern, which is the conjunction of gyri coming from three directions in cortical folding. Many studies demonstrated that 3HGs can be reliable nodes when constructing brain networks or connectome since they simultaneously possess commonality and individuality across different individual brains and populations. However, 3HGs are identified and validated within individual spaces, making it difficult to directly serve as the brain network nodes due to the absence of cross-subject correspondence. The 3HG correspondences represent the intrinsic regulation of brain organizational architecture, traditional image-based registration methods tend to fail because individual anatomical properties need to be fully respected. To address this challenge, we propose a novel self-supervised framework for anatomical feature embedding of the 3HGs to build the correspondences among different brains. The core component of this framework is to construct a structural similarity-enhanced multi-hop feature encoding strategy based on the recently developed Kolmogorov-Arnold network (KAN) for anatomical feature embedding. Extensive experiments suggest that our approach can effectively establish robust cross-subject correspondences when no one-to-one mapping exists.

Large Language Models (LLMs) are increasingly employed for simulations, enabling applications in role-playing agents and Computational Social Science (CSS). However, the reliability of these simulations is under-explored, which raises concerns about the trustworthiness of LLMs in these applications. In this paper, we aim to answer ``How reliable is LLM-based simulation?'' To address this, we introduce TrustSim, an evaluation dataset covering 10 CSS-related topics, to systematically investigate the reliability of the LLM simulation. We conducted experiments on 14 LLMs and found that inconsistencies persist in the LLM-based simulated roles. In addition, the consistency level of LLMs does not strongly correlate with their general performance. To enhance the reliability of LLMs in simulation, we proposed Adaptive Learning Rate Based ORPO (AdaORPO), a reinforcement learning-based algorithm to improve the reliability in simulation across 7 LLMs. Our research provides a foundation for future studies to explore more robust and trustworthy LLM-based simulations.

Autonomous mobile app interaction has become increasingly important with growing complexity of mobile applications. Developing intelligent agents that can effectively navigate and interact with mobile apps remains a significant challenge. In this paper, we propose an Explainable Behavior Cloning LLM Agent (EBC-LLMAgent), a novel approach that combines large language models (LLMs) with behavior cloning by learning demonstrations to create intelligent and explainable agents for autonomous mobile app interaction. EBC-LLMAgent consists of three core modules: Demonstration Encoding, Code Generation, and UI Mapping, which work synergistically to capture user demonstrations, generate executable codes, and establish accurate correspondence between code and UI elements. We introduce the Behavior Cloning Chain Fusion technique to enhance the generalization capabilities of the agent. Extensive experiments on five popular mobile applications from diverse domains demonstrate the superior performance of EBC-LLMAgent, achieving high success rates in task completion, efficient generalization to unseen scenarios, and the generation of meaningful explanations., Comment: 20 pages

Low-Rank Adaptation (LoRA) is a parameter-efficient technique for rapidly fine-tuning foundation models. In standard LoRA training dynamics, models tend to quickly converge to a local optimum near the initialization. However, this local optimum may not be ideal for out-of-distribution data or tasks such as merging and pruning. In this work, we propose a novel progressive training strategy for LoRA with random layer dropping. This strategy also optimizes the Shapley value of LoRA parameters in each layer, treating each layer as a player in a cooperative game. We refer to this method as Cooperative LoRA (CopRA). Our experimental results demonstrate that parameters trained with CopRA exhibit linear mode connectivity, which enables efficient model merging. This also paves the way for federated learning and multi-task learning via LoRA merging. Additionally, by optimizing the Shapley value, CopRA shows superior performance in pruning tasks., Comment: Published in UniReps Workshop (Extended Abstract Track), NeurIPS 2024

Quantum spin Hall effect is characterized by topologically protected helical edge states. Here we study the thermal dissipation of helical edge states by considering two types of dissipation sources. The results show that the helical edge states are dissipationless for normal dissipation sources with or without Rashba spin-orbit coupling in the system, but they are dissipative for spin dissipation sources. Further studies on the energy distribution show that electrons with spin-up and spin-down are both in their own equilibrium without dissipation sources. Spin dissipation sources can couple the two subsystems together to induce voltage drop and nonequilibrium distribution, leading to thermal dissipation, while normal dissipation sources cannot. With the increase of thermal dissipation, the subsystems of electrons with spin-up and spin-down evolve from non-equilibrium finally to mutual equilibrium. In addition, the effects of disorder on thermal dissipation are also discussed. Our work provides clues to reduce thermal dissipation in the quantum spin Hall systems, Comment: 18 pages, 7 figures

Nonuniform strain in graphene can induce a pseudo-magnetic field (PMF) preserving time-reversal symmetry, generating pseudo-Landau levels under zero real magnetic field (MF). The different natures between PMF and real MF lead to the counterpropagating valley-polarized edge states under the PMF and unidirectionally chiral edge states under the real MF. In this work, we find, due to the valley mixing on the armchair edges, the quantum valley Hall edge states only exist at the zigzag edges but not at armchair edges in a uniaxial strained graphene, very different from the case that chiral quantum Hall edge states exist at all edges in pristine graphene under a real MF. We theoretically demonstrate it through the wave function distributions, multi-terminal transport measurements and the electron local occupations, respectively. The interface state in a p-n junction under PMF is further proposed to transport electrons between the conductive zigzag boundaries, which could be used as a valley-polarized single pole double throw switch., Comment: 13 pages, 7 figures

The effectiveness of feedback in enhancing learning outcomes is well documented within Educational Data Mining (EDM). Various prior research has explored methodologies to enhance the effectiveness of feedback. Recent developments in Large Language Models (LLMs) have extended their utility in enhancing automated feedback systems. This study aims to explore the potential of LLMs in facilitating automated feedback in math education. We examine the effectiveness of LLMs in evaluating student responses by comparing 3 different models: Llama, SBERT-Canberra, and GPT4 model. The evaluation requires the model to provide both a quantitative score and qualitative feedback on the student's responses to open-ended math problems. We employ Mistral, a version of Llama catered to math, and fine-tune this model for evaluating student responses by leveraging a dataset of student responses and teacher-written feedback for middle-school math problems. A similar approach was taken for training the SBERT model as well, while the GPT4 model used a zero-shot learning approach. We evaluate the model's performance in scoring accuracy and the quality of feedback by utilizing judgments from 2 teachers. The teachers utilized a shared rubric in assessing the accuracy and relevance of the generated feedback. We conduct both quantitative and qualitative analyses of the model performance. By offering a detailed comparison of these methods, this study aims to further the ongoing development of automated feedback systems and outlines potential future directions for leveraging generative LLMs to create more personalized learning experiences., Comment: 12 pages including references, 4 figures, 9 tables

The wavefront dislocation is an important and ubiquitous phenomenon in wave fields. It is closely related to the phase singularity in a wave function. Some recent studies have verified that the wavefront dislocations in the local density of states (LDOS) map can well manifest the intrinsic topological characteristics in graphene and some topological systems. Different from these previous schemes, we raise a transport method to measure such wavefront dislocations in monolayer and Bernal-stacked bilayer graphene. Combining analytical analysis and numerical calculation, we find phase singularities naturally appear in the transmission coefficients between different sublattices, due to the intervalley interference on the electron propagating paths. These phase singularities could contribute wavefront dislocations in the conductance map. Additionally, in bilayer graphene, the wavefront dislocations are found to remain robust even though the tip is coupled to multiple sublattices. Biased bilayer graphene is also explored. Our scheme provides a new transport routine to explore valley-related topological properties of materials., Comment: 15 pages, 9 figures

The photon-assisted tunneling of Majorana bound states in a Majorana nanowire driven by the periodic field is studied both theoretically and numerically. We find that Majorana bound states exhibit an anomalous photon-assisted tunneling signal which is different from an ordinary fermionic state : the height of photonic sidebands is related to the degree of Majorana nonlocality. Moreover, we show that the Bardeen-Cooper-Schrieffer (BCS) charge and spin components of subgap states can be well revealed by the local conductance. Our work illuminates the effect of driving field on Majorana bound states and provides a systematic scheme to directly measure BCS information of overlapping Majorana bound states, Comment: 14 pages, 8 figures

The quantum Hall (QH) effect, the quantum spin Hall (QSH) effect and the quantum valley Hall (QVH) effect are three peculiar topological insulating phases in graphene. They are characterized by three different types of edge states. These three effects are caused by the external magnetic field, the intrinsic spin-orbit coupling (SOC) and the strain-induced pseudomagnetic field, respectively. Here we theoretically study phase diagrams when these effects coexist and analyze how the edge states evolve between the three. We find the real magnetic field and the pseudomagnetic field will compete above the SOC energy gap while the QSH effect is almost unaffected within the SOC energy gap. The edge states transition from the QH effect or the QVH effect to the QSH effect directly relies on the arrangement of the zeroth Landau levels. Using edge states transitions, we raise a device similar to a spin field effect transistor (spin-FET) and also design a spintronics multiple-way switch., Comment: 16 pages, 10 figures

This paper presents an accurate and robust Structure-from-Motion (SfM) pipeline named LiVisSfM, which is an SfM-based reconstruction system that fully combines LiDAR and visual cues. Unlike most existing LiDAR-inertial odometry (LIO) and LiDAR-inertial-visual odometry (LIVO) methods relying heavily on LiDAR registration coupled with Inertial Measurement Unit (IMU), we propose a LiDAR-visual SfM method which innovatively carries out LiDAR frame registration to LiDAR voxel map in a Point-to-Gaussian residual metrics, combined with a LiDAR-visual BA and explicit loop closure in a bundle optimization way to achieve accurate and robust LiDAR pose estimation without dependence on IMU incorporation. Besides, we propose an incremental voxel updating strategy for efficient voxel map updating during the process of LiDAR frame registration and LiDAR-visual BA optimization. Experiments demonstrate the superior effectiveness of our LiVisSfM framework over state-of-the-art LIO and LIVO works on more accurate and robust LiDAR pose recovery and dense point cloud reconstruction of both public KITTI benchmark and a variety of self-captured dataset., Comment: 18 pages, 9 figures, 2 tables

We prove that the set of thick points of the log-correlated Gaussian field contains an unbounded path in sufficiently high dimensions. This contrasts with the two-dimensional case, where Aru, Papon, and Powell (2023) showed that the set of thick points is totally disconnected. This result has an interesting implication for the exponential metric of the log-correlated Gaussian field: in sufficiently high dimensions, when the parameter $\xi$ is large, the set-to-set distance exponent (if it exists) is negative. This suggests that a new phase may emerge for the exponential metric, which does not appear in two dimensions. In addition, we establish similar results for the set of thick points of the branching random walk. As an intermediate result, we also prove that the critical probability for fractal percolation converges to 0 as $d \to \infty$., Comment: 38 pages, 5 figures

In recent years, quantum computation has been rapidly advancing, driving a technological revolution with significant potential across various sectors, particularly in finance. Despite this, the insurance industry, an essential tool for mitigating unforeseen risks and losses, has received limited attention. This paper provides an initial exploration into the realm of quantum computational insurance and actuarial science. After introducing key insurance models and challenges, we examine quantum algorithms designed to address complex insurance issues. Our study includes experimental and numerical demonstrations of quantum applications in non-life insurance, life insurance, and reinsurance. Additionally, we explore the timeline for quantum insurance, the development of quantum-enhanced insurance products, and the challenges posed by quantum computational advancements., Comment: 10 pages, 5 figures

Despite the impressive generative abilities of black-box large language models (LLMs), their inherent opacity hinders further advancements in capabilities such as reasoning, planning, and personalization. Existing works aim to enhance LLM capabilities via domain-specific adaptation or in-context learning, which require additional training on accessible model parameters, an infeasible option for black-box LLMs. To address this challenge, we introduce Matryoshika, a lightweight white-box LLM controller that guides a large-scale black-box LLM generator by decomposing complex tasks into a series of intermediate outputs. Specifically, we consider the black-box LLM as an environment, with Matryoshika serving as a policy to provide intermediate guidance through prompts for driving the black-box LLM. Matryoshika is trained to pivot the outputs of the black-box LLM aligning with preferences during iterative interaction, which enables controllable multi-turn generation and self-improvement in optimizing intermediate guidance. Empirical evaluations on three diverse tasks demonstrate that Matryoshika effectively enhances the capabilities of black-box LLMs in complex, long-horizon tasks, including reasoning, planning, and personalization. By leveraging this pioneering controller-generator framework to mitigate dependence on model parameters, Matryoshika provides a transparent and practical solution for improving black-box LLMs through controllable multi-turn generation using white-box LLMs., Comment: Work in Progress

Test-Time Adaptation (TTA) aims to help pre-trained model bridge the gap between source and target datasets using only the pre-trained model and unlabelled test data. A key objective of TTA is to address domain shifts in test data caused by corruption, such as weather changes, noise, or sensor malfunctions. Multi-Modal Continual Test-Time Adaptation (MM-CTTA), an extension of TTA with better real-world applications, further allows pre-trained models to handle multi-modal inputs and adapt to continuously-changing target domains. MM-CTTA typically faces challenges including error accumulation, catastrophic forgetting, and reliability bias, with few existing approaches effectively addressing these issues in multi-modal corruption scenarios. In this paper, we propose a novel approach, Multi-modality Dynamic Analytic Adapter (MDAA), for MM-CTTA tasks. We innovatively introduce analytic learning into TTA, using the Analytic Classifiers (ACs) to prevent model forgetting. Additionally, we develop Dynamic Selection Mechanism (DSM) and Soft Pseudo-label Strategy (SPS), which enable MDAA to dynamically filter reliable samples and integrate information from different modalities. Extensive experiments demonstrate that MDAA achieves state-of-the-art performance on MM-CTTA tasks while ensuring reliable model adaptation.

High-energy vortex $\gamma$ photons have significant applications in many fields, however, their generation and angular momentum manipulation are still great challenges. Here, we first investigated the generation of vortex $\gamma$ photons with controllable spin and orbital angular momenta via nonlinear Compton scattering of two-color counter-rotating circularly polarized (CP) laser fields. The radiation probability of vortex photons are calculated using the semiclassical approach that resolves angular momenta of emitted photons. We find that the angular momenta transferred to emitted photons are determined by the dominating photon absorption channel, leading to a structured spectrum with alternations in chirality and twisted direction. By tuning the relative intensity ratio of the two-color CP laser fields, the polarization and vortex charge of the emitted $\gamma$ photons can be controlled, enabling the generation of circularly polarized vortex $\gamma$ photons with a user-defined polarization and topological charge, which may have a plenty of applications in nuclear physics, astrophysics, particle physics, etc.

In this paper, we study the evolution of the zero-temperature random field Ising model as the mean of the external field $M$ increases from $-\infty$ to $\infty$. We focus on two types of evolutions: the ground state evolution and the Glauber evolution. For the ground state evolution, we investigate the occurrence of global avalanche, a moment where a large fraction of spins flip simultaneously from minus to plus. In two dimensions, no global avalanche occurs, while in three or higher dimensions, there is a phase transition: a global avalanche happens when the noise intensity is small, but not when it is large. Additionally, we study the zero-temperature Glauber evolution, where spins are updated locally to minimize the Hamiltonian. Our results show that for small noise intensity, in dimensions $d =2$ or $3$, most spins flip around a critical time $c_d = \frac{2 \sqrt{d}}{1 + \sqrt{d}}$ (but we cannot decide whether such flipping occurs simultaneously or not). We also connect this process to polluted bootstrap percolation and solve an open problem on it., Comment: 41 pages, 9 figures; updated references on polluted bootstrap percolation

GPT-4o is an autoregressive omni model that accepts as input any combination of text, audio, image, and video, and generates any combination of text, audio, and image outputs. It's trained end-to-end across text, vision, and audio, meaning all inputs and outputs are processed by the same neural network. GPT-4o can respond to audio inputs in as little as 232 milliseconds, with an average of 320 milliseconds, which is similar to human response time in conversation. It matches GPT-4 Turbo performance on text in English and code, with significant improvement on text in non-English languages, while also being much faster and 50\% cheaper in the API. GPT-4o is especially better at vision and audio understanding compared to existing models. In line with our commitment to building AI safely and consistent with our voluntary commitments to the White House, we are sharing the GPT-4o System Card, which includes our Preparedness Framework evaluations. In this System Card, we provide a detailed look at GPT-4o's capabilities, limitations, and safety evaluations across multiple categories, focusing on speech-to-speech while also evaluating text and image capabilities, and measures we've implemented to ensure the model is safe and aligned. We also include third-party assessments on dangerous capabilities, as well as discussion of potential societal impacts of GPT-4o's text and vision capabilities.

Accurate and robust initialization is essential for Visual-Inertial Odometry (VIO), as poor initialization can severely degrade pose accuracy. During initialization, it is crucial to estimate parameters such as accelerometer bias, gyroscope bias, initial velocity, and gravity, etc. The IMU sensor requires precise estimation of gyroscope bias because gyroscope bias affects rotation, velocity and position. Most existing VIO initialization methods adopt Structure from Motion (SfM) to solve for gyroscope bias. However, SfM is not stable and efficient enough in fast motion or degenerate scenes. To overcome these limitations, we extended the rotation-translation-decoupling framework by adding new uncertainty parameters and optimization modules. First, we adopt a gyroscope bias optimizer that incorporates probabilistic normal epipolar constraints. Second, we fuse IMU and visual measurements to solve for velocity, gravity, and scale efficiently. Finally, we design an additional refinement module that effectively diminishes gravity and scale errors. Extensive initialization tests on the EuRoC dataset show that our method reduces the gyroscope bias and rotation estimation error by an average of 16% and 4% respectively. It also significantly reduces the gravity error, with an average reduction of 29%.

In this paper, we show that for a non-trivial quasi-symmetric $2$-design $\mathcal{D}$ with two block intersection numbers $x=0$ and $2\leq y\leq10$, if $G\leq \mathrm{Aut}(\mathcal{D})$ is flag-transitive and point-primitive, then $G$ is either of affine type or almost simple type. Moreover, we prove that the socle of $G$ cannot be an alternating group. If the socle of $G$ is a sporadic group, then $\mathcal{D}$ and $G$ must be one of the following: $\mathcal{D}$ is a $2$-$(12,6,5)$ design with block intersection numbers $0,3$ and $G=\mathrm{M}_{11}$, or $\mathcal{D}$ is a $2$-$(22,6,5)$ design with block intersection numbers $0,2$ and $G=\mathrm{M}_{22}$ or $\mathrm{M}_{22}:2$.

As trustworthy AI continues to advance, the fairness issue in recommendations has received increasing attention. A recommender system is considered unfair when it produces unequal outcomes for different user groups based on user-sensitive attributes (e.g., age, gender). Some researchers have proposed data augmentation-based methods aiming at alleviating user-level unfairness by altering the skewed distribution of training data among various user groups. Despite yielding promising results, they often rely on fairness-related assumptions that may not align with reality, potentially reducing the data quality and negatively affecting model effectiveness. To tackle this issue, in this paper, we study how to implement high-quality data augmentation to improve recommendation fairness. Specifically, we propose FairDgcl, a dynamic graph adversarial contrastive learning framework aiming at improving fairness in recommender system. First, FairDgcl develops an adversarial contrastive network with a view generator and a view discriminator to learn generating fair augmentation strategies in an adversarial style. Then, we propose two dynamic, learnable models to generate contrastive views within contrastive learning framework, which automatically fine-tune the augmentation strategies. Meanwhile, we theoretically show that FairDgcl can simultaneously generate enhanced representations that possess both fairness and accuracy. Lastly, comprehensive experiments conducted on four real-world datasets demonstrate the effectiveness of the proposed FairDgcl., Comment: 12 pages, submitted to TKDE

Multimodal Large Language Models (MLLMs) have shown promising progress in understanding and analyzing video content. However, processing long videos remains a significant challenge constrained by LLM's context size. To address this limitation, we propose LongVU, a spatiotemporal adaptive compression mechanism thats reduces the number of video tokens while preserving visual details of long videos. Our idea is based on leveraging cross-modal query and inter-frame dependencies to adaptively reduce temporal and spatial redundancy in videos. Specifically, we leverage DINOv2 features to remove redundant frames that exhibit high similarity. Then we utilize text-guided cross-modal query for selective frame feature reduction. Further, we perform spatial token reduction across frames based on their temporal dependencies. Our adaptive compression strategy effectively processes a large number of frames with little visual information loss within given context length. Our LongVU consistently surpass existing methods across a variety of video understanding benchmarks, especially on hour-long video understanding tasks such as VideoMME and MLVU. Given a light-weight LLM, our LongVU also scales effectively into a smaller size with state-of-the-art video understanding performance., Comment: Project page: https://vision-cair.github.io/LongVU

Zero-Shot Composed Image Retrieval (ZS-CIR) supports diverse tasks with a broad range of visual content manipulation intentions that can be related to domain, scene, object, and attribute. A key challenge for ZS-CIR is to accurately map image representation to a pseudo-word token that captures the manipulation intention relevant image information for generalized CIR. However, existing methods between the retrieval and pre-training stages lead to significant redundancy in the pseudo-word tokens. In this paper, we propose a novel denoising image-to-word mapping approach, named Denoise-I2W, for mapping images into denoising pseudo-word tokens that, without intention-irrelevant visual information, enhance accurate ZS-CIR. Specifically, a pseudo triplet construction module first automatically constructs pseudo triples (\textit{i.e.,} a pseudo-reference image, a pseudo-manipulation text, and a target image) for pre-training the denoising mapping network. Then, a pseudo-composed mapping module maps the pseudo-reference image to a pseudo-word token and combines it with the pseudo-manipulation text with manipulation intention. This combination aligns with the target image, facilitating denoising intention-irrelevant visual information for mapping. Our proposed Denoise-I2W is a model-agnostic and annotation-free approach. It demonstrates strong generalization capabilities across three state-of-the-art ZS-CIR models on four benchmark datasets. By integrating Denoise-I2W with existing best models, we obtain consistent and significant performance boosts ranging from 1.45\% to 4.17\% over the best methods without increasing inference costs. and achieve new state-of-the-art results on ZS-CIR. Our code is available at \url{https://github.com/Pter61/denoise-i2w-tmm}., Comment: This work was submitted to IJCAI 2024, with a score of weak accept and borderline accept