Search

Recently, large language model (LLM)-based agents have made significant advances across various fields. One of the most popular research areas involves applying these agents to video games. Traditionally, these methods have relied on game APIs to access in-game environmental and action data. However, this approach is limited by the availability of APIs and does not reflect how humans play games. With the advent of vision language models (VLMs), agents now have enhanced visual understanding capabilities, enabling them to interact with games using only visual inputs. Despite these advances, current approaches still face challenges in action-oriented tasks, particularly in action role-playing games (ARPGs), where reinforcement learning methods are prevalent but suffer from poor generalization and require extensive training. To address these limitations, we select an ARPG, ``Black Myth: Wukong'', as a research platform to explore the capability boundaries of existing VLMs in scenarios requiring visual-only input and complex action output. We define 12 tasks within the game, with 75% focusing on combat, and incorporate several state-of-the-art VLMs into this benchmark. Additionally, we will release a human operation dataset containing recorded gameplay videos and operation logs, including mouse and keyboard actions. Moreover, we propose a novel VARP (Vision Action Role-Playing) agent framework, consisting of an action planning system and a visual trajectory system. Our framework demonstrates the ability to perform basic tasks and succeed in 90% of easy and medium-level combat scenarios. This research aims to provide new insights and directions for applying multimodal agents in complex action game environments. The code and datasets will be made available at https://varp-agent.github.io/.

In this work, we develop a new computational framework to solve sequential Bayesian experimental design (SBOED) problems constrained by large-scale partial differential equations with infinite-dimensional random parameters. We propose an adaptive terminal formulation of the optimality criteria for SBOED to achieve adaptive global optimality. We also establish an equivalent optimization formulation to achieve computational simplicity enabled by Laplace and low-rank approximations of the posterior. To accelerate the solution of the SBOED problem, we develop a derivative-informed latent attention neural operator (LANO), a new neural network surrogate model that leverages (1) derivative-informed dimension reduction for latent encoding, (2) an attention mechanism to capture the dynamics in the latent space, (3) an efficient training in the latent space augmented by projected Jacobian, which collectively lead to an efficient, accurate, and scalable surrogate in computing not only the parameter-to-observable (PtO) maps but also their Jacobians. We further develop the formulation for the computation of the MAP points, the eigenpairs, and the sampling from posterior by LANO in the reduced spaces and use these computations to solve the SBOED problem. We demonstrate the superior accuracy of LANO compared to two other neural architectures and the high accuracy of LANO compared to the finite element method (FEM) for the computation of MAP points in solving the SBOED problem with application to the experimental design of the time to take MRI images in monitoring tumor growth.

Differential spatial modulation (DSM) exploits the time dimension to facilitate the differential modulation, which can perfectly avoid the challenge in acquiring of heavily entangled channel state information of visible light communication (VLC) system. However, it has huge search space and high complexity for large number of transmitters. In this paper, a novel vector correction (VC)-based orthogonal matching pursuit (OMP) detection algorithm is proposed to reduce the complexity, which exploits the sparsity and relativity of all transmitters, and then employs a novel correction criterion by correcting the index vectors of the error estimation for improving the demodulation performance. To overcome the local optimum dilemma in the atoms searching, an OMP-assisted genetic algorithm is also proposed to further improve the bit error rate (BER) performance of the VLC-DSM system. Simulation results demonstrate that the proposed schemes can significantly reduce the computational complexity at least by 62.5% while achieving an excellent BER performance as compared with traditional maximum likelihood based receiver., Comment: This paper has been accepted by 2024 IEEE 24th International Conference on Communication Technology (ICCT 2024)

Locating manipulation maps, i.e., pixel-level annotation of forgery cues, is crucial for providing interpretable detection results in face forgery detection. Related learning objects have also been widely adopted as auxiliary tasks to improve the classification performance of detectors whereas they require comparisons between paired real and forged faces to obtain manipulation maps as supervision. This requirement restricts their applicability to unpaired faces and contradicts real-world scenarios. Moreover, the used comparison methods annotate all changed pixels, including noise introduced by compression and upsampling. Using such maps as supervision hinders the learning of exploitable cues and makes models prone to overfitting. To address these issues, we introduce a weakly supervised model in this paper, named Forgery Cue Discovery (FoCus), to locate forgery cues in unpaired faces. Unlike some detectors that claim to locate forged regions in attention maps, FoCus is designed to sidestep their shortcomings of capturing partial and inaccurate forgery cues. Specifically, we propose a classification attentive regions proposal module to locate forgery cues during classification and a complementary learning module to facilitate the learning of richer cues. The produced manipulation maps can serve as better supervision to enhance face forgery detectors. Visualization of the manipulation maps of the proposed FoCus exhibits superior interpretability and robustness compared to existing methods. Experiments on five datasets and four multi-task models demonstrate the effectiveness of FoCus in both in-dataset and cross-dataset evaluations., Comment: TIFS 2024

Topological photonics has revolutionized manipulations of electromagnetic waves by leveraging various topological phases proposed originally in condensed matters, leading to robust and error-immune signal processing. Despite considerable efforts, a critical challenge remains in devising frequency routers operating at a broadband frequency range with limited crosstalk. Previous designs usually relied on fine tuning of parameters and are difficult to be integrated efficiently and compactly. Here, targeting the demand for frequency-selective applications in on-chip photonics, we explore a topological approach to photonic frequency router via valley-Hall metacrystals. Diverging from the majority of studies which focuses on zigzag interfaces, our research shifts the attention to armchair interfaces within an ABA sandwich-like structure, where a single column of type-B metacrystal acts as a perturbation in the background type-A metacrystal. Essentially, through tuning a single geometric parameter of the type-B metacrystal, this configuration gives rise to interface states within a customized frequency band, enabling signal routing with limited crosstalk to meet specified demands. Moreover, this concept is practically demonstrated through a photonic frequency router with three distinct channels, experimentally exhibiting robust wave transmissions with excellent agreement with the design. This investigation manifests possible applications of the armchair interfaces in valley-Hall photonic systems and advances development of photonic devices that are both compact and efficient. Notably, the approach is naturally compatible with on-chip photonics and integration, which could benefit telecommunications and optical computing applications.

Accurate modeling and prediction of complex physical systems often rely on data assimilation techniques to correct errors inherent in model simulations. Traditional methods like the Ensemble Kalman Filter (EnKF) and its variants as well as the recently developed Ensemble Score Filters (EnSF) face significant challenges when dealing with high-dimensional and nonlinear Bayesian filtering problems with sparse observations, which are ubiquitous in real-world applications. In this paper, we propose a novel data assimilation method, Latent-EnSF, which leverages EnSF with efficient and consistent latent representations of the full states and sparse observations to address the joint challenges of high dimensionlity in states and high sparsity in observations for nonlinear Bayesian filtering. We introduce a coupled Variational Autoencoder (VAE) with two encoders to encode the full states and sparse observations in a consistent way guaranteed by a latent distribution matching and regularization as well as a consistent state reconstruction. With comparison to several methods, we demonstrate the higher accuracy, faster convergence, and higher efficiency of Latent-EnSF for two challenging applications with complex models in shallow water wave propagation and medium-range weather forecasting, for highly sparse observations in both space and time., Comment: 13 pages, 10 figures, 1 table

This work considers the computation of risk measures for quantities of interest governed by PDEs with Gaussian random field parameters using Taylor approximations. While efficient, Taylor approximations are local to the point of expansion, and hence may degrade in accuracy when the variances of the input parameters are large. To address this challenge, we approximate the underlying Gaussian measure by a mixture of Gaussians with reduced variance in a dominant direction of parameter space. Taylor approximations are constructed at the means of each Gaussian mixture component, which are then combined to approximate the risk measures. The formulation is presented in the setting of infinite-dimensional Gaussian random parameters for risk measures including the mean, variance, and conditional value-at-risk. We also provide detailed analysis of the approximations errors arising from two sources: the Gaussian mixture approximation and the Taylor approximations. Numerical experiments are conducted for a semilinear advection-diffusion-reaction equation with a random diffusion coefficient field and for the Helmholtz equation with a random wave speed field. For these examples, the proposed approximation strategy can achieve less than $1\%$ relative error in estimating CVaR with only $\mathcal{O}(10)$ state PDE solves, which is comparable to a standard Monte Carlo estimate with $\mathcal{O}(10^4)$ samples, thus achieving significant reduction in computational cost. The proposed method can therefore serve as a way to rapidly and accurately estimate risk measures under limited computational budgets., Comment: 34 Pages, 13 Figures, 1 Table

In this article we investigate $L^p$ boundedness of the spherical maximal operator $\mathfrak{m}^\alpha$ of (complex) order $\alpha$ on the $n$-dimensional hyperbolic space $\mathbb{H}^n$, which was introduced and studied by Kohen [13]. We prove that when $n\geq 2$, for $\alpha\in\mathbb{R}$ and $11-n+n/p$ for $1 \max \{{(2-n)/p}-{1/(p p_n)}, \ {(2-n)/p} - (p-2)/ [p p_n(p_n-2) ] \} $ for $2\leq p\leq \infty$, with $p_n=2(n+1)/(n-1)$ for $n\geq 3$ and $p_n=4$ for $n=2$.

Big mobility datasets (BMD) have shown many advantages in studying human mobility and evaluating the performance of transportation systems. However, the quality of BMD remains poorly understood. This study evaluates biases in BMD and develops mitigation methods. Using Google and Apple mobility data as examples, this study compares them with benchmark data from governmental agencies. Spatio-temporal discrepancies between BMD and benchmark are observed and their impacts on transportation applications are investigated, emphasizing the urgent need to address these biases to prevent misguided policymaking. This study further proposes and tests a bias mitigation method. It is shown that the mitigated BMD could generate valuable insights into large-scale public transit systems across 100+ US counties, revealing regional disparities of the recovery of transit systems from the COVID-19. This study underscores the importance of caution when using BMD in transportation research and presents effective mitigation strategies that would benefit practitioners., Comment: 25 pages, 10 figures. Transportation Letters. August 2024

We apply a Lindeberg principle under the Markov process setting to approximate the Wright-Fisher model with neutral $r$-alleles using a diffusion process, deriving an error rate based on a function class distance involving fourth-order bounded differentiable functions. This error rate consists of a linear combination of the maximum mutation rate and the reciprocal of the population size. Our result improves the error bound in the seminal work [PNAS,1977], where only the special case $r=2$ was studied.

This paper investigates a data-locality-aware task assignment and scheduling problem aimed at minimizing job completion times for distributed job executions. Without prior knowledge of future job arrivals, we propose an optimal balanced task assignment algorithm (OBTA) that minimizes the completion time of each arriving job. We significantly reduce OBTA's computational overhead by narrowing the search space of potential solutions. Additionally, we extend an approximate algorithm known as water-filling (WF) and nontrivially prove that its approximation factor equals the number of task groups in the job assignment. We also design a novel heuristic, replica-deletion (RD), which outperforms WF. To further reduce the completion time of each job, we expand the problem to include job reordering, where we adjust the order of outstanding jobs following the shortest-estimated-time-first policy. Extensive trace-driven evaluations validate the performance and efficiency of the proposed algorithms.

In the field of evolutionary multi-objective optimization, the approximation of the Pareto front (PF) is achieved by utilizing a collection of representative candidate solutions that exhibit desirable convergence and diversity. Although several multi-objective evolutionary algorithms (MOEAs) have been designed, they still have difficulties in keeping balance between convergence and diversity of population. To better solve multi-objective optimization problems (MOPs), this paper proposes a Two-stage Evolutionary Framework For Multi-objective Optimization (TEMOF). Literally, algorithms are divided into two stages to enhance the search capability of the population. During the initial half of evolutions, parental selection is exclusively conducted from the primary population. Additionally, we not only perform environmental selection on the current population, but we also establish an external archive to store individuals situated on the first PF. Subsequently, in the second stage, parents are randomly chosen either from the population or the archive. In the experiments, one classic MOEA and two state-of-the-art MOEAs are integrated into the framework to form three new algorithms. The experimental results demonstrate the superior and robust performance of the proposed framework across a wide range of MOPs. Besides, the winner among three new algorithms is compared with several existing MOEAs and shows better results. Meanwhile, we conclude the reasons that why the two-stage framework is effect for the existing benchmark functions., Comment: Accepted by the CEC conference of WCCI2024

Understanding how elemental abundances evolve during solar flares helps shed light on the mass and energy transfer between different solar atmospheric layers. However, prior studies have mostly concentrated on averaged abundances or specific flare phases, leaving a gap in exploring the comprehensive observations throughout the entire flare process. Consequently, investigations into this area are relatively scarce. Exploiting the Solar X-ray Monitor data obtained from the Chang'E-2 lunar orbiter, we present two comprehensive soft X-ray spectroscopic observations of flares in active regions, AR 11149 and 11158, demonstrating elemental abundance evolutions under different conditions. Our findings unveil the inverse first ionization potential (IFIP) effect during flares for Fe for the first time, and reaffirm its existence for Si. Additionally, we observed a rare depletion of elemental abundances, marking the second IFIP effect in flare decay phases. Our study offers a CSHKP model-based interpretation to elucidate the formation of both the FIP and IFIP effects in flare dynamics, with the inertia effect being incorporated into the ponderomotive force fractionation model., Comment: Accepted ApJ

We present Ksformer, utilizing Multi-scale Key-select Routing Attention (MKRA) for intelligent selection of key areas through multi-channel, multi-scale windows with a top-k operator, and Lightweight Frequency Processing Module (LFPM) to enhance high-frequency features, outperforming other dehazing methods in tests., Comment: 5 pages,4 figures,IEICE Trans. Information and Systems

Vertical Federated Learning (VFL) facilitates collaborative machine learning without the need for participants to share raw private data. However, recent studies have revealed privacy risks where adversaries might reconstruct sensitive features through data leakage during the learning process. Although data reconstruction methods based on gradient or model information are somewhat effective, they reveal limitations in VFL application scenarios. This is because these traditional methods heavily rely on specific model structures and/or have strict limitations on application scenarios. To address this, our study introduces the Unified InverNet Framework into VFL, which yields a novel and flexible approach (dubbed UIFV) that leverages intermediate feature data to reconstruct original data, instead of relying on gradients or model details. The intermediate feature data is the feature exchanged by different participants during the inference phase of VFL. Experiments on four datasets demonstrate that our methods significantly outperform state-of-the-art techniques in attack precision. Our work exposes severe privacy vulnerabilities within VFL systems that pose real threats to practical VFL applications and thus confirms the necessity of further enhancing privacy protection in the VFL architecture.

In this paper, we propose a textless acoustic model with a self-supervised distillation strategy for noise-robust expressive speech-to-speech translation (S2ST). Recently proposed expressive S2ST systems have achieved impressive expressivity preservation performances by cascading unit-to-speech (U2S) generator to the speech-to-unit translation model. However, these systems are vulnerable to the presence of noise in input speech, which is an assumption in real-world translation scenarios. To address this limitation, we propose a U2S generator that incorporates a distillation with no label (DINO) self-supervised training strategy into it's pretraining process. Because the proposed method captures noise-agnostic expressivity representation, it can generate qualified speech even in noisy environment. Objective and subjective evaluation results verified that the proposed method significantly improved the performance of the expressive S2ST system in noisy environments while maintaining competitive performance in clean environments., Comment: Accepted to ACL 2024 (findings)

Estimating the parameters of compact binaries which coalesce and produce gravitational waves is a challenging Bayesian inverse problem. Gravitational-wave parameter estimation lies within the class of multifidelity problems, where a variety of models with differing assumptions, levels of fidelity, and computational cost are available for use in inference. In an effort to accelerate the solution of a Bayesian inverse problem, cheaper surrogates for the best models may be used to reduce the cost of likelihood evaluations when sampling the posterior. Importance sampling can then be used to reweight these samples to represent the true target posterior, incurring a reduction in the effective sample size. In cases when the problem is high dimensional, or when the surrogate model produces a poor approximation of the true posterior, this reduction in effective samples can be dramatic and render multifidelity importance sampling ineffective. We propose a novel method of tempered multifidelity importance sampling in order to remedy this issue. With this method the biasing distribution produced by the low-fidelity model is tempered, allowing for potentially better overlap with the target distribution. There is an optimal temperature which maximizes the efficiency in this setting, and we propose a low-cost strategy for approximating this optimal temperature using samples from the untempered distribution. In this paper, we motivate this method by applying it to Gaussian target and biasing distributions. Finally, we apply it to a series of problems in gravitational wave parameter estimation and demonstrate improved efficiencies when applying the method to real gravitational wave detections., Comment: 19 pages, 10 figures, 1 table

With the increasing collection of time series data from various domains, there arises a strong demand for general time series forecasting models pre-trained on a large number of time-series datasets to support a variety of downstream prediction tasks. Enabling general time series forecasting faces two challenges: how to obtain unified representations from multi-domian time series data, and how to capture domain-specific features from time series data across various domains for adaptive transfer in downstream tasks. To address these challenges, we propose a Register Assisted General Time Series Forecasting Model with Decomposed Frequency Learning (ROSE), a novel pre-trained model for time series forecasting. ROSE employs Decomposed Frequency Learning for the pre-training task, which decomposes coupled semantic and periodic information in time series with frequency-based masking and reconstruction to obtain unified representations across domains. We also equip ROSE with a Time Series Register, which learns to generate a register codebook to capture domain-specific representations during pre-training and enhances domain-adaptive transfer by selecting related register tokens on downstream tasks. After pre-training on large-scale time series data, ROSE achieves state-of-the-art forecasting performance on 8 real-world benchmarks. Remarkably, even in few-shot scenarios, it demonstrates competitive or superior performance compared to existing methods trained with full data.

Editing objects within a scene is a critical functionality required across a broad spectrum of applications in computer vision and graphics. As 3D Gaussian Splatting (3DGS) emerges as a frontier in scene representation, the effective modification of 3D Gaussian scenes has become increasingly vital. This process entails accurately retrieve the target objects and subsequently performing modifications based on instructions. Though available in pieces, existing techniques mainly embed sparse semantics into Gaussians for retrieval, and rely on an iterative dataset update paradigm for editing, leading to over-smoothing or inconsistency issues. To this end, this paper proposes a systematic approach, namely TIGER, for coherent text-instructed 3D Gaussian retrieval and editing. In contrast to the top-down language grounding approach for 3D Gaussians, we adopt a bottom-up language aggregation strategy to generate a denser language embedded 3D Gaussians that supports open-vocabulary retrieval. To overcome the over-smoothing and inconsistency issues in editing, we propose a Coherent Score Distillation (CSD) that aggregates a 2D image editing diffusion model and a multi-view diffusion model for score distillation, producing multi-view consistent editing with much finer details. In various experiments, we demonstrate that our TIGER is able to accomplish more consistent and realistic edits than prior work.

In the realm of autonomous driving, robust perception under out-of-distribution conditions is paramount for the safe deployment of vehicles. Challenges such as adverse weather, sensor malfunctions, and environmental unpredictability can severely impact the performance of autonomous systems. The 2024 RoboDrive Challenge was crafted to propel the development of driving perception technologies that can withstand and adapt to these real-world variabilities. Focusing on four pivotal tasks -- BEV detection, map segmentation, semantic occupancy prediction, and multi-view depth estimation -- the competition laid down a gauntlet to innovate and enhance system resilience against typical and atypical disturbances. This year's challenge consisted of five distinct tracks and attracted 140 registered teams from 93 institutes across 11 countries, resulting in nearly one thousand submissions evaluated through our servers. The competition culminated in 15 top-performing solutions, which introduced a range of innovative approaches including advanced data augmentation, multi-sensor fusion, self-supervised learning for error correction, and new algorithmic strategies to enhance sensor robustness. These contributions significantly advanced the state of the art, particularly in handling sensor inconsistencies and environmental variability. Participants, through collaborative efforts, pushed the boundaries of current technologies, showcasing their potential in real-world scenarios. Extensive evaluations and analyses provided insights into the effectiveness of these solutions, highlighting key trends and successful strategies for improving the resilience of driving perception systems. This challenge has set a new benchmark in the field, providing a rich repository of techniques expected to guide future research in this field., Comment: ICRA 2024; 32 pages, 24 figures, 5 tables; Code at https://robodrive-24.github.io/

To accurately predict the synthesis cross-sections of superheavy elements, identifying the optimal projectile-target combinations and the evaporation channels at specific collision energies, we have attempted to utilize high-quality microscopic nuclear masses (HQMNM) within the dinuclear system (DNS) model, which are obtained by fitting experimental data with the Skyrme energy density functional theory (DFT), as published in Phys. Lett. B 851 (2024) 138578. The atomic nuclear mass serves as a crucial input for the DNS model, as the Q-values and separation energies it generates directly influence the calculated fusion and survival probabilities. Our calculations have reproduced the experimental data for hot fusion and have been compared with results based on the finite-range droplet model (FRDM12) mass calculations. Compared to the FRDM12 mass results, we have found that the HQMNM provides a better fit to the experimental outcomes. For the specific reaction of \(^{48}\rm{Ca} + ^{243}\rm{Am} \rightarrow ^{291}\rm{Mc}^*\), we have conducted a detailed calculation of capture, fusion, and survival based on the HQMNM model and compared these with calculations based on other mass models. Based on these findings, we have systematically calculated available projectile target combinations for the synthesis of elements 119 and 120, and identified the optimal combinations. We provided the synthesis cross-sections, collision energies, and evaporation channels, offering a reference for conducting experiments on the synthesis of superheavy elements., Comment: 8 pages, 5 figures

A novel multicast communication system with movable antennas (MAs) is proposed, where the antenna position optimization is exploited to enhance the transmission rate. Specifically, an MA-assisted two-user multicast multiple-input single-input system is considered. The joint optimization of the transmit beamforming vector and transmit MA positions is studied by modeling the motion of the MA elements as discrete movements. A low-complexity greedy search-based algorithm is proposed to tackle this non-convex inter-programming problem. A branch-and-bound (BAB)-based method is proposed to achieve the optimal multicast rate with a reduced time complexity than the brute-force search by assuming the two users suffer similar line-of-sight path losses. Numerical results reveal that the proposed MA systems significantly improve the multicast rate compared to conventional fixed-position antennas (FPAs)-based systems., Comment: 5 pages

For $p\ge 2$, and $\lambda>\max\{n|\tfrac 1p-\tfrac 12|-\tfrac12, 0\}$, we prove the pointwise convergence of cone multipliers, i.e. $$ \lim_{t\to\infty}T_t^\lambda(f)\to f \text{ a.e.},$$ where $f\in L^p(\mathbb R^n)$ satisfies $supp\ \widehat f\subset\{\xi\in\mathbb R^n:\ 1<|\xi_n|<2\}$. Our main tools are weighted estimates for maximal cone operators, which are consequences of trace inequalities for cones.

To assess the potential of cold-fusion for synthesizing superheavy nuclei (SHN) with proton numbers 104-113, we systematically calculated 145 naturally occurring projectile-target combinations within the DNS model. Reactions predominantly show maximum cross-sections in the 1n to 2n channels, peaking near the Coulomb barrier with a sum of barrier and Q-value within 30 MeV. The maximum cross-section occurs below the Bass barrier, suggesting either the Bass model's limitation or significant deformation reducing the effective Coulomb barrier. Our calculations align well with experimental data, revealing that more neutron-rich projectiles slightly enhance fusion, though the effect is minor. For fixed targets (Pb, Bi), evaporation residue cross-sections decrease linearly with increasing projectile proton number, attributed to reduced fusion probability and lower fission barriers in heavier SHN. The touching potential $V_{\rm in}$ shows a linear trend with the product of projectile-target proton numbers, with neutron-rich systems exhibiting lower $V_{\rm in}$. Some reactions with $V_{\rm in} < V_{\rm S}$ may involve nucleon transfer before capture. Based on the DNS model, we identified optimal combinations and collision energies for synthesizing SHN with significant cross-sections. Collectively, our findings indicate that cold fusion is a promising avenue for creating proton-rich SHN around the drip line in the Z=104-113 region, offering distinct advantages over alternative mechanisms., Comment: 10 pages, 6 figures

Cross-modality images that integrate visible-infrared spectra cues can provide richer complementary information for object detection. Despite this, existing visible-infrared object detection methods severely degrade in severe weather conditions. This failure stems from the pronounced sensitivity of visible images to environmental perturbations, such as rain, haze, and snow, which frequently cause false negatives and false positives in detection. To address this issue, we introduce a novel and challenging task, termed visible-infrared object detection under adverse weather conditions. To foster this task, we have constructed a new Severe Weather Visible-Infrared Dataset (SWVID) with diverse severe weather scenes. Furthermore, we introduce the Cross-modality Fusion Mamba with Weather-removal (CFMW) to augment detection accuracy in adverse weather conditions. Thanks to the proposed Weather Removal Diffusion Model (WRDM) and Cross-modality Fusion Mamba (CFM) modules, CFMW is able to mine more essential information of pedestrian features in cross-modality fusion, thus could transfer to other rarer scenarios with high efficiency and has adequate availability on those platforms with low computing power. To the best of our knowledge, this is the first study that targeted improvement and integrated both Diffusion and Mamba modules in cross-modality object detection, successfully expanding the practical application of this type of model with its higher accuracy and more advanced architecture. Extensive experiments on both well-recognized and self-created datasets conclusively demonstrate that our CFMW achieves state-of-the-art detection performance, surpassing existing benchmarks. The dataset and source code will be made publicly available at https://github.com/lhy-zjut/CFMW., Comment: The dataset and source code will be made publicly available at https://github.com/lhy-zjut/CFMW

We investigated the optimal control of a continuous variable system, focusing on entanglement generation in an optomechanical system without utilizing Fock basis cutoffs. Using the Krotov algorithm to optimize the dynamics of the covariance matrix, we illustrated how to design a control objective function to manipulate the dynamics of the system to generate a desirable target state. We showed that entanglement between the macroscopic mechanical mirror and the quantum optical cavity can be reliably generated through imposing the control on the detuning of the external laser field. It has be shown that the control may be still achieved when imposing spectral constraints on the external field to restrict it to low-frequency components. In addition, we systematically studies the effects of quantum control on non-Markovian open system dynamics. We observed that memory effects can play a beneficial role in mitigating the detrimental impact of environmental noises. Specifically, the entanglement generated shows reduced decay in the presence of these memory effects., Comment: 10 pages, 5 figures

We propose a quantum metrology protocol based on a two-step joint evolution of the probe system and an ancillary qubit and a single-shot projective measurement. With an optimized initialization of the ancillary qubit, the quantum Fisher information (QFI) about the phase parameter encoded in the probe system is found to be determined by the expectation value of the square of a time-optimized phase generator, independent of the probe state. Therefore, QFI can approach the Heisenberg scaling $N^2$ with respect to the quantum number $N$, even when the probe system is prepared in a classical state. We find that this scaling behavior is robust against the imperfections in preparing the ancillary qubit and controlling the evolution time. Using the time-reversal strategy, the classical Fisher information (CFI) in our metrology protocol is saturated with its quantum counterpart. Our work thus paves an economical way to realize the Heisenberg-scaling limit in metrology precision with no use of entanglement or squeezing.

Attention-based models are proliferating in the space of image analytics, including segmentation. The standard method of feeding images to transformer encoders is to divide the images into patches and then feed the patches to the model as a linear sequence of tokens. For high-resolution images, e.g. microscopic pathology images, the quadratic compute and memory cost prohibits the use of an attention-based model, if we are to use smaller patch sizes that are favorable in segmentation. The solution is to either use custom complex multi-resolution models or approximate attention schemes. We take inspiration from Adapative Mesh Refinement (AMR) methods in HPC by adaptively patching the images, as a pre-processing step, based on the image details to reduce the number of patches being fed to the model, by orders of magnitude. This method has a negligible overhead, and works seamlessly with any attention-based model, i.e. it is a pre-processing step that can be adopted by any attention-based model without friction. We demonstrate superior segmentation quality over SoTA segmentation models for real-world pathology datasets while gaining a geomean speedup of $6.9\times$ for resolutions up to $64K^2$, on up to $2,048$ GPUs.

Exploration of new dielectrics with large capacitive coupling is an essential topic in modern electronics when conventional dielectrics suffer from the leakage issue near breakdown limit. To address this looming challenge, we demonstrate that rare-earth-metal fluorides with extremely-low ion migration barriers can generally exhibit an excellent capacitive coupling over 20 $\mu$F cm$^{-2}$ (with an equivalent oxide thickness of ~0.15 nm and a large effective dielectric constant near 30) and great compatibility with scalable device manufacturing processes. Such static dielectric capability of superionic fluorides is exemplified by MoS$_2$ transistors exhibiting high on/off current ratios over 10$^8$, ultralow subthreshold swing of 65 mV dec$^{-1}$, and ultralow leakage current density of ~10$^{-6}$ A cm$^{-2}$. Therefore, the fluoride-gated logic inverters can achieve significantly higher static voltage gain values, surpassing ~167, compared to conventional dielectric. Furthermore, the application of fluoride gating enables the demonstration of NAND, NOR, AND, and OR logic circuits with low static energy consumption. Notably, the superconductor-to-insulator transition at the clean-limit Bi$_2$Sr$_2$CaCu$_2$O$_{8+\delta}$ can also be realized through fluoride gating. Our findings highlight fluoride dielectrics as a pioneering platform for advanced electronics applications and for tailoring emergent electronic states in condensed matters., Comment: 33 pages, 5 figures

With the leaping advances in autonomous vehicles and transportation infrastructure, dual function radar-communication (DFRC) systems have become attractive due to the size, cost and resource efficiency. A frequency modulated continuous waveform (FMCW)-based radar-communication system (FRaC) utilizing both sparse multiple-input and multiple-output (MIMO) arrays and index modulation (IM) has been proposed to form a DFRC system specifically designed for vehicular applications. In this paper, the three-dimensional (3D) parameter estimation problem in the FRaC is considered. Since the 3D-parameters including range, direction of arrival (DOA) and velocity are coupled in the estimating matrix of the FRaC system, the existing estimation algorithms cannot estimate the 3D-parameters accurately. Hence, a novel decomposed decoupled atomic norm minimization (DANM) method is proposed by splitting the 3D-parameter estimating matrix into multiple 2D matrices with sparsity constraints. Then, the 3D-parameters are estimated and efficiently and separately with the optimized decoupled estimating matrix. Moreover, the Cram\'{e}r-Rao lower bound (CRLB) of the 3D-parameter estimation are derived, and the computational complexity of the proposed algorithm is analyzed. Simulation results show that the proposed decomposed DANM method exploits the advantage of the virtual aperture in the existence of coupling caused by IM and sparse MIMO array and outperforms the co-estimation algorithm with lower computation complexity.

Applying iterative solvers on sparsity-constrained optimization (SCO) requires tedious mathematical deduction and careful programming/debugging that hinders these solvers' broad impact. In the paper, the library skscope is introduced to overcome such an obstacle. With skscope, users can solve the SCO by just programming the objective function. The convenience of skscope is demonstrated through two examples in the paper, where sparse linear regression and trend filtering are addressed with just four lines of code. More importantly, skscope's efficient implementation allows state-of-the-art solvers to quickly attain the sparse solution regardless of the high dimensionality of parameter space. Numerical experiments reveal the available solvers in skscope can achieve up to 80x speedup on the competing relaxation solutions obtained via the benchmarked convex solver. skscope is published on the Python Package Index (PyPI) and Conda, and its source code is available at: https://github.com/abess-team/skscope., Comment: 4 pages;add experiment

Frequency diverse array multiple-input multiple-output (FDA-MIMO) radar differs from the traditional phased array (PA) radar, and can form range-angle-dependent beampattern and differentiate between closely spaced targets sharing the same angle but occupying distinct range cells. In the FDA-MIMO radar, target range estimation is achieved by employing a subtle frequency variation between adjacent array antennas, so the estimation performance is degraded severely in a practical scenario with frequency offset. In this paper, the range-angle estimation problem for FDA-MIMO radar is considered with frequency offsets in both transmitting and receiving arrays. First, we build a system model for the FDA-MIMO radar with transmitting and receiving frequency offsets. Then, the frequency offset is transferred into an equalized additional noise. The noise characteristics are analyzed in detail theoretically, together with the influence on the range-angle estimation. Moreover, since the effect of the transmitting frequency offset is similar to additional colored noise, denoising algorithms are introduced to mitigate the performance deterioration caused by the frequency offset. Finally, Cram\'{e}r-Rao lower bounds (CRLB) for the range-angle estimation are derived in the scenario with the frequency offsets. Simulation results show the analysis of frequency offset and the corresponding estimation performance using different algorithms.

Deep operator networks (DeepONets), a class of neural operators that learn mappings between function spaces, have recently been developed as surrogate models for parametric partial differential equations (PDEs). In this work we propose a derivative-enhanced deep operator network (DE-DeepONet), which leverages the derivative information to enhance the prediction accuracy, and provide a more accurate approximation of the derivatives, especially when the training data are limited. DE-DeepONet incorporates dimension reduction of input into DeepONet and includes two types of derivative labels in the loss function for training, that is, the directional derivatives of the output function with respect to the input function and the gradient of the output function with respect to the physical domain variables. We test DE-DeepONet on three different equations with increasing complexity to demonstrate its effectiveness compared to the vanilla DeepONet.

Bayesian optimal experimental design (OED) seeks to conduct the most informative experiment under budget constraints to update the prior knowledge of a system to its posterior from the experimental data in a Bayesian framework. Such problems are computationally challenging because of (1) expensive and repeated evaluation of some optimality criterion that typically involves a double integration with respect to both the system parameters and the experimental data, (2) suffering from the curse-of-dimensionality when the system parameters and design variables are high-dimensional, (3) the optimization is combinatorial and highly non-convex if the design variables are binary, often leading to non-robust designs. To make the solution of the Bayesian OED problem efficient, scalable, and robust for practical applications, we propose a novel joint optimization approach. This approach performs simultaneous (1) training of a scalable conditional normalizing flow (CNF) to efficiently maximize the expected information gain (EIG) of a jointly learned experimental design (2) optimization of a probabilistic formulation of the binary experimental design with a Bernoulli distribution. We demonstrate the performance of our proposed method for a practical MRI data acquisition problem, one of the most challenging Bayesian OED problems that has high-dimensional (320 $\times$ 320) parameters at high image resolution, high-dimensional (640 $\times$ 386) observations, and binary mask designs to select the most informative observations.

Neural implicit fields have been a de facto standard in novel view synthesis. Recently, there exist some methods exploring fusing multiple modalities within a single field, aiming to share implicit features from different modalities to enhance reconstruction performance. However, these modalities often exhibit misaligned behaviors: optimizing for one modality, such as LiDAR, can adversely affect another, like camera performance, and vice versa. In this work, we conduct comprehensive analyses on the multimodal implicit field of LiDAR-camera joint synthesis, revealing the underlying issue lies in the misalignment of different sensors. Furthermore, we introduce AlignMiF, a geometrically aligned multimodal implicit field with two proposed modules: Geometry-Aware Alignment (GAA) and Shared Geometry Initialization (SGI). These modules effectively align the coarse geometry across different modalities, significantly enhancing the fusion process between LiDAR and camera data. Through extensive experiments across various datasets and scenes, we demonstrate the effectiveness of our approach in facilitating better interaction between LiDAR and camera modalities within a unified neural field. Specifically, our proposed AlignMiF, achieves remarkable improvement over recent implicit fusion methods (+2.01 and +3.11 image PSNR on the KITTI-360 and Waymo datasets) and consistently surpasses single modality performance (13.8% and 14.2% reduction in LiDAR Chamfer Distance on the respective datasets)., Comment: CVPR2024

Within the framework of the dinuclear system model, this study delves into the impact of various nuclear mass models on evaluating the fusion probability of superheavy nuclei. Nuclear mass models, as crucial inputs to the DNS model, exhibit slight variations in binding energy, quadrupole deformation, and extrapolation ability; these subtle differences can significantly influence the model's outcomes. Specifically, the study finds that nuclear mass plays a pivotal role in determining fusion probability, and Q-value. By numerically solving a set of master equations, the study examines how binding energies from different mass models affect the fusion probability of colliding nuclei, taking the example of $^{48}$Ca + $^{243}$Am $\rightarrow$ $^{291}$Mc. A careful analysis of the potential energy surface (PES) reveals that the inner fusion barriers lead to variations in fusion probabilities. Importantly, the study demonstrates that the synthesis cross sections of superheavy nuclei calculated using different nuclear mass models align well with experimental data, falling within an error range of one order of magnitude. This finding underscores the reliability of our model predictions. Looking ahead, the study utilizes five distinct nuclear mass models to predict the synthesis cross sections of superheavy elements 119 and 120, along with their associated uncertainties. These predictions offer valuable insights into the feasibility of synthesizing these elusive elements and pave the way for future experimental explorations.

Animating virtual characters has always been a fundamental research problem in virtual reality (VR). Facial animations play a crucial role as they effectively convey emotions and attitudes of virtual humans. However, creating such facial animations can be challenging, as current methods often involve utilization of expensive motion capture devices or significant investments of time and effort from human animators in tuning animation parameters. In this paper, we propose a holistic solution to automatically animate virtual human faces. In our solution, a deep learning model was first trained to retarget the facial expression from input face images to virtual human faces by estimating the blendshape coefficients. This method offers the flexibility of generating animations with characters of different appearances and blendshape topologies. Second, a practical toolkit was developed using Unity 3D, making it compatible with the most popular VR applications. The toolkit accepts both image and video as input to animate the target virtual human faces and enables users to manipulate the animation results. Furthermore, inspired by the spirit of Human-in-the-loop (HITL), we leveraged user feedback to further improve the performance of the model and toolkit, thereby increasing the customization properties to suit user preferences. The whole solution, for which we will make the code public, has the potential to accelerate the generation of facial animations for use in VR applications., Comment: 9 pages. To appear in IEEE-VR

Deep denoisers have shown excellent performance in solving inverse problems in signal and image processing. In order to guarantee the convergence, the denoiser needs to satisfy some Lipschitz conditions like non-expansiveness. However, enforcing such constraints inevitably compromises recovery performance. This paper introduces a novel training strategy that enforces a weaker constraint on the deep denoiser called pseudo-contractiveness. By studying the spectrum of the Jacobian matrix, relationships between different denoiser assumptions are revealed. Effective algorithms based on gradient descent and Ishikawa process are derived, and further assumptions of strict pseudo-contractiveness yield efficient algorithms using half-quadratic splitting and forward-backward splitting. The proposed algorithms theoretically converge strongly to a fixed point. A training strategy based on holomorphic transformation and functional calculi is proposed to enforce the pseudo-contractive denoiser assumption. Extensive experiments demonstrate superior performance of the pseudo-contractive denoiser compared to related denoisers. The proposed methods are competitive in terms of visual effects and quantitative values.

Transformers for time series forecasting mainly model time series from limited or fixed scales, making it challenging to capture different characteristics spanning various scales. We propose Pathformer, a multi-scale Transformer with adaptive pathways. It integrates both temporal resolution and temporal distance for multi-scale modeling. Multi-scale division divides the time series into different temporal resolutions using patches of various sizes. Based on the division of each scale, dual attention is performed over these patches to capture global correlations and local details as temporal dependencies. We further enrich the multi-scale Transformer with adaptive pathways, which adaptively adjust the multi-scale modeling process based on the varying temporal dynamics of the input, improving the accuracy and generalization of Pathformer. Extensive experiments on eleven real-world datasets demonstrate that Pathformer not only achieves state-of-the-art performance by surpassing all current models but also exhibits stronger generalization abilities under various transfer scenarios. The code is made available at https://github.com/decisionintelligence/pathformer., Comment: Accepted by the 12th International Conference on Learning Representations (ICLR 2024)

Electrical manipulation of magnetic order by current-induced spin torques lays the foundation for spintronics. One promising approach is encoding information in the N\'eel vector of antiferromagnetic (AFM) materials, particularly to collinear antiferromagnets with the perpendicular magnetic anisotropy (PMA), as the negligible stray fields and terahertz spin dynamics can enable memory devices with higher integration density and ultrafast speed. Here we demonstrate that the N\'eel order information in a prototypical collinear AFM insulator with PMA, Cr2O3, can be reliably readout via the anomalous Hall effect and efficiently switched by the spin-orbit torque (SOT) effect with a low current density of 5.8*106 A/cm2. Moreover, using Cr2O3 as a mediator, we electrically switch the magnetization of a Y3Fe5O12 film exchange-coupled to the Cr2O3 layer, unambiguously confirming the N\'eel order switching of the Cr2O3 layer. This work provides a significant basis for developing AFM memory devices based on collinear AFM materials with PMA.

Charge density wave (CDW) is a collective quantum phenomenon with a charge modulation in solids1-2. Condensation of electron and hole pairs with finite momentum will lead to such an ordered state3-7. However, lattice symmetry breaking manifested as the softening of phonon modes can occur simultaneously, which makes it difficult to disentangle the origin of the transition8-14. Here, we report a condensed phase in low dimensional HfTe2, whereas angle-resolved photoemission spectroscopy (ARPES) measurements show a metal-insulator transition by lowering the temperature in single triatomic layer (TL) HfTe2. A full gap opening, renormalization of the bands, and emergence of replica bands at the M point are observed in the low temperatures, indicating formation of a CDW in the ground state.Raman spectroscopy shows no sign of lattice distortion within the detection limit. The results are corroborated by first-principles calculations, demonstrating the electronic origin of the CDW. By adding more layers, the phase transition is suppressed and completely destroyed at 3 TL because of the increased screening around the Fermi surface. Interestingly, a small amount of electron doping in 1 TL film during the growth significantly raises the transition temperature (TC), which is attributed to a reduced screening effect and a more balanced electron and hole carrier density. Our results indicate a CDW formation mechanism consistent with the excitonic insulator phase in low dimensional HfTe2 and open up opportunity for realization of novel quantum states based on exciton condensation., Comment: https://www.nature.com/articles/s41567-023-02349-0 published in Nature Physics

Many existing coverless steganography methods establish a mapping relationship between cover images and hidden data. There exists an issue that the number of images stored in the database grows exponentially as the steganographic capacity rises. The need for a high steganographic capacity makes it challenging to build an image database. To improve the image library utilization and anti-attack capability of the steganography system, we present an efficient coverless scheme based on dynamically matched substrings. YOLO is employed for selecting optimal objects, and a mapping dictionary is established between these objects and scrambling factors. With the aid of this dictionary, each image is effectively assigned to a specific scrambling factor, which is used to scramble the receiver's sequence key. To achieve sufficient steganography capability based on a limited image library, all substrings of the scrambled sequences hold the potential to hide data. After completing the secret information matching, the ideal number of stego images will be obtained from the database. According to experimental results, this technology outperforms most previous works on data load, transmission security, and hiding capacity. Under typical geometric attacks, it can recover 79.85\% of secret information on average. Furthermore, only approximately 200 random images are needed to meet a capacity of 19 bits per image.