Search

Low-precision training is considered an effective strategy for reducing both training and downstream inference costs. Previous scaling laws for precision mainly focus on integer quantization, which pay less attention to the constituents in floating-point quantization and thus cannot well fit the LLM losses in this scenario. In contrast, while floating-point quantization training is more commonly implemented in production, the research on it has been relatively superficial. In this paper, we thoroughly explore the effects of floating-point quantization targets, exponent bits, mantissa bits, and the calculation granularity of the scaling factor in floating-point quantization training performance of LLM models. While presenting an accurate floating-point quantization unified scaling law, we also provide valuable suggestions for the community: (1) Exponent bits contribute slightly more to the model performance than mantissa bits. We provide the optimal exponent-mantissa bit ratio for different bit numbers, which is available for future reference by hardware manufacturers; (2) We discover the formation of the critical data size in low-precision LLM training. Too much training data exceeding the critical data size will inversely bring in degradation of LLM performance; (3) The optimal floating-point quantization precision is directly proportional to the computational power, but within a wide computational power range, we estimate that the best cost-performance precision lies between 4-8 bits.

The spatial characteristics and structure of an axisymmetric turbulent boundary layer under strong adverse pressure gradient and weak transverse curvature are investigated using incompressible large-eddy simulation. The boundary layer is on a $20^{\circ}$ tail cone of a body of revolution at a length-based Reynolds number of $1.9\times10^6$. The simulation results are in agreement with the experimental measurements of Balantrapu et al. (J. Fluid Mech., vol. 929, 2021) and significantly expand the experimental results with new flow-field details and physical insights. The mean streamwise velocity profiles exhibit a shortened logarithmic region and a longer wake region compared with planar boundary layers at zero pressure gradient. With the embedded-shear-layer scaling, self-similarity is observed for the mean velocity and all three components of turbulence intensity. The azimuthal-wavenumber spectra of streamwise velocity fluctuations possess two peaks in the wall-normal direction, an inner peak at the wavelength of approximately 100 wall units and an outer peak in the wake region with growing strength, wavelength and distance to the wall in the downstream direction. Two-point correlations of streamwise velocity fluctuations show significant downstream growth and elongation of turbulence structures with increasing inclination angle. However, relative to the boundary-layer thickness, the correlation structures decrease in size in the downstream direction. The distributions of streamwise and wall-normal integral lengths across the boundary-layer thickness resemble those of zero-pressure-gradient planar boundary layers, whereas the azimuthal integral length deviates from the planar boundary-layer behavior at downstream stations.

The gravitational decoupling method systematically generates hairy modifications to the solutions in general relativity due to new gravitational sources. In view of the recent advances in astronomical observations, these hairy solutions are expected to be testable in the near-term observations. In this paper, we study the radiative property and optical appearance of the thin accretion disk around the rotating hairy black holes obtained by gravitational decoupling. We numerically compute the radiative flux, temperature, and differential luminosity of the thin accretion disk, and we also show its bolometric image by the ray-tracing method. By comparing with the results for the Kerr metric, we found that the deviations of the observational properties of the thin accretion disk from those of Kerr metric becomes significant in the rapid rotating case, or in the inner region of the disk. These results guide the observational investigations on the rotating hairy black hole., Comment: 12 pages, 7 figures

For superlinear heat equations with the Dirichlet boundary condition, the $L^\infty$ estimates of radially symmetric solutions are studied. In particular, the uniform boundedness of global solutions and the non-existence of solutions with type II blow-up are proved. For the space dimension greater than $9$, our results are shown under the condition that an exponent representing the growth rate of a nonlinear term is between the Sobolev exponent and the Joseph-Lundgren exponent. In the case where the space dimension is greater than $2$ and smaller than $10$, our results are applicable for nonlinear terms growing extremely faster than the exponential function.

Reinforcement Learning (RL) based methods have been increasingly explored for robot learning. However, RL based methods often suffer from low sampling efficiency in the exploration phase, especially for long-horizon manipulation tasks, and generally neglect the semantic information from the task level, resulted in a delayed convergence or even tasks failure. To tackle these challenges, we propose a Temporal-Logic-guided Hybrid policy framework (HyTL) which leverages three-level decision layers to improve the agent's performance. Specifically, the task specifications are encoded via linear temporal logic (LTL) to improve performance and offer interpretability. And a waypoints planning module is designed with the feedback from the LTL-encoded task level as a high-level policy to improve the exploration efficiency. The middle-level policy selects which behavior primitives to execute, and the low-level policy specifies the corresponding parameters to interact with the environment. We evaluate HyTL on four challenging manipulation tasks, which demonstrate its effectiveness and interpretability. Our project is available at: https://sites.google.com/view/hytl-0257/., Comment: Accepted by IROS 2024. Code:https://github.com/Charlie0257/HyTL

Quantum computing is an emerging technology that offers exponential speedups for certain problems. At the core of quantum-centric supercomputing is advanced middleware that manages the interaction between quantum hardware and classical computing infrastructure. Circuit knitting is a technique that leverages classical computation to offload some of the computational burden from quantum circuits, enabling them to exceed the capacity of current Noisy Intermediate-Scale Quantum (NISQ) devices. This is done by partitioning large circuits into smaller subcircuits, though at the cost of classical reconstruction and increased sampling overhead. Despite significant advancements in reducing the theoretical costs of circuit knitting, efficiently deploying these techniques across a broad range of quantum algorithms remains a challenge. In this work, we propose CiFold, a novel graph-based system that, at the individual qubit's level, identifies and leverages repeated structures within quantum circuits. By folding these repeated modules in parallel, CiFold constructs a meta-graph that guides the partitioning process, optimizing the cutting strategy through the integration of advanced circuit knitting techniques. Our system has been extensively evaluated across various quantum algorithms, achieving up to 799.2\% reduction in quantum resource usage, demonstrating its scalability and substantial performance improvements over state-of-the-art approaches.

Direction reasoning is essential for intelligent systems to understand the real world. While existing work focuses primarily on spatial reasoning, compass direction reasoning remains underexplored. To address this, we propose the Compass Direction Reasoning (CDR) benchmark, designed to evaluate the direction reasoning capabilities of multimodal language models (MLMs). CDR includes three types images to test spatial (up, down, left, right) and compass (north, south, east, west) directions. Our evaluation reveals that most MLMs struggle with direction reasoning, often performing at random guessing levels. Experiments show that training directly with CDR data yields limited improvements, as it requires an understanding of real-world physical rules. We explore the impact of mixdata and CoT fine-tuning methods, which significantly enhance MLM performance in compass direction reasoning by incorporating diverse data and step-by-step reasoning, improving the model's ability to understand direction relationships.

Data sharing is ubiquitous in the metaverse, which adopts blockchain as its foundation. Blockchain is employed because it enables data transparency, achieves tamper resistance, and supports smart contracts. However, securely sharing data based on blockchain necessitates further consideration. Ciphertext-policy attribute-based encryption (CP-ABE) is a promising primitive to provide confidentiality and fine-grained access control. Nonetheless, authority accountability and key abuse are critical issues that practical applications must address. Few studies have considered CP-ABE key confidentiality and authority accountability simultaneously. To our knowledge, we are the first to fill this gap by integrating non-interactive zero-knowledge (NIZK) proofs into CP-ABE keys and outsourcing the verification process to a smart contract. To meet the decentralization requirement, we incorporate a decentralized CP-ABE scheme into the proposed data sharing system. Additionally, we provide an implementation based on smart contract to determine whether an access control policy is satisfied by a set of CP-ABE keys. We also introduce an open incentive mechanism to encourage honest participation in data sharing. Hence, the key abuse issue is resolved through the NIZK proof and the incentive mechanism. We provide a theoretical analysis and conduct comprehensive experiments to demonstrate the feasibility and efficiency of the data sharing system. Based on the proposed accountable approach, we further illustrate an application in GameFi, where players can play to earn or contribute to an accountable DAO, fostering a thriving metaverse ecosystem.

The Ambient Internet of Things (A-IoT) has emerged as a critical direction for achieving effective connectivity as the IoT system evolves to 6G. However, the introduction of A-IoT technologies, particularly involving backscatter modulation, poses numerous challenges for system design and network operations. This paper surveys current standardization efforts, highlights potential challenges, and explores future directions for A-IoT. It begins with a comprehensive overview of ongoing standardization initiatives by the 3rd Generation Partnership Project (3GPP) on A-IoT, providing a solid foundation for further technical research in both industry and academia. Building upon this groundwork, the paper conducts an analysis of critical enabling technologies for A-IoT. Moreover, a comprehensive A-IoT demonstration system is designed to showcase the practical viability and efficiency of A-IoT techniques, supported by field experiments. We finally address ongoing challenges associated with A-IoT technologies, providing valuable insights for future research endeavors.

Noisy labels can negatively impact the performance of deep neural networks. One common solution is label refurbishment, which involves reconstructing noisy labels through predictions and distributions. However, these methods may introduce problematic semantic associations, a phenomenon that we identify as Semantic Contamination. Through an analysis of Robust LR, a representative label refurbishment method, we found that utilizing the logits of views for refurbishment does not adequately balance the semantic information of individual classes. Conversely, using the logits of models fails to maintain consistent semantic relationships across models, which explains why label refurbishment methods frequently encounter issues related to Semantic Contamination. To address this issue, we propose a novel method called Collaborative Cross Learning, which utilizes semi-supervised learning on refurbished labels to extract appropriate semantic associations from embeddings across views and models. Experimental results show that our method outperforms existing approaches on both synthetic and real-world noisy datasets, effectively mitigating the impact of label noise and Semantic Contamination., Comment: AAAI2025

In this paper, our focus lies on the study of the second-order variational analysis of orthogonally invariant matrix functions. It is well-known that an orthogonally invariant matrix function is an extended-real-value function defined on ${\mathbb M}_{m,n}\,(n \leqslant m)$ of the form $f \circ \sigma$ for an absolutely symmetric function $f \colon \R^n \rightarrow [-\infty,+\infty]$ and the singular values $\sigma \colon {\mathbb M}_{m,n} \rightarrow \R^{n}$. We establish several second-order properties of orthogonally invariant matrix functions, such as parabolic epi-differentiability, parabolic regularity, and twice epi-differentiability when their associated absolutely symmetric functions enjoy some properties. Specifically, we show that the nuclear norm of a real $m \times n$ matrix is twice epi-differentiable and we derive an explicit expression of its second-order epi-derivative. Moreover, for a convex orthogonally invariant matrix function, we calculate its second subderivative and present sufficient conditions for twice epi-differentiability. This enables us to establish second-order optimality conditions for a class of matrix optimization problems.

Attribute-based encryption (ABE) is a generalization of public-key encryption that enables fine-grained access control in cloud services. Recently, Hohenberger et al. (Eurocrypt 2023) introduced the notion of registered ABE, which is an ABE scheme without a trusted central authority. Instead, users generate their own public/secret keys and then register their keys and attributes with a key curator. The key curator is a transparent and untrusted entity and its behavior needs to be audited for malicious registration. In addition, pairing-based registered ABE still suffers the heavy decryption overhead like ABE. A general approach to address this issue is to outsource decryption to a decryption cloud service (DCS).In this work, we propose BA-ORABE, the first fully auditable registered ABE with reliable outsourced decryption scheme based on blockchain. First, we utilize a verifiable tag mechanism to achieve verifiability of ciphertext transformation, and the exemptibility which enables the honest DCS to escape from wrong claims is guaranteed by zero knowledge fraud proof under optimistic assumption. Additionally, our system achieves fairness and decentralized outsourcing to protect the interests of all parties and the registration and outsourcing process are transparent and fully auditable through blockchain. Finally, we give security analysis, implement and evaluate our scheme on Ethereum to demonstrate its feasibility and efficiency, and show its advantages in real application of decentralized finance., Comment: 15pages,add application

Federated Adversarial Learning (FAL) is a robust framework for resisting adversarial attacks on federated learning. Although some FAL studies have developed efficient algorithms, they primarily focus on convergence performance and overlook generalization. Generalization is crucial for evaluating algorithm performance on unseen data. However, generalization analysis is more challenging due to non-smooth adversarial loss functions. A common approach to addressing this issue is to leverage smoothness approximation. In this paper, we develop algorithm stability measures to evaluate the generalization performance of two popular FAL algorithms: \textit{Vanilla FAL (VFAL)} and {\it Slack FAL (SFAL)}, using three different smooth approximation methods: 1) \textit{Surrogate Smoothness Approximation (SSA)}, (2) \textit{Randomized Smoothness Approximation (RSA)}, and (3) \textit{Over-Parameterized Smoothness Approximation (OPSA)}. Based on our in-depth analysis, we answer the question of how to properly set the smoothness approximation method to mitigate generalization error in FAL. Moreover, we identify RSA as the most effective method for reducing generalization error. In highly data-heterogeneous scenarios, we also recommend employing SFAL to mitigate the deterioration of generalization performance caused by heterogeneity. Based on our theoretical results, we provide insights to help develop more efficient FAL algorithms, such as designing new metrics and dynamic aggregation rules to mitigate heterogeneity.

Multimodal Large Language Models (MLLMs) excel in vision--language tasks by pre-training solely on coarse-grained concept annotations (e.g., image captions). We hypothesize that integrating fine-grained concept annotations (e.g., object labels and object regions) will further improve performance, as both data granularities complement each other in terms of breadth and depth in concept representation. We introduce a new dataset featuring Multimodal Multi-Grained Concept annotations (MMGiC) for MLLMs. In constructing MMGiC, we explore the impact of different data recipes on multimodal comprehension and generation. Our analyses reveal that multi-grained concept annotations integrate and complement each other, under our structured template and a general MLLM framework. We clearly explore and demonstrate the potential of MMGiC to help MLLMs better locate and learn concepts, aligning vision and language at multiple granularities. We further validate our hypothesis by investigating the fair comparison and effective collaboration between MMGiC and image--caption data on 12 multimodal comprehension and generation benchmarks, e.g., their appropriate combination achieve 3.95% and 2.34% absolute improvements over image--caption data alone on POPE and SEED-Bench. Code, data and models will be available at https://github.com/LooperXX/MMGiC., Comment: A manuscript that should have been Arxived in May :)

The application of deep learning methods, particularly foundation models, in biological research has surged in recent years. These models can be text-based or trained on underlying biological data, especially omics data of various types. However, comparing the performance of these models consistently has proven to be a challenge due to differences in training data and downstream tasks. To tackle this problem, we developed an architecture-agnostic benchmarking approach that, instead of evaluating the models directly, leverages entity representation vectors from each model and trains simple predictive models for each benchmarking task. This ensures that all types of models are evaluated using the same input and output types. Here we focus on gene properties collected from professionally curated bioinformatics databases. These gene properties are categorized into five major groups: genomic properties, regulatory functions, localization, biological processes, and protein properties. Overall, we define hundreds of tasks based on these databases, which include binary, multi-label, and multi-class classification tasks. We apply these benchmark tasks to evaluate expression-based models, large language models, protein language models, DNA-based models, and traditional baselines. Our findings suggest that text-based models and protein language models generally outperform expression-based models in genomic properties and regulatory functions tasks, whereas expression-based models demonstrate superior performance in localization tasks. These results should aid in the development of more informed artificial intelligence strategies for biological understanding and therapeutic discovery. To ensure the reproducibility and transparency of our findings, we have made the source code and benchmark data publicly accessible for further investigation and expansion at github.com/BiomedSciAI/gene-benchmark.

Aerosol forecasting is essential for air quality warnings, health risk assessment, and climate change mitigation. However, it is more complex than weather forecasting due to the intricate interactions between aerosol physicochemical processes and atmospheric dynamics, resulting in significant uncertainty and high computational costs. Here, we develop an artificial intelligence-driven global aerosol-meteorology forecasting system (AI-GAMFS), which provides reliable 5-day, 3-hourly forecasts of aerosol optical components and surface concentrations at a 0.5{\deg} x 0.625{\deg} resolution. AI-GAMFS combines Vision Transformer and U-Net in a backbone network, robustly capturing the complex aerosol-meteorology interactions via global attention and spatiotemporal encoding. Trained on 42 years of advanced aerosol reanalysis data and initialized with GEOS Forward Processing (GEOS-FP) analyses, AI-GAMFS delivers operational 5-day forecasts in one minute. It outperforms the Copernicus Atmosphere Monitoring Service (CAMS) global forecasting system, GEOS-FP forecasts, and several regional dust forecasting systems in forecasting most aerosol variables including aerosol optical depth and dust components. Our results mark a significant step forward in leveraging AI to refine physics-based aerosol forecasting, facilitating more accurate global warnings for aerosol pollution events, such as dust storms and wildfires., Comment: 37 pages, 14 figures

Cities play a pivotal role in human development and sustainability, yet studying them presents significant challenges due to the vast scale and complexity of spatial-temporal data. One such challenge is the need to uncover universal urban patterns, such as the urban scaling law, across thousands of cities worldwide. In this study, we propose a novel large-scale geospatial data processing system that enables city analysis on an unprecedented scale. We demonstrate the system's capabilities by revisiting the urban scaling law across 21,280 cities globally, using a range of open-source datasets including road networks, nighttime light intensity, built-up areas, and population statistics. Analyzing the characteristics of 21,280 cities involves querying over half a billion geospatial data points, a task that traditional Geographic Information Systems (GIS) would take several days to process. In contrast, our cloud-based system accelerates the analysis, reducing processing time to just minutes while significantly lowering resource consumption. Our findings reveal that the urban scaling law varies across cities in under-developed, developing, and developed regions, extending the insights gained from previous studies focused on hundreds of cities. This underscores the critical importance of cloud-based big data processing for efficient, large-scale geospatial analysis. As the availability of satellite imagery and other global datasets continues to grow, the potential for scientific discovery expands exponentially. Our approach not only demonstrates how such large-scale tasks can be executed efficiently but also offers a powerful solution for data scientists and researchers working in the fields of city and geospatial science.

Two-dimensional transition metal carbides and nitrides (MXenes) have gained popularity in fields such as energy storage, catalysis, and electromagnetic interference due to their diverse elemental compositions and variable surface terminations (T). Generally, the synthesis of MXene materials involves etching the weak M-A metallic bonds in the ternary layered transition metal carbides and nitrides (MAX phase) using HF acid or Lewis acid molten salts, while the strong M-X covalent bonds preserve the two-dimensional framework structure of MXenes. On the other hand, the MAX phase material family also includes a significant class of members where the A site is occupied by non-metal main group elements (such as sulfur and phosphorus), in which both M-A and M-X are covalent bond-type sublayers. The aforementioned etching methods cannot be used to synthesize MXene materials from these parent phases. In this work, we discovered that the covalent bond-type M-A and M-X sublayers exhibit different reactivity with some inorganic materials in a high-temperature molten state. By utilizing this difference in reactivity, we can structurally modify these covalent sublayers, allowing for the substitution of elements at the X site (from B to Se, S, P, C) and converting non-metal A site atoms in non-van der Waals (non-vdW) MAX phases into surface atoms in vdW layered materials. This results in a family of early transition metal Xide chalcogenides (TMXCs) that exhibit lattice characteristics of both MXenes and transition metal chalcogenides. Using electron-donor chemical scissors, these TMXC layered materials can be further exfoliated into monolayer nanosheets. The atomic configurations of each atom in these monolayer TMXCs are the same as those of conventional MXenes, but the oxidation states of the M-site atoms can be regulated by both X-site atoms and intercalated cations.

We consider the nonparametric maximum likelihood estimation for the underlying event time based on mixed-case interval-censored data, under a log-concavity assumption on its distribution function. This generalized framework relaxes the assumptions of a log-concave density function or a concave distribution function considered in the literature. A log-concave distribution function is fulfilled by many common parametric families in survival analysis and also allows for multi-modal and heavy-tailed distributions. We establish the existence, uniqueness and consistency of the log-concave nonparametric maximum likelihood estimator. A computationally efficient procedure that combines an active set algorithm with the iterative convex minorant algorithm is proposed. Numerical studies demonstrate the advantages of incorporating additional shape constraint compared to the unconstrained nonparametric maximum likelihood estimator. The results also show that our method achieves a balance between efficiency and robustness compared to assuming log-concavity in the density. An R package iclogcondist is developed to implement our proposed method.

Offline batch inference, which leverages the flexibility of request batching to achieve higher throughput and lower costs, is becoming more popular for latency-insensitive applications. Meanwhile, recent progress in model capability and modality makes requests more diverse in compute and memory demands, creating unique opportunities for throughput improvement by resource overlapping. However, a request schedule that maximizes resource overlapping can conflict with the schedule that maximizes prefix sharing, a widely-used performance optimization, causing sub-optimal inference throughput. We present BlendServe, a system that maximizes resource utilization of offline batch inference by combining the benefits of resource overlapping and prefix sharing using a resource-aware prefix tree. BlendServe exploits the relaxed latency requirements in offline batch inference to reorder and overlap requests with varied resource demands while ensuring high prefix sharing. We evaluate BlendServe on a variety of synthetic multi-modal workloads and show that it provides up to $1.44\times$ throughput boost compared to widely-used industry standards, vLLM and SGLang.

The coupling between dual-band or multi-band quasi-bound states in the continuum (q-BICs) is of great interest for their rich physics and promising applications. Here, we report tunable collective electromagnetic induced transparency-like (EIT-like) phenomenon due to coupling between dual-band collective electric dipolar and magnetic quadrupolar q-BICs, which are supported by an all-dielectric metasurface composed of periodic tilted silicon quadrumers. We show that this collective EIT-like phenomenon with strong slow light effect can be realized by varying the nanodisk diameter or the tilt angle, and that the transparency window wavelength, the quality factor, and the group index can all be tuned by changing the nanodisk size. We further find that as the nanodisk size decreases, the slow light effect becomes stronger, and higher sensitivity can be obtained for the refractive index sensing. Interestingly, the sensitivity first increases exponentially and then reaches a plateau as the nanodisk size decreases, or equivalently as the group index increases. We therefore expect this work will advance the understanding of the collective EIT-like effect due to coupling between q-BICs, and the findings will have potential applications in slow-light enhanced biochemical sensing., Comment: 12 pages, 9 figures

Large Vision-Language Model (LVLM) systems have demonstrated impressive vision-language reasoning capabilities but suffer from pervasive and severe hallucination issues, posing significant risks in critical domains such as healthcare and autonomous systems. Despite previous efforts to mitigate hallucinations, a persistent issue remains: visual defect from vision-language misalignment, creating a bottleneck in visual processing capacity. To address this challenge, we develop Complementary Adaptive Token-level Contrastive Decoding to Mitigate Hallucinations in LVLMs (CATCH), based on the Information Bottleneck theory. CATCH introduces Complementary Visual Decoupling (CVD) for visual information separation, Non-Visual Screening (NVS) for hallucination detection, and Adaptive Token-level Contrastive Decoding (ATCD) for hallucination mitigation. CATCH addresses issues related to visual defects that cause diminished fine-grained feature perception and cumulative hallucinations in open-ended scenarios. It is applicable to various visual question-answering tasks without requiring any specific data or prior knowledge, and generalizes robustly to new tasks without additional training, opening new possibilities for advancing LVLM in various challenging applications.

The problem of interfacing quantum mechanics and gravity has long been an unresolved issue in physics. Recent advances in precision measurement technology suggest that detecting gravitational effects in massive quantum systems, particularly gravity-induced entanglement (GIE) in the oscillator system, could provide crucial empirical evidence for revealing the quantum nature of the gravitational field. However, thermal decoherence imposes strict constraints on system parameters. For entanglement to occur, the inequality $2 \gamma_m k_B T < \hbar G \Lambda \rho$ must be satisfied, linking mechanical dissipation $\gamma_m$, effective temperature $T$, oscillator density $\rho$ and form factor $\Lambda$ determined by the geometry and spatial arrangement of the oscillators. This inequality, based on the inherent property of the noise model of GIE, is considered universally across experimental systems and cannot be improved by quantum control. Given the challenges in further optimizing $\gamma_m$, $\rho$, and $T$ near their limits, optimizing the form factor $\Lambda$ may reduce demands on other parameters. In this work, we prove that the form factor $\Lambda$ has a supremum of $2\pi$, revealing a fundamental limit of the oscillator system. We propose design schemes that enable the form factor to approach this supremum, which is nearly an order of magnitude higher than typical spherical oscillators. This optimization may ease experimental constraints, bringing GIE-based validation of quantum gravity closer to realization.

Microservices architecture has become the dominant architecture in cloud computing paradigm with its advantages of facilitating development, deployment, modularity and scalability. The workflow of microservices architecture is transparent to the users, who are concerned with the quality of service (QoS). Taking Service Level Objective (SLO) as an important indicator of system resource scaling can effectively ensure user's QoS, but how to quickly allocate end-to-end SLOs to each microservice in a complete service so that it can obtain the optimal SLO resource allocation scheme is still a challenging problem. Existing microservice autoscaling frameworks based on SLO resources often have heavy and complex models that demand substantial time and computational resources to get a suitable resource allocation scheme. Moreover, when the system environment or microservice application changes, these methods require significant time and resources for model retraining. In this paper, we propose LSRAM, a lightweight SLO resource allocation management framework based on the gradient descent method to overcome the limitation of existing methods in terms of heavy model, time-consuming, poor scalability, and difficulty in retraining. LSRAM has two stages: at stage one, the lightweight SLO resource allocation model from LSRAM can quickly compute the appropriate SLO resources for each microservice; at stage two, LSRAM's SLO resource update model enables the entire framework to quickly adapt to changes in the cluster environment (e.g. load and applications). Additionally, LSRAM can effectively handle bursty traffic and highly fluctuating load application scenarios. Compared to state-of-the-art SLO allocation frameworks, LSRAM not only guarantees users' QoS but also reduces resource usage by 17%., Comment: 22 pages

How turbulent convective fluctuations organise to form large-scale structures in planetary atmospheres remains a question that eludes quantitative answers. The assumption that this process is the result of an inverse cascade was suggested half a century ago in two-dimensional fluids, but its applicability to atmospheric and oceanic flows remains heavily debated, hampering our understanding of the energy balance in planetary systems. We show with direct numerical simulations of spatial resolutions of 122882 $\times$ 384 points that rotating and stratified flows can support a bidirectional cascade of energy, in three dimensions, with a ratio of Rossby to Froude numbers comparable to that of the Earth's atmosphere. Our results establish that in dry atmospheres spontaneous order can arise via an inverse cascade to the largest spatial scales.

Virtual reality (VR) environments have greatly expanded opportunities for immersive exploration, yet physically navigating these digital spaces remains a significant challenge. In this paper, we present the conceptual framework of NAVIS (Navigating Virtual Spaces with Immersive Scooters), a novel system that utilizes a scooter-based interface to enhance both navigation and interaction within virtual environments. NAVIS combines real-time physical mobility, haptic feedback, and CAVE-like (Cave Automatic Virtual Environment) technology to create a realistic sense of travel and movement, improving both spatial awareness and the overall immersive experience. By offering a more natural and physically engaging method of exploration, NAVIS addresses key limitations found in traditional VR locomotion techniques, such as teleportation or joystick control, which can detract from immersion and realism. This approach highlights the potential of combining physical movement with virtual environments to provide a more intuitive and enjoyable experience for users, opening up new possibilities for applications in gaming, education, and beyond.

Large Language Models (LLMs) have demonstrated impressive capabilities in reasoning using Chain-of-Thought (CoT) prompting. However, CoT can be biased by users' instruction. In this work, we study the reasoning robustness of LLMs to typographical errors, which can naturally occur in users' queries. We design an Adversarial Typo Attack ($\texttt{ATA}$) algorithm that iteratively samples typos for words that are important to the query and selects the edit that is most likely to succeed in attacking. It shows that LLMs are sensitive to minimal adversarial typographical changes. Notably, with 1 character edit, Mistral-7B-Instruct's accuracy drops from 43.7% to 38.6% on GSM8K, while with 8 character edits the performance further drops to 19.2%. To extend our evaluation to larger and closed-source LLMs, we develop the $\texttt{R$^2$ATA}$ benchmark, which assesses models' $\underline{R}$easoning $\underline{R}$obustness to $\underline{\texttt{ATA}}$. It includes adversarial typographical questions derived from three widely used reasoning datasets-GSM8K, BBH, and MMLU-by applying $\texttt{ATA}$ to open-source LLMs. $\texttt{R$^2$ATA}$ demonstrates remarkable transferability and causes notable performance drops across multiple super large and closed-source LLMs.

The ubiquitous noise in quantum system hinders the advancement of quantum information processing and has driven the emergence of different hardware-efficient quantum error correction protocols. Among them, qubits with structured noise, especially with biased noise, are one of the most promising platform to achieve fault-tolerance due to the high error thresholds of quantum error correction codes tailored for them. Nevertheless, their quantum operations are challenging and the demonstration of their performance beyond the fault-tolerant threshold remain incomplete. Here, we leverage Schr\"odinger cat states in a scalable planar superconducting nonlinear oscillator to thoroughly characterize the high-fidelity single-qubit quantum operations with systematic quantum tomography and benchmarking tools, demonstrating the state-of-the-art performance of operations crossing the fault-tolerant threshold of the XZZX surface code. These results thus embody a transformative milestone in the exploration of quantum systems with structured error channels. Notably, our framework is extensible to other types of structured-noise systems, paving the way for systematic characterization and validation of novel quantum platforms with structured noise., Comment: 19 pages, 12 figures

High resolution is crucial for precise segmentation in fundus images, yet handling high-resolution inputs incurs considerable GPU memory costs, with diminishing performance gains as overhead increases. To address this issue while tackling the challenge of segmenting tiny objects, recent studies have explored local-global fusion methods. These methods preserve fine details using local regions and capture long-range context information from downscaled global images. However, the necessity of multiple forward passes inevitably incurs significant computational overhead, adversely affecting inference speed. In this paper, we propose HRDecoder, a simple High-Resolution Decoder network for fundus lesion segmentation. It integrates a high-resolution representation learning module to capture fine-grained local features and a high-resolution fusion module to fuse multi-scale predictions. Our method effectively improves the overall segmentation accuracy of fundus lesions while consuming reasonable memory and computational overhead, and maintaining satisfying inference speed. Experimental results on the IDRID and DDR datasets demonstrate the effectiveness of our method. Code is available at https://github.com/CVIU-CSU/HRDecoder., Comment: 11 pages, 3 figures, accepted by MICCAI 2024, the revised version

Attribute-Based Encryption (ABE) is a promising solution for access control in cloud services. However, the heavy decryption overhead hinders its widespread adoption. A general approach to address this issue is to outsource decryption to decryption cloud service(DCS). Existing schemes have utilized various methods to enable users to verify outsourced results; however, they lack an effective mechanism to achieve exemptibility which enables the honest DCS to escape from wrong claims. And it is impractical to assume that the DCS will provide free services. In this paper, we propose a blockchain-based payable outsourced decryption ABE scheme that achieves both verifiability and exemptibility without adding redundant information to ABE ciphertext. We use zero-knowledge proof to verify outsourced results on blockchain and introduce an optional single-round challenge game under optimistic assumption to address the high cost of proof generation. Moreover, our system achieves fairness and decentralized outsourcing to protect the interests of all parties. Finally, we implement and evaluate our scheme on Ethereum to demonstrate its feasibility and efficiency, the gas usage in attribute numbers from 5 to 60 is 11$\times$ to 140$\times$ in the happy case and 4$\times$ to 55$\times$ in the challenge case lower than the scheme of Ge et al. (TDSC'23)., Comment: 12 pages, 5 figures

In this paper, we introduce Hunyuan-Large, which is currently the largest open-source Transformer-based mixture of experts model, with a total of 389 billion parameters and 52 billion activation parameters, capable of handling up to 256K tokens. We conduct a thorough evaluation of Hunyuan-Large's superior performance across various benchmarks including language understanding and generation, logical reasoning, mathematical problem-solving, coding, long-context, and aggregated tasks, where it outperforms LLama3.1-70B and exhibits comparable performance when compared to the significantly larger LLama3.1-405B model. Key practice of Hunyuan-Large include large-scale synthetic data that is orders larger than in previous literature, a mixed expert routing strategy, a key-value cache compression technique, and an expert-specific learning rate strategy. Additionally, we also investigate the scaling laws and learning rate schedule of mixture of experts models, providing valuable insights and guidances for future model development and optimization. The code and checkpoints of Hunyuan-Large are released to facilitate future innovations and applications. Codes: https://github.com/Tencent/Hunyuan-Large Models: https://huggingface.co/tencent/Tencent-Hunyuan-Large, Comment: 17 pages, 4 Figures

Exploring low-cost applications is paramount to creating value in early fault-tolerant quantum computers. Here we optimize both gate and qubit counts of recent algorithms for simulating the Fermi-Hubbard model. We further devise and compile algorithms to simulate established models of cuprate and pnictide high-temperature superconductors, which include beyond-nearest-neighbor hopping terms and multi-orbital interactions that are absent in the Fermi-Hubbard model. We find plenty classically difficult instances with Toffoli counts much lower than commonly considered molecular simulations in quantum chemistry. We believe our results pave the way towards studying high-temperature superconductors on early fault-tolerant quantum computers., Comment: 30 pages, 5 figures, 7 tables

Large vision-language models (LVLMs) suffer from hallucination, resulting in misalignment between the output textual response and the input visual content. Recent research indicates that the over-reliance on the Large Language Model (LLM) backbone, as one cause of the LVLM hallucination, inherently introduces bias from language priors, leading to insufficient context attention to the visual inputs. We tackle this issue of hallucination by mitigating such over-reliance through preference learning. We propose Vision-guided Direct Preference Optimization (V-DPO) to enhance visual context learning at training time. To interpret the effectiveness and generalizability of V-DPO on different types of training data, we construct a synthetic dataset containing both response- and image-contrast preference pairs, compared against existing human-annotated hallucination samples. Our approach achieves significant improvements compared with baseline methods across various hallucination benchmarks. Our analysis indicates that V-DPO excels in learning from image-contrast preference data, demonstrating its superior ability to elicit and understand nuances of visual context. Our code is publicly available at https://github.com/YuxiXie/V-DPO., Comment: EMNLP 2024 Findings; 9 pages, 6 figures, 5 tables (16 pages, 8 figures, 8 tables including references and appendices)

Propagation of interplanetary (IP) shocks, particularly those driven by coronal mass ejections (CMEs), is still an outstanding question in heliophysics and space weather forecasting. Here we address effects of the ambient solar wind on the propagation of two similar CME-driven shocks from the Sun to Earth. The two shock events (CME03: April 3, 2010 and CME12: July 12, 2012) have the following properties: Both events (1) were driven by a halo CME (i.e., source location is near the Sun-Earth line), (2) had a CME source in the southern hemisphere, (3) had a similar transit time (~2 days) to Earth, (4) occurred in a non-quiet solar period, and (5) led to a severe geomagnetic storm. The initial (near the Sun) propagation speed, as measured by coronagraph images, was slower (by ~300 km/s) for CME03 than CME12, but it took about the same amount of traveling time for both events to reach Earth. According to the in-situ solar wind observations from the Wind spacecraft, the CME03-driven shock was associated with a faster solar wind upstream of the shock than the CME12-driven shock. This is also demonstrated in our global MHD simulations. Analysis of our simulation result indicates that the drag force indirectly plays an important role in the shock propagation. The present study suggests that in addition to the initial CME propagation speed near the Sun the shock speed (in the inertial frame) and the ambient solar wind condition, in particular the solar wind speed, are the key to timing the arrival of CME-driven-shock events., Comment: in press

We present Multi-expert Prompting, a novel enhancement of ExpertPrompting (Xu et al., 2023), designed to improve the large language model (LLM) generation. Specifically, it guides an LLM to fulfill an input instruction by simulating multiple experts, aggregating their responses, and selecting the best among individual and aggregated responses. This process is performed in a single chain of thoughts through our seven carefully designed subtasks derived from the Nominal Group Technique (Ven and Delbecq, 1974), a well-established decision-making framework. Our evaluations demonstrate that Multi-expert Prompting significantly outperforms ExpertPrompting and comparable baselines in enhancing the truthfulness, factuality, informativeness, and usefulness of responses while reducing toxicity and hurtfulness. It further achieves state-of-the-art truthfulness by outperforming the best baseline by 8.69% with ChatGPT. Multi-expert Prompting is efficient, explainable, and highly adaptable to diverse scenarios, eliminating the need for manual prompt construction., Comment: EMNLP 2024 Main Conference

The rise of social media and short-form video (SFV) has facilitated a breeding ground for misinformation. With the emergence of large language models, significant research has gone into curbing this misinformation problem with automatic false claim detection for text. Unfortunately, the automatic detection of misinformation in SFV is a more complex problem that remains largely unstudied. While text samples are monomodal (only containing words), SFVs comprise three different modalities: words, visuals, and non-linguistic audio. In this work, we introduce Video Masked Autoencoders for Misinformation Guarding (ViMGuard), the first deep-learning architecture capable of fact-checking an SFV through analysis of all three of its constituent modalities. ViMGuard leverages a dual-component system. First, Video and Audio Masked Autoencoders analyze the visual and non-linguistic audio elements of a video to discern its intention; specifically whether it intends to make an informative claim. If it is deemed that the SFV has informative intent, it is passed through our second component: a Retrieval Augmented Generation system that validates the factual accuracy of spoken words. In evaluation, ViMGuard outperformed three cutting-edge fact-checkers, thus setting a new standard for SFV fact-checking and marking a significant stride toward trustworthy news on social platforms. To promote further testing and iteration, VimGuard was deployed into a Chrome extension and all code was open-sourced on GitHub., Comment: 7 pages, 2 figures

Motivated by the surge of interest in Koopman operator theory, we propose a machine-learning alternative based on a functional Bayesian perspective for operator-theoretic modeling of unknown, data-driven, nonlinear dynamical systems. This formulation is directly done in an infinite-dimensional space of linear operators or Hilbert space with universal approximation property. The theory of reproducing kernel Hilbert space (RKHS) allows the lifting of nonlinear dynamics to a potentially infinite-dimensional space via linear embeddings, where a general nonlinear function is represented as a set of linear functions or operators in the functional space. This allows us to apply classical linear Bayesian methods such as the Kalman filter directly in the Hilbert space, yielding nonlinear solutions in the original input space. This kernel perspective on the Koopman operator offers two compelling advantages. First, the Hilbert space can be constructed deterministically, agnostic to the nonlinear dynamics. The Gaussian kernel is universal, approximating uniformly an arbitrary continuous target function over any compact domain. Second, Bayesian filter is an adaptive, linear minimum-variance algorithm, allowing the system to update the Koopman operator and continuously track the changes across an extended period of time, ideally suited for modern data-driven applications such as real-time machine learning using streaming data. In this paper, we present several practical implementations to obtain a finite-dimensional approximation of the functional Bayesian filter (FBF). Due to the rapid decay of the Gaussian kernel, excellent approximation is obtained with a small dimension. We demonstrate that this practical approach can obtain accurate results and outperform finite-dimensional Koopman decomposition.

We introduce the Cuntz-Thomsen picture of $\mathcal{C}$-equivariant Kasparov theory, denoted $\mathrm{KK}^\mathcal{C}$, for a unitary tensor category $\mathcal{C}$ with countably many isomorphism classes of simple objects. We use this description of $\mathrm{KK}^\mathcal{C}$ to prove the stable uniqueness theorem in this setting.

Global magnetohydrodynamic (MHD) models play an important role in the infrastructure of space weather forecasting. Validating such models commonly utilizes in situ solar wind measurements made near the orbit of the Earth. The purpose of this study is to test the performance of G3DMHD (a data driven, time-dependent, 3-D MHD model of the solar wind) with Parker Solar Probe (PSP) measurements. Since its launch in August 2018, PSP has traversed the inner heliosphere at different radial distances sunward of the Earth (the closest approach ~13.3 solar radii), thus providing a good opportunity to study evolution of the solar wind and to validate heliospheric models of the solar wind. The G3DMHD model simulation is driven by a sequence of maps of photospheric field extrapolated to the assumed source surface (2.5 Rs) using the potential field model from 2018 to 2022, which covers the first 15 PSP orbits. The Pearson correlation coefficient (cc) and the mean absolute squared error (MASE) are used as the metrics to evaluate the model performance. It is found that the model performs better for both magnetic intensity (cc = 0.75; MASE = 0.60) and the solar wind density (cc = 0.73; MASE = 0.50) than for the solar wind speed (cc = 0.15; MASE = 1.29) and temperature (cc = 0.28; MASE = 1.14). This is due primarily to lack of accurate boundary conditions. The well-known underestimate of the magnetic field in solar minimum years is also present. Assuming that the radial magnetic field becomes uniformly distributed with latitude at or below 18 Rs (the inner boundary of the computation do-main), the agreement in the magnetic intensity significantly improves (cc = 0.83; MASE = 0.49)., Comment: in press

We analytically study isothermal spheres in the light of nonextensive statistical mechanics. The equations for the isothermal spheres are derived from the grand partition function of the gravitating particle system in the Tsallis statistical mechanics. The effect of nonextensive statistics appears in relatively dense state, which appears at the center of the isothermal sphere. The stability of the isothermal sphere in the general relativistic system is found to be sensitive to the parameter q in the Tsallis statistics., Comment: 16 pages, 10 figures

We propose an efficient decomposition scheme for a quantum receiver that attains the Helstrom bound in the low-photon regime for discriminating binary coherent states. Our method, which avoids feedback as used in Dolinar's case, breaks down nonlinear operations into basic gates used in continuous-variable quantum computation. We account for realistic conditions by examining the impact of photon loss and imperfect photon detection, including the presence of dark counts, while presenting squeezing as a technique to mitigate these noise sources and maintain the advantage over SQL. Our scheme motivates testing quantum advantages with cubic-phase gates and designing photonic quantum computers to optimize symbol-by-symbol measurements in optical communication., Comment: 8 pages, 7 figures

Optical circuit systems, unlike other systems, have the potential to perform quantum information processing (QIP) at higher clock rate than conventional processing. The approach utilizing the electromagnetic field of light allows deterministic QIP by feedforward process, which counteracts the quantum randomness by performing adaptive quantum operation according to the measurement result of an entangled state. However, conventional feedforward with electronic measurement devices has limited the clock rate of the QIP down to around 100 MHz. In this paper, we demonstrate a variable squeezing gate with a clock rate of 1.3 THz by all-optical measurement-device-free feedforward. We utilize a periodically poled lithium niobate (PPLN) waveguide as an optical parametric amplifier, which eliminates the need for electronic measuring devices and enables ultra-fast feedforward. Experimental results demonstrate that our all-optical QIP operates at a THz clock rate, representing a major step toward a true optical quantum computer which opens the curtain to a new era of ultra-fast information processing., Comment: 15 pages, 4 figures

We introduce DataTales, a novel benchmark designed to assess the proficiency of language models in data narration, a task crucial for transforming complex tabular data into accessible narratives. Existing benchmarks often fall short in capturing the requisite analytical complexity for practical applications. DataTales addresses this gap by offering 4.9k financial reports paired with corresponding market data, showcasing the demand for models to create clear narratives and analyze large datasets while understanding specialized terminology in the field. Our findings highlights the significant challenge that language models face in achieving the necessary precision and analytical depth for proficient data narration, suggesting promising avenues for future model development and evaluation methodologies.

Multiple stationary, localized structures were recently found for inclined porous medium convection with constant-temperature boundaries. We analyze traveling asymmetric, localized convective structures, consisting of 1 to 5 pulses, in a 2D inclined porous layer with fixed temperature at the bottom and an imperfectly conducting boundary at the top, such that midplane reflection symmetry is broken. Direct numerical simulations (DNS) are performed with different Biot numbers at the top boundary. The drift velocity $c$ of pulses is measured for different values of the symmetry parameter $\kappa\geq0$ based on the Biot number, with perfect midplane reflection symmetry and $c=0$ at $\kappa=0$. In small domains, the drift velocity $c>0$ (upslope), increases monotonically with $\kappa$, while in large domains $c$ changes sign depending on parameters. We show that pulse tails, controlling interactions, depend on the dominant spatial eigenvalues, whose real part is closest to zero, with a transition at $\kappa_c>0$: below $\kappa_c$, both dominant eigenvalues are complex, and the tails are oscillatory. Above $\kappa_c$, the dominant spatial eigenvalue with a positive real part becomes real, and the downslope tail transitions from oscillatory to monotonic. Below $\kappa_c$, bound states with different numbers of pulses exist whose collisions are studied. Well above $\kappa_c$, adjacent pulses repel each other, spreading out and becoming equispaced in the domain. A reduced model is proposed based on the interaction via tails of adjacent pulses, reproducing the repulsion and collisions from DNS. The model shows that the transition from bound states to equidistant spreading occurs when the monotonic/oscillatory tails have the same slope. This study elucidates the motion of localized patterns in moderate-Rayleigh number convection in an inclined porous layer with an imperfectly conducting boundary.

Large language models have revolutionized data processing in numerous domains, with their ability to handle extended context reasoning receiving notable recognition. To speed up inference, maintaining a key-value (KV) cache memory is essential. Nonetheless, the growing demands for KV cache memory create significant hurdles for efficient implementation. This work introduces a novel architecture, Cross-Layer Latent Attention (CLLA), aimed at compressing the KV cache to less than 2% of its original size while maintaining comparable performance levels. CLLA integrates multiple aspects of KV cache compression, including attention head/dimension reduction, layer sharing, and quantization techniques, into a cohesive framework. Our extensive experiments demonstrate that CLLA achieves lossless performance on most tasks while utilizing minimal KV cache, marking a significant advancement in practical KV cache compression.