Search

Global allocations in the upper mid-band spectrum (4--24 GHz) necessitate a comprehensive exploration of the propagation behavior to meet the promise of coverage and capacity. This paper presents an extensive Urban Microcell (UMi) outdoor propagation measurement campaign at 6.75 GHz and 16.95 GHz conducted in Downtown Brooklyn, USA, using a 1 GHz bandwidth sliding correlation channel sounder over 40--880 m propagation distance, encompassing 7 Line of Sight (LOS) and 13 Non-Line of Sight (NLOS) locations. Analysis of the path loss (PL) reveals lower directional and omnidirectional PL exponents compared to mmWave and sub-THz frequencies in the UMi environment, using the close-in PL model with a 1 m reference distance. Additionally, a decreasing trend in root mean square (RMS) delay spread (DS) and angular spread (AS) with increasing frequency was observed. The NLOS RMS DS and RMS AS mean values are obtained consistently lower compared to 3GPP model predictions. Point data for all measured statistics at each TX-RX location are provided to supplement the models and results. The spatio-temporal statistics evaluated here offer valuable insights for the design of next-generation wireless systems and networks., Comment: 6 pages, 4 figures, 6 tables

In this and a companion paper [arXiv:2410.09277], we give a general and comprehensive theory for nonequilibrium (NEQ) network forces and flows (Caliber Force Theory, CFT). It follows the "Two Laws" structure of Equilibrium Thermodynamics, where a First Law asserts conservation constraints and a Second Law is a variational predictor of a system's status, that, when taken together, give a rich set of prediction tools. The novel results for network flows here are that: (1) CFT defines forces, in this case for the three independent dynamic quantities: node distribution, edge traffic, and cycle flux, allowing for treating complex nets with conflicting constraints, actions, costs and benefits. (2) CFT is not limited to approximations and assumptions about nearness to equilibria, Local Detailed Balance or heat baths. (3) CFT shows that Kirchhoff current and voltage laws are an incomplete set; we find that a third such law applies when fluctuations are important.

We present SpectroMotion, a novel approach that combines 3D Gaussian Splatting (3DGS) with physically-based rendering (PBR) and deformation fields to reconstruct dynamic specular scenes. Previous methods extending 3DGS to model dynamic scenes have struggled to accurately represent specular surfaces. Our method addresses this limitation by introducing a residual correction technique for accurate surface normal computation during deformation, complemented by a deformable environment map that adapts to time-varying lighting conditions. We implement a coarse-to-fine training strategy that significantly enhances both scene geometry and specular color prediction. We demonstrate that our model outperforms prior methods for view synthesis of scenes containing dynamic specular objects and that it is the only existing 3DGS method capable of synthesizing photorealistic real-world dynamic specular scenes, outperforming state-of-the-art methods in rendering complex, dynamic, and specular scenes., Comment: Project page: https://cdfan0627.github.io/spectromotion/

Motivated by the first observation of a doubly charmed tetraquark candidate $T_{cc}(3875)^+$, we perform a systematic calculation of the mass spectra of doubly charmed tetraquark states from $1S$ to $2P$ excitations in a nonrelativistic constituent quark potential model. In terms of the quark-quark potential, the mass of the charmed spin-$1$ diquark is predicted $\sim 3500$ MeV. The masses of the light ``good'' and ``bad'' antidiquark are calculated with $\sim 670$ MeV and $\sim 840$ MeV, respectively. The mass difference between light ``good'' and ``bad'' diquarks is $\sim 170$ MeV, which is consistent with previous phenomenological analyses. The interaction between the charmed diquark and the light antidiquark is then modulated by the quark-antiquark potential in the model, and the mass spectra of doubly charmed tetraquarks from $1S$ to $2P$ excitations are calculated with the calculated masses of diquarks/antidiquarks and refitted parameters from $T_{cc}(3875)^+$ and $X(3872)$. The $1S-1P$ and $1S-2S$ mass splittings of doubly charmed tetraquarks are about $410-430$ MeV and $610-650$ MeV, respectively, and this mass splittings pattern is similar to that for ordinary $D$ mesons. The mass splittings of doubly charmed tetraquarks with isospin-$0$ is about $10-25$ MeV higher than the corresponding mass splittings of doubly charmed tetraquarks with isospin-$1$ from $1S$ to $2P$ excitations. $T_{cc}(3875)^+$ is possible to be assumed as a $IJ^P=01^+$ ground doubly charmed tetraquark candidate with a light ``good'' antidiquark, and the next $P$-wave excited doubly charmed tetraquarks are predicted to have masses with an average $420-430$ MeV higher. A $J^P=0^+$ ground doubly charmed tetraquark $T_{cc}(3834)$ with isospin-$1$ located around $3834$ MeV is expected., Comment: 10 pages, 8 tables, PDFLatex

FlashAttention series has been widely applied in the inference of large language models (LLMs). However, FlashAttention series only supports the high-level GPU architectures, e.g., Ampere and Hopper. At present, FlashAttention series is not easily transferrable to NPUs and low-resource GPUs. Moreover, FlashAttention series is inefficient for multi- NPUs or GPUs inference scenarios. In this work, we propose FastAttention which pioneers the adaptation of FlashAttention series for NPUs and low-resource GPUs to boost LLM inference efficiency. Specifically, we take Ascend NPUs and Volta-based GPUs as representatives for designing our FastAttention. We migrate FlashAttention series to Ascend NPUs by proposing a novel two-level tiling strategy for runtime speedup, tiling-mask strategy for memory saving and the tiling-AllReduce strategy for reducing communication overhead, respectively. Besides, we adapt FlashAttention for Volta-based GPUs by redesigning the operands layout in shared memory and introducing a simple yet effective CPU-GPU cooperative strategy for efficient memory utilization. On Ascend NPUs, our FastAttention can achieve a 10.7$\times$ speedup compared to the standard attention implementation. Llama-7B within FastAttention reaches up to 5.16$\times$ higher throughput than within the standard attention. On Volta architecture GPUs, FastAttention yields 1.43$\times$ speedup compared to its equivalents in \texttt{xformers}. Pangu-38B within FastAttention brings 1.46$\times$ end-to-end speedup using FasterTransformer. Coupled with the propose CPU-GPU cooperative strategy, FastAttention supports a maximal input length of 256K on 8 V100 GPUs. All the codes will be made available soon.

Reinforcement learning (RL) policies are prone to high-frequency oscillations, especially undesirable when deploying to hardware in the real-world. In this paper, we identify, categorize, and compare methods from the literature that aim to mitigate high-frequency oscillations in deep RL. We define two broad classes: loss regularization and architectural methods. At their core, these methods incentivize learning a smooth mapping, such that nearby states in the input space produce nearby actions in the output space. We present benchmarks in terms of policy performance and control smoothness on traditional RL environments from the Gymnasium and a complex manipulation task, as well as three robotics locomotion tasks that include deployment and evaluation with real-world hardware. Finally, we also propose hybrid methods that combine elements from both loss regularization and architectural methods. We find that the best-performing hybrid outperforms other methods, and improves control smoothness by 26.8% over the baseline, with a worst-case performance degradation of just 2.8%., Comment: Presented in IROS 2024

An extreme ultraviolet (EUV) close-up view of the Sun offers unprecedented detail of heating events in the solar corona. Enhanced temporal and spatial images obtained by the Solar Orbiter during its first science perihelion enabled us to identify clustered EUV bright tadpoles (CEBTs) occurring near the footpoints of coronal loops. Combining SDO/AIA observations, we determine the altitudes of six distinct CEBTs by stereoscopy, ranging from $\sim$1300 to 3300 km. We then notice a substantial presence of dark, cooler filamentary structures seemingly beneath the CEBTs, displaying periodic up-and-down motions lasting 3 to 5 minutes. This periodic behavior suggests an association of the majority of CEBTs with Type I spicules. Out of the ten selected CEBTs with fast downward velocity, six exhibit corrected velocities close to or exceeding 50 km $s^{-1}$. These velocities notably surpass the typical speeds of Type I spicules. We explore the generation of such velocities. It indicates that due to the previous limited observations of spicules in the EUV wavelengths, they may reveal novel observational features beyond our current understanding. Gaining insights into these features contributes to a better comprehension of small-scale coronal heating dynamics., Comment: 17 pages, 10 figures, accepted for publication in RAA

Large Language Models (LLMs) have emerged as powerful tools in mathematical theorem proving, particularly when utilizing formal languages such as LEAN. The major learning paradigm is expert iteration, which necessitates a pre-defined dataset comprising numerous mathematical problems. In this process, LLMs attempt to prove problems within the dataset and iteratively refine their capabilities through self-training on the proofs they discover. We propose to use large scale LEAN problem datasets Lean-workbook for expert iteration with more than 20,000 CPU days. During expert iteration, we found log-linear trends between solved problem amount with proof length and CPU usage. We train a critic model to select relatively easy problems for policy models to make trials and guide the model to search for deeper proofs. InternLM2.5-StepProver achieves open-source state-of-the-art on MiniF2F, Lean-Workbook-Plus, ProofNet, and Putnam benchmarks. Specifically, it achieves a pass of 65.9% on the MiniF2F-test and proves (or disproves) 17.0% of problems in Lean-Workbook-Plus which shows a significant improvement compared to only 9.5% of problems proved when Lean-Workbook-Plus was released. We open-source our models and searched proofs at https://github.com/InternLM/InternLM-Math and https://huggingface.co/datasets/internlm/Lean-Workbook.

Image dehazing has drawn a significant attention in recent years. Learning-based methods usually require paired hazy and corresponding ground truth (haze-free) images for training. However, it is difficult to collect real-world image pairs, which prevents developments of existing methods. Although several works partially alleviate this issue by using synthetic datasets or small-scale real datasets. The haze intensity distribution bias and scene homogeneity in existing datasets limit the generalization ability of these methods, particularly when encountering images with previously unseen haze intensities. In this work, we present LMHaze, a large-scale, high-quality real-world dataset. LMHaze comprises paired hazy and haze-free images captured in diverse indoor and outdoor environments, spanning multiple scenarios and haze intensities. It contains over 5K high-resolution image pairs, surpassing the size of the biggest existing real-world dehazing dataset by over 25 times. Meanwhile, to better handle images with different haze intensities, we propose a mixture-of-experts model based on Mamba (MoE-Mamba) for dehazing, which dynamically adjusts the model parameters according to the haze intensity. Moreover, with our proposed dataset, we conduct a new large multimodal model (LMM)-based benchmark study to simulate human perception for evaluating dehazed images. Experiments demonstrate that LMHaze dataset improves the dehazing performance in real scenarios and our dehazing method provides better results compared to state-of-the-art methods.

The reference field method, known as the difference formulation, is a key variance reduction technique for Monte Carlo simulations of thermal radiation transport problems. When the material temperature is relatively high and the spatial temperature gradient is moderate, this method demonstrates significant advantages in reducing variance compared to classical Monte Carlo methods. However, in problems with larger temperature gradients, this method has not only been found ineffective at reducing statistical noise, but in some cases, it even increases noise compared to classical Monte Carlo methods. The global optimal reference field method, a recently proposed variance reduction technique, effectively reduces the average energy weight of Monte Carlo particles, thereby decreasing variance. Its effectiveness has been validated both theoretically and numerically, demonstrating a significant reduction in statistical errors for problems with large temperature gradients. In our previous work, instead of computing the exact global optimal reference field, we developed an approximate, physically motivated method to find a relatively better reference field using a selection scheme. In this work, we reformulate the problem of determining the global optimal reference field as a linear programming problem and solve it exactly. To further enhance computational efficiency, we use the MindOpt solver, which leverages graph neural network methods. Numerical experiments demonstrate that the MindOpt solver not only solves linear programming problems accurately but also significantly outperforms the Simplex and interior-point methods in terms of computational efficiency. The global optimal reference field method combined with the MindOpt solver not only improves computational efficiency but also substantially reduces statistical errors.

Approximate Nearest Neighbor Search (ANNS), which enables efficient semantic similarity search in large datasets, has become a fundamental component of critical applications such as information retrieval and retrieval-augmented generation (RAG). However, ANNS is a well-known I/O-intensive algorithm with a low compute-to-I/O ratio, often requiring massive storage due to the large volume of high-dimensional data. This leads to I/O bottlenecks on CPUs and memory limitations on GPUs. DRAM-based Processing-in-Memory (DRAM-PIM) architecture, which offers high bandwidth, large-capacity memory, and the ability to perform efficient computation in or near the data, presents a promising solution for ANNS. In this work, we investigate the use of commercial DRAM-PIM for ANNS for the first time and propose DRIM-ANN, an optimized ANNS engine based on DRAM-PIMs from UPMEM. Notably, given that the target DRAM-PIM exhibits an even lower compute-to-I/O ratio than basic ANNS, we leverage lookup tables (LUTs) to replace more multiplications with I/O operations. We then systematically tune ANNS to search optimized configurations with lower computational load, aligning the compute-to-I/O ratio of ANNS with that of DRAM-PIMs while maintaining accuracy constraints. Building on this tuned ANNS algorithm, we further explore implementation optimizations to fully utilize the two thousand parallel processing units with private local memory in DRAM-PIMs. To address the load imbalance caused by ANNS requests distributed across different clusters of large datasets, we propose a load-balancing strategy that combines static data layout optimization with dynamic runtime request scheduling. Experimental results on representative datasets show that DRIM-ANN achieves an average performance speedup of 2.92x compared to a 32-thread CPU counterpart.

Cross-modal metric learning is a prominent research topic that bridges the semantic heterogeneity between vision and language. Existing methods frequently utilize simple cosine or complex distance metrics to transform the pairwise features into a similarity score, which suffers from an inadequate or inefficient capability for distance measurements. Consequently, we propose a Generalized Structural Sparse Function to dynamically capture thorough and powerful relationships across modalities for pair-wise similarity learning while remaining concise but efficient. Specifically, the distance metric delicately encapsulates two formats of diagonal and block-diagonal terms, automatically distinguishing and highlighting the cross-channel relevancy and dependency inside a structured and organized topology. Hence, it thereby empowers itself to adapt to the optimal matching patterns between the paired features and reaches a sweet spot between model complexity and capability. Extensive experiments on cross-modal and two extra uni-modal retrieval tasks (image-text retrieval, person re-identification, fine-grained image retrieval) have validated its superiority and flexibility over various popular retrieval frameworks. More importantly, we further discover that it can be seamlessly incorporated into multiple application scenarios, and demonstrates promising prospects from Attention Mechanism to Knowledge Distillation in a plug-and-play manner. Our code is publicly available at: https://github.com/Paranioar/GSSF., Comment: 12 pages, 9 figures, Accepted by TIP2024

This research is part of a systematic study of scientific time series. In the last three years, hundreds of papers and over fifty new deep-learning models have been described for time series models. These mainly focus on the key aspect of time dependence, whereas in some scientific time series, the situation is more complex with multiple locations, each location having multiple observed and target time-dependent streams and multiple exogenous (known) properties that are either constant or time-dependent. Here, we analyze the hydrology time series using the CAMELS and Caravan global datasets on catchment rainfall and runoff. Together, these have up to 6 observed streams and up to 209 static parameters defined at each of about 8000 locations. This analysis is fully open source with a Jupyter Notebook running on Google Colab for both an LSTM-based analysis and the data engineering preprocessing. Our goal is to investigate the importance of exogenous data, which we look at using eight different choices on representative hydrology tasks. Increasing the exogenous information significantly improves the data representation, with the mean square error decreasing to 60% of its initial value in the largest dataset examined. We present the initial results of studies of other deep-learning neural network architectures where the approaches that can use the full observed and exogenous observations outperform less flexible methods, including Foundation models. Using the natural annual periodic exogenous time series produces the largest impact, but the static and other periodic exogenous streams are also important. Our analysis is intended to be valuable as an educational resource and benchmark.

Style transfer aims to fuse the artistic representation of a style image with the structural information of a content image. Existing methods train specific networks or utilize pre-trained models to learn content and style features. However, they rely solely on textual or spatial representations that are inadequate to achieve the balance between content and style. In this work, we propose a novel and training-free approach for style transfer, combining textual embedding with spatial features and separating the injection of content or style. Specifically, we adopt the BLIP-2 encoder to extract the textual representation of the style image. We utilize the DDIM inversion technique to extract intermediate embeddings in content and style branches as spatial features. Finally, we harness the step-by-step property of diffusion models by separating the injection of content and style in the target branch, which improves the balance between content preservation and style fusion. Various experiments have demonstrated the effectiveness and robustness of our proposed DiffeseST for achieving balanced and controllable style transfer results, as well as the potential to extend to other tasks., Comment: Accepted to ACMMM Asia 2024. Code is available at https://github.com/I2-Multimedia-Lab/DiffuseST

We study the finite horizon Restless Multi-Armed Bandit (RMAB) problem with $N$ homogeneous arms, focusing on the challenges posed by degenerate RMABs, which are prevalent in practical applications. While previous work has shown that Linear Programming (LP)-based policies achieve exponentially fast convergence relative to the LP upper bound in non-degenerate models, applying these LP-based policies to degenerate RMABs results in slower convergence rates of $O(1/\sqrt{N})$. We construct a diffusion system that incorporates both the mean and variance of the stochastic processes, in contrast to the fluid system from the LP, which only accounts for the mean, thereby providing a more accurate representation of RMAB dynamics. Consequently, our novel diffusion-resolving policy achieves an optimality gap of $O(1/N)$ relative to the true optimal value, rather than the LP upper bound, revealing that the fluid approximation and the LP upper bound are too loose in degenerate settings. These insights pave the way for constructing policies that surpass the $O(1/\sqrt{N})$ optimality gap for any RMAB, whether degenerate or not., Comment: 31 pages, 6 figures

We establish a high-resolution, high-performance, and high-confidence compressible multiphysics system in a Cartesian grid with irregular boundary topologies to simulate intensive blast waves propagating in large-scale and extremely complex environments. The multiphysics system is modeled by a multi-component model solved using a generalized Godunov method and a classical material point method in a combination of Lagrangian particles and a rigid material model. An artificial neural network equation of state (EOS) is proposed based on experimental data to simulate the intensive explosion products and real gas under extreme pressure and temperature. To improve computational accuracy and efficiency, a deepMTBVD reconstruction scheme of our previous work is extended to the multiphysics system. With the aid of high-performance parallel computation, several large-scale blast wave applications, such as blast wave propagating in a local and entire urban city, are simulated in a reasonable time period, which can validate numerical schemes and lead to more practical engineering applications.

Lung cancer remains one of the leading causes of morbidity and mortality worldwide, making early diagnosis critical for improving therapeutic outcomes and patient prognosis. Computer-aided diagnosis (CAD) systems, which analyze CT images, have proven effective in detecting and classifying pulmonary nodules, significantly enhancing the detection rate of early-stage lung cancer. Although traditional machine learning algorithms have been valuable, they exhibit limitations in handling complex sample data. The recent emergence of deep learning has revolutionized medical image analysis, driving substantial advancements in this field. This review focuses on recent progress in deep learning for pulmonary nodule detection, segmentation, and classification. Traditional machine learning methods, such as SVM and KNN, have shown limitations, paving the way for advanced approaches like Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), and Generative Adversarial Networks (GAN). The integration of ensemble models and novel techniques is also discussed, emphasizing the latest developments in lung cancer diagnosis. Deep learning algorithms, combined with various analytical techniques, have markedly improved the accuracy and efficiency of pulmonary nodule analysis, surpassing traditional methods, particularly in nodule classification. Although challenges remain, continuous technological advancements are expected to further strengthen the role of deep learning in medical diagnostics, especially for early lung cancer detection and diagnosis. A comprehensive list of lung cancer detection models reviewed in this work is available at https://github.com/CaiGuoHui123/Awesome-Lung-Cancer-Detection

Exceptional points, a remarkable phenomenon in physical systems, have been exploited for sensing applications. It has been demonstrated recently that it can also utilize as sensory threshold in which the interplay between exceptional-point dynamics and noise can lead to enhanced performance. Most existing works focused on second-order exceptional points. We investigate the stochastic dynamics associated with high-order exceptional points with a particular eye towards optimizing sensing performance by developing a theoretical framework based on pseudo-Hermiticity. Our analysis reveals three distinct types of frequency responses to external perturbations. A broad type of stochastic resonance is uncovered where, as the noise amplitude increases, the signal-to-noise ratio reaches a global maximum rapidly but with a slow decaying process afterwards, indicating achievable high performance in a wide range of the noise level. These results suggest that stochastic high-order exceptional-point dynamics can be exploited for applications in signal processing and sensor technologies., Comment: 13 pages, 9 figures

With the development of social economy and the improvement of people's attention to health, the growth and development of children and adolescents has become an important indicator to measure the level of national health. Therefore, accurate and timely assessment of children's growth and development has become increasingly important. At the same time, global health inequalities, especially child malnutrition and stunting in developing countries, urgently require effective assessment tools to monitor and intervene. In recent years, the rapid development of technologies such as big data, artificial intelligence, and cloud computing, and the cross-integration of multiple disciplines such as biomedicine, statistics, and computer science have promoted the rapid development of large-scale models for growth and development assessment. However, there are still problems such as too single evaluation factors, inaccurate diagnostic results, and inability to give accurate and reasonable recommendations. The multi-modal growth and development assessment model uses the public data set of RSNA ( North American College of Radiology ) as the training set, and the data set of the Department of Pediatrics of Huaibei People's Hospital as the open source test set. The embedded ICL module enables the model to quickly adapt and identify the tasks that need to be done to ensure that under the premise of considering multiple evaluation factors, accurate diagnosis results and reasonable medical recommendations are given, so as to provide solutions to the above problems and promote the development of the medical field., Comment: 7 Pages 7 Figures

Multi-camera systems are indispensable in movies, TV shows, and other media. Selecting the appropriate camera at every timestamp has a decisive impact on production quality and audience preferences. Learning-based view recommendation frameworks can assist professionals in decision-making. However, they often struggle outside of their training domains. The scarcity of labeled multi-camera view recommendation datasets exacerbates the issue. Based on the insight that many videos are edited from the original multi-camera videos, we propose transforming regular videos into pseudo-labeled multi-camera view recommendation datasets. Promisingly, by training the model on pseudo-labeled datasets stemming from videos in the target domain, we achieve a 68% relative improvement in the model's accuracy in the target domain and bridge the accuracy gap between in-domain and never-before-seen domains., Comment: Accepted to VCIP 2024. Project page: https://eric11220.github.io/publication/VCIP24/

Recent advancements in large language models (LLMs) have demonstrated that progressive refinement, rather than providing a single answer, results in more accurate and thoughtful outputs. However, existing methods often rely heavily on supervision signals to evaluate previous responses, making it difficult to assess output quality in more open-ended scenarios effectively. Additionally, these methods are typically designed for specific tasks, which limits their generalization to new domains. To address these limitations, we propose Progressive Thought Refinement (PTR), a framework that enables LLMs to refine their responses progressively. PTR operates in two phases: (1) Thought data construction stage: We propose a weak and strong model collaborative selection strategy to build a high-quality progressive refinement dataset to ensure logical consistency from thought to answers, and the answers are gradually refined in each round. (2) Thought-Mask Fine-Tuning Phase: We design a training structure to mask the "thought" and adjust loss weights to encourage LLMs to refine prior thought, teaching them to implicitly understand "how to improve" rather than "what is correct." Experimental results show that PTR significantly enhances LLM performance across ten diverse tasks (avg. from 49.6% to 53.5%) without task-specific fine-tuning. Notably, in more open-ended tasks, LLMs also demonstrate substantial improvements in the quality of responses beyond mere accuracy, suggesting that PTR truly teaches LLMs to self-improve over time., Comment: 10 pages, 4 figures

The complexity and nonlinearity of active distribution network (ADN), coupled with the fast-changing renewable energy (RE), necessitate advanced real-time and safe dispatch approach. This paper proposes a complementary reinforcement learning (RL) and optimization approach, namely SA2CO, to address the coordinated dispatch of the energy storage systems (ESSs) in the ADN. The proposed approach leverages RL's capability to make fast decision and address the model inaccuracies, while optimization methods ensure the ADN security. Furthermore, a hybrid data-driven and expert-experience auxiliary neural network is formulated as a rapid security assessment component in the SA2CO algorithm, enabling dynamic switching between RL and optimization methodologies. Simulation results demonstrate the proposed method's effectiveness and scalability in achieving real-time, safe, and economical dispatch of multiple ESSs in the ADN, surpassing the performance of the state-of-the-art RL and optimization methods.

This work demonstrates a novel energy-efficient tunnel FET (TFET)-CMOS hybrid foundry platform for ultralow-power AIoT applications. By utilizing the proposed monolithic integration process, the novel complementary n and p-type Si TFET technology with dopant segregated source junction and self-aligned drain underlap design is successfully integrated into a 300mm CMOS baseline process without CMOS performance penalty and any new materials, experimentally demonstrating the large Ion and record high Ion/Ioff ratio of 10^7 among TFETs by industry-manufacturers. The device performance and variability are also co-optimized for high-volume production. Further circuit-level implementations are presented based on the calibrated compact model. The proposed TFET-CMOS hybrid logic and SRAM topologies show significant energy efficiency improvement with comparable operation speed compared with standard CMOS circuits, indicating its great potential for power-constraint AIoT applications.

Deterministic creation of quantum emitters with high single-photon-purity and excellent indistinguishability is essential for practical applications in quantum information science. Many successful attempts have been carried out in hexagonal boron nitride showing its capability of hosting room temperature quantum emitters. However, most of the existing methods produce emitters with heterogeneous optical properties and unclear creation mechanisms. Here, the authors report a deterministic creation of identical room temperature quantum emitters using masked-carbon-ion implantation on freestanding hBN flakes. Quantum emitters fabricated by our approach showed thermally limited monochromaticity with an emission center wavelength distribution of 590.7 +- 2.7 nm, a narrow full width half maximum of 7.1 +- 1.7 nm, excellent brightness (1MHz emission rate), and extraordinary stability. Our method provides a reliable platform for characterization and fabrication research on hBN based quantum emitters, helping to reveal the origins of the single-photon-emission behavior in hBN and favoring practical applications, especially the industrial-scale production of quantum technology., Comment: 29 pages, 5 figures, research article

In this work, we first develop a unified Bhatnagar-Gross-Krook lattice Boltzmann (BGK-LB) model for the $d$($d\geq 1$)-dimensional linear hyperbolic equation (L-HE), where the natural moments and the D$d$Q$(2d^2+1)$ [($2d^2+1$) discrete velocities in $d$-dimensional space] lattice structure are considered. Subsequently, at the acoustic scaling, we conduct an accuracy analysis on the developed BGK-LB model by the direct Taylor expansion (DTE) method, and present the second- and third-order moments of the equilibrium distribution functions (EDFs) to ensure that the BGK-LB model can be fourth-order consistent with the L-HE. And on this basis, when considering the Dirichlet boundary condition, the fourth-order full-way and half-way boundary schemes are proposed to approximate the unknown distribution functions to ensure that the BGK-LB model can be overall fourth-order accurate. Thereafter, based on the kinetic entropy theory, we derive the conditions that the fourth-order moments of the EDFs should satisfy to ensure the microscopic entropy stability of the BGK-LB model. In addition, with the aid of the von Neumann stability analysis, we also discuss the $L^2$ stability of the BGK-LB model and numerically plot the stability regions. In particular, from a numerical perspective, we find that the region of microscopic entropy stability is identical to that of $L^2$ stability. Finally, we carry out some numerical experiments to test the accuracy and stability of the BGK-LB model, and the numerical results are in agreement with our theoretical analysis. In addition, we compare the developed full-way and half-way boundary schemes for the Dirichlet boundary condition, which shows that the full-way boundary scheme is more stable.

We study the interplay between repulsive interaction and superfluidity in flat band system. We consider spatially indirect excitons in the Lieb lattice bilayer as an example. We show that due to the presence of repulsive interaction, the excitons may form an incoherent fluid. By increasing the quantum metric, the exciton fluid experiences a transition into the superfluid phase. Such transition may be captured by a model of Josephson junction array, which is beyond mean field description., Comment: Maintext: 6 pages, 4 figures; Supplemental material: 8 pages, 1 figure

Recent self-rewarding large language models (LLM) have successfully applied LLM-as-a-Judge to iteratively improve the alignment performance without the need of human annotations for preference data. These methods commonly utilize the same LLM to act as both the policy model (which generates responses) and the reward model (which scores and ranks those responses). The ranked responses are then used as preference pairs to train the LLM via direct alignment technologies (e.g. DPO). However, it is noteworthy that throughout this process, there is no guarantee of accuracy in the rewarding and ranking, which is critical for ensuring accurate rewards and high-quality preference data. Empirical results from relatively small LLMs (e.g., 7B parameters) also indicate that improvements from self-rewarding may diminish after several iterations in certain situations, which we hypothesize is due to accumulated bias in the reward system. This bias can lead to unreliable preference data for training the LLM. To address this issue, we first formulate and analyze the generalized iterative preference fine-tuning framework for self-rewarding language model. We then introduce the regularization to this generalized framework to mitigate the overconfident preference labeling in the self-rewarding process. Based on this theoretical insight, we propose a Consistency Regularized sElf-rewarding lAnguage Model (CREAM) that leverages the rewarding consistency across different iterations to regularize the self-rewarding training, helping the model to learn from more reliable preference data. With this explicit regularization, our empirical results demonstrate the superiority of CREAM in improving both reward consistency and alignment performance. The code is publicly available at https://github.com/Raibows/CREAM.

The Willmore Problem seeks the surface in $\mathbb{S}^3\subset\mathbb{R}^4$ of a given topological type minimizing the squared-mean-curvature energy $W = \int |H_{\mathbb{R}^4}|^2 = area + \int |H_{\mathbb{S}^3}|^2$. The longstanding Willmore Conjecture that the Clifford torus minimizes $W$ among genus-$1$ surfaces is now a theorem of Marques and Neves [22], but the general conjecture \cite[12] that Lawson's [18] minimal surface $\xi_{g,1}\subset\mathbb{S}^3$ minimizes $W$ among surfaces of genus $g>1$ remains open. Here we prove this conjecture under the additional assumption that the competitor surfaces $M\subset\mathbb{S}^3$ share the ambient symmetries $\widehat{G}_{g,1}$ of $\xi_{g,1}$. In fact, we show each Lawson surface $\xi_{m,k}$ satisfies the analogous $W$-minimizing property under a smaller symmetry group $\widetilde{G}_{m,k}=\widehat{G}_{m,k}\cap SO(4)$. We also describe a genus 2 example where known methods do not ensure the existence of a $W$-minimizer among surfaces with its symmetry., Comment: 16 pages, 8 figures. This supersedes our previous paper arXiv:2103.09432

Advancements in reinforcement learning have led to the development of sophisticated models capable of learning complex decision-making tasks. However, efficiently integrating world models with decision transformers remains a challenge. In this paper, we introduce a novel approach that combines the Dreamer algorithm's ability to generate anticipatory trajectories with the adaptive learning strengths of the Online Decision Transformer. Our methodology enables parallel training where Dreamer-produced trajectories enhance the contextual decision-making of the transformer, creating a bidirectional enhancement loop. We empirically demonstrate the efficacy of our approach on a suite of challenging benchmarks, achieving notable improvements in sample efficiency and reward maximization over existing methods. Our results indicate that the proposed integrated framework not only accelerates learning but also showcases robustness in diverse and dynamic scenarios, marking a significant step forward in model-based reinforcement learning.

Quantization is a widely-used compression technology to reduce the overhead of serving large language models (LLMs) on terminal devices and in cloud data centers. However, prevalent quantization methods, such as 8-bit weight-activation or 4-bit weight-only quantization, achieve limited performance improvements due to poor support for low-precision (e.g., 4-bit) activation. This work, for the first time, realizes practical W4A4KV4 serving for LLMs, fully utilizing the INT4 tensor cores on modern GPUs and reducing the memory bottleneck caused by the KV cache. Specifically, we propose a novel fine-grained mixed-precision quantization algorithm (FMPQ) that compresses most activations into 4-bit with negligible accuracy loss. To support mixed-precision matrix multiplication for W4A4 and W4A8, we develop a highly optimized W4Ax kernel. Our approach introduces a novel mixed-precision data layout to facilitate access and fast dequantization for activation and weight tensors, utilizing the GPU's software pipeline to hide the overhead of data loading and conversion. Additionally, we propose fine-grained streaming multiprocessor (SM) scheduling to achieve load balance across different SMs. We integrate the optimized W4Ax kernel into our inference framework, COMET, and provide efficient management to support popular LLMs such as LLaMA-3-70B. Extensive evaluations demonstrate that, when running LLaMA family models on a single A100-80G-SMX4, COMET achieves a kernel-level speedup of \textbf{$2.88\times$} over cuBLAS and a \textbf{$2.02 \times$} throughput improvement compared to TensorRT-LLM from an end-to-end framework perspective., Comment: 14 pages, 12 figures

Graph representation learning, involving both node features and graph structures, is crucial for real-world applications but often encounters pervasive noise. State-of-the-art methods typically address noise by focusing separately on node features with large language models (LLMs) and on graph structures with graph structure learning models (GSLMs). In this paper, we introduce LangGSL, a robust framework that integrates the complementary strengths of pre-trained language models and GSLMs to jointly enhance both node feature and graph structure learning. In LangGSL, we first leverage LLMs to filter noise in the raw data and extract valuable cleaned information as features, enhancing the synergy of downstream models. During the mutual learning phase in LangGSL, the core idea is to leverage the relatively small language model (LM) to process local attributes and generate reliable pseudo-labels and informative node embeddings, which are then integrated into the GSLM's prediction phase. This approach enriches the global context and enhances overall performance. Meanwhile, GSLM refines the evolving graph structure constructed from the LM's output, offering updated labels back to the LM as additional guidance, thus facilitating a more effective mutual learning process. The LM and GSLM work synergistically, complementing each other's strengths and offsetting weaknesses within a variational information-maximizing framework, resulting in enhanced node features and a more robust graph structure. Extensive experiments on diverse graph datasets of varying scales and across different task scenarios demonstrate the scalability and effectiveness of the proposed approach., Comment: Graph structure learning, Graph representation learning, Large language models, Graph neural networks

Recent progress in blind face restoration has resulted in producing high-quality restored results for static images. However, efforts to extend these advancements to video scenarios have been minimal, partly because of the absence of benchmarks that allow for a comprehensive and fair comparison. In this work, we first present a fair evaluation benchmark, in which we first introduce a Real-world Low-Quality Face Video benchmark (RFV-LQ), evaluate several leading image-based face restoration algorithms, and conduct a thorough systematical analysis of the benefits and challenges associated with extending blind face image restoration algorithms to degraded face videos. Our analysis identifies several key issues, primarily categorized into two aspects: significant jitters in facial components and noise-shape flickering between frames. To address these issues, we propose a Temporal Consistency Network (TCN) cooperated with alignment smoothing to reduce jitters and flickers in restored videos. TCN is a flexible component that can be seamlessly plugged into the most advanced face image restoration algorithms, ensuring the quality of image-based restoration is maintained as closely as possible. Extensive experiments have been conducted to evaluate the effectiveness and efficiency of our proposed TCN and alignment smoothing operation. Project page: https://wzhouxiff.github.io/projects/FIR2FVR/FIR2FVR., Comment: Accepted by TIP'2024; Project page: https://wzhouxiff.github.io/projects/FIR2FVR/FIR2FVR

Despite the progress made by multimodal large language models (MLLMs) in computational pathology, they remain limited by a predominant focus on patch-level analysis, missing essential contextual information at the whole-slide level. The lack of large-scale instruction datasets and the gigapixel scale of whole slide images (WSIs) pose significant developmental challenges. In this paper, we present SlideChat, the first vision-language assistant capable of understanding gigapixel whole-slide images, exhibiting excellent multimodal conversational capability and response complex instruction across diverse pathology scenarios. To support its development, we created SlideInstruction, the largest instruction-following dataset for WSIs consisting of 4.2K WSI captions and 176K VQA pairs with multiple categories. Furthermore, we propose SlideBench, a multimodal benchmark that incorporates captioning and VQA tasks to assess SlideChat's capabilities in varied clinical settings such as microscopy, diagnosis. Compared to both general and specialized MLLMs, SlideChat exhibits exceptional capabilities achieving state-of-the-art performance on 18 of 22 tasks. For example, it achieved an overall accuracy of 81.17% on SlideBench-VQA (TCGA), and 54.15% on SlideBench-VQA (BCNB). We will fully release SlideChat, SlideInstruction and SlideBench as open-source resources to facilitate research and development in computational pathology.

The advent of pre-trained vision-language foundation models has revolutionized the field of zero/few-shot (i.e., low-shot) image recognition. The key challenge to address under the condition of limited training data is how to fine-tune pre-trained vision-language models in a parameter-efficient manner. Previously, numerous approaches tackling this challenge have been proposed. Meantime, a few survey papers are also published to summarize these works. However, there still lacks a unified computational framework to integrate existing methods together, identify their nature and support in-depth comparison. As such, this survey paper first proposes a unified computational framework from the perspective of Representer Theorem and then derives many of the existing methods by specializing this framework. Thereafter, a comparative analysis is conducted to uncover the differences and relationships between existing methods. Based on the analyses, some possible variants to improve the existing works are presented. As a demonstration, we extend existing methods by modeling inter-class correlation between representers in reproducing kernel Hilbert space (RKHS), which is implemented by exploiting the closed-form solution of kernel ridge regression. Extensive experiments on 11 datasets are conducted to validate the effectiveness of this method. Toward the end of this paper, we discuss the limitations and provide further research directions.

Cyber Threat Intelligence (CTI) reports are factual records compiled by security analysts through their observations of threat events or their own practical experience with attacks. In order to utilize CTI reports for attack detection, existing methods have attempted to map the content of reports onto system-level attack provenance graphs to clearly depict attack procedures. However, existing studies on constructing graphs from CTI reports suffer from problems such as weak natural language processing (NLP) capabilities, discrete and fragmented graphs, and insufficient attack semantic representation. Therefore, we propose a system called CRUcialG for the automated reconstruction of attack scenario graphs (ASGs) by CTI reports. First, we use NLP models to extract systematic attack knowledge from CTI reports to form preliminary ASGs. Then, we propose a four-phase attack rationality verification framework from the tactical phase with attack procedure to evaluate the reasonability of ASGs. Finally, we implement the relation repair and phase supplement of ASGs by adopting a serialized graph generation model. We collect a total of 10,607 CTI reports and generate 5,761 complete ASGs. Experimental results on CTI reports from 30 security vendors and DARPA show that the similarity of ASG reconstruction by CRUcialG can reach 84.54%. Compared with SOTA (EXTRACTOR and AttackG), the recall of CRUcialG (extraction of real attack events) can reach 88.13% and 94.46% respectively, which is 40% higher than SOTA on average. The F1-score of attack phase verification is able to reach 90.04%.

Cooperation as a self-organized collective behavior plays a significant role in the evolution of ecosystems and human society. Reinforcement learning (RL) offers a new perspective, distinct from imitation learning in evolutionary games, for exploring the mechanisms underlying its emergence. However, most existing studies with the public good game (PGG) employ a self-regarding setup or are on pairwise interaction networks. Players in the real world, however, optimize their policies based not only on their histories but also on the histories of their co-players, and the game is played in a group manner. In the work, we investigate the evolution of cooperation in the PGG under the other-regarding reinforcement learning evolutionary game (OR-RLEG) on hypergraph by combining the Q-learning algorithm and evolutionary game framework, where other players' action history is incorporated and the game is played on hypergraphs. Our results show that as the synergy factor increases, the parameter interval is divided into three distinct regions, the absence of cooperation (AC), medium cooperation (MC), and high cooperation (HC), accompanied by two abrupt transitions in the cooperation level near two transition points, respectively. Interestingly, we identify regular and anti-coordinated chessboard structures in the spatial pattern that positively contribute to the first cooperation transition but adversely affect the second. Furthermore, we provide a theoretical treatment for the first transition with an approximated first transition point and reveal that players with a long-sighted perspective and low exploration rate are more likely to reciprocate kindness with each other, thus facilitating the emergence of cooperation. Our findings contribute to understanding the evolution of human cooperation, where other-regarding information and group interactions are commonplace.

Despite the remarkable advancements in Visual Question Answering (VQA), the challenge of mitigating the language bias introduced by textual information remains unresolved. Previous approaches capture language bias from a coarse-grained perspective. However, the finer-grained information within a sentence, such as context and keywords, can result in different biases. Due to the ignorance of fine-grained information, most existing methods fail to sufficiently capture language bias. In this paper, we propose a novel causal intervention training scheme named CIBi to eliminate language bias from a finer-grained perspective. Specifically, we divide the language bias into context bias and keyword bias. We employ causal intervention and contrastive learning to eliminate context bias and improve the multi-modal representation. Additionally, we design a new question-only branch based on counterfactual generation to distill and eliminate keyword bias. Experimental results illustrate that CIBi is applicable to various VQA models, yielding competitive performance.

Humanoid robots capable of autonomous operation in diverse environments have long been a goal for roboticists. However, autonomous manipulation by humanoid robots has largely been restricted to one specific scene, primarily due to the difficulty of acquiring generalizable skills. Recent advances in 3D visuomotor policies, such as the 3D Diffusion Policy (DP3), have shown promise in extending these capabilities to wilder environments. However, 3D visuomotor policies often rely on camera calibration and point-cloud segmentation, which present challenges for deployment on mobile robots like humanoids. In this work, we introduce the Improved 3D Diffusion Policy (iDP3), a novel 3D visuomotor policy that eliminates these constraints by leveraging egocentric 3D visual representations. We demonstrate that iDP3 enables a full-sized humanoid robot to autonomously perform skills in diverse real-world scenarios, using only data collected in the lab. Videos are available at: https://humanoid-manipulation.github.io, Comment: Project website: https://humanoid-manipulation.github.io

In this work, we introduce FlexGen, a flexible framework designed to generate controllable and consistent multi-view images, conditioned on a single-view image, or a text prompt, or both. FlexGen tackles the challenges of controllable multi-view synthesis through additional conditioning on 3D-aware text annotations. We utilize the strong reasoning capabilities of GPT-4V to generate 3D-aware text annotations. By analyzing four orthogonal views of an object arranged as tiled multi-view images, GPT-4V can produce text annotations that include 3D-aware information with spatial relationship. By integrating the control signal with proposed adaptive dual-control module, our model can generate multi-view images that correspond to the specified text. FlexGen supports multiple controllable capabilities, allowing users to modify text prompts to generate reasonable and corresponding unseen parts. Additionally, users can influence attributes such as appearance and material properties, including metallic and roughness. Extensive experiments demonstrate that our approach offers enhanced multiple controllability, marking a significant advancement over existing multi-view diffusion models. This work has substantial implications for fields requiring rapid and flexible 3D content creation, including game development, animation, and virtual reality. Project page: https://xxu068.github.io/flexgen.github.io/., Comment: 16 pages, 13 figures

This paper addresses differential inference in time-varying parametric probabilistic models, like graphical models with changing structures. Instead of estimating a high-dimensional model at each time and inferring changes later, we directly learn the differential parameter, i.e., the time derivative of the parameter. The main idea is treating the time score function of an exponential family model as a linear model of the differential parameter for direct estimation. We use time score matching to estimate parameter derivatives. We prove the consistency of a regularized score matching objective and demonstrate the finite-sample normality of a debiased estimator in high-dimensional settings. Our methodology effectively infers differential structures in high-dimensional graphical models, verified on simulated and real-world datasets., Comment: Daniel J. Williams and Leyang Wang contributed equally to this work

Molecular generation and molecular property prediction are both crucial for drug discovery, but they are often developed independently. Inspired by recent studies, which demonstrate that diffusion model, a prominent generative approach, can learn meaningful data representations that enhance predictive tasks, we explore the potential for developing a unified generative model in the molecular domain that effectively addresses both molecular generation and property prediction tasks. However, the integration of these tasks is challenging due to inherent inconsistencies, making simple multi-task learning ineffective. To address this, we propose UniGEM, the first unified model to successfully integrate molecular generation and property prediction, delivering superior performance in both tasks. Our key innovation lies in a novel two-phase generative process, where predictive tasks are activated in the later stages, after the molecular scaffold is formed. We further enhance task balance through innovative training strategies. Rigorous theoretical analysis and comprehensive experiments demonstrate our significant improvements in both tasks. The principles behind UniGEM hold promise for broader applications, including natural language processing and computer vision., Comment: 11 pages, 5 figures

Existing benchmarks like NLGraph and GraphQA evaluate LLMs on graphs by focusing mainly on pairwise relationships, overlooking the high-order correlations found in real-world data. Hypergraphs, which can model complex beyond-pairwise relationships, offer a more robust framework but are still underexplored in the context of LLMs. To address this gap, we introduce LLM4Hypergraph, the first comprehensive benchmark comprising 21,500 problems across eight low-order, five high-order, and two isomorphism tasks, utilizing both synthetic and real-world hypergraphs from citation networks and protein structures. We evaluate six prominent LLMs, including GPT-4o, demonstrating our benchmark's effectiveness in identifying model strengths and weaknesses. Our specialized prompting framework incorporates seven hypergraph languages and introduces two novel techniques, Hyper-BAG and Hyper-COT, which enhance high-order reasoning and achieve an average 4% (up to 9%) performance improvement on structure classification tasks. This work establishes a foundational testbed for integrating hypergraph computational capabilities into LLMs, advancing their comprehension. The source codes are at https://github.com/iMoonLab/LLM4Hypergraph.

This paper investigates stability estimates for inverse source problems in the stochastic polyharmonic wave equation, where the source is represented by white noise. The study examines the well-posedness of the direct problem and derives stability estimates for identifying the strength of the random source. Assuming a priori information of the regularity and support of the source strength, the H\"{o}lder stability is established in the absence of a potential. In the more challenging case where a potential is present, the logarithmic stability estimate is obtained by constructing specialized solutions to the polyharmonic wave equation.

We present a comprehensive QCD sum rules analysis for all types of light $J^P=0^{+}$ four-quark states at next-to-leading order. Most of them have masses around $1-2\text{GeV}$ and can be interpreted as the $0^+$ mesons observed in experiments. We find a category of four-quark nonets with masses $\lesssim1\text{GeV}$, potentially corresponding to the light $0^+$ mesons $f_0(500)$, $K^*_0(700)$, $f_0(980)$ and $a_0(980)$. Additionally, the 27-fold states may also exist, which are heavier than the $0^+$ nonets. The main uncertainty in the results arises from the factorization deviation of the multi-quark condensates. We also find that the factorization of dimension-8 condensates involves an ambiguity larger than $O(1/N_C^2)$. As a byproduct, a simple trick for renormalizing multi-quark operators at one-loop level is proposed in this paper., Comment: 56 pages, 40 figures, 8 tables

Energy band theory is a fundamental cornerstone of condensed matter physics. According to conventional wisdom, discrete translational symmetry is mandatory for defining energy bands. Here, we illustrate that, in fact, the concept of energy band can be generalized to incommensurate systems lacking such symmetry, thus transcending the traditional paradigm of energy band. The validity of our theory is verified by extensive numerical calculations in the celebrated Aubry-Andr\'e-Harper model and a two-dimensional incommensurate model of graphene. Building upon the proposed concept of incommensurate energy bands, we further develop a theory of angle-resolved photoemission spectroscopy (ARPES) for incommensurate systems, providing a clear physical picture for the incommensurate ARPES spectra. Our work establishes a comprehensive energy band theory for incommensurate systems., Comment: 8 pages, 3 figures

Person re-identification (ReID) has recently benefited from large pretrained vision-language models such as Contrastive Language-Image Pre-Training (CLIP). However, the absence of concrete descriptions necessitates the use of implicit text embeddings, which demand complicated and inefficient training strategies. To address this issue, we first propose one straightforward solution by leveraging existing image captioning models to generate pseudo captions for person images, and thereby boost person re-identification with large vision language models. Using models like the Large Language and Vision Assistant (LLAVA), we generate high-quality captions based on fixed templates that capture key semantic attributes such as gender, clothing, and age. By augmenting ReID training sets from uni-modality (image) to bi-modality (image and text), we introduce CLIP-SCGI, a simple yet effective framework that leverages synthesized captions to guide the learning of discriminative and robust representations. Built on CLIP, CLIP-SCGI fuses image and text embeddings through two modules to enhance the training process. To address quality issues in generated captions, we introduce a caption-guided inversion module that captures semantic attributes from images by converting relevant visual information into pseudo-word tokens based on the descriptions. This approach helps the model better capture key information and focus on relevant regions. The extracted features are then utilized in a cross-modal fusion module, guiding the model to focus on regions semantically consistent with the caption, thereby facilitating the optimization of the visual encoder to extract discriminative and robust representations. Extensive experiments on four popular ReID benchmarks demonstrate that CLIP-SCGI outperforms the state-of-the-art by a significant margin.

In this work, we explore the mechanism of in-context learning (ICL) on out-of-distribution (OOD) tasks that were not encountered during training. To achieve this, we conduct synthetic experiments where the objective is to learn OOD mathematical functions through ICL using a GPT-2 model. We reveal that Transformers may struggle to learn OOD task functions through ICL. Specifically, ICL performance resembles implementing a function within the pretraining hypothesis space and optimizing it with gradient descent based on the in-context examples. Additionally, we investigate ICL's well-documented ability to learn unseen abstract labels in context. We demonstrate that such ability only manifests in the scenarios without distributional shifts and, therefore, may not serve as evidence of new-task-learning ability. Furthermore, we assess ICL's performance on OOD tasks when the model is pretrained on multiple tasks. Both empirical and theoretical analyses demonstrate the existence of the \textbf{low-test-error preference} of ICL, where it tends to implement the pretraining function that yields low test error in the testing context. We validate this through numerical experiments. This new theoretical result, combined with our empirical findings, elucidates the mechanism of ICL in addressing OOD tasks., Comment: Preprint, under review

In this technical report, we introduce OpenR, an open-source framework designed to integrate key components for enhancing the reasoning capabilities of large language models (LLMs). OpenR unifies data acquisition, reinforcement learning training (both online and offline), and non-autoregressive decoding into a cohesive software platform. Our goal is to establish an open-source platform and community to accelerate the development of LLM reasoning. Inspired by the success of OpenAI's o1 model, which demonstrated improved reasoning abilities through step-by-step reasoning and reinforcement learning, OpenR integrates test-time compute, reinforcement learning, and process supervision to improve reasoning in LLMs. Our work is the first to provide an open-source framework that explores the core techniques of OpenAI's o1 model with reinforcement learning, achieving advanced reasoning capabilities beyond traditional autoregressive methods. We demonstrate the efficacy of OpenR by evaluating it on the MATH dataset, utilising publicly available data and search methods. Our initial experiments confirm substantial gains, with relative improvements in reasoning and performance driven by test-time computation and reinforcement learning through process reward models. The OpenR framework, including code, models, and datasets, is accessible at https://openreasoner.github.io.

Recently, spatial-temporal forecasting technology has been rapidly developed due to the increasing demand for traffic management and travel planning. However, existing traffic forecasting models still face the following limitations. On one hand, most previous studies either focus too much on real-world geographic information, neglecting the potential traffic correlation between different regions, or overlook geographical position and only model the traffic flow relationship. On the other hand, the importance of different time slices is ignored in time modeling. Therefore, we propose a Fusion Matrix Prompt Enhanced Self-Attention Spatial-Temporal Interactive Traffic Forecasting Framework (FMPESTF), which is composed of spatial and temporal modules for down-sampling traffic data. The network is designed to establish a traffic fusion matrix considering spatial-temporal heterogeneity as a query to reconstruct a data-driven dynamic traffic data structure, which accurately reveal the flow relationship of nodes in the traffic network. In addition, we introduce attention mechanism in time modeling, and design hierarchical spatial-temporal interactive learning to help the model adapt to various traffic scenarios. Through extensive experimental on six real-world traffic datasets, our method is significantly superior to other baseline models, demonstrating its efficiency and accuracy in dealing with traffic forecasting problems., Comment: THE WEB CONFERENCE 2025

The acceleration of the young solar wind is studied using the first 17 encounters of Parker Solar Probe. We identify wind intervals from different source regions: coronal hole (CH) interiors, streamers, and low Mach number boundary layers (LMBLs), i.e. the inner boundaries of coronal holes. We present their statistical trends in the acceleration process. Most of the observations can be reproduced by a two-fluid hydrodynamic model with realistic corona temperatures. In such a model, the solar wind is accelerated by the combined thermal pressures of protons and electrons,but it is mainly the difference in the proton pressure that leads to the difference in the solar wind speed. The proton pressure is the highest in the fastest CH wind, with a high initial proton temperature that decreases slowly. It is lower in the relatively slow LMBL wind, and the lowest in the slowest streamer wind. The proton temperature is quadratically correlated with the wind speed when scaled to the same distance. In contrast, the electron temperature shows no significant differences for different wind types or wind speeds, indicating more similar contributions from the electron pressure. The model gives reasonable locations for the sonic critical point, which is on average at 3.6-7.3 solar radii and can also extend to large distances when the proton temperature is extremely low, as in the LMBL wind. In addition to the thermal pressure, we raise the possibility that Alfv\'en waves may contribute to the solar wind acceleration, especially for the fast CH wind., Comment: Accepted for publication by ApJ Letters