Search

In the burgeoning field of large language models (LLMs), the assessment of fundamental knowledge remains a critical challenge, particularly for models tailored to Chinese language and culture. This paper introduces FoundaBench, a pioneering benchmark designed to rigorously evaluate the fundamental knowledge capabilities of Chinese LLMs. FoundaBench encompasses a diverse array of 3354 multiple-choice questions across common sense and K-12 educational subjects, meticulously curated to reflect the breadth and depth of everyday and academic knowledge. We present an extensive evaluation of 12 state-of-the-art LLMs using FoundaBench, employing both traditional assessment methods and our CircularEval protocol to mitigate potential biases in model responses. Our results highlight the superior performance of models pre-trained on Chinese corpora, and reveal a significant disparity between models' reasoning and memory recall capabilities. The insights gleaned from FoundaBench evaluations set a new standard for understanding the fundamental knowledge of LLMs, providing a robust framework for future advancements in the field.

In this report, we introduce InternVL 1.5, an open-source multimodal large language model (MLLM) to bridge the capability gap between open-source and proprietary commercial models in multimodal understanding. We introduce three simple improvements: (1) Strong Vision Encoder: we explored a continuous learning strategy for the large-scale vision foundation model -- InternViT-6B, boosting its visual understanding capabilities, and making it can be transferred and reused in different LLMs. (2) Dynamic High-Resolution: we divide images into tiles ranging from 1 to 40 of 448$\times$448 pixels according to the aspect ratio and resolution of the input images, which supports up to 4K resolution input. (3) High-Quality Bilingual Dataset: we carefully collected a high-quality bilingual dataset that covers common scenes, document images, and annotated them with English and Chinese question-answer pairs, significantly enhancing performance in OCR- and Chinese-related tasks. We evaluate InternVL 1.5 through a series of benchmarks and comparative studies. Compared to both open-source and proprietary models, InternVL 1.5 shows competitive performance, achieving state-of-the-art results in 8 of 18 benchmarks. Code has been released at https://github.com/OpenGVLab/InternVL., Comment: Technical report

Lane detection has made significant progress in recent years, but there is not a unified architecture for its two sub-tasks: 2D lane detection and 3D lane detection. To fill this gap, we introduce B\'{e}zierFormer, a unified 2D and 3D lane detection architecture based on B\'{e}zier curve lane representation. B\'{e}zierFormer formulate queries as B\'{e}zier control points and incorporate a novel B\'{e}zier curve attention mechanism. This attention mechanism enables comprehensive and accurate feature extraction for slender lane curves via sampling and fusing multiple reference points on each curve. In addition, we propose a novel Chamfer IoU-based loss which is more suitable for the B\'{e}zier control points regression. The state-of-the-art performance of B\'{e}zierFormer on widely-used 2D and 3D lane detection benchmarks verifies its effectiveness and suggests the worthiness of further exploration., Comment: ICME 2024, 11 pages, 8 figures

Supersolid is an exotic quantum state of matter that spontaneously hosts the features of both solid and superfluid, which breaks the translation and U(1) gauge symmetries. Here we study the spin dynamics in the triangular-lattice compound Na$_2$BaCo(PO$_4$)$_2$, which is revealed in [Xiang et al., Nature 625, 270-275 (2024)] as a quantum magnetic analog of supersolid. We simulate the easy-axis Heisenberg model with tensor network approach and uncover unique dynamic traits. These features are manifested in two branches of excitations that can be associated with the spin solidity and superfluidity, respectively. One branch contains the U(1) Goldstone and roton modes, while the other comprises pseudo-Goldstone and roton modes. The gapless Goldstone modes of the in-plane superfluid order are confirmed by our inelastic neutron scattering measurements. Together with the evident out-of-plane solid order indicated by the magnetic Bragg peaks, our findings provide spectroscopic evidence for spin supersolidity in this easy-axis antiferromagnet. Akin to the role of phonon-roton modes -- Landau elementary excitations -- in shaping the helium superfluid thermodynamics, the intriguing double magnon-roton dispersion here determines the low-temperature thermodynamics of spin supersolid down to sub-Kelvin regime, explaining the recently observed giant magnetocaloric effect in Na$_2$BaCo(PO$_4$)$_2$., Comment: 8+9 pages, 4+9 figures

Long baseline diffraction-limited optical aperture synthesis technology by interferometry plays an important role in scientific study and practical application. In contrast to amplitude (phase) interferometry, intensity interferometry -- which exploits the quantum nature of light to measure the photon bunching effect in thermal light -- is robust against atmospheric turbulence and optical defects. However, a thermal light source typically has a significant divergence angle and a low average photon number per mode, forestalling the applicability over long ranges. Here, we propose and demonstrate active intensity interferometry for super-resolution imaging over the kilometer range. Our scheme exploits phase-independent multiple laser emitters to produce the thermal illumination and uses an elaborate computational algorithm to reconstruct the image. In outdoor environments, we image two-dimension millimeter-level targets over 1.36 kilometers at a resolution of 14 times the diffraction limit of a single telescope. High-resolution optical imaging and sensing are anticipated by applying long-baseline active intensity interferometry in general branches of physics and metrology., Comment: 42 pages, 11 figures

Research of the nonlinear optical characteristics of transition metal dichalcogenides in the presence of photoactive particles, plasmonic nanocavities, waveguides, and metamaterials is still in its early stages. This investigation delves into the high-order harmonic generation (HHG) from laser induced plasma of MoS2 nanosheets in the presence of semiconductor photoactive medium such as CdSe and CdSe/V2O5 core/shell quantum dots. Our comprehensive findings shed light on the counteractive coupling impact of both bare and passivated quantum dots on MoS2 nanosheets, as evidenced by the emission of higher-order harmonics. Significantly, the intensity of harmonics and their cut-off were notably enhanced in the MoS2-CdSe and MoS2-V-CdSe configurations compared to pristine MoS2 nanosheets. These advancements hold promise for applications requiring the emission of coherent short-wavelength radiation., Comment: 8 pages, 4 figures

The studies of the nonlinear optical (NLO) properties of the transition metal dichalcogenides (TMDs) coupled with photoactive particles, plasmonic nanocavities, waveguides, and metamaterials remain in their infancy. This study investigates the third-order NLO properties of MoS$_2$ nanosheets in the presence of a semiconductor photoactive medium. Our extensive studies and the obtained results reveal the counteractive coupling effect of bare and passivated quantum dots on the MoS$_2$ nanosheet, as made evident by the analysis of the NLO processes. The enhanced NLO properties of MoS$_2$ nanosheets functionalized with CdSe and CdSe-V2O5 quantum dots are helpful for applications as saturable absorbers in laser applications and the emission of coherent short-wavelength radiation. The multiphoton-excitation resonance energy transfer mechanism exploiting remote dipole dipole coupling, and ultrafast charge transfer pathways emerges as another plausible way to alter the NLO properties in TMDs., Comment: 16 pages, 4 figures

The Kondo lattice, describing a grid of the local magnetic moments coupling to itinerant electrons, is a fertile ground of strongly correlated states in condensed matter physics. While the Kagome lattice has long been predicted to host Kondo physics with exotic magnetism and nontrivial topology, no experimental realization has been achieved. Here, we report the discovery of CsCr6Sb6, a van der Waals-like Kagome Kondo lattice featuring extremely flat, isolated bands at the Fermi level (EF) that composed entirely of Cr-3d electrons. We observe heavy fermions with the effective mass over 100 times greater than those of its vanadium counterpart. We also observe Kondo insulating behavior in an ultra-low carrier density of 1019 cm-3 and dimensionality-induced Kondo breakdown. More interestingly, the frustrated magnetism observed in the bulk give way to a hidden A-type antiferromagnetic ordering in few layers, in sharp contrast to the common sense of weakened magnetism with thinning. The realization of Kondo physics in Kagome lattice opens avenues for exploring diverse quantum criticalities in a strongly-correlated frustrated system., Comment: 13 pages, 4 figures

This paper provides a comprehensive review of the NTIRE 2024 challenge, focusing on efficient single-image super-resolution (ESR) solutions and their outcomes. The task of this challenge is to super-resolve an input image with a magnification factor of x4 based on pairs of low and corresponding high-resolution images. The primary objective is to develop networks that optimize various aspects such as runtime, parameters, and FLOPs, while still maintaining a peak signal-to-noise ratio (PSNR) of approximately 26.90 dB on the DIV2K_LSDIR_valid dataset and 26.99 dB on the DIV2K_LSDIR_test dataset. In addition, this challenge has 4 tracks including the main track (overall performance), sub-track 1 (runtime), sub-track 2 (FLOPs), and sub-track 3 (parameters). In the main track, all three metrics (ie runtime, FLOPs, and parameter count) were considered. The ranking of the main track is calculated based on a weighted sum-up of the scores of all other sub-tracks. In sub-track 1, the practical runtime performance of the submissions was evaluated, and the corresponding score was used to determine the ranking. In sub-track 2, the number of FLOPs was considered. The score calculated based on the corresponding FLOPs was used to determine the ranking. In sub-track 3, the number of parameters was considered. The score calculated based on the corresponding parameters was used to determine the ranking. RLFN is set as the baseline for efficiency measurement. The challenge had 262 registered participants, and 34 teams made valid submissions. They gauge the state-of-the-art in efficient single-image super-resolution. To facilitate the reproducibility of the challenge and enable other researchers to build upon these findings, the code and the pre-trained model of validated solutions are made publicly available at https://github.com/Amazingren/NTIRE2024_ESR/., Comment: The report paper of NTIRE2024 Efficient Super-resolution, accepted by CVPRW2024

Polar topologies, such as vortex and skyrmion, have attracted significant interest due to their unique physical properties and promising applications in high-density memory devices. Currently, most polar vortices are observed in heterostructures containing ferroelectric materials and constrained by substrates. In this study, we unravel arrays of polar vortices formed in twisted freestanding bilayers composed of SrTiO3, a quantum-paraelectric material. Depth-resolved structures of the bilayers are measured with deep-sub-angstrom resolution and one picometer accuracy using multislice ptychography, enabling identification of the three-dimensional variations of polarization topology. Our findings reveal the evolution of the polar vortices in the twisted overlapping layers, demonstrating the reverse of rotation manner in the depth direction. Twisted freestanding bilayers provide a unique platform for exploration and modulation of novel polar topologies.

The Large Vision-Language Model (LVLM) field has seen significant advancements, yet its progression has been hindered by challenges in comprehending fine-grained visual content due to limited resolution. Recent efforts have aimed to enhance the high-resolution understanding capabilities of LVLMs, yet they remain capped at approximately 1500 x 1500 pixels and constrained to a relatively narrow resolution range. This paper represents InternLM-XComposer2-4KHD, a groundbreaking exploration into elevating LVLM resolution capabilities up to 4K HD (3840 x 1600) and beyond. Concurrently, considering the ultra-high resolution may not be necessary in all scenarios, it supports a wide range of diverse resolutions from 336 pixels to 4K standard, significantly broadening its scope of applicability. Specifically, this research advances the patch division paradigm by introducing a novel extension: dynamic resolution with automatic patch configuration. It maintains the training image aspect ratios while automatically varying patch counts and configuring layouts based on a pre-trained Vision Transformer (ViT) (336 x 336), leading to dynamic training resolution from 336 pixels to 4K standard. Our research demonstrates that scaling training resolution up to 4K HD leads to consistent performance enhancements without hitting the ceiling of potential improvements. InternLM-XComposer2-4KHD shows superb capability that matches or even surpasses GPT-4V and Gemini Pro in 10 of the 16 benchmarks. The InternLM-XComposer2-4KHD model series with 7B parameters are publicly available at https://github.com/InternLM/InternLM-XComposer., Comment: Code and models are publicly available at https://github.com/InternLM/InternLM-XComposer

Transformer is leading a trend in the field of image processing. Despite the great success that existing lightweight image processing transformers have achieved, they are tailored to FLOPs or parameters reduction, rather than practical inference acceleration. In this paper, we present a latency-aware image processing transformer, termed LIPT. We devise the low-latency proportion LIPT block that substitutes memory-intensive operators with the combination of self-attention and convolutions to achieve practical speedup. Specifically, we propose a novel non-volatile sparse masking self-attention (NVSM-SA) that utilizes a pre-computing sparse mask to capture contextual information from a larger window with no extra computation overload. Besides, a high-frequency reparameterization module (HRM) is proposed to make LIPT block reparameterization friendly, which improves the model's detail reconstruction capability. Extensive experiments on multiple image processing tasks (e.g., image super-resolution (SR), JPEG artifact reduction, and image denoising) demonstrate the superiority of LIPT on both latency and PSNR. LIPT achieves real-time GPU inference with state-of-the-art performance on multiple image SR benchmarks.

Knowledge distillation (KD) is a promising yet challenging model compression technique that transfers rich learning representations from a well-performing but cumbersome teacher model to a compact student model. Previous methods for image super-resolution (SR) mostly compare the feature maps directly or after standardizing the dimensions with basic algebraic operations (e.g. average, dot-product). However, the intrinsic semantic differences among feature maps are overlooked, which are caused by the disparate expressive capacity between the networks. This work presents MiPKD, a multi-granularity mixture of prior KD framework, to facilitate efficient SR model through the feature mixture in a unified latent space and stochastic network block mixture. Extensive experiments demonstrate the effectiveness of the proposed MiPKD method.

Model generalization ability upon incrementally acquiring dynamically updating knowledge from sequentially arriving tasks is crucial to tackle the sensitivity-stability dilemma in Continual Learning (CL). Weight loss landscape sharpness minimization seeking for flat minima lying in neighborhoods with uniform low loss or smooth gradient is proven to be a strong training regime improving model generalization compared with loss minimization based optimizer like SGD. Yet only a few works have discussed this training regime for CL, proving that dedicated designed zeroth-order sharpness optimizer can improve CL performance. In this work, we propose a Continual Flatness (C-Flat) method featuring a flatter loss landscape tailored for CL. C-Flat could be easily called with only one line of code and is plug-and-play to any CL methods. A general framework of C-Flat applied to all CL categories and a thorough comparison with loss minima optimizer and flat minima based CL approaches is presented in this paper, showing that our method can boost CL performance in almost all cases. Code is available at https://github.com/WanNaa/C-Flat.

The recent discovery of the spin supersolid candidate Na$_2$BaCo(PO$_4$)$_2$ stimulates numerous research interest on the triangular-lattice transition-metal phosphates. Here we report a comprehensive study on the structural, magnetic and magnetocaloric properties of polycrystalline Na$_2$$A$$T$(PO$_4$)$_2$ ($A$ = Ba, Sr; $T$ = Co, Ni, Mn). X-ray and neutron diffraction measurements confirm that Na$_2$Ba$T$(PO$_4$)$_2$ (NB$T$P) crystallizes in a trigonal structure, while Na$_2$Sr$T$(PO$_4$)$_2$ (NS$T$P) forms a monoclinic structure with a slight distortion of the triangular network of $T^{2+}$ ions. The dc magnetization data show that all six compounds order antiferromagnetically below 2 K, and the N\'{e}el temperatures of NS$T$P are consistently higher than those of NB$T$P for $T$ = Co, Ni, and Mn, due to the release of geometrical frustration by monoclinic distortions. Further magnetocaloric measurements show that trigonal NB$T$P can reach a lower temperature in the quasi-adiabatic demagnetization process and thus shows a better performance in the magnetic refrigeration, compared with monoclinic NS$T$P. Our findings highlight the outstanding magnetocaloric performances of the trigonal transition-metal phosphates, and disclose two necessary ingredients for a superior magnetic coolant that can reach an ultra-low temperature, including a perfect geometrically frustrated lattice and a small effective spin number associated with the magnetic ions., Comment: 10 Pages, 6 figures, accepted for publication in Physical Review Materials

Recent advances have demonstrated the powerful capability of transformer architecture in image restoration. However, our analysis indicates that existing transformerbased methods can not establish both exact global and local dependencies simultaneously, which are much critical to restore the details and missing content of degraded images. To this end, we present an efficient image processing transformer architecture with hierarchical attentions, called IPTV2, adopting a focal context self-attention (FCSA) and a global grid self-attention (GGSA) to obtain adequate token interactions in local and global receptive fields. Specifically, FCSA applies the shifted window mechanism into the channel self-attention, helps capture the local context and mutual interaction across channels. And GGSA constructs long-range dependencies in the cross-window grid, aggregates global information in spatial dimension. Moreover, we introduce structural re-parameterization technique to feed-forward network to further improve the model capability. Extensive experiments demonstrate that our proposed IPT-V2 achieves state-of-the-art results on various image processing tasks, covering denoising, deblurring, deraining and obtains much better trade-off for performance and computational complexity than previous methods. Besides, we extend our method to image generation as latent diffusion backbone, and significantly outperforms DiTs.

Non-Abelian topological orders offer an intriguing path towards fault-tolerant quantum computation, where information can be encoded and manipulated in a topologically protected manner immune to arbitrary local noises and perturbations. However, realizing non-Abelian topologically ordered states is notoriously challenging in both condensed matter and programmable quantum systems, and it was not until recently that signatures of non-Abelian statistics were observed through digital quantum simulation approaches. Despite these exciting progresses, none of them has demonstrated the appropriate type of topological orders and associated non-Abelian anyons whose braidings alone support universal quantum computation. Here, we report the realization of non-Abelian topologically ordered states of the Fibonacci string-net model and demonstrate braidings of Fibonacci anyons featuring universal computational power, with a superconducting quantum processor. We exploit efficient quantum circuits to prepare the desired states and verify their nontrivial topological nature by measuring the topological entanglement entropy. In addition, we create two pairs of Fibonacci anyons and demonstrate their fusion rule and non-Abelian braiding statistics by applying unitary gates on the underlying physical qubits. Our results establish a versatile digital approach to exploring exotic non-Abelian topological states and their associated braiding statistics with current noisy intermediate-scale quantum processors.

Neutron stars are among the main targets for gravitational wave observatories, however, their equation of state is still not well established. Mainly phenomenological models with many parameters are widely used by far, while theoretical models are not so practical. In arXiv:1902.08477, a theoretical equation of state with only one parameter is derived from Witten-Sakai-Sugimoto model, as an application of AdS/QCD, where pointlike instanton case is taken into consideration. When the tidal deformability constraint from gravitational wave event is satisfied, the maximum mass is about 1.7 solar masses. Now we upgrade this model to instanton gas, with one more variable, the instanton width. This is not naively a free parameter, but a function of the chemical potential. Thus we end up with a more complicated and accurate model, but still with only one adjustable parameter. In this case, we find the maximum mass becomes 1.85 solar masses. This is an encouraging result, as a theoretically derived model., Comment: 10 pages, 8 figures; v2: published version, with parameter sensitivity analysis added in Appendix C

The evolution of Large Language Models (LLMs) like ChatGPT and GPT-4 has sparked discussions on the advent of Artificial General Intelligence (AGI). However, replicating such advancements in open-source models has been challenging. This paper introduces InternLM2, an open-source LLM that outperforms its predecessors in comprehensive evaluations across 6 dimensions and 30 benchmarks, long-context modeling, and open-ended subjective evaluations through innovative pre-training and optimization techniques. The pre-training process of InternLM2 is meticulously detailed, highlighting the preparation of diverse data types including text, code, and long-context data. InternLM2 efficiently captures long-term dependencies, initially trained on 4k tokens before advancing to 32k tokens in pre-training and fine-tuning stages, exhibiting remarkable performance on the 200k ``Needle-in-a-Haystack" test. InternLM2 is further aligned using Supervised Fine-Tuning (SFT) and a novel Conditional Online Reinforcement Learning from Human Feedback (COOL RLHF) strategy that addresses conflicting human preferences and reward hacking. By releasing InternLM2 models in different training stages and model sizes, we provide the community with insights into the model's evolution.

Effectively predicting transonic unsteady flow over an aerofoil poses inherent challenges. In this study, we harness the power of deep neural network (DNN) models using the attention U-Net architecture. Through efficient training of these models, we achieve the capability to capture the complexities of transonic and unsteady flow dynamics at high resolution, even when faced with previously unseen conditions. We demonstrate that by leveraging the differentiability inherent in neural network representations, our approach provides a framework for assessing fundamental physical properties via global instability analysis. This integration bridges deep neural network models and traditional modal analysis, offering valuable insights into transonic flow dynamics and enhancing the interpretability of neural network models in flowfield diagnostics.

The impressive performance of large language models (LLMs) on code-related tasks has shown the potential of fully automated software development. In light of this, we introduce a new software engineering task, namely Natural Language to code Repository (NL2Repo). This task aims to generate an entire code repository from its natural language requirements. To address this task, we propose a simple yet effective framework CodeS, which decomposes NL2Repo into multiple sub-tasks by a multi-layer sketch. Specifically, CodeS includes three modules: RepoSketcher, FileSketcher, and SketchFiller. RepoSketcher first generates a repository's directory structure for given requirements; FileSketcher then generates a file sketch for each file in the generated structure; SketchFiller finally fills in the details for each function in the generated file sketch. To rigorously assess CodeS on the NL2Repo task, we carry out evaluations through both automated benchmarking and manual feedback analysis. For benchmark-based evaluation, we craft a repository-oriented benchmark, SketchEval, and design an evaluation metric, SketchBLEU. For feedback-based evaluation, we develop a VSCode plugin for CodeS and engage 30 participants in conducting empirical studies. Extensive experiments prove the effectiveness and practicality of CodeS on the NL2Repo task., Comment: https://github.com/NL2Code/CodeS

In image restoration (IR), leveraging semantic priors from segmentation models has been a common approach to improve performance. The recent segment anything model (SAM) has emerged as a powerful tool for extracting advanced semantic priors to enhance IR tasks. However, the computational cost of SAM is prohibitive for IR, compared to existing smaller IR models. The incorporation of SAM for extracting semantic priors considerably hampers the model inference efficiency. To address this issue, we propose a general framework to distill SAM's semantic knowledge to boost exiting IR models without interfering with their inference process. Specifically, our proposed framework consists of the semantic priors fusion (SPF) scheme and the semantic priors distillation (SPD) scheme. SPF fuses two kinds of information between the restored image predicted by the original IR model and the semantic mask predicted by SAM for the refined restored image. SPD leverages a self-distillation manner to distill the fused semantic priors to boost the performance of original IR models. Additionally, we design a semantic-guided relation (SGR) module for SPD, which ensures semantic feature representation space consistency to fully distill the priors. We demonstrate the effectiveness of our framework across multiple IR models and tasks, including deraining, deblurring, and denoising.

The study of research trends is pivotal for understanding scientific development on specific topics. Traditionally, this involves keyword analysis within scholarly literature, yet comprehensive tools for such analysis are scarce, especially those capable of parsing large datasets with precision. pyKCN, a Python toolkit, addresses this gap by automating keyword cleaning, extraction and trend analysis from extensive academic corpora. It is equipped with modules for text processing, deduplication, extraction, and advanced keyword co-occurrence and analysis, providing a granular view of research trends. This toolkit stands out by enabling researchers to visualize keyword relationships, thereby identifying seminal works and emerging trends. Its application spans diverse domains, enhancing scholars' capacity to understand developments within their fields. The implications of using pyKCN are significant. It offers an empirical basis for predicting research trends, which can inform funding directions, policy-making, and academic curricula. The code source and details can be found on: https://github.com/zhenyuanlu/pyKCN

Bird's eye view (BEV) representation has emerged as a dominant solution for describing 3D space in autonomous driving scenarios. However, objects in the BEV representation typically exhibit small sizes, and the associated point cloud context is inherently sparse, which leads to great challenges for reliable 3D perception. In this paper, we propose IS-Fusion, an innovative multimodal fusion framework that jointly captures the Instance- and Scene-level contextual information. IS-Fusion essentially differs from existing approaches that only focus on the BEV scene-level fusion by explicitly incorporating instance-level multimodal information, thus facilitating the instance-centric tasks like 3D object detection. It comprises a Hierarchical Scene Fusion (HSF) module and an Instance-Guided Fusion (IGF) module. HSF applies Point-to-Grid and Grid-to-Region transformers to capture the multimodal scene context at different granularities. IGF mines instance candidates, explores their relationships, and aggregates the local multimodal context for each instance. These instances then serve as guidance to enhance the scene feature and yield an instance-aware BEV representation. On the challenging nuScenes benchmark, IS-Fusion outperforms all the published multimodal works to date. Code is available at: https://github.com/yinjunbo/IS-Fusion., Comment: Accepted to CVPR 2024; Code: https://github.com/yinjunbo/IS-Fusion

We introduce CHARM, the first benchmark for comprehensively and in-depth evaluating the commonsense reasoning ability of large language models (LLMs) in Chinese, which covers both globally known and Chinese-specific commonsense. We evaluated 7 English and 12 Chinese-oriented LLMs on CHARM, employing 5 representative prompt strategies for improving LLMs' reasoning ability, such as Chain-of-Thought. Our findings indicate that the LLM's language orientation and the task's domain influence the effectiveness of the prompt strategy, which enriches previous research findings. We built closely-interconnected reasoning and memorization tasks, and found that some LLMs struggle with memorizing Chinese commonsense, affecting their reasoning ability, while others show differences in reasoning despite similar memorization performance. We also evaluated the LLMs' memorization-independent reasoning abilities and analyzed the typical errors. Our study precisely identified the LLMs' strengths and weaknesses, providing the clear direction for optimization. It can also serve as a reference for studies in other fields. We will release CHARM at https://github.com/opendatalab/CHARM ., Comment: Equal contribution: Jiaxing Sun, Weiquan Huang, Jiang Wu; Corresponding author: Conghui He

The use of neural networks to approximate partial differential equations (PDEs) has gained significant attention in recent years. However, the approximation of PDEs with localised phenomena, e.g., sharp gradients and singularities, remains a challenge, due to ill-defined cost functions in terms of pointwise residual sampling or poor numerical integration. In this work, we introduce $h$-adaptive finite element interpolated neural networks. The method relies on the interpolation of a neural network onto a finite element space that is gradually adapted to the solution during the training process to equidistribute a posteriori error indicator. The use of adaptive interpolation is essential in preserving the non-linear approximation capabilities of the neural networks to effectively tackle problems with localised features. The training relies on a gradient-based optimisation of a loss function based on the (dual) norm of the finite element residual of the interpolated neural network. Automatic mesh adaptation (i.e., refinement and coarsening) is performed based on a posteriori error indicators till a certain level of accuracy is reached. The proposed methodology can be applied to indefinite and nonsymmetric problems. We carry out a detailed numerical analysis of the scheme and prove several a priori error estimates, depending on the expressiveness of the neural network compared to the interpolation mesh. Our numerical experiments confirm the effectiveness of the method in capturing sharp gradients and singularities for forward PDE problems, both in 2D and 3D scenarios. We also show that the proposed preconditioning strategy (i.e., using a dual residual norm of the residual as a cost function) enhances training robustness and accelerates convergence.

CLIP (Contrastive Language-Image Pre-training) uses contrastive learning from noise image-text pairs to excel at recognizing a wide array of candidates, yet its focus on broad associations hinders the precision in distinguishing subtle differences among fine-grained items. Conversely, Multimodal Large Language Models (MLLMs) excel at classifying fine-grained categories, thanks to their substantial knowledge from pre-training on web-level corpora. However, the performance of MLLMs declines with an increase in category numbers, primarily due to growing complexity and constraints of limited context window size. To synergize the strengths of both approaches and enhance the few-shot/zero-shot recognition abilities for datasets characterized by extensive and fine-grained vocabularies, this paper introduces RAR, a Retrieving And Ranking augmented method for MLLMs. We initially establish a multi-modal retriever based on CLIP to create and store explicit memory for different categories beyond the immediate context window. During inference, RAR retrieves the top-k similar results from the memory and uses MLLMs to rank and make the final predictions. Our proposed approach not only addresses the inherent limitations in fine-grained recognition but also preserves the model's comprehensive knowledge base, significantly boosting accuracy across a range of vision-language recognition tasks. Notably, our approach demonstrates a significant improvement in performance on 5 fine-grained visual recognition benchmarks, 11 few-shot image recognition datasets, and the 2 object detection datasets under the zero-shot recognition setting., Comment: Project: https://github.com/Liuziyu77/RAR

Low-dimensional quantum magnets, particularly those with strong spin frustration, are characterized by their notable spin fluctuations. Nuclear magnetic resonance (NMR) serves as a sensitive probe of low-energy fluctuations that offers valuable insight into rich magnetic phases and emergent phenomena in quantum magnets. Although experimentally accessible, the numerical simulation of NMR relaxation rates, specifically the spin-lattice relaxation rate $1/T_1$, remains a significant challenge. Analytical continuation based on Monte Carlo calculations are hampered by the notorious negative sign for frustrated systems, and the real-time simulations incur significant costs to capture low-energy fluctuations. Here we propose computing the relaxation rate using thermal tensor networks (TTNs), which provides a streamlined approach by calculating its imaginary-time proxy. We showcase the accuracy and versatility of our methodology by applying it to one-dimensional spin chains and two-dimensional lattices, where we find that the critical exponents $\eta$ and $z\nu$ can be extracted from the low-temperature scalings of the simulated $1/T_1$ near quantum critical points. Our results also provide insights into the low-dimensional and frustrated magnetic materials, elucidating universal scaling behaviors in the Ising chain compound CoNb$_2$O$_6$ and revealing the renormalized classical behaviors in the triangular-lattice antiferromagnet Ba$_8$CoNb$_6$O$_{24}$. We apply the approach to effective model of the family of frustrated magnets AYbCh$_2$ (A = Na, K, Cs, and Ch = O, S, Se), and find dramatic changes from spin ordered to the proposed quantum spin liquid phase. Overall, with high reliability and accuracy, the TTN methodology offers a systematic strategy for studying the intricate dynamics observed across a broad spectrum of quantum magnets and related fields., Comment: 15 pages, 12 figures