Search

Reinforcement Learning from Human Feedback (RLHF) is an effective approach for aligning language models to human preferences. Central to RLHF is learning a reward function for scoring human preferences. Two main approaches for learning a reward model are 1) training an EXplicit Reward Model (EXRM) as in RLHF, and 2) using an implicit reward learned from preference data through methods such as Direct Preference Optimization (DPO). Prior work has shown that the implicit reward model of DPO (denoted as DPORM) can approximate an EXRM in the limit. DPORM's effectiveness directly implies the optimality of the learned policy, and also has practical implication for LLM alignment methods including iterative DPO. However, it is unclear how well DPORM empirically matches the performance of EXRM. This work studies the accuracy at distinguishing preferred and rejected answers for both DPORM and EXRM. Our findings indicate that even though DPORM fits the training dataset comparably, it generalizes less effectively than EXRM, especially when the validation datasets contain distribution shifts. Across five out-of-distribution settings, DPORM has a mean drop in accuracy of 3% and a maximum drop of 7%. These findings highlight that DPORM has limited generalization ability and substantiates the integration of an explicit reward model in iterative DPO approaches., Comment: 12 pages, 8 tables, 2 figures

Model merging offers an effective strategy to combine the strengths of multiple finetuned models into a unified model that preserves the specialized capabilities of each. Existing methods merge models in a global manner, performing arithmetic operations across all model parameters. However, such global merging often leads to task interference, degrading the performance of the merged model. In this work, we introduce Localize-and-Stitch, a novel approach that merges models in a localized way. Our algorithm works in two steps: i) Localization: identify tiny ($1\%$ of the total parameters) localized regions in the finetuned models containing essential skills for the downstream tasks, and ii) Stitching: reintegrate only these essential regions back into the pretrained model for task synergy. We demonstrate that our approach effectively locates sparse regions responsible for finetuned performance, and the localized regions could be treated as compact and interpretable representations of the finetuned models (tasks). Empirically, we evaluate our method on various vision and language benchmarks, showing that it outperforms existing model merging methods under different data availability scenarios. Beyond strong empirical performance, our algorithm also facilitates model compression and preserves pretrained knowledge, enabling flexible and continual skill composition from multiple finetuned models with minimal storage and computational overhead. Our code is available at https://github.com/yifei-he/Localize-and-Stitch.

Heterophilic Graph Neural Networks (HGNNs) have shown promising results for semi-supervised learning tasks on graphs. Notably, most real-world heterophilic graphs are composed of a mixture of nodes with different neighbor patterns, exhibiting local node-level homophilic and heterophilic structures. However, existing works are only devoted to designing better HGNN backbones or architectures for node classification tasks on heterophilic and homophilic graph benchmarks simultaneously, and their analyses of HGNN performance with respect to nodes are only based on the determined data distribution without exploring the effect caused by this structural difference between training and testing nodes. How to learn invariant node representations on heterophilic graphs to handle this structure difference or distribution shifts remains unexplored. In this paper, we first discuss the limitations of previous graph-based invariant learning methods from the perspective of data augmentation. Then, we propose \textbf{HEI}, a framework capable of generating invariant node representations through incorporating heterophily information to infer latent environments without augmentation, which are then used for invariant prediction, under heterophilic graph structure distribution shifts. We theoretically show that our proposed method can achieve guaranteed performance under heterophilic graph structure distribution shifts. Extensive experiments on various benchmarks and backbones can also demonstrate the effectiveness of our method compared with existing state-of-the-art baselines., Comment: 20 pages, 7 figures

In this paper, we first define a discrete version of the fractional Laplace operator $(-\Delta)^{s}$ through the heat semigroup on a stochastically complete, connected, locally finite graph $G = (V, E, \mu, w)$. Secondly, we define the fractional divergence and give another form of $(-\Delta)^s$. The third point, and the foremost, is the introduction of the fractional Sobolev space $W^{s,2}(V)$, which is necessary when we study problems involving $(-\Delta)^{s}$. Finally, using the mountain-pass theorem and the Nehari manifold, we obtain multiplicity solutions to a discrete fractional Schr\"{o}dinger equation on $G$. We caution the readers that though these existence results are well known in the continuous case, the discrete case is quite different., Comment: 23 pages

We investigate the feasibility of measuring the proton charge radius through dimuon photoproduction off a proton target. Our findings indicate that the Bethe-Heitler mechanism, which dominates at small momentum transfers, allows for an extraction of the proton electromagnetic form factors in the extremely low $Q^2$ region below $10^{-3}$ GeV$^2$ in the spacelike region, when the incident photon beam energy exceeds several hundred MeV. The optimal kinematical region and a sensitivity study of the proton charge radius from dimuon photoproduction are presented. Such a measurement is expected to provide an alternative to the elastic muon-proton scattering measurements such as MUSE at PSI and AMBER at CERN., Comment: 6 pages, 4 figures. Comments are welcome

Reward models trained on human preference data have been proven to be effective for aligning Large Language Models (LLMs) with human intent within the reinforcement learning from human feedback (RLHF) framework. However, the generalization capabilities of current reward models to unseen prompts and responses are limited. This limitation can lead to an unexpected phenomenon known as reward over-optimization, where excessive optimization of rewards results in a decline in actual performance. While previous research has advocated for constraining policy optimization, our study proposes a novel approach to enhance the reward model's generalization ability against distribution shifts by regularizing the hidden states. Specifically, we retain the base model's language model head and incorporate a suite of text-generation losses to preserve the hidden states' text generation capabilities, while concurrently learning a reward head behind the same hidden states. Our experimental results demonstrate that the introduced regularization technique markedly improves the accuracy of learned reward models across a variety of out-of-distribution (OOD) tasks and effectively alleviate the over-optimization issue in RLHF, offering a more reliable and robust preference learning paradigm., Comment: 21 pages

This paper aims to compute and estimate the eigenvalues of the Hodge Laplacians on directed graphs. We have devised a new method for computing Hodge spectra with the following two ingredients. (I) We have observed that the product rule does work for the so-called normalized Hodge operator, denoted by $\Delta _{p}^{(a)},$ where $a$ refers to the weight that is used to redefine the inner product in the spaces $\Omega _{p}$. This together with the K\"{u}nneth formula for product allows us to compute inductively the spectra of all normalized Hodge operators $\Delta _{p}^{(a)}$ on Cartesian powers including $n$-cubes and $n$-tori. (II) We relate in a certain way the spectra of $\Delta _{p}$ and $\Delta_{p}^{(a)}$ to those of operators $\mathcal{L}_{p}=\partial ^{\ast }\partial$ also acting on $\Omega _{p}$. Knowing the spectra of $\Delta_{p}^{(a)}$ for all values of $p$, we compute the spectra of $\mathcal{L}_{p} $ and then the spectra of $\Delta _{p}.$ This program yields the spectra of all operators $\Delta _{p}$ on all $n$-cubes and $n$-tori.

Underwater Image Enhancement (UIE) is critical for marine research and exploration but hindered by complex color distortions and severe blurring. Recent deep learning-based methods have achieved remarkable results, yet these methods struggle with high computational costs and insufficient global modeling, resulting in locally under- or over- adjusted regions. We present PixMamba, a novel architecture, designed to overcome these challenges by leveraging State Space Models (SSMs) for efficient global dependency modeling. Unlike convolutional neural networks (CNNs) with limited receptive fields and transformer networks with high computational costs, PixMamba efficiently captures global contextual information while maintaining computational efficiency. Our dual-level strategy features the patch-level Efficient Mamba Net (EMNet) for reconstructing enhanced image feature and the pixel-level PixMamba Net (PixNet) to ensure fine-grained feature capturing and global consistency of enhanced image that were previously difficult to obtain. PixMamba achieves state-of-the-art performance across various underwater image datasets and delivers visually superior results. Code is available at: https://github.com/weitunglin/pixmamba.

By the connection graph we mean an underlying weighted graph with a connection which associates edge set with an orthogonal group. This paper centers its investigation on the connection heat kernels on connection lattices and connection discrete torus. For one dimensional connection lattice, we derive the connection heat kernel expression by doing the Taylor expansion on the exponential function involving normalized connection Laplacian. We introduce a novel connection called product connection and prove that the connection heat kernel on arbitrary high dimensional lattice with product connection equals the Kronecker sum of one dimensional connection lattices' connection heat kernels. Furthermore, if the connection graph is consistent, we substantiate the interrelation between its connection heat kernel and its underlying graph's heat kernel. We define a connection called quotient connection such that discrete torus with quotient connection can be considered as a quotient graph of connection lattice, whose connection heat kernel is demonstrated to be the sum of connection lattices' connection heat kernels. In addition, we derive an alternative expression of connection heat kernel on discrete torus whenever its quotient connection is a constant connection, yielding an equation as a connection graph's trace formula.

Nowadays a great attention has been focused on the discrete fractional Laplace operator as the natural counterpart of the continuous one. In this paper, we discretize the fractional Laplace operator $(-\Delta)^{s}$ for an arbitrary finite graph and any positive real number $s$. It is shown that $(-\Delta)^{s}$ can be explicitly represented by eigenvalues and eigenfunctions of the Laplace operator $-\Delta$. Moreover, we study its important properties, such as $(-\Delta)^{s}$ converges to $-\Delta$ as $s$ tends to $1$; while $(-\Delta)^{s}$ converges to the identity map as $s$ tends to $0$ on a specific function space. For related problems involving the fractional Laplace operator, we consider the fractional Kazdan-Warner equation and obtain several existence results via variational principles and the method of upper and lower solutions., Comment: 25 pages

In recent years, machine learning models have achieved success based on the independently and identically distributed assumption. However, this assumption can be easily violated in real-world applications, leading to the Out-of-Distribution (OOD) problem. Understanding how modern over-parameterized DNNs behave under non-trivial natural distributional shifts is essential, as current theoretical understanding is insufficient. Existing theoretical works often provide meaningless results for over-parameterized models in OOD scenarios or even contradict empirical findings. To this end, we are investigating the performance of the over-parameterized model in terms of OOD generalization under the general benign overfitting conditions. Our analysis focuses on a random feature model and examines non-trivial natural distributional shifts, where the benign overfitting estimators demonstrate a constant excess OOD loss, despite achieving zero excess in-distribution (ID) loss. We demonstrate that in this scenario, further increasing the model's parameterization can significantly reduce the OOD loss. Intuitively, the variance term of ID loss remains low due to orthogonality of long-tail features, meaning overfitting noise during training generally doesn't raise testing loss. However, in OOD cases, distributional shift increases the variance term. Thankfully, the inherent shift is unrelated to individual x, maintaining the orthogonality of long-tail features. Expanding the hidden dimension can additionally improve this orthogonality by mapping the features into higher-dimensional spaces, thereby reducing the variance term. We further show that model ensembles also improve OOD loss, akin to increasing model capacity. These insights explain the empirical phenomenon of enhanced OOD generalization through model ensembles, supported by consistent simulations with theoretical results.

Large vision language models, such as CLIPs, have revolutionized modern machine learning. CLIPs have demonstrated great generalizability under distribution shifts, supported by an increasing body of literature. However, the evaluation datasets for CLIPs are variations primarily designed for ImageNet benchmarks, which may not fully reflect the extent to which CLIPs, e.g., pre-trained on LAION, robust to spurious correlations. To bridge the gap, we collect a real-world dataset called CounterAnimal that contains realistic spurious features found in animal photos. CounterAnimal consists of a) the common group: comprising animals on common backgrounds, and b) the counter group: including animals on unusual backgrounds. The performance drops from the common to counter groups quantify the reliance of models on spurious features (i.e., backgrounds) to predict the animals. We find that CLIPs trained on either LAION or the OpenAI data exhibit notable performance drops on the counter group. Surprisingly, we observe that single-modal models trained on ImageNet are more robust than CLIPs. We provide both theoretical and empirical explanations for why CLIPs still learn spurious features. Our findings suggest that distribution shifts remain an open problem for CLIPs, and one needs to be cautious about test setups when evaluating foundation models pre-trained on a significantly different scale and distribution., Comment: Qizhou Wang, Yong Lin, and Yongqiang Chen contributed equally. Project page: https://counteranimal.github.io

This paper presents a comprehensive analysis of the spectral properties of the connection Laplacian for both real and discrete tori. We introduce novel methods to examine these eigenvalues by employing parallel orthonormal basis in the pullback bundle on universal covering spaces. Our main results reveal that the eigenvalues of the connection Laplacian on a real torus can be expressed in terms of standard Laplacian eigenvalues, with a unique twist encapsulated in the torsion matrix. This connection is further investigated in the context of discrete tori, where we demonstrate similar results. A significant portion of the paper is dedicated to exploring the convergence properties of a family of discrete tori towards a real torus. We extend previous findings on the spectrum of the standard Laplacian to include the connection Laplacian, revealing that the rescaled eigenvalues of discrete tori converge to those of the real torus. Furthermore, our analysis of the discrete torus occurs within a broader context, where it is not constrained to being a product of cyclic groups. Additionally, we delve into the theta functions associated with these structures, providing a detailed analysis of their behavior and convergence. The paper culminates in a study of the regularized log-determinant of the connection Laplacian and the converging results of it. We derive formulae for both real and discrete tori, emphasizing their dependence on the spectral zeta function and theta functions.

Fine-grained control over large language models (LLMs) remains a significant challenge, hindering their adaptability to diverse user needs. While Reinforcement Learning from Human Feedback (RLHF) shows promise in aligning LLMs, its reliance on scalar rewards often limits its ability to capture diverse user preferences in real-world applications. To address this limitation, we introduce the Directional Preference Alignment (DPA) framework. Unlike the scalar-reward RLHF, DPA incorporates multi-objective reward modeling to represent diverse preference profiles. Additionally, DPA models user preferences as directions (i.e., unit vectors) in the reward space to achieve user-dependent preference control. Our method involves training a multi-objective reward model and then fine-tuning the LLM with a preference-conditioned variant of Rejection Sampling Finetuning (RSF), an RLHF method adopted by Llama 2. This method enjoys a better performance trade-off across various reward objectives. In comparison with the scalar-reward RLHF, DPA offers users intuitive control over LLM generation: they can arithmetically specify their desired trade-offs (e.g., more helpfulness with less verbosity). We also validate the effectiveness of DPA with real-world alignment experiments on Mistral-7B. Our method provides straightforward arithmetic control over the trade-off between helpfulness and verbosity while maintaining competitive performance with strong baselines such as Direct Preference Optimization (DPO)., Comment: The code and model are released at https://github.com/Haoxiang-Wang/directional-preference-alignment

In this paper, we study the Bakry-\'Emery Ricci flow on finite graphs. Our main result is the local existence and uniqueness of solutions to the Ricci flow. We prove the long-time convergence or finite-time blow up for the Bakry-\'Emery Ricci flow on finite trees and circles.

Large language models (LLMs) have recently experienced remarkable progress, where the advent of multi-modal large language models (MLLMs) has endowed LLMs with visual capabilities, leading to impressive performances in various multi-modal tasks. However, those powerful MLLMs such as GPT-4V still fail spectacularly when presented with certain image and text inputs. In this paper, we identify a typical class of inputs that baffles MLLMs, which consist of images that are highly relevant but inconsistent with answers, causing MLLMs to suffer from hallucination. To quantify the effect, we propose CorrelationQA, the first benchmark that assesses the hallucination level given spurious images. This benchmark contains 7,308 text-image pairs across 13 categories. Based on the proposed CorrelationQA, we conduct a thorough analysis on 9 mainstream MLLMs, illustrating that they universally suffer from this instinctive bias to varying degrees. We hope that our curated benchmark and evaluation results aid in better assessments of the MLLMs' robustness in the presence of misleading images. The resource is available in https://github.com/MasaiahHan/CorrelationQA.

We comment on the puzzling status of the proton magnetic radius determinations., Comment: 3 pages, 1 figure, accepted for publication in Science Bulletin, News and Views section, irrelevant formula removed

Large language models (LLMs) have revolutionized numerous domains with their impressive performance but still face their challenges. A predominant issue is the propensity for these models to generate non-existent facts, a concern termed hallucination. Our research is motivated by the observation that previous instruction tuning methods force the model to complete a sentence no matter whether the model knows the knowledge or not. When the question is out of the parametric knowledge, it will try to make up something and fail to indicate when it lacks knowledge. In this paper, we present a new approach called Refusal-Aware Instruction Tuning (R-Tuning). This approach is formalized by first identifying the disparity in knowledge encompassed by pre-trained parameters compared to that of instruction tuning data. Then, we construct the refusal-aware data based on the knowledge intersection, to tune LLMs to refrain from responding to questions beyond its parametric knowledge. Experimental results demonstrate R-Tuning effectively improves a model's ability to answer known questions and refrain from answering unknown questions. Furthermore, when tested on out-of-domain datasets, the refusal ability was found to be a meta-skill that could be generalized to other tasks. Further analysis surprisingly finds that learning the uncertainty results in better calibration and an improved ability to estimate the uncertainty than uncertainty-based testing. Our code is available at https://github.com/shizhediao/R-Tuning., Comment: NAACL 2024

Invariance learning methods aim to learn invariant features in the hope that they generalize under distributional shifts. Although many tasks are naturally characterized by continuous domains, current invariance learning techniques generally assume categorically indexed domains. For example, auto-scaling in cloud computing often needs a CPU utilization prediction model that generalizes across different times (e.g., time of a day and date of a year), where `time' is a continuous domain index. In this paper, we start by theoretically showing that existing invariance learning methods can fail for continuous domain problems. Specifically, the naive solution of splitting continuous domains into discrete ones ignores the underlying relationship among domains, and therefore potentially leads to suboptimal performance. To address this challenge, we then propose Continuous Invariance Learning (CIL), which extracts invariant features across continuously indexed domains. CIL is a novel adversarial procedure that measures and controls the conditional independence between the labels and continuous domain indices given the extracted features. Our theoretical analysis demonstrates the superiority of CIL over existing invariance learning methods. Empirical results on both synthetic and real-world datasets (including data collected from production systems) show that CIL consistently outperforms strong baselines among all the tasks.

Generalization to out-of-distribution (OOD) data is a critical challenge in machine learning. Ensemble-based methods, like weight space ensembles that interpolate model parameters, have been shown to achieve superior OOD performance. However, the underlying mechanism for their effectiveness remains unclear. In this study, we closely examine WiSE-FT, a popular weight space ensemble method that interpolates between a pre-trained and a fine-tuned model. We observe an unexpected ``FalseFalseTrue" phenomenon, in which WiSE-FT successfully corrects many cases where each individual model makes incorrect predictions, which contributes significantly to its OOD effectiveness. To gain further insights, we conduct theoretical analysis in a multi-class setting with a large number of spurious features. Our analysis predicts the above phenomenon and it further shows that ensemble-based models reduce prediction errors in the OOD settings by utilizing a more diverse set of spurious features. Contrary to the conventional wisdom that focuses on learning invariant features for better OOD performance, our findings suggest that incorporating a large number of diverse spurious features weakens their individual contributions, leading to improved overall OOD generalization performance. Additionally, our findings provide the first explanation for the mysterious phenomenon of weight space ensembles outperforming output space ensembles in OOD. Empirically we demonstrate the effectiveness of utilizing diverse spurious features on a MultiColorMNIST dataset, and our experimental results are consistent with the theoretical analysis. Building upon the new theoretical insights into the efficacy of ensemble methods, we further propose a novel averaging method called BAlaNced averaGing (BANG) which significantly enhances the OOD performance of WiSE-FT., Comment: ICLR 2024

We propose a novel method for measuring the charge radii of charged stable hadrons, with which the first measurement of the charge radii of the $\Sigma^+$ and the $\Xi^-$ is foreseen. The method explores the facts that the Dalitz decay $\psi(2S) \to Y\bar{Y}e^+e^-$ contains the hyperon form factors and the lowest measurable four-momentum transfer squared can be as low as $\sim 4m_e^2= 1.05\times10^{-6}\,{\rm GeV^2}$ in the time-like region. We identify a kinematic region where the hyperon form factors are essential and propose a method for subtracting the background from the data. It is estimated that the hyperon charge radii can be measured to a precision of about {0.2~fm} with the BES\Rom{3} experiment and one order of magnitude better at the future Super $\tau$-Charm Facility. Moreover, the same method can be used to measure the charge radius of the proton, which provides an independent cross-check on the extraction of proton radius from elastic $ep$ scattering or leptonic hydrogen spectroscopy., Comment: 10 pages, 6 figures; PLB accepted version

LLMs acquire a wide range of abilities during pre-training, but aligning LLMs under Reinforcement Learning with Human Feedback (RLHF) can lead to forgetting, which is also known as the alignment tax. To empirically verify this hypothesis, we conducted experiments with existing RLHF algorithms using OpenLLaMA-3B, which revealed a pronounced alignment tax in NLP tasks. On the other hand, despite various techniques to mitigate forgetting, they are often at odds with the RLHF performance, leading to a trade-off between reward maximization and forgetting mitigation. In light of the above pressing issue in aligning LLMs, in this paper we explore model averaging, which interpolates between pre and post RLHF model weights, to achieve a more efficient reward-tax Pareto front. To understand its effectiveness, We offer theoretical insights into model averaging, revealing that it enhances performance Pareto front by increasing feature diversity on the layers where tasks share overlapped feature spaces. Empirical evidence corroborates our analysis by showing the benefits of averaging low-level transformer layers. Building on the analysis and the observation that averaging different layers of the transformer leads to significantly different reward-tax trade-offs, we propose Adaptive Model Averaging (AMA) to adaptively find various combination ratios of model layers. AMA seeks to maximize the alignment reward while incurring minimal alignment tax. Moreover, we validate AMA's performance across a range of RLHF algorithms over OpenLLaMA-3B and further extend our findings to Mistral-7B., Comment: 28 Pages

Modern deep learning heavily relies on large labeled datasets, which often comse with high costs in terms of both manual labeling and computational resources. To mitigate these challenges, researchers have explored the use of informative subset selection techniques, including coreset selection and active learning. Specifically, coreset selection involves sampling data with both input ($\bx$) and output ($\by$), active learning focuses solely on the input data ($\bx$). In this study, we present a theoretically optimal solution for addressing both coreset selection and active learning within the context of linear softmax regression. Our proposed method, COPS (unCertainty based OPtimal Sub-sampling), is designed to minimize the expected loss of a model trained on subsampled data. Unlike existing approaches that rely on explicit calculations of the inverse covariance matrix, which are not easily applicable to deep learning scenarios, COPS leverages the model's logits to estimate the sampling ratio. This sampling ratio is closely associated with model uncertainty and can be effectively applied to deep learning tasks. Furthermore, we address the challenge of model sensitivity to misspecification by incorporating a down-weighting approach for low-density samples, drawing inspiration from previous works. To assess the effectiveness of our proposed method, we conducted extensive empirical experiments using deep neural networks on benchmark datasets. The results consistently showcase the superior performance of COPS compared to baseline methods, reaffirming its efficacy.

Offline goal-conditioned RL (GCRL) offers a way to train general-purpose agents from fully offline datasets. In addition to being conservative within the dataset, the generalization ability to achieve unseen goals is another fundamental challenge for offline GCRL. However, to the best of our knowledge, this problem has not been well studied yet. In this paper, we study out-of-distribution (OOD) generalization of offline GCRL both theoretically and empirically to identify factors that are important. In a number of experiments, we observe that weighted imitation learning enjoys better generalization than pessimism-based offline RL method. Based on this insight, we derive a theory for OOD generalization, which characterizes several important design choices. We then propose a new offline GCRL method, Generalizable Offline goAl-condiTioned RL (GOAT), by combining the findings from our theoretical and empirical studies. On a new benchmark containing 9 independent identically distributed (IID) tasks and 17 OOD tasks, GOAT outperforms current state-of-the-art methods by a large margin., Comment: Accepted by International Conference on Machine Learning (ICML), 2023

The internal structure of the charm-strange mesons $D_{s0}^*(2317)$ and $D_{s1}(2460)$ are subject of intensive studies. Their widths are small because they decay dominantly through isospin-breaking hadronic channels $D_{s0}^*(2317)^+\to D_s^+\pi^0$ and $D_{s1}(2460)^+\to D_s^{*+}\pi^0$. The $D_{s1}(2460)$ can also decay into the hadronic final states $D_s^+\pi\pi$, conserving isospin. In that case there is, however, a strong suppression from phase space. We study the transition $D_{s1}(2460)^+\to D_s^+\pi^+\pi^-$ in the scenario that the $D_{s1}(2460)$ is a $D^*K$ hadronic molecule. The $\pi\pi$ final state interaction is taken into account through dispersion relations. We find that the ratio of the partial widths of the $\Gamma(D_{s1}(2460)^+\to D_s^+\pi^+\pi^-)/\Gamma(D_{s1}(2460)^+\to D_s^{*+}\pi^0)$ obtained in the molecular picture is consistent with the existing experimental measurement. More interestingly, we demonstrate that the $\pi^+\pi^-$ invariant mass distribution shows a double bump structure, which can be used to disentangle the hadronic molecular picture from the compact state picture for the $D_{s1}(2460)^+$. Predictions on the $B_{s1}^0\to B_s^0\pi^+\pi^-$ are also made., Comment: 16 pages, 5 figures; the published version

The increasing scale of large language models (LLMs) brings emergent abilities to various complex tasks requiring reasoning, such as arithmetic and commonsense reasoning. It is known that the effective design of task-specific prompts is critical for LLMs' ability to produce high-quality answers. In particular, an effective approach for complex question-and-answer tasks is example-based prompting with chain-of-thought (CoT) reasoning, which significantly improves the performance of LLMs. However, current CoT methods rely on a fixed set of human-annotated exemplars, which are not necessarily the most effective examples for different tasks. This paper proposes a new method, Active-Prompt, to adapt LLMs to different tasks with task-specific example prompts (annotated with human-designed CoT reasoning). For this purpose, we propose a solution to the key problem of determining which questions are the most important and helpful ones to annotate from a pool of task-specific queries. By borrowing ideas from the related problem of uncertainty-based active learning, we introduce several metrics to characterize the uncertainty so as to select the most uncertain questions for annotation. Experimental results demonstrate the superiority of our proposed method, achieving state-of-the-art on eight complex reasoning tasks. Further analyses of different uncertainty metrics, pool sizes, zero-shot learning, and accuracy-uncertainty relationship demonstrate the effectiveness of our method. Our code will be available at https://github.com/shizhediao/active-prompt., Comment: Published in ACL 2024