Search

As large language models (LLMs) continue to advance, aligning these models with human preferences has emerged as a critical challenge. Traditional alignment methods, relying on human or LLM annotated datasets, are limited by their resource-intensive nature, inherent subjectivity, and the risk of feedback loops that amplify model biases. To overcome these limitations, we introduce WildFeedback, a novel framework that leverages real-time, in-situ user interactions to create preference datasets that more accurately reflect authentic human values. WildFeedback operates through a three-step process: feedback signal identification, preference data construction, and user-guided evaluation. We applied this framework to a large corpus of user-LLM conversations, resulting in a rich preference dataset that reflects genuine user preferences. This dataset captures the nuances of user preferences by identifying and classifying feedback signals within natural conversations, thereby enabling the construction of more representative and context-sensitive alignment data. Our extensive experiments demonstrate that LLMs fine-tuned on WildFeedback exhibit significantly improved alignment with user preferences, as evidenced by both traditional benchmarks and our proposed user-guided evaluation. By incorporating real-time feedback from actual users, WildFeedback addresses the scalability, subjectivity, and bias challenges that plague existing approaches, marking a significant step toward developing LLMs that are more responsive to the diverse and evolving needs of their users. In summary, WildFeedback offers a robust, scalable solution for aligning LLMs with true human values, setting a new standard for the development and evaluation of user-centric language models., Comment: 24 pages

Combining multiple predictors obtained from distributed data sources to an accurate meta-learner is promising to achieve enhanced performance in lots of prediction problems. As the accuracy of each predictor is usually unknown, integrating the predictors to achieve better performance is challenging. Conventional ensemble learning methods assess the accuracy of predictors based on extensive labeled data. In practical applications, however, the acquisition of such labeled data can prove to be an arduous task. Furthermore, the predictors under consideration may exhibit high degrees of correlation, particularly when similar data sources or machine learning algorithms were employed during their model training. In response to these challenges, this paper introduces a novel structured unsupervised ensemble learning model (SUEL) to exploit the dependency between a set of predictors with continuous predictive scores, rank the predictors without labeled data and combine them to an ensembled score with weights. Two novel correlation-based decomposition algorithms are further proposed to estimate the SUEL model, constrained quadratic optimization (SUEL.CQO) and matrix-factorization-based (SUEL.MF) approaches. The efficacy of the proposed methods is rigorously assessed through both simulation studies and real-world application of risk genes discovery. The results compellingly demonstrate that the proposed methods can efficiently integrate the dependent predictors to an ensemble model without the need of ground truth data.

Analytical and semi-analytical models for stream depletion with transient stream stage drawdown induced by groundwater pumping are developed to address a deficiency in existing models, namely, the use of a fixed stream stage condition at the stream-aquifer interface. Field data are presented to demonstrate that stream stage drawdown does indeed occur in response to groundwater pumping near aquifer connected streams. A model that predicts stream depletion with transient stream drawdown is developed, based on stream channel mass conservation and finite stream channel storage. The resulting models are shown to reduce to existing fixed-stage models in the limit as stream channel storage becomes infinitely large, and to the confined aquifer flow with a no-flow boundary at the streambed in the limit as stream storage becomes vanishingly small. The model is applied to field measurements of aquifer and stream drawdown, giving estimates of aquifer hydraulic parameters, streambed conductance and a measure of stream channel storage. The results of the modeling and data analysis presented herein have implications for sustainable groundwater management., Comment: 13 figure, 4 tables

Purpose. It is a long standing pursuit in sodium (23Na) MRI to separate signals between mono and bi exponential T2 decays in the human brain, due to lack of clinically translational solutions under the restriction of intrinsically low signal to noise ratio (SNR). Here we propose a new technique called multi TE single quantum (MSQ) sodium MRI to address the challenge. Methods. We exploit an intrinsic difference in T2 decay between mono and bi exponential sodium signals by acquiring SQ images at multiple TEs and performing voxel based matrix inversions on these SQ images. The MSQ method was then investigated on numerical models, agar phantoms, and human brains for the feasibility on clinical scanners at 3T. Results. The whole brain T2* spectrum of FID signals from the study subjects showed sparse peaks (2 to 4 peaks), suggesting a global set of T2* values (T2*fr, T2*bs, T2*bl) applicable to the separation. The simulations indicated a small impact (3.9 to 5.6 percent) of T2* variation on accuracy of the separation, and the phantom experiments showed a high accuracy of the separation, 95.8 percent for mono T2 sodium and 72.5 to 80.4 percent for biT2 sodium. The human studies demonstrated feasibility of the separation and potentials of highlighting abnormal brain regions in the biT2 sodium images. Conclusion. The MSQ technique has been shown, via the numerical simulations, phantom experiments, and human brain studies, to be able to separate mono and bi T2 sodium signals using a two TE sampling scheme and a global set of T2* values. However, MSQ has limitations and requires cautions in practice. Keywords. sodium MRI, single quantum MRI, triple quantum MRI, neuroimaging, neurodegeneration, Comment: 37 pages and 14 figures

We prove that twisted groupoid C*-algebras are characterised, up to isomorphism, by having Cartan semigroups, a natural generalisation of normaliser semigroups of Cartan subalgebras. This extends the classic Kumjian-Renault theory to general twisted \'etale groupoid C*-algebras, even non-reduced C*-algebras of non-effective groupoids., Comment: 39 pages

Online learning has demonstrated notable potential to dynamically allocate limited resources to monitor a large population of processes, effectively balancing the exploitation of processes yielding high rewards, and the exploration of uncertain processes. However, most online learning algorithms were designed under 1) a centralized setting that requires data sharing across processes to obtain an accurate prediction or 2) a homogeneity assumption that estimates a single global model from the decentralized data. To facilitate the online learning of heterogeneous processes from the decentralized data, we propose a federated collaborative online monitoring method, which captures the latent representative models inherent in the population through representation learning and designs a novel federated collaborative UCB algorithm to estimate the representative models from sequentially observed decentralized data. The efficiency of our method is illustrated through theoretical analysis, simulation studies, and decentralized cognitive degradation monitoring in Alzheimer's disease.

Accurate and interpretable user satisfaction estimation (USE) is critical for understanding, evaluating, and continuously improving conversational systems. Users express their satisfaction or dissatisfaction with diverse conversational patterns in both general-purpose (ChatGPT and Bing Copilot) and task-oriented (customer service chatbot) conversational systems. Existing approaches based on featurized ML models or text embeddings fall short in extracting generalizable patterns and are hard to interpret. In this work, we show that LLMs can extract interpretable signals of user satisfaction from their natural language utterances more effectively than embedding-based approaches. Moreover, an LLM can be tailored for USE via an iterative prompting framework using supervision from labeled examples. The resulting method, Supervised Prompting for User satisfaction Rubrics (SPUR), not only has higher accuracy but is more interpretable as it scores user satisfaction via learned rubrics with a detailed breakdown.

In this paper, without requiring any constraint qualifications, we establish tight error bounds for the log-determinant cone, which is the closure of the hypograph of the perspective function of the log-determinant function. This error bound is obtained using the recently developed framework based on one-step facial residual functions., Comment: 32 pages, comments welcome

We present CFEVER, a Chinese dataset designed for Fact Extraction and VERification. CFEVER comprises 30,012 manually created claims based on content in Chinese Wikipedia. Each claim in CFEVER is labeled as "Supports", "Refutes", or "Not Enough Info" to depict its degree of factualness. Similar to the FEVER dataset, claims in the "Supports" and "Refutes" categories are also annotated with corresponding evidence sentences sourced from single or multiple pages in Chinese Wikipedia. Our labeled dataset holds a Fleiss' kappa value of 0.7934 for five-way inter-annotator agreement. In addition, through the experiments with the state-of-the-art approaches developed on the FEVER dataset and a simple baseline for CFEVER, we demonstrate that our dataset is a new rigorous benchmark for factual extraction and verification, which can be further used for developing automated systems to alleviate human fact-checking efforts. CFEVER is available at https://ikmlab.github.io/CFEVER., Comment: AAAI-24

We propose a classification of BPS states in holographic CFTs into monotone and fortuitous, based on their behaviors in the large $N$ limit. Intuitively, monotone BPS states form infinite sequences with increasing rank $N$, while fortuitous ones exist within finite ranges of consecutive ranks. A precise definition is formulated using supercharge cohomology. We conjecture that under the AdS/CFT correspondence, monotone BPS states are dual to smooth horizonless geometries, and fortuitous ones are responsible for typical black hole microstates and give dominant contributions to the entropy. We present supporting evidence for our conjectures in the ${\cal N}=4$ SYM and symmetric product orbifolds., Comment: 23 pages, 1 figure; v2: minor revision, Appendix A added

We investigate the existence of $\frac18$-BPS black hole microstates in the $\mathfrak{su}(1,1|2)$ sector of Type IIB string theory on $\mathrm{AdS}_5 \times \mathrm{S}^5$. As will be explained, these states are in one-to-one correspondence with the Schur operators comprising the chiral algebra of $\mathcal{N}=4$ super-Yang-Mills, and a conjecture of Beem et al. implies that the Schur sector only contains graviton operators and hence $\frac18$-BPS black holes do not exist. We scrutinize this conjecture from multiple angles. Concerning the macroscopic counting, we rigorously prove that the flavored Schur index cannot exhibit black hole entropy growth, and provide numerical evidence that the flavored MacDonald index also does not exhibit such growth. Next, we go beyond counting to examine the algebraic structure, beginning by presenting evidence for the well-definedness of the super-$\mathcal{W}$ algebra of Beem et al., then using modular differential equations to argue for an upper bound on the lightest non-graviton operator if existent, and finally performing a systematic construction of cohomologies to recover only gravitons. Along the way, we clarify key aspects of the 4d/2d correspondence using the formalism of the holomorphic topological twist., Comment: 37+15 pages, 5 figures, 6 tables; v2: minor revision, references added

Adaptive monitoring of a large population of dynamic processes is critical for the timely detection of abnormal events under limited resources in many healthcare and engineering systems. Examples include the risk-based disease screening and condition-based process monitoring. However, existing adaptive monitoring models either ignore the dependency among processes or overlook the uncertainty in process modeling. To design an optimal monitoring strategy that accurately monitors the processes with poor health conditions and actively collects information for uncertainty reduction, a novel online collaborative learning method is proposed in this study. The proposed method designs a collaborative learning-based upper confidence bound (CL-UCB) algorithm to optimally balance the exploitation and exploration of dependent processes under limited resources. Efficiency of the proposed method is demonstrated through theoretical analysis, simulation studies and an empirical study of adaptive cognitive monitoring in Alzheimer's disease.

In this note, we present criteria that are equivalent to a locally compact Hausdorff groupoid $G$ being effective. One of these conditions is that $G$ satisfies the "C*-algebraic local bisection hypothesis"; that is, that every normaliser in the reduced twisted groupoid C*-algebra is supported on an open bisection. The semigroup of normalisers plays a fundamental role in our proof, as does the semigroup of normalisers in cyclic group C*-algebras., Comment: 11 pages. This version matches the version in Semigroup Forum

We consider BPS states in M theory compactified on a Calabi-Yau threefold with one K\"ahler parameter, and investigate their relation to blackholes and blackrings. On the microscopic side, a characterization of the BPS degeneracy can be obtained from the non-perturbative part of the topological string free energy according to the Gopakumar-Vafa conjecture. On the macroscopic side, the Wald entropy of relevant black objects in the four-dimensional IIA description is computed from the perturbative part of the same topological string free energy following the work of Cardoso-de Wit-Mohaupt and then lifted to five-dimensional M theory through the Gaiotto-Strominger-Yin correspondence. For a given value of the M2 brane charge, we find that for small angular momenta, the microscopic BPS degeneracy matches the entropy of a rotating blackhole, whereas an apparent transition occurs at a critical value of the angular momentum. Beyond this value, we find encouraging evidence that the microscopic counting is well approximated by the entropy of a blackring. We conclude by formulating a new puzzle involving the corrections due to degenerate worldsheet instantons., Comment: 53 pages

Building on the recent discovery of the first candidate black hole operator in $\mathcal{N}=4$ super-Yang-Mills, we explore the near-supersymmetric aspects of the theory that capture lightly excited, highly stringy black holes. We extend the superspace formalism describing the classically supersymmetric (1/16-BPS) sector of $\mathcal{N}=4$ super-Yang-Mills and compute a large number of one-loop anomalous dimensions. Despite being in the highly stringy regime, we find hints of a gap in the spectrum, similar to that found by a gravitational path integral. We also determine the actual expression of the first candidate black hole operator at weak gauge coupling, going beyond the cohomological construction., Comment: 22+14 pages, 1 table, 3 figures; v2: minor revision

This work introduces the first small-loss and gradual-variation regret bounds for online portfolio selection, marking the first instances of data-dependent bounds for online convex optimization with non-Lipschitz, non-smooth losses. The algorithms we propose exhibit sublinear regret rates in the worst cases and achieve logarithmic regrets when the data is "easy," with per-iteration time almost linear in the number of investment alternatives. The regret bounds are derived using novel smoothness characterizations of the logarithmic loss, a local norm-based analysis of following the regularized leader (FTRL) with self-concordant regularizers, which are not necessarily barriers, and an implicit variant of optimistic FTRL with the log-barrier., Comment: 37 pages, typos fixed, NeurIPS 2023