Search

Automatic citation generation for sentences in a document or report is paramount for intelligence analysts, cybersecurity, news agencies, and education personnel. In this research, we investigate whether large language models (LLMs) are capable of generating references based on two forms of sentence queries: (a) Direct Queries, LLMs are asked to provide author names of the given research article, and (b) Indirect Queries, LLMs are asked to provide the title of a mentioned article when given a sentence from a different article. To demonstrate where LLM stands in this task, we introduce a large dataset called REASONS comprising abstracts of the 12 most popular domains of scientific research on arXiv. From around 20K research articles, we make the following deductions on public and proprietary LLMs: (a) State-of-the-art, often called anthropomorphic GPT-4 and GPT-3.5, suffers from high pass percentage (PP) to minimize the hallucination rate (HR). When tested with Perplexity.ai (7B), they unexpectedly made more errors; (b) Augmenting relevant metadata lowered the PP and gave the lowest HR; (c) Advance retrieval-augmented generation (RAG) using Mistral demonstrates consistent and robust citation support on indirect queries and matched performance to GPT-3.5 and GPT-4. The HR across all domains and models decreased by an average of 41.93% and the PP was reduced to 0% in most cases. In terms of generation quality, the average F1 Score and BLEU were 68.09% and 57.51%, respectively; (d) Testing with adversarial samples showed that LLMs, including the Advance RAG Mistral, struggle to understand context, but the extent of this issue was small in Mistral and GPT-4-Preview. Our study con tributes valuable insights into the reliability of RAG for automated citation generation tasks., Comment: Submitted to ACL ARR April 2024

As the burden of herbicide resistance grows and the environmental repercussions of excessive herbicide use become clear, new ways of managing weed populations are needed. This is particularly true for cereal crops, like wheat and barley, that are staple food crops and occupy a globally significant portion of agricultural land. Even small improvements in weed management practices across these major food crops worldwide would yield considerable benefits for both the environment and global food security. Blackgrass is a major grass weed which causes particular problems in cereal crops in north-west Europe, a major cereal production area, because it has high levels of of herbicide resistance and is well adapted to agronomic practice in this region. With the use of machine vision and multispectral imaging, we investigate the effectiveness of state-of-the-art methods to identify blackgrass in wheat and barley crops. As part of this work, we provide a large dataset with which we evaluate several key aspects of blackgrass weed recognition. Firstly, we determine the performance of different CNN and transformer-based architectures on images from unseen fields. Secondly, we demonstrate the role that different spectral bands have on the performance of weed classification. Lastly, we evaluate the role of dataset size in classification performance for each of the models trialled. We find that even with a fairly modest quantity of training data an accuracy of almost 90% can be achieved on images from unseen fields., Comment: 19 pages, 6 figures

Recently a certain thought experiment was discussed which involves the decoherence of a quantum system due to a black hole. Here we show how this phenomenon is consistent with standard ideas about quantum black holes. In other words, modeling the black hole as a quantum system at finite temperature one obtains the same answer. We demonstrate this by analyzing the problem in terms of an effective theory that can apply both for the black hole case and for an ordinary matter system, showing that the same qualitative effect is present for ordinary matter at finite temperature.

This paper introduces a framework for post-processing machine learning models so that their predictions satisfy multi-group fairness guarantees. Based on the celebrated notion of multicalibration, we introduce $(\mathbf{s},\mathcal{G}, \alpha)-$GMC (Generalized Multi-Dimensional Multicalibration) for multi-dimensional mappings $\mathbf{s}$, constraint set $\mathcal{G}$, and a pre-specified threshold level $\alpha$. We propose associated algorithms to achieve this notion in general settings. This framework is then applied to diverse scenarios encompassing different fairness concerns, including false negative rate control in image segmentation, prediction set conditional uncertainty quantification in hierarchical classification, and de-biased text generation in language models. We conduct numerical studies on several datasets and tasks., Comment: 28 pages, 8 figures, accepted by ICML2024

Binary Neural Networks emerged as a cost-effective and energy-efficient solution for computer vision tasks by binarizing either network weights or activations. However, common binary activations, such as the Sign activation function, abruptly binarize the values with a single threshold, losing fine-grained details in the feature outputs. This work proposes an activation that applies multiple thresholds following dithering principles, shifting the Sign activation function for each pixel according to a spatially periodic threshold kernel. Unlike literature methods, the shifting is defined jointly for a set of adjacent pixels, taking advantage of spatial correlations. Experiments over the classification task demonstrate the effectiveness of the designed dithering Sign activation function as an alternative activation for binary neural networks, without increasing the computational cost. Further, DeSign balances the preservation of details with the efficiency of binary operations., Comment: 7 pages

We show that there exists a quasi-isometric embedding of the product of $n$ copies of $\mathbb{H}_{\mathbb{R}}^2$ into any symmetric space of non-compact type of rank $n$, and there exists a bi-Lipschitz embedding of the product of $n$ copies of the $3$-regular tree $T_3$ into any thick Euclidean building of rank $n$ with co-compact affine Weyl group. This extends a previous result of Fisher--Whyte. The proof is purely geometrical, and the result also applies to the non Bruhat--Tits buildings., Comment: 23 pages, 6 figures. Comments are welcome!

Operator learning is a variant of machine learning that is designed to approximate maps between function spaces from data. The Fourier Neural Operator (FNO) is a common model architecture used for operator learning. The FNO combines pointwise linear and nonlinear operations in physical space with pointwise linear operations in Fourier space, leading to a parameterized map acting between function spaces. Although FNOs formally involve convolutions of functions on a continuum, in practice the computations are performed on a discretized grid, allowing efficient implementation via the FFT. In this paper, the aliasing error that results from such a discretization is quantified and algebraic rates of convergence in terms of the grid resolution are obtained as a function of the regularity of the input. Numerical experiments that validate the theory and describe model stability are performed.

This paper presents a framework for evaluating fairness in recommender systems powered by Large Language Models (RecLLMs), addressing the need for a unified approach that spans various fairness dimensions including sensitivity to user attributes, intrinsic fairness, and discussions of fairness based on underlying benefits. In addition, our framework introduces counterfactual evaluations and integrates diverse user group considerations to enhance the discourse on fairness evaluation for RecLLMs. Our key contributions include the development of a robust framework for fairness evaluation in LLM-based recommendations and a structured method to create \textit{informative user profiles} from demographic data, historical user preferences, and recent interactions. We argue that the latter is essential for enhancing personalization in such systems, especially in temporal-driven scenarios. We demonstrate the utility of our framework through practical applications on two datasets, LastFM-1K and ML-1M. We conduct experiments on a subsample of 80 users from each dataset, testing and assessing the effectiveness of various prompt construction scenarios and in-context learning, comprising more than 50 scenarios. This results in more than 4000 recommendations (80 * 50 = 4000). Our study reveals that while there are no significant unfairness issues in scenarios involving sensitive attributes, some concerns remain. However, in terms of intrinsic fairness, which does not involve direct sensitivity, unfairness across demographic groups remains significant. The code and data used for this paper are available at: \url{https://shorturl.at/awBFM}.

There has been recently considerable progress in understanding the nature of perturbation theory in UV free and gapped $2d$ integrable field theories with renormalon singularities. Thanks to Bethe ansatz and large $N$ techniques, non-perturbative corrections can also be computed and lead to the reconstruction of the trans-series for the free energy in presence of a chemical potential. This is an ideal arena to test resurgence in QFT and determine if and how the exact result can be reconstructed from the knowledge of the perturbative series only. In these notes we give a pedagogical introduction to this subject starting from the basics. In the first lecture we give an overview of applications in QFT of Borel resummations before the advent of resurgence. The second lecture introduces the key concepts of resurgence and finally in the third lecture we discuss a specific application in the context of the principal chiral field model. Extended version of three lectures given at IHES and review talks given at Les Diablerets and Mainz, in 2023., Comment: 45 pages, 7 figures

During the last 25 years, hundreds of binary stars and planets have been discovered towards the Galactic Bulge by microlensing surveys. Thanks to a new generation of large-sky surveys, it is now possible to regularly detect microlensing events across the entire sky. The OMEGA Key Projet at the Las Cumbres Observatory carries out automated follow-up observations of microlensing events alerted by these surveys with the aim of identifying and characterizing exoplanets as well as stellar remnants. In this study, we present the analysis of the binary lens event Gaia20bof. By automatically requesting additional observations, the OMEGA Key Project obtained dense time coverage of an anomaly near the peak of the event, allowing characterization of the lensing system. The observed anomaly in the lightcurve is due to a binary lens. However, several models can explain the observations. Spectroscopic observations indicate that the source is located at $\le2.0$ kpc, in agreement with the parallax measurements from Gaia. While the models are currently degenerate, future observations, especially the Gaia astrometric time series as well as high-resolution imaging, will provide extra constraints to distinguish between them., Comment: Accepted in AJ