Search

Recently, there have been several advancements in quantum algorithms for Gibbs sampling. These algorithms simulate the dynamics generated by an artificial Lindbladian, which is meticulously constructed to obey a detailed-balance condition with the Gibbs state of interest, ensuring it is a stationary point of the evolution, while simultaneously having efficiently implementable time steps. The overall complexity then depends primarily on the mixing time of the Lindbladian, which can vary drastically, but which has been previously bounded in the regime of high enough temperatures [Rouz\'e {\it et al.}~arXiv:2403.12691 and arXiv:2411.04885]. In this work, we calculate the spectral gap of the Lindbladian for free fermions using third quantisation, and then prove a constant gap of the perturbed Lindbladian corresponding to interacting fermions up to some maximal coupling strength. This is achieved by using theorems about stability of the gap for lattice fermions. Our methods apply at any constant temperature and independently of the system size. The gap then provides an upper bound on the mixing time, and hence on the overall complexity of the quantum algorithm, proving that the purified Gibbs state of weakly interacting (quasi-)local fermionic systems of any dimension can be prepared in $\widetilde{\mathcal{O}} (n^3 \operatorname{polylog}(1/\epsilon))$ time on $\mathcal{O}(n)$ qubits, where $n$ denotes the size of the system and $\epsilon$ the desired accuracy. We provide exact numerical simulations for small system sizes supporting the theory and also identify different suitable jump operators and filter functions for the sought-after regime of intermediate coupling in the Fermi-Hubbard model.

This summary of the second Terrestrial Very-Long-Baseline Atom Interferometry (TVLBAI) Workshop provides a comprehensive overview of our meeting held in London in April 2024, building on the initial discussions during the inaugural workshop held at CERN in March 2023. Like the summary of the first workshop, this document records a critical milestone for the international atom interferometry community. It documents our concerted efforts to evaluate progress, address emerging challenges, and refine strategic directions for future large-scale atom interferometry projects. Our commitment to collaboration is manifested by the integration of diverse expertise and the coordination of international resources, all aimed at advancing the frontiers of atom interferometry physics and technology, as set out in a Memorandum of Understanding signed by over 50 institutions., Comment: Summary of the second Terrestrial Very-Long-Baseline Atom Interferometry Workshop held at Imperial College London: https://indico.cern.ch/event/1369392/

Betting markets are gaining in popularity. Mean beliefs generally differ from prices in prediction markets. Logarithmic utility is employed to study the risk and return adjustments to prices. Some consequences are described. A modified payout structure is proposed. A simple asset price model based on flipping biased coins is investigated. It is shown using the Kullback-Leibler divergence how the misjudgment of the bias and the miscalculation of the investment fraction influence the portfolio growth rate., Comment: 6 pages

This note is a companion paper to arXiv:1608.01630 [math.CA]. Here we generalize some of the geometric results of arXiv:1608.01630 [math.CA] to the case of a $3\times 3$ matrix function $A(x)\approx \mathrm{diag}\{1,f(x_1), g(x_1)\}$. More precisely, we make explicit calculations of the geodesics in the Carnot-Carath\'{e}odory space associated to $A$, and provide estimates on the Lebesgue measures of metric balls centered at the origin in that space.

The collection of speech data carried out in Sociolinguistics has the potential to enhance large language models due to its quality and representativeness. In this paper, we examine the ethical considerations associated with the gathering and dissemination of such data. Additionally, we outline strategies for addressing the sensitivity of speech data, as it may facilitate the identification of informants who contributed with their speech., Comment: in Portuguese language. Paper accepted to LAAI-Ethics 2024

Language models (LMs) are increasingly used as simulacra for people, yet their ability to match the distribution of views of a specific demographic group and be \textit{distributionally aligned} remains uncertain. This notion of distributional alignment is complex, as there is significant variation in the types of attributes that are simulated. Prior works have underexplored the role of three critical variables -- the question domain, steering method, and distribution expression method -- which motivates our contribution of a benchmark explicitly addressing these dimensions. We construct a dataset expanding beyond political values, create human baselines for this task, and evaluate the extent to which an LM can align with a particular group's opinion distribution to inform design choices of such simulation systems. Our analysis reveals open problems regarding if, and how, LMs can be used to simulate humans, and that LLMs can more accurately describe the opinion distribution than simulate such distributions.

This study examines the applicability of the Cloze test, a widely used tool for assessing text comprehension proficiency, while highlighting its challenges in large-scale implementation. To address these limitations, an automated correction approach was proposed, utilizing Natural Language Processing (NLP) techniques, particularly word embeddings (WE) models, to assess semantic similarity between expected and provided answers. Using data from Cloze tests administered to students in Brazil, WE models for Brazilian Portuguese (PT-BR) were employed to measure the semantic similarity of the responses. The results were validated through an experimental setup involving twelve judges who classified the students' answers. A comparative analysis between the WE models' scores and the judges' evaluations revealed that GloVe was the most effective model, demonstrating the highest correlation with the judges' assessments. This study underscores the utility of WE models in evaluating semantic similarity and their potential to enhance large-scale Cloze test assessments. Furthermore, it contributes to educational assessment methodologies by offering a more efficient approach to evaluating reading proficiency.

Linguistic diversity is a human attribute which, with the advance of generative AIs, is coming under threat. This paper, based on the contributions of sociolinguistics, examines the consequences of the variety selection bias imposed by technological applications and the vicious circle of preserving a variety that becomes dominant and standardized because it has linguistic documentation to feed the large language models for machine learning., Comment: in Portuguese language. paper aceepted to LAAI-Ethics 2024

Surprisal theory posits that the cognitive effort required to comprehend a word is determined by its contextual predictability, quantified as surprisal. Traditionally, surprisal theory treats words as distinct entities, overlooking any potential similarity between them. Giulianelli et al. (2023) address this limitation by introducing information value, a measure of predictability designed to account for similarities between communicative units. Our work leverages Ricotta and Szeidl's (2006) diversity index to extend surprisal into a metric that we term similarity-adjusted surprisal, exposing a mathematical relationship between surprisal and information value. Similarity-adjusted surprisal aligns with information value when considering graded similarities and reduces to standard surprisal when words are treated as distinct. Experimental results with reading time data indicate that similarity-adjusted surprisal adds predictive power beyond standard surprisal for certain datasets, suggesting it serves as a complementary measure of comprehension effort., Comment: EMNLP 2024 main conference proceedings

Quantum repeaters and satellite-based optical links are complementary technological approaches to overcome the exponential photon loss in optical fibers and thus allow quantum communication on a global scale. We analyze architectures which combine these approaches and use satellites as quantum repeater nodes to distribute entanglement to distant optical ground stations. Here we simulate dynamic, three-dimensional ground station passes, going beyond previous studies that typically consider static satellite links. For this, we numerically solve the equations of motion of the dynamic system consisting of three satellites in low Earth orbit. The model of the optical link takes into account atmospheric attenuation, single-mode fiber coupling, beam wandering and broadening, as well as adaptive optics effects. We derive analytical expressions for the Bell state measurement and associated error rates for quantum memory assisted communications, including retrieval efficiency and state coherence. We consider downlink and uplink architectures for continental and intercontinental connections and evaluate the impact of satellite altitude and inter-satellite distance on the expected entanglement swapping rate. Our simulation model enables us to design different orbital configurations for the satellite constellation and analyze the annual performance of the quantum repeater under realistic conditions., Comment: 41 pages, 12 figures

Different of biases are reproduced in LLM-generated responses, including dialectal biases. A study based on prompt engineering was carried out to uncover how LLMs discriminate varieties of Brazilian Portuguese, specifically if sociolinguistic rules are taken into account in four LLMs: GPT 3.5, GPT-4o, Gemini, and Sabi.-2. The results offer sociolinguistic contributions for an equity fluent NLP technology., Comment: 8 pages, XI Jornada de Descri\c{c}\~ao do Portugu\^es

We consider a general class of statistical experiments, in which an $n$-dimensional centered Gaussian random variable is observed and its covariance matrix is the parameter of interest. The covariance matrix is assumed to be well-approximable in a linear space of lower dimension $K_n$ with eigenvalues uniformly bounded away from zero and infinity. We prove asymptotic equivalence of this experiment and a class of $K_n$-dimensional Gaussian models with informative expectation in Le Cam's sense when $n$ tends to infinity and $K_n$ is allowed to increase moderately in $n$ at a polynomial rate. For this purpose we derive a new localization technique for non-i.i.d. data and a novel high-dimensional Central Limit Law in total variation distance. These results are key ingredients to show asymptotic equivalence between the experiments of locally stationary Gaussian time series and a bivariate Wiener process with the log spectral density as its drift. Therein a novel class of matrices is introduced which generalizes circulant Toeplitz matrices traditionally used for strictly stationary time series.

We explore the important task of applying a phase $exp(i f(x))$ to a computational basis state $\left| x \right>$. The closely related task of rotating a target qubit by an angle depending on $f(x)$ is also studied. Such operations are key in many quantum subroutines, and often the function $f$ can be well-approximated by a piecewise linear composition. Examples range from the application of diagonal Hamiltonian terms (such as the Coulomb interaction) in grid-based many-body simulation, to derivative pricing algorithms. Here we exploit a parallelisation of the piecewise approach so that all constituent elementary rotations are performed simultaneously, that is, we achieve a total rotation depth of one. Moreover, we explore the use of recursive catalyst 'towers' to implement these elementary rotations efficiently. Depending on the choice of implementation strategy, we find a depth as low as $O(log n + log S)$ for a register of $n$ qubits and a piecewise approximation of $S$ sections. In the limit of multiple repetitions of the oracle, we find that catalyst tower approaches have an $O(S \cdot n)$ T-count, whereas linear interpolation with QROM has an $O(n^{log_2(3)})$ T-count., Comment: 14 pages, references added, table I updated, code for table I added

This article is about the neural conundrum behind the slowness of human behavior. The information throughput of a human being is about 10 bits/s. In comparison, our sensory systems gather data at ~10^9 bits/s. The stark contrast between these numbers remains unexplained and touches on fundamental aspects of brain function: What neural substrate sets this speed limit on the pace of our existence? Why does the brain need billions of neurons to process 10 bits/s? Why can we only think about one thing at a time? The brain seems to operate in two distinct modes: the "outer" brain handles fast high-dimensional sensory and motor signals, whereas the "inner" brain processes the reduced few bits needed to control behavior. Plausible explanations exist for the large neuron numbers in the outer brain, but not for the inner brain, and we propose new research directions to remedy this., Comment: 23 pages, 1 table. Accepted at NEURON

Expected values weighted by the inverse of a multivariate density or, equivalently, Lebesgue integrals of regression functions with multivariate regressors occur in various areas of applications, including estimating average treatment effects, nonparametric estimators in random coefficient regression models or deconvolution estimators in Berkson errors-in-variables models. The frequently used nearest-neighbor and matching estimators suffer from bias problems in multiple dimensions. By using polynomial least squares fits on each cell of the $K^{\text{th}}$-order Voronoi tessellation for sufficiently large $K$, we develop novel modifications of nearest-neighbor and matching estimators which again converge at the parametric $\sqrt n $-rate under mild smoothness assumptions on the unknown regression function and without any smoothness conditions on the unknown density of the covariates. We stress that in contrast to competing methods for correcting for the bias of matching estimators, our estimators do not involve nonparametric function estimators and in particular do not rely on sample-size dependent smoothing parameters. We complement the upper bounds with appropriate lower bounds derived from information-theoretic arguments, which show that some smoothness of the regression function is indeed required to achieve the parametric rate. Simulations illustrate the practical feasibility of the proposed methods.

Language models (LMs) estimate a probability distribution over strings in a natural language; these distributions are crucial for computing perplexity and surprisal in linguistics research. While we are usually concerned with measuring these values for words, most LMs operate over subwords. Despite seemingly straightforward, accurately computing probabilities over one unit given probabilities over the other requires care. Indeed, we show here that many recent linguistic studies have been incorrectly computing these values. This paper derives the correct methods for computing word probabilities, highlighting issues when relying on language models that use beginning-of-word (bow)-marking tokenisers, e.g., the GPT family. Empirically, we show that correcting the widespread bug in probability computations affects measured outcomes in sentence comprehension and lexical optimisation analyses., Comment: Camera ready version for EMNLP 2024. Our code is available in https://github.com/tpimentelms/probability-of-a-word

The gas fee, paid for inclusion in the blockchain, is analyzed in two parts. First, we consider how effort in terms of resources required to process and store a transaction turns into a gas limit, which, through a fee, comprised of the base and priority fee in the current version of Ethereum, is converted into the cost paid by the user. We adhere closely to the Ethereum protocol to simplify the analysis and to constrain the design choices when considering multidimensional gas. Second, we assume that the gas price is given deus ex machina by a fractional Ornstein-Uhlenbeck process and evaluate various derivatives. These contracts can, for example, mitigate gas cost volatility. The ability to price and trade forwards besides the existing spot inclusion into the blockchain could enable users to hedge against future cost fluctuations. Overall, this paper offers a comprehensive analysis of gas fee dynamics on the Ethereum blockchain, integrating supply-side constraints with demand-side modelling to enhance the predictability and stability of transaction costs., Comment: 30 pages, 10 figures; very similar to published version in 'Frontiers in Blockchain, Section Blockchain Economics', Volume 7 - 2024

Understanding memorisation in language models has practical and societal implications, e.g., studying models' training dynamics or preventing copyright infringements. Prior work defines memorisation as the causal effect of training with an instance on the model's ability to predict that instance. This definition relies on a counterfactual: the ability to observe what would have happened had the model not seen that instance. Existing methods struggle to provide computationally efficient and accurate estimates of this counterfactual. Further, they often estimate memorisation for a model architecture rather than for a specific model instance. This paper fills an important gap in the literature, proposing a new, principled, and efficient method to estimate memorisation based on the difference-in-differences design from econometrics. Using this method, we characterise a model's memorisation profile--its memorisation trends across training--by only observing its behaviour on a small set of instances throughout training. In experiments with the Pythia model suite, we find that memorisation (i) is stronger and more persistent in larger models, (ii) is determined by data order and learning rate, and (iii) has stable trends across model sizes, thus making memorisation in larger models predictable from smaller ones., Comment: Published at the ACL 2024 Conference (main)

Since 2019, GRAVITY has provided direct observations of giant planets and brown dwarfs at separations of down to 95 mas from the host star. Some of these observations have provided the first direct confirmation of companions previously detected by indirect techniques (astrometry and radial velocities). We want to improve the observing strategy and data reduction in order to lower the inner working angle of GRAVITY in dual-field on-axis mode. We also want to determine the current limitations of the instrument when observing faint companions with separations in the 30-150 mas range. To improve the inner working angle, we propose a fiber off-pointing strategy during the observations to maximize the ratio of companion-light-to-star-light coupling in the science fiber. We also tested a lower-order model for speckles to decouple the companion light from the star light. We then evaluated the detection limits of GRAVITY using planet injection and retrieval in representative archival data. We compare our results to theoretical expectations. We validate our observing and data-reduction strategy with on-sky observations; first in the context of brown dwarf follow-up on the auxiliary telescopes with HD 984 B, and second with the first confirmation of a substellar candidate around the star Gaia DR3 2728129004119806464. With synthetic companion injection, we demonstrate that the instrument can detect companions down to a contrast of $8\times 10^{-4}$ ($\Delta \mathrm{K}= 7.7$ mag) at a separation of 35 mas, and a contrast of $3\times 10^{-5}$ ($\Delta \mathrm{K}= 11$ mag) at 100 mas from a bright primary (K<6.5), for 30 min exposure time. With its inner working angle and astrometric precision, GRAVITY has a unique reach in direct observation parameter space. This study demonstrates the promising synergies between GRAVITY and Gaia for the confirmation and characterization of substellar companions., Comment: 16 pages, 14 figures. Submitted to A&A

Decoders that provide an estimate of the probability of a logical failure conditioned on the error syndrome ("soft-output decoders") can reduce the overhead cost of fault-tolerant quantum memory and computation. In this work, we construct efficient soft-output decoders for the surface code derived from the Minimum-Weight Perfect Matching and Union-Find decoders. We show that soft-output decoding can improve the performance of a "hierarchical code," a concatenated scheme in which the inner code is the surface code, and the outer code is a high-rate quantum low-density parity-check code. Alternatively, the soft-output decoding can improve the reliability of fault-tolerant circuit sampling by flagging those runs that should be discarded because the probability of a logical error is intolerably large., Comment: Update lit. review

Event-triggered control is often argued to lower the average triggering rate compared to time-triggered control while still achieving a desired control goal, e.g., the same performance level. However, this property, often called consistency, cannot be taken for granted and can be hard to analyze in many settings. In particular, although numerous decentralized event-triggered control schemes have been proposed in the past years, their performance properties with respect to time-triggered control remain mostly unexplored. In this paper, we therefore examine the performance properties of event-triggered control (relative to time-triggered control) for a single-integrator consensus problem with a level-triggering rule. We consider the long-term average quadratic deviation from consensus as a performance measure. For this setting, we show that enriching the information the local controllers use improves the performance of the consensus algorithm but renders a previously consistent event-triggered control scheme inconsistent. In addition, we do so while deploying optimal control inputs which we derive for both information cases and all triggering schemes. With this insight, we can furthermore explain the relationship between two contrasting consistency results from the literature on decentralized event-triggered control. We support our theoretical findings with simulation results.

Linguistic ambiguity continues to represent a significant challenge for natural language processing (NLP) systems, notwithstanding the advancements in architectures such as Transformers and BERT. Inspired by the recent success of instructional models like ChatGPT and Gemini (In 2023, the artificial intelligence was called Bard.), this study aims to analyze and discuss linguistic ambiguity within these models, focusing on three types prevalent in Brazilian Portuguese: semantic, syntactic, and lexical ambiguity. We create a corpus comprising 120 sentences, both ambiguous and unambiguous, for classification, explanation, and disambiguation. The models capability to generate ambiguous sentences was also explored by soliciting sets of sentences for each type of ambiguity. The results underwent qualitative analysis, drawing on recognized linguistic references, and quantitative assessment based on the accuracy of the responses obtained. It was evidenced that even the most sophisticated models, such as ChatGPT and Gemini, exhibit errors and deficiencies in their responses, with explanations often providing inconsistent. Furthermore, the accuracy peaked at 49.58 percent, indicating the need for descriptive studies for supervised learning., Comment: in Portuguese language, 16 p\'aginas, 5 p\'aginas de ap\^endice e 4 imagens

Quantum key distribution (QKD) allows the generation of cryptographic keys beyond the computational hardness paradigm and is befitting for secure data transmission requiring long-term security. The communication distance of fibre-based QKD, however, is limited to a few hundred kilometers due to the exponential scaling of signal attenuation. Satellite QKD (SatQKD) can instead leverage free-space optical links to establish long-range connections and enable global-scale QKD. In this work we review the manifold of design choices that concur to form the set of possible SatQKD architectures. These include the choice of the QKD protocol and its physical implementation, but also the satellite orbit, the optical link direction, and whether or not to use trusted-node relays. The possible SatQKD architectures are then evaluated in terms of key generation throughput, latency and maximum reachable communication distance, but also the system-level security and implementation complexity. Given the technical challenges of realising SatQKD systems it is paramount, for near-future satellite missions, to adhere to the simplest possible architecture that still allows to deliver the QKD service. We thus identify as advisable options the use of low-Earth orbit satellites as trusted nodes for prepare-and-measure discrete-variable QKD downlinks with weak laser pulses. The decoy-state version of BB84 is found to be the most promising QKD protocols due to the maturity of the security proofs, the high key generation rate and low system complexity. These findings are confirmed by the multitude of current and planned SatQKD missions that are adopting these architectural choices., Comment: 37 pages, 7 figures

For nearly three decades, language models derived from the $n$-gram assumption held the state of the art on the task. The key to their success lay in the application of various smoothing techniques that served to combat overfitting. However, when neural language models toppled $n$-gram models as the best performers, $n$-gram smoothing techniques became less relevant. Indeed, it would hardly be an understatement to suggest that the line of inquiry into $n$-gram smoothing techniques became dormant. This paper re-opens the role classical $n$-gram smoothing techniques may play in the age of neural language models. First, we draw a formal equivalence between label smoothing, a popular regularization technique for neural language models, and add-$\lambda$ smoothing. Second, we derive a generalized framework for converting any $n$-gram smoothing technique into a regularizer compatible with neural language models. Our empirical results find that our novel regularizers are comparable to and, indeed, sometimes outperform label smoothing on language modeling and machine translation., Comment: NAACL 2024

In this paper, we propose a collision avoidance safety filter for autonomous electric scooters to enable safe operation of such vehicles in pedestrian areas. In particular, we employ multiple low-cost ultrasonic sensors to detect a wide range of possible obstacles in front of the e-scooter. Based on possibly faulty distance measurements, we design a filter to mitigate measurement noise and missing values as well as a gain-scheduled controller to limit the velocity commanded to the e-scooter when required due to imminent collisions. The proposed controller structure is able to prevent collisions with unknown obstacles by deploying a reduced safe velocity ensuring a sufficiently large safety distance. The collision avoidance approach is designed such that it may be easily deployed in similar applications of general micromobility vehicles. The effectiveness of our proposed safety filter is demonstrated in real-world experiments., Comment: Accepted for presentation at the 17th IFAC Symposium on Control in Transportation Systems (CTS 2024)

Quantum error correction is believed to be essential for scalable quantum computation, but its implementation is challenging due to its considerable space-time overhead. Motivated by recent experiments demonstrating efficient manipulation of logical qubits using transversal gates (Bluvstein et al., Nature 626, 58-65 (2024)), we show that the performance of logical algorithms can be substantially improved by decoding the qubits jointly to account for physical error propagation during transversal entangling gates. We find that such correlated decoding improves the performance of both Clifford and non-Clifford transversal entangling gates, and explore two decoders offering different computational runtimes and accuracies. By considering deep logical Clifford circuits, we find that correlated decoding can significantly improve the space-time cost by reducing the number of rounds of noisy syndrome extraction per gate. These results demonstrate that correlated decoding provides a major advantage in early fault-tolerant computation, and indicate it has considerable potential to reduce the space-time cost in large-scale logical algorithms., Comment: 7+12 pages, 5+3 figures

Ultracold atomic gases hold unique promise for space science by capitalizing on quantum advantages and extended freefall, afforded in a microgravity environment, to enable next-generation precision sensors. Atom interferometers are a class of quantum sensors which can use freely falling gases of atoms cooled to sub-photon-recoil temperatures to provide unprecedented sensitivities to accelerations, rotations, and gravitational forces, and are currently being developed for space-based applications in gravitational, earth, and planetary sciences, as well as to search for subtle forces that could signify physics beyond General Relativity and the Standard Model. NASA's Cold Atom Lab (CAL) operates onboard the International Space Station as a multi-user facility for studies of ultracold atoms and to mature quantum technologies, including atom interferometry, in persistent microgravity. In this paper, we report on path-finding experiments utilizing ultracold $^{87}$Rb atoms in the CAL atom interferometer, which was enabled by an on-orbit upgrade of the CAL science module: A three-pulse Mach-Zehnder interferometer was studied to understand limitations from the influence of ISS vibrations. Additionally, Ramsey shear-wave interferometry was used to manifest interference patterns in a single run that were observable for over 150 ms free-expansion time. Finally, the CAL atom interferometer was used to remotely measure the photon recoil from the atom interferometer laser as a demonstration of the first quantum sensor using matter-wave interferometry in space., Comment: 14 pages, 5 figures

Learning time-series models is useful for many applications, such as simulation and forecasting. In this study, we consider the problem of actively learning time-series models while taking given safety constraints into account. For time-series modeling we employ a Gaussian process with a nonlinear exogenous input structure. The proposed approach generates data appropriate for time series model learning, i.e. input and output trajectories, by dynamically exploring the input space. The approach parametrizes the input trajectory as consecutive trajectory sections, which are determined stepwise given safety requirements and past observations. We analyze the proposed algorithm and evaluate it empirically on a technical application. The results show the effectiveness of our approach in a realistic technical use case., Comment: Clarification / Errata of article originally published at NeurIPS: https://proceedings.neurips.cc/paper/2018/hash/b197ffdef2ddc3308584dce7afa3661b-Abstract.html

With the proliferation of increasingly complicated Deep Learning architectures, data synthesis is a highly promising technique to address the demand of data-hungry models. However, reliably assessing the quality of a 'synthesiser' model's output is an open research question with significant associated risks for high-stake domains. To address this challenge, we propose a unique synthesis algorithm that generates data from high-confidence feature space regions based on the Conformal Prediction framework. We support our proposed algorithm with a comprehensive exploration of the core parameter's influence, an in-depth discussion of practical advice, and an extensive empirical evaluation of five benchmark datasets. To show our approach's versatility on ubiquitous real-world challenges, the datasets were carefully selected for their variety of difficult characteristics: low sample count, class imbalance, and non-separability. In all trials, training sets extended with our confident synthesised data performed at least as well as the original set and frequently significantly improved Deep Learning performance by up to 61 percentage points F1-score., Comment: Accepted for publication in the Machine Learning journal special issue "Conformal Prediction and Distribution-Free Uncertainty Quantification"

Zipf (1935) posited that wordforms are optimized to minimize utterances' communicative costs. Under the assumption that cost is given by an utterance's length, he supported this claim by showing that words' lengths are inversely correlated with their frequencies. Communicative cost, however, can be operationalized in different ways. Piantadosi et al. (2011) claim that cost should be measured as the distance between an utterance's information rate and channel capacity, which we dub the channel capacity hypothesis (CCH) here. Following this logic, they then proposed that a word's length should be proportional to the expected value of its surprisal (negative log-probability in context). In this work, we show that Piantadosi et al.'s derivation does not minimize CCH's cost, but rather a lower bound, which we term CCH-lower. We propose a novel derivation, suggesting an improved way to minimize CCH's cost. Under this method, we find that a language's word lengths should instead be proportional to the surprisal's expectation plus its variance-to-mean ratio. Experimentally, we compare these three communicative cost functions: Zipf's, CCH-lower , and CCH. Across 13 languages and several experimental settings, we find that length is better predicted by frequency than either of the other hypotheses. In fact, when surprisal's expectation, or expectation plus variance-to-mean ratio, is estimated using better language models, it leads to worse word length predictions. We take these results as evidence that Zipf's longstanding hypothesis holds., Comment: Published at EMNLP 2023

As businesses, products, and services spring up around large language models, the trustworthiness of these models hinges on the verifiability of their outputs. However, methods for explaining language model outputs largely fall across two distinct fields of study which both use the term "attribution" to refer to entirely separate techniques: citation generation and training data attribution. In many modern applications, such as legal document generation and medical question answering, both types of attributions are important. In this work, we argue for and present a unified framework of large language model attributions. We show how existing methods of different types of attribution fall under the unified framework. We also use the framework to discuss real-world use cases where one or both types of attributions are required. We believe that this unified framework will guide the use case driven development of systems that leverage both types of attribution, as well as the standardization of their evaluation., Comment: NeurIPS ATTRIB Workshop 2023

An adjoint-based procedure to determine weaknesses, or, more generally, the material properties of structures is developed and tested. Given a series of load cases and corresponding displacement/strain measurements, the material properties are obtained by minimizing the weighted differences between the measured and computed values. In a subsequent step, techniques to minimize the number of load cases and sensors are proposed and tested. Several examples show the viability, accuracy and efficiency of the proposed methodology and its potential use for high fidelity digital twins., Comment: arXiv admin note: substantial text overlap with arXiv:2303.15329

Large language models have become one of the most commonly deployed NLP inventions. In the past half-decade, their integration into core natural language processing tools has dramatically increased the performance of such tools, and they have entered the public discourse surrounding artificial intelligence. Consequently, it is important for both developers and researchers alike to understand the mathematical foundations of large language models, as well as how to implement them. These notes are the accompaniment to the theoretical portion of the ETH Z\"urich course on large language models, covering what constitutes a language model from a formal, theoretical perspective.