Search

We propose a mathematical walk around the gcd of the values $A(n)$ and $B(n)$ of two polynomials evaluated at an integer $n$. This is an opportunity to use a very powerful tool: the resultant., Comment: The final version will appear in the American Mathematical Monthly

Data analytics is essential for extracting valuable insights from data that can assist organizations in making effective decisions. We introduce InsightBench, a benchmark dataset with three key features. First, it consists of 31 datasets representing diverse business use cases such as finance and incident management, each accompanied by a carefully curated set of insights planted in the datasets. Second, unlike existing benchmarks focusing on answering single queries, InsightBench evaluates agents based on their ability to perform end-to-end data analytics, including formulating questions, interpreting answers, and generating a summary of insights and actionable steps. Third, we conducted comprehensive quality assurance to ensure that each dataset in the benchmark had clear goals and included relevant and meaningful questions and analysis. Furthermore, we implement a two-way evaluation mechanism using LLaMA-3-Eval as an effective, open-source evaluator method to assess agents' ability to extract insights. We also propose AgentPoirot, our baseline data analysis agent capable of performing end-to-end data analytics. Our evaluation on InsightBench shows that AgentPoirot outperforms existing approaches (such as Pandas Agent) that focus on resolving single queries. We also compare the performance of open- and closed-source LLMs and various evaluation strategies. Overall, this benchmark serves as a testbed to motivate further development in comprehensive data analytics and can be accessed here: https://github.com/ServiceNow/insight-bench.

The ability of large language models (LLMs) to mimic human-like intelligence has led to a surge in LLM-based autonomous agents. Though recent LLMs seem capable of planning and reasoning given user instructions, their effectiveness in applying these capabilities for autonomous task solving remains underexplored. This is especially true in enterprise settings, where automated agents hold the promise of a high impact. To fill this gap, we propose WorkArena++, a novel benchmark consisting of 682 tasks corresponding to realistic workflows routinely performed by knowledge workers. WorkArena++ is designed to evaluate the planning, problem-solving, logical/arithmetic reasoning, retrieval, and contextual understanding abilities of web agents. Our empirical studies across state-of-the-art LLMs and vision-language models (VLMs), as well as human workers, reveal several challenges for such models to serve as useful assistants in the workplace. In addition to the benchmark, we provide a mechanism to effortlessly generate thousands of ground-truth observation/action traces, which can be used for fine-tuning existing models. Overall, we expect this work to serve as a useful resource to help the community progress toward capable autonomous agents. The benchmark can be found at https://github.com/ServiceNow/WorkArena/tree/workarena-plus-plus.

Heterogeneous systems with analog CMOS circuits integrated with nanoscale memristive devices enable efficient deployment of neural networks on neuromorphic hardware. CMOS Neuron with low footprint can emulate slow temporal dynamics by operating with extremely low current levels. Nevertheless, the current read from the memristive synapses can be higher by several orders of magnitude, and performing impedance matching between neurons and synapses is mandatory. In this paper, we implement an analog leaky integrate and fire (LIF) neuron with a voltage regulator and current attenuator for interfacing CMOS neurons with memristive synapses. In addition, the neuron design proposes a dual leakage that could enable the implementation of local learning rules such as voltage-dependent synaptic plasticity. We also propose a connection scheme to implement adaptive LIF neurons based on two-neuron interaction. The proposed circuits can be used to interface with a variety of synaptic devices and process signals of diverse temporal dynamics., Comment: Accepted to International Conference on Neuromorphic Systems (ICONS 2024)

This study presents a machine-learning-based procedure to automate the charge tuning of semiconductor spin qubits with minimal human intervention, addressing one of the significant challenges in scaling up quantum dot technologies. This method exploits artificial neural networks to identify noisy transition lines in stability diagrams, guiding a robust exploration strategy leveraging neural networks' uncertainty estimations. Tested across three distinct offline experimental datasets representing different single quantum dot technologies, the approach achieves over 99% tuning success rate in optimal cases, where more than 10% of the success is directly attributable to uncertainty exploitation. The challenging constraints of small training sets containing high diagram-to-diagram variability allowed us to evaluate the capabilities and limits of the proposed procedure., Comment: 12 pages (main) + 13 pages (supplementary)

The tetragonal heavy-fermion superconductor CeRh$_2$As$_2$ ($T_{\rm c}=0.3$ K) exhibits an exceptionally high critical field of 14 T for $\textbf{B} \parallel \textbf{c}$. It undergoes a field-driven first-order phase transition between superconducting (SC) states, potentially transitioning from spin-singlet to spin-triplet superconductivity. To elucidate the underlying pairing mechanism, we probe spin fluctuations in CeRh$_2$As$_2$ using neutron scattering. We find dynamic $(\pi,\pi)$ antiferromagnetic spin correlations with an anisotropic quasi-two-dimensional correlation volume. Our data place an upper limit of 0.31 $\mu_{\rm B}$ on the staggered magnetization of corresponding N\'{e}el orders at $T=0.08$ K. Density functional theory (DFT) calculations, treating Ce $4f$ electrons as core states, show that the AFM wave vector connects significant areas of the Fermi surface. Our findings show the dominant excitations in CeRh$_2$As$_2$ for $\hbar\omega< 1.2$~meV are magnetic and indicate superconductivity in CeRh$_2$As$_2$ is mediated by AFM spin fluctuations associated with a proximate quantum critical point., Comment: 7+6 pages, 4+4 figures

Cryogenic memristor-based DC sources offer a promising avenue for in situ biasing of quantum dot arrays. In this study, we present experimental results and discuss the scaling potential for such DC sources. We first demonstrate the operation of a commercial discrete operational amplifier down to 1.2K which is used on the DC source prototype. Then, the tunability of the memristor-based DC source is validated by performing several 250mV-DC sweeps with a resolution of 10mV at room temperature and at 1.2K. Additionally, the DC source prototype exhibits a limited output drift of $\approx1\mathrm{\mu Vs^{-1}}$ at 1.2K. This showcases the potential of memristor-based DC sources for quantum dot biasing. Limitations in power consumption and voltage resolution using discrete components highlight the need for a fully integrated and scalable complementary metal-oxide-semiconductor-based (CMOS-based) approach. To address this, we propose to monolithically co-integrate emerging non-volatile memories (eNVMs) and 65nm CMOS circuitry. Simulations reveal a reduction in power consumption, down to $\mathrm{10\mu W}$ per DC source and in footprint. This allows for the integration of up to one million eNVM-based DC sources at the 4.2K stage of a dilution fridge, paving the way for near term large-scale quantum computing applications.

Numerous decision-making tasks require estimating causal effects under interventions on different parts of a system. As practitioners consider using large language models (LLMs) to automate decisions, studying their causal reasoning capabilities becomes crucial. A recent line of work evaluates LLMs ability to retrieve commonsense causal facts, but these evaluations do not sufficiently assess how LLMs reason about interventions. Motivated by the role that interventions play in causal inference, in this paper, we conduct empirical analyses to evaluate whether LLMs can accurately update their knowledge of a data-generating process in response to an intervention. We create benchmarks that span diverse causal graphs (e.g., confounding, mediation) and variable types, and enable a study of intervention-based reasoning. These benchmarks allow us to isolate the ability of LLMs to accurately predict changes resulting from their ability to memorize facts or find other shortcuts. Our analysis on four LLMs highlights that while GPT- 4 models show promising accuracy at predicting the intervention effects, they remain sensitive to distracting factors in the prompts., Comment: 17 pages

We study the use of large language model-based agents for interacting with software via web browsers. Unlike prior work, we focus on measuring the agents' ability to perform tasks that span the typical daily work of knowledge workers utilizing enterprise software systems. To this end, we propose WorkArena, a remote-hosted benchmark of 33 tasks based on the widely-used ServiceNow platform. We also introduce BrowserGym, an environment for the design and evaluation of such agents, offering a rich set of actions as well as multimodal observations. Our empirical evaluation reveals that while current agents show promise on WorkArena, there remains a considerable gap towards achieving full task automation. Notably, our analysis uncovers a significant performance disparity between open and closed-source LLMs, highlighting a critical area for future exploration and development in the field., Comment: 21 pages, 11 figures, preprint

We present broadband dual frequency comb laser absorption measurements of 2% H$_2$O (natural isotopic abundance of 99.7% H$_2^{16}$O) in air from 6600-7650 cm$^{-1}$ (1307-1515 nm) with a spectral point spacing of 0.0068 cm$^{-1}$. Twenty-nine datasets were collected at temperatures between 300 and 1300 K ($\pm$0.82% average uncertainty) and pressures ranging from 20 to 600 Torr ($\pm$0.25%) with an average residual absorbance noise of 8.0E-4 across the spectrum for all measurements. We fit measurements using a quadratic speed-dependent Voigt profile to determine 7088 absorption parameters for 3366 individual transitions found in HITRAN2020. These measurements build on the line strength, line center, self-broadening, and self-shift parameters determined in the Part I companion of this work. Here we measure air-broadened width (with temperature- and speed-dependence) and air pressure shift (with temperature dependence) parameters. Various trends are explored for extrapolation to weak transitions that were not covered in this work. Improvements made in this work are predominantly due to the inclusion of air pressure shift temperature dependence values. In aggregate, these updates improved RMS absorbance error by a factor of 4.2 on average, and the remaining residual is predominantly spectral noise. This updated database improves high temperature spectroscopic knowledge across the 6600 7650 cm$^{-1}$ region of H$_2$O absorption., Comment: Database files available upon request. Will be included with published manuscript following review process

The extraction of a small number of relevant insights from vast amounts of data is a crucial component of data-driven decision-making. However, accomplishing this task requires considerable technical skills, domain expertise, and human labor. This study explores the potential of using Large Language Models (LLMs) to automate the discovery of insights in data, leveraging recent advances in reasoning and code generation techniques. We propose a new evaluation methodology based on a "capture the flag" principle, measuring the ability of such models to recognize meaningful and pertinent information (flags) in a dataset. We further propose two proof-of-concept agents, with different inner workings, and compare their ability to capture such flags in a real-world sales dataset. While the work reported here is preliminary, our results are sufficiently interesting to mandate future exploration by the community., Comment: 14 pages, 1 figure, Foundation Models for Decision Making Workshop at NeurIPS 2023

Over the past years, foundation models have caused a paradigm shift in machine learning due to their unprecedented capabilities for zero-shot and few-shot generalization. However, despite the success of foundation models in modalities such as natural language processing and computer vision, the development of foundation models for time series forecasting has lagged behind. We present Lag-Llama, a general-purpose foundation model for univariate probabilistic time series forecasting based on a decoder-only transformer architecture that uses lags as covariates. Lag-Llama is pretrained on a large corpus of diverse time series data from several domains, and demonstrates strong zero-shot generalization capabilities compared to a wide range of forecasting models on downstream datasets across domains. Moreover, when fine-tuned on relatively small fractions of such previously unseen datasets, Lag-Llama achieves state-of-the-art performance, outperforming prior deep learning approaches, emerging as the best general-purpose model on average. Lag-Llama serves as a strong contender to the current state-of-art in time series forecasting and paves the way for future advancements in foundation models tailored to time series data., Comment: First two authors contributed equally. All data, models and code used are open-source. GitHub: https://github.com/time-series-foundation-models/lag-llama

The presence of competing interactions due to geometry leads to frustration in quantum spin models. As a consequence, the ground state of such systems often displays a large degeneracy that can be lifted due to thermal or quantum effects. One such example is the antiferromagnetic Ising model on the Kagome lattice. It was shown that while the same model on the triangular lattice is ordered at zero temperature for small transverse field due to an order by disorder mechanism, the Kagome lattice resists any such effects and exhibits only short range spin correlations and a trivial paramagnetic phase. We embed this model on the latest architecture of D-Wave's quantum annealer, the Advantage2 prototype, which uses the highly connected Zephyr graph. Using advanced embedding and calibration techniques, we are able to embed a Kagome lattice with mixed open and periodic boundary conditions of 231 sites on the full graph of the currently available prototype. Through forward annealing experiments, we show that under a finite longitudinal field the system exhibits a one-third magnetization plateau, consistent with a classical spin liquid state of reduced entropy. An anneal-pause-quench protocol is then used to extract an experimental ensemble of states resulting from the equilibration of the model at finite transverse and longitudinal field. This allows us to construct a partial phase diagram and confirm that the system exits the constrained Hilbert space of the classical spin liquid when subjected to a transverse field. We connect our results to previous theoretical results and quantum Monte Carlo simulation, which helps us confirm the validity of the quantum simulation realized here, thereby extracting insight into the performance of the D-Wave quantum annealer to simulate non-trivial quantum systems in equilibrium., Comment: 12 + 6 pages, 7 + 8 figures, updated paper to be published in PRB

We introduce a new model for multivariate probabilistic time series prediction, designed to flexibly address a range of tasks including forecasting, interpolation, and their combinations. Building on copula theory, we propose a simplified objective for the recently-introduced transformer-based attentional copulas (TACTiS), wherein the number of distributional parameters now scales linearly with the number of variables instead of factorially. The new objective requires the introduction of a training curriculum, which goes hand-in-hand with necessary changes to the original architecture. We show that the resulting model has significantly better training dynamics and achieves state-of-the-art performance across diverse real-world forecasting tasks, while maintaining the flexibility of prior work, such as seamless handling of unaligned and unevenly-sampled time series. Code is made available at https://github.com/ServiceNow/TACTiS., Comment: 28 pages, 15 figures, The Twelfth International Conference on Learning Representations (ICLR 2024)

The exploration of memristors' behavior at cryogenic temperatures has become crucial due to the growing interest in quantum computing and cryogenic electronics. In this context, our study focuses on the characterization at cryogenic temperatures (4.2 K) of TiO$_\textrm{2-x}$-based memristors fabricated with a CMOS-compatible etch-back process. We demonstrate a so-called cryogenic reforming (CR) technique performed at 4.2 K to overcome the well-known metal-insulator transition (MIT) which limits the analog behavior of memristors at low temperatures. This cryogenic reforming process was found to be reproducible and led to a durable suppression of the MIT. This process allowed to reduce by approximately 20% the voltages required to perform DC resistive switching at 4.2 K. Additionally, conduction mechanism studies of memristors before and after cryogenic reforming from 4.2 K to 300 K revealed different behaviors above 100 K, indicating a potential change in the conductive filament stoichiometry. The reformed devices exhibit a conductance level that is 50 times higher than ambient-formed memristor, and the conduction drop between 300 K and 4.2 K is 100 times smaller, indicating the effectiveness of the reforming process. More importantly, CR enables analog programming at 4.2 K with typical read voltages. Suppressing the MIT improved the analog switching dynamics of the memristor leading to approximately 250% larger on/off ratios during long-term depression (LTD)/long-term potentiation (LTP) resistance tuning. This enhancement opens up the possibility of using TiO$_{\textrm{2-x}}$-based memristors to be used as synapses in neuromorphic computing at cryogenic temperatures.

Neural decoders for quantum error correction (QEC) rely on neural networks to classify syndromes extracted from error correction codes and find appropriate recovery operators to protect logical information against errors. Despite the good performance of neural decoders, important practical requirements remain to be achieved, such as minimizing the decoding time to meet typical rates of syndrome generation in repeated error correction schemes, and ensuring the scalability of the decoding approach as the code distance increases. Designing a dedicated integrated circuit to perform the decoding task in co-integration with a quantum processor appears necessary to reach these decoding time and scalability requirements, as routing signals in and out of a cryogenic environment to be processed externally leads to unnecessary delays and an eventual wiring bottleneck. In this work, we report the design and performance analysis of a neural decoder inference accelerator based on an in-memory computing (IMC) architecture, where crossbar arrays of resistive memory devices are employed to both store the synaptic weights of the decoder neural network and perform analog matrix-vector multiplications during inference. In proof-of-concept numerical experiments supported by experimental measurements, we investigate the impact of TiO$_\textrm{x}$-based memristive devices' non-idealities on decoding accuracy. Hardware-aware training methods are developed to mitigate the loss in accuracy, allowing the memristive neural decoders to achieve a pseudo-threshold of $9.23\times 10^{-4}$ for the distance-three surface code, whereas the equivalent digital neural decoder achieves a pseudo-threshold of $1.01\times 10^{-3}$. This work provides a pathway to scalable, fast, and low-power cryogenic IMC hardware for integrated QEC.

The practical utility of causality in decision-making is widespread and brought about by the intertwining of causal discovery and causal inference. Nevertheless, a notable gap exists in the evaluation of causal discovery methods, where insufficient emphasis is placed on downstream inference. To address this gap, we evaluate seven established baseline causal discovery methods including a newly proposed method based on GFlowNets, on the downstream task of treatment effect estimation. Through the implementation of a distribution-level evaluation, we offer valuable and unique insights into the efficacy of these causal discovery methods for treatment effect estimation, considering both synthetic and real-world scenarios, as well as low-data scenarios. The results of our study demonstrate that some of the algorithms studied are able to effectively capture a wide range of useful and diverse ATE modes, while some tend to learn many low-probability modes which impacts the (unrelaxed) recall and precision., Comment: Peer-reviewed and Accepted to ICML 2023 Workshop on Structured Probabilistic Inference & Generative Modeling

Understanding the causal relationships that underlie a system is a fundamental prerequisite to accurate decision-making. In this work, we explore how expert knowledge can be used to improve the data-driven identification of causal graphs, beyond Markov equivalence classes. In doing so, we consider a setting where we can query an expert about the orientation of causal relationships between variables, but where the expert may provide erroneous information. We propose strategies for amending such expert knowledge based on consistency properties, e.g., acyclicity and conditional independencies in the equivalence class. We then report a case study, on real data, where a large language model is used as an imperfect expert., Comment: Peer reviewed and accepted for presentation at the Structured Probabilistic Inference & Generative Modeling (SPIGM) workshop at ICML 2023, Hawaii, USA

Rule-based models, such as decision trees, appeal to practitioners due to their interpretable nature. However, the learning algorithms that produce such models are often vulnerable to spurious associations and thus, they are not guaranteed to extract causally-relevant insights. In this work, we build on ideas from the invariant causal prediction literature to propose Invariant Causal Set Covering Machines, an extension of the classical Set Covering Machine algorithm for conjunctions/disjunctions of binary-valued rules that provably avoids spurious associations. We demonstrate both theoretically and empirically that our method can identify the causal parents of a variable of interest in polynomial time.

Recent progress in self-supervision has shown that pre-training large neural networks on vast amounts of unsupervised data can lead to substantial increases in generalization to downstream tasks. Such models, recently coined foundation models, have been transformational to the field of natural language processing. Variants have also been proposed for image data, but their applicability to remote sensing tasks is limited. To stimulate the development of foundation models for Earth monitoring, we propose a benchmark comprised of six classification and six segmentation tasks, which were carefully curated and adapted to be both relevant to the field and well-suited for model evaluation. We accompany this benchmark with a robust methodology for evaluating models and reporting aggregated results to enable a reliable assessment of progress. Finally, we report results for 20 baselines to gain information about the performance of existing models. We believe that this benchmark will be a driver of progress across a variety of Earth monitoring tasks., Comment: arXiv admin note: text overlap with arXiv:2112.00570

Passive resistive random access memory (ReRAM) crossbar arrays, a promising emerging technology used for analog matrix-vector multiplications, are far superior to their active (1T1R) counterparts in terms of the integration density. However, current transfers of neural network weights into the conductance state of the memory devices in the crossbar architecture are accompanied by significant losses in precision due to hardware variabilities such as sneak path currents, biasing scheme effects and conductance tuning imprecision. In this work, training approaches that adapt techniques such as dropout, the reparametrization trick and regularization to TiO2 crossbar variabilities are proposed in order to generate models that are better adapted to their hardware transfers. The viability of this approach is demonstrated by comparing the outputs and precision of the proposed hardware-aware network with those of a regular fully connected network over a few thousand weight transfers using the half moons dataset in a simulation based on experimental data. For the neural network trained using the proposed hardware-aware method, 79.5% of the test set's data points can be classified with an accuracy of 95% or higher, while only 18.5% of the test set's data points can be classified with this accuracy by the regularly trained neural network., Comment: 15 pages, 11 figures

In this work we report a study and a co-design methodology of an analog SNN crossbar output circuit designed in a 28nm FD-SOI technology node that comprises a tunable current attenuator and a leak-integrate and fire neurons that would enable the integration of emerging non-volatile memories (eNVMs) for synaptic arrays based on various technologies including phase change (PCRAM), oxide-based (OxRAM), spin transfer and spin orbit torque magnetic memories (STT, SOT-MRAM). Circuit SPICE simulation results and eNVM experimental data are used to showcase and estimate the neurons fan-in for each type of eNVM considering the technology constraints and design trade-offs that set its limits such as membrane capacitance and supply voltage, etc., Comment: 7 pages, 6 figures

Methodical aerial population surveys monitoring critically endangered species in Canadian North Atlantic waters are instrumental in influencing government policies both in economic and conservational efforts. The primary factor hindering the success of these missions is poor visibility caused by glare. This paper builds off our foundational paper [1] and pushes the envelope toward a data-driven glare modelling system. Said data-driven system makes use of meteorological and astronomical data to assist aircraft in navigating in order to mitigate acquisition errors and optimize the quality of acquired data. It is found that reliably extracting aircraft orientation is critical to our approach, to that end, we present a GPS alignment methodology which makes use of the fusion of two GPS signals. Using the complementary strengths and weaknesses of these two signals a synthetic interpolation of fused data is used to generate more reliable flight tracks, substantially improving glare modelling. This methodology could be applied to any other applications with similar signal restrictions., Comment: 12 pages, 11 figures, IEEE/ION Position Location and Navigation Symposium (PLANS 2023)

Multivariate probabilistic time series forecasts are commonly evaluated via proper scoring rules, i.e., functions that are minimal in expectation for the ground-truth distribution. However, this property is not sufficient to guarantee good discrimination in the non-asymptotic regime. In this paper, we provide the first systematic finite-sample study of proper scoring rules for time-series forecasting evaluation. Through a power analysis, we identify the "region of reliability" of a scoring rule, i.e., the set of practical conditions where it can be relied on to identify forecasting errors. We carry out our analysis on a comprehensive synthetic benchmark, specifically designed to test several key discrepancies between ground-truth and forecast distributions, and we gauge the generalizability of our findings to real-world tasks with an application to an electricity production problem. Our results reveal critical shortcomings in the evaluation of multivariate probabilistic forecasts as commonly performed in the literature., Comment: 47 pages, 37 figures, camera-ready version, Fortieth International Conference on Machine Learning (ICML 2023)

Critically endangered species in Canadian North Atlantic waters are systematically surveyed to estimate species populations which influence governing policies. Due to its impact on policy, population accuracy is important. This paper lays the foundation towards a data-driven glare modelling system, which will allow surveyors to preemptively minimize glare. Surveyors use a detection function to estimate megafauna populations which are not explicitly seen. A goal of the research is to maximize useful imagery collected, to that end we will use our glare model to predict glare and optimize for glare-free data collection. To build this model, we leverage a small labelled dataset to perform semi-supervised learning. The large dataset is labelled with a Cascading Random Forest Model using a na\"ive pseudo-labelling approach. A reflectance model is used, which pinpoints features of interest, to populate our datasets which allows for context-aware machine learning models. The pseudo-labelled dataset is used on two models: a Multilayer Perceptron and a Recurrent Neural Network. With this paper, we lay the foundation for data-driven mission planning; a glare modelling system which allows surveyors to preemptively minimize glare and reduces survey reliance on the detection function as an estimator of whale populations during periods of poor subsurface visibility., Comment: 15 pages, 4 figures, 5th ICPR Workshop on Computer Vison for Automated Analysis of Underwater Imagery (CVAUI 2022)