Search

The design and performance of wave union TDC implemented in a Lattice CertusPro-NX FPGA is discussed. This FPGA is available for radiation tolerant applications. The TDC is implemented with 16-channels and a 200 MHz reference clock. Each channel is able to record at an event rate of > 1 MHz. The performance of the TDC is assessed over voltage and temperature. Typical TDC performance has a resolution of 10.9 ps. Typical INL is +/-3 LSB peak-to-peak. Typical DNL is (+1.13,-0.77) LSB. Typical differential performance between two channels is 20 ps (1-sigma).

Robotic wire arc additive manufacturing has been widely adopted due to its high deposition rates and large print volume relative to other metal additive manufacturing processes. For complex geometries, printing with variable height within layers offer the advantage of producing overhangs without the need for support material or geometric decomposition. This approach has been demonstrated for steel using precomputed robot speed profiles to achieve consistent geometric quality. In contrast, aluminum exhibits a bead geometry that is tightly coupled to the temperature of the previous layer, resulting in significant changes to the height of the deposited material at different points in the part. This paper presents a closed-loop approach to correcting for variations in the height of the deposited material between layers. We use an IR camera mounted on a separate robot to track the welding flame and estimate the height of deposited material. The robot velocity profile is then updated to account for the error in the previous layer and the nominal planned height profile while factoring in process and system constraints. Implementation of this framework showed significant improvement over the open-loop case and demonstrated robustness to inaccurate model parameters., Comment: 8 pages, 17 figures

Nearly fifty years ago, Roberts (1978) postulated that Earth's magnetic field, which is generated by turbulent motions of liquid metal in its outer core, likely results from a subcritical dynamo instability characterised by a dominant balance between Coriolis, pressure and Lorentz forces. Here we numerically explore the generation of subcritical geomagnetic fields using techniques from optimal control and dynamical systems theory to uncover the nonlinear dynamical landscape underlying dynamo action. Through nonlinear optimisation, via direct-adjoint looping, we identify the minimal seed - the smallest magnetic field that attracts to a nonlinear dynamo solution. Additionally, using the Newton-hookstep algorithm, we converge stable and unstable travelling wave solutions to the governing equations. By combining these two techniques, complex nonlinear pathways between attracting states are revealed, providing insight into a potential subcritical origin of the geodynamo. This paper showcases these methods on the widely studied benchmark of Christensen et al. (2001), laying the foundations for future studies in more extreme and realistic parameter regimes. We show that the minimal seed reaches a nonlinear dynamo solution by first attracting to an unstable travelling wave solution, which acts as an edge state separating a hydrodynamic solution from a magnetohydrodynamic one. Furthermore, by carefully examining the choice of cost functional, we establish a robust optimisation procedure that can systematically locate dynamo solutions on short time horizons with no prior knowledge of its structure., Comment: 21 pages, 8 figures

Forecasting is a critical task in decision-making across numerous domains. While historical numerical data provide a start, they fail to convey the complete context for reliable and accurate predictions. Human forecasters frequently rely on additional information, such as background knowledge and constraints, which can efficiently be communicated through natural language. However, in spite of recent progress with LLM-based forecasters, their ability to effectively integrate this textual information remains an open question. To address this, we introduce "Context is Key" (CiK), a time-series forecasting benchmark that pairs numerical data with diverse types of carefully crafted textual context, requiring models to integrate both modalities; crucially, every task in CiK requires understanding textual context to be solved successfully. We evaluate a range of approaches, including statistical models, time series foundation models, and LLM-based forecasters, and propose a simple yet effective LLM prompting method that outperforms all other tested methods on our benchmark. Our experiments highlight the importance of incorporating contextual information, demonstrate surprising performance when using LLM-based forecasting models, and also reveal some of their critical shortcomings. This benchmark aims to advance multimodal forecasting by promoting models that are both accurate and accessible to decision-makers with varied technical expertise. The benchmark can be visualized at https://servicenow.github.io/context-is-key-forecasting/v0/., Comment: Preprint; under review. First two authors contributed equally

In graph theory, the minimum directed feedback vertex set (FVS) problem consists in identifying the smallest subsets of vertices in a directed graph whose deletion renders the directed graph acyclic. Although being known as NP-hard since 1972, this problem can be solved in a reasonable time on small instances, or on instances having special combinatorial structure. In this paper we investigate graph reductions preserving all or some minimum FVS and focus on their properties, especially the Church-Rosser property, also called confluence. The Church-Rosser property implies the irrelevance of reduction order, leading to a unique directed graph. The study seeks the largest subset of reductions with the Church-Rosser property and explores the adaptability of reductions to meet this criterion. Addressing these questions is crucial, as it may impact algorithmic implications, allowing for parallelization and speeding up sequential algorithms., Comment: In Proceedings GASCom 2024, arXiv:2406.14588

Large Language Models (LLMs) are trained on vast amounts of data, most of which is automatically scraped from the internet. This data includes encyclopedic documents that harbor a vast amount of general knowledge (e.g., Wikipedia) but also potentially overlap with benchmark datasets used for evaluating LLMs. Consequently, evaluating models on test splits that might have leaked into the training set is prone to misleading conclusions. To foster sound evaluation of language models, we introduce a new test dataset named RepLiQA, suited for question-answering and topic retrieval tasks. RepLiQA is a collection of five splits of test sets, four of which have not been released to the internet or exposed to LLM APIs prior to this publication. Each sample in RepLiQA comprises (1) a reference document crafted by a human annotator and depicting an imaginary scenario (e.g., a news article) absent from the internet; (2) a question about the document's topic; (3) a ground-truth answer derived directly from the information in the document; and (4) the paragraph extracted from the reference document containing the answer. As such, accurate answers can only be generated if a model can find relevant content within the provided document. We run a large-scale benchmark comprising several state-of-the-art LLMs to uncover differences in performance across models of various types and sizes in a context-conditional language modeling setting. Released splits of RepLiQA can be found here: https://huggingface.co/datasets/ServiceNow/repliqa.

Conservation laws are well-established in the context of Euclidean gradient flow dynamics, notably for linear or ReLU neural network training. Yet, their existence and principles for non-Euclidean geometries and momentum-based dynamics remain largely unknown. In this paper, we characterize "all" conservation laws in this general setting. In stark contrast to the case of gradient flows, we prove that the conservation laws for momentum-based dynamics exhibit temporal dependence. Additionally, we often observe a "conservation loss" when transitioning from gradient flow to momentum dynamics. Specifically, for linear networks, our framework allows us to identify all momentum conservation laws, which are less numerous than in the gradient flow case except in sufficiently over-parameterized regimes. With ReLU networks, no conservation law remains. This phenomenon also manifests in non-Euclidean metrics, used e.g. for Nonnegative Matrix Factorization (NMF): all conservation laws can be determined in the gradient flow context, yet none persists in the momentum case., Comment: Accepted to ICML 2024

In confined systems, the entrapment of a gas volume with an equivalent spherical diameter greater than the dimension of the channel can form extended bubbles that obstruct fluid circuits and compromise performance. Notably, in sealed vertical tubes, buoyant long bubbles -- called Taylor bubbles -- cannot rise if the inner tube radius is below a critical value near the capillary length. This critical threshold for steady ascent is determined by geometric constraints related to matching the upper cap shape with the lubricating film in the elongated part of the bubble. Developing strategies to overcome this threshold and release stuck bubbles is essential for applications involving narrow liquid channels. Effective strategies involve modifying matching conditions with an external force field to facilitate bubble ascent. However, it's unclear how changes in acceleration conditions affect the motion onset of buoyancy-driven long bubbles. This study investigates the mobility of elongated bubbles in sealed tubes with an inner radius near the critical value inhibiting bubble motion in a vertical setting. Two strategies are explored to tune bubble motion, leveraging variations in axial and transversal accelerations: tube rotation around its axis and tube inclination relative to gravity. By revising the geometrical constraints of the simple vertical setting, the study predicts new thresholds based on rotational speed and tilt angle, respectively, providing forecasts for the bubble rising velocity under modified apparent gravity. Experimental measurements of motion threshold and rising velocity compare well with theoretical developments, thus suggesting practical approaches to control and tune bubble motion in confined environments.

In-context learning (ICL) approaches typically leverage prompting to condition decoder-only language model generation on reference information. Just-in-time processing of a context is inefficient due to the quadratic cost of self-attention operations, and caching is desirable. However, caching transformer states can easily require almost as much space as the model parameters. When the right context isn't known in advance, caching ICL can be challenging. This work addresses these limitations by introducing models that, inspired by the encoder-decoder architecture, use cross-attention to condition generation on reference text without the prompt. More precisely, we leverage pre-trained decoder-only models and only train a small number of added layers. We use Question-Answering (QA) as a testbed to evaluate the ability of our models to perform conditional generation and observe that they outperform ICL, are comparable to fine-tuned prompted LLMs, and drastically reduce the space footprint relative to standard KV caching by two orders of magnitude.

We develop a linear theory for the prediction of excitation wave quenching -- the construction of minimal perturbations which return stable excitations to quiescence -- for localized pulse solutions in models of excitable media. The theory accounts for an additional equivariance compared to the homogeneous ignition problem, and thus requires a reconsideration of heuristics for choosing optimal reference states from their group representation. We compare predictions made with the linear theory to direct numerical simulations across a family of perturbations and assess their accuracy for several models with distinct stable excitation structures. We find that the theory achieves qualitative predictive power with only the effort of continuing a scalar root, and achieves quantitative predictive power in many circumstances. Finally, we compare the computational cost of our prediction technique to other numerical methods for the determination of transitions in extended excitable systems., Comment: 17 pages, 11 figures

We introduce a new model for multivariate probabilistic time series prediction, designed to flexibly address a range of tasks including forecasting, interpolation, and their combinations. Building on copula theory, we propose a simplified objective for the recently-introduced transformer-based attentional copulas (TACTiS), wherein the number of distributional parameters now scales linearly with the number of variables instead of factorially. The new objective requires the introduction of a training curriculum, which goes hand-in-hand with necessary changes to the original architecture. We show that the resulting model has significantly better training dynamics and achieves state-of-the-art performance across diverse real-world forecasting tasks, while maintaining the flexibility of prior work, such as seamless handling of unaligned and unevenly-sampled time series. Code is made available at https://github.com/ServiceNow/TACTiS., Comment: 28 pages, 15 figures, The Twelfth International Conference on Learning Representations (ICLR 2024)

Hyperspectral unmixing aims at decomposing a given signal into its spectral signatures and its associated fractional abundances. To improve the accuracy of this decomposition, algorithms have included different assumptions depending on the application. The goal of this study is to develop a new unmixing algorithm that can be applied for the calibration of multi-point scintillation dosimeters used in the field of radiation therapy. This new algorithm is based on a non-negative matrix factorization. It incorporates a partial prior knowledge on both the abundances and the endmembers of a given signal. It is shown herein that, following a precise calibration routine, it is possible to use partial prior information about the fractional abundances, as well as on the endmembers, in order to perform a simplified yet precise calibration of these dosimeters. Validation and characterization of this algorithm is made using both simulations and experiments. The experimental validation shows an improvement in accuracy compared to previous algorithms with a mean spectral angle distance (SAD) on the estimated endmembers of 0.0766, leading to an average error of $(0.25 \pm 0.73)$ % on dose measurements., Comment: 12 pages, 6 figures

The fundamental goal of small molecule discovery is to generate chemicals with target functionality. While this often proceeds through structure-based methods, we set out to investigate the practicality of orthogonal methods that leverage the extensive corpus of chemical literature. We hypothesize that a sufficiently large text-derived chemical function dataset would mirror the actual landscape of chemical functionality. Such a landscape would implicitly capture complex physical and biological interactions given that chemical function arises from both a molecule's structure and its interacting partners. To evaluate this hypothesis, we built a Chemical Function (CheF) dataset of patent-derived functional labels. This dataset, comprising 631K molecule-function pairs, was created using an LLM- and embedding-based method to obtain functional labels for approximately 100K molecules from their corresponding 188K unique patents. We carry out a series of analyses demonstrating that the CheF dataset contains a semantically coherent textual representation of the functional landscape congruent with chemical structural relationships, thus approximating the actual chemical function landscape. We then demonstrate that this text-based functional landscape can be leveraged to identify drugs with target functionality using a model able to predict functional profiles from structure alone. We believe that functional label-guided molecular discovery may serve as an orthogonal approach to traditional structure-based methods in the pursuit of designing novel functional molecules., Comment: Under review

About 30% of all traffic crash fatalities in the United States involve drunk drivers, making the prevention of drunk driving paramount to vehicle safety in the US and other locations which have a high prevalence of driving while under the influence of alcohol. Driving impairment can be monitored through active use of sensors (when drivers are asked to engage in providing breath samples to a vehicle instrument or when pulled over by a police officer), but a more passive and robust mechanism of sensing may allow for wider adoption and benefit of intelligent systems that reduce drunk driving accidents. This could assist in identifying impaired drivers before they drive, or early in the driving process (before a crash or detection by law enforcement). In this research, we introduce a study which adopts a multi-modal ensemble of visual, thermal, audio, and chemical sensors to (1) examine the impact of acute alcohol administration on driving performance in a driving simulator, and (2) identify data-driven methods for detecting driving under the influence of alcohol. We describe computer vision and machine learning models for analyzing the driver's face in thermal imagery, and introduce a pipeline for training models on data collected from drivers with a range of breath-alcohol content levels, including discussion of relevant machine learning phenomena which can help in future experiment design for related studies., Comment: Withdrawn at the request of industry research collaborators, per contract agreement

Neural decoders for quantum error correction (QEC) rely on neural networks to classify syndromes extracted from error correction codes and find appropriate recovery operators to protect logical information against errors. Its ability to adapt to hardware noise and long-term drifts make neural decoders a promising candidate for inclusion in a fault-tolerant quantum architecture. However, given their limited scalability, it is prudent that small-scale (local) neural decoders are treated as first stages of multi-stage decoding schemes for fault-tolerant quantum computers with millions of qubits. In this case, minimizing the decoding time to match the stabilization measurements frequency and a tight co-integration with the QPUs is highly desired. Cryogenic realizations of neural decoders can not only improve the performance of higher stage decoders, but they can minimize communication delays, and alleviate wiring bottlenecks. In this work, we design and analyze a neural decoder based on an in-memory computation (IMC) architecture, where crossbar arrays of resistive memory devices are employed to both store the synaptic weights of the neural decoder and perform analog matrix-vector multiplications. In simulations supported by experimental measurements, we investigate the impact of TiOx-based memristive devices' non-idealities on decoding fidelity. We develop hardware-aware re-training methods to mitigate the fidelity loss, restoring the ideal decoder's pseudo-threshold for the distance-3 surface code. This work provides a pathway to scalable, fast, and low-power cryogenic IMC hardware for integrated fault-tolerant QEC.

Understanding the geometric properties of gradient descent dynamics is a key ingredient in deciphering the recent success of very large machine learning models. A striking observation is that trained over-parameterized models retain some properties of the optimization initialization. This "implicit bias" is believed to be responsible for some favorable properties of the trained models and could explain their good generalization properties. The purpose of this article is threefold. First, we rigorously expose the definition and basic properties of "conservation laws", that define quantities conserved during gradient flows of a given model (e.g. of a ReLU network with a given architecture) with any training data and any loss. Then we explain how to find the maximal number of independent conservation laws by performing finite-dimensional algebraic manipulations on the Lie algebra generated by the Jacobian of the model. Finally, we provide algorithms to: a) compute a family of polynomial laws; b) compute the maximal number of (not necessarily polynomial) independent conservation laws. We provide showcase examples that we fully work out theoretically. Besides, applying the two algorithms confirms for a number of ReLU network architectures that all known laws are recovered by the algorithm, and that there are no other independent laws. Such computational tools pave the way to understanding desirable properties of optimization initialization in large machine learning models.

The Tayler-Spruit dynamo mechanism has been proposed two decades ago as a plausible mechanism to transport angular momentum in radiative stellar layers. Direct numerical simulations are still needed to understand its trigger conditions and the saturation mechanisms. The present study follows up on (Petitdemange et al. 2023), where we reported the first numerical simulations of a Tayler-Spruit dynamo cycle. Here we extend the explored parameter space to assess in particular the influence of stratification on the dynamo solutions. We also present numerical verification of theoretical assumptions made in (Spruit 2002), which are instrumental in deriving the classical prescription for angular momentum transport implemented in stellar evolution codes. A simplified radiative layer is modeled numerically by considering the dynamics of a stably-stratified, differentially rotating, magnetized fluid in a spherical shell. Our simulations display a diversity of magnetic field topologies and amplitudes depending on the flow parameters, including hemispherical solutions. The Tayler-Spruit dynamos reported here are found to satisfy magnetostrophic equilibrium and achieve efficient turbulent transport of angular momentum, following Spruit's heuristic prediction.

The reconstruction of electrical excitation patterns through the unobserved depth of the tissue is essential to realizing the potential of computational models in cardiac medicine. We have utilized experimental optical-mapping recordings of cardiac electrical excitation on the epicardial and endocardial surfaces of a canine ventricle as observations directing a local ensemble transform Kalman Filter (LETKF) data assimilation scheme. We demonstrate that the inclusion of explicit information about the stimulation protocol can marginally improve the confidence of the ensemble reconstruction and the reliability of the assimilation over time. Likewise, we consider the efficacy of stochastic modeling additions to the assimilation scheme in the context of experimentally derived observation sets. Approximation error is addressed at both the observation and modeling stages, through the uncertainty of observations and the specification of the model used in the assimilation ensemble. We find that perturbative modifications to the observations have marginal to deleterious effects on the accuracy and robustness of the state reconstruction. Further, we find that incorporating additional information from the observations into the model itself (in the case of stimulus and stochastic currents) has a marginal improvement on the reconstruction accuracy over a fully autonomous model, while complicating the model itself and thus introducing potential for new types of model error. That the inclusion of explicit modeling information has negligible to negative effects on the reconstruction implies the need for new avenues for optimization of data assimilation schemes applied to cardiac electrical excitation., Comment: main text: 18 pages, 10 figures; supplement: 5 pages, 9 figures, 2 movies

Multivariate probabilistic time series forecasts are commonly evaluated via proper scoring rules, i.e., functions that are minimal in expectation for the ground-truth distribution. However, this property is not sufficient to guarantee good discrimination in the non-asymptotic regime. In this paper, we provide the first systematic finite-sample study of proper scoring rules for time-series forecasting evaluation. Through a power analysis, we identify the "region of reliability" of a scoring rule, i.e., the set of practical conditions where it can be relied on to identify forecasting errors. We carry out our analysis on a comprehensive synthetic benchmark, specifically designed to test several key discrepancies between ground-truth and forecast distributions, and we gauge the generalizability of our findings to real-world tasks with an application to an electricity production problem. Our results reveal critical shortcomings in the evaluation of multivariate probabilistic forecasts as commonly performed in the literature., Comment: 47 pages, 37 figures, camera-ready version, Fortieth International Conference on Machine Learning (ICML 2023)

We study the free surface response in a cylindrical container undergoing an elliptic periodic orbit. For small forcing amplitudes and deep liquid layers, we quantify the effect of orbit's aspect ratio onto the surface dynamics in the vicinity of the fluid system's lowest natural frequency. We provide experimental evidences of the existence of a frequency range where stable swirling can be either co- or counter-directed with respect to the container's direction of motion. Our findings are successfully predicted by an inviscid asymptotic model, amended with a heuristic damping.

Resonant sloshing in circular cylinders was studied by Faltinsen et al. (2016), whose theory was used to describe steady-state resonant waves due to time-harmonic container's elliptic orbits. In the limit of longitudinal container motions, a symmetry-breaking of the planar wave solution occurs, with clockwise and anti-clockwise swirling equally likely. In addition to this primary harmonic dynamics, previous experiments have unveiled that diverse super-harmonic dynamics are observable far from primary resonances. Among these, the so-called double-crest (DC) dynamics, first observed by Reclari et al. (2014) for rotary sloshing, is particularly relevant, as its manifestation is the most favored by the spatial structure of the external driving. Following Bongarzone et al. (2022a), in this work we develop a weakly nonlinear (WNL) analysis to describe the system response to super-harmonic longitudinal forcing. The resulting system of amplitude equations predicts that a planar wave symmetry-breaking via stable swirling may also occur under super-harmonic excitation. This finding is confirmed by our experimental observations, which identify three possible super-harmonic regimes, i.e. (i) stable planar DC waves, (ii) irregular motion and (iii) stable swirling DC waves, whose corresponding stability boundaries in the forcing frequency-amplitude plane quantitatively match the present theoretical estimates.

Many physical questions in fluid dynamics can be recast in terms of norm constrained optimisation problems; which in-turn, can be further recast as unconstrained problems on spherical manifolds. Due to the nonlinearities of the governing PDEs, and the computational cost of performing optimal control on such systems, improving the numerical convergence of the optimisation procedure is crucial. Borrowing tools from the optimisation on manifolds community we outline a numerically consistent, discrete formulation of the direct-adjoint looping method accompanied by gradient descent and line-search algorithms with global convergence guarantees. We numerically demonstrate the robustness of this formulation on three example problems of relevance in fluid dynamics and provide an accompanying library SphereManOpt

The evolution of a star is influenced by its internal rotation dynamics through transport and mixing mechanisms, which are poorly understood. Magnetic fields can play a role in transporting angular momentum and chemical elements, but the origin of magnetism in radiative stellar layers is unclear. Using global numerical simulations, we identify a subcritical transition to turbulence due to the generation of a magnetic dynamo. Our results have many of the properties of the theoretically-proposed Tayler-Spruit dynamo mechanism, which strongly enhances transport of angular momentum in radiative zones. It generates deep toroidal fields that are screened by the stellar outer layers. This mechanism could produce strong magnetic fields inside radiative stars, without an observable field on their surface.

The interface between a liquid and a solid is the location of plethora of intrincate mechanisms at the nanoscale, at the root of their specific emerging properties in natural processes or technological applications. However, while the structural properties and chemistry of interfaces have been intensively explored, the effect of the solid-state electronic transport at the fluid interface has been broadly overlooked up to now. It has been reported that water flowing against carbon-based nanomaterials, such as carbon nanotubes or graphene sheets, does induce electronic currents, but the mechanism at stake remains controversial. Here, we unveil the molecular mechanisms underlying the hydro-electronic couplings by investigating the electronic conversion under flow at the nanoscale. We use a tuning fork-Atomic Force Microscope (AFM) to deposit and displace a micrometric droplet of both ionic and non-ionic liquids on a multilayer graphene sample, while recording the electrical current across the carbon flake. We report measurements of an oscillation-induced current which is several orders of magnitude larger than previously reported for water on carbon , and further boosted by the presence of surface wrinkles on the carbon layer. Our results point to a peculiar momentum transfer mechanism between fluid molecules and charge carriers in the carbon walls mediated by phonon excitations in the solid. Our findings pave the way for active control of fluid transfer at the nanoscale by harnessing the complex interplay between collective excitations in the solid and the molecules in the fluid., Comment: Accepted for publication at PRX

Over the past three decades, children from low-income families and those from more affluent families have increasingly been attending different public schools. While recent work has helped us understand patterns of income segregation between districts and schools within districts, we know very little about segregation of students as they experience school: in the classroom. In this paper, we attempt to advance knowledge of trends in the segregation of students by income at the classroom level. We make use of detailed, student-level administrative data from North Carolina which provides a measure of a student's free/reduced price lunch eligibility, which we refer to as economically disadvantaged (ED) status, along with information on classroom assignments. Since we know the ED status of each student in each classroom, we assess whether ED students are assigned to classes in the same pattern as other students or if are clustered/segregated into different classrooms. We know very little about the magnitude of income-based segregation, and almost nothing about whether this has changed over time, so we provide novel evidence on the question of whether segregation of students by socioeconomic status has increased within schools. We find that within-school segregation has risen by about 10 percent between 2007 and 2014 in elementary and middle schools we study. Further, we find that segregation of ED students within schools is correlated with the level of segregation between schools in districts, and this relationship grew stronger over our panel.

The estimation of time-varying quantities is a fundamental component of decision making in fields such as healthcare and finance. However, the practical utility of such estimates is limited by how accurately they quantify predictive uncertainty. In this work, we address the problem of estimating the joint predictive distribution of high-dimensional multivariate time series. We propose a versatile method, based on the transformer architecture, that estimates joint distributions using an attention-based decoder that provably learns to mimic the properties of non-parametric copulas. The resulting model has several desirable properties: it can scale to hundreds of time series, supports both forecasting and interpolation, can handle unaligned and non-uniformly sampled data, and can seamlessly adapt to missing data during training. We demonstrate these properties empirically and show that our model produces state-of-the-art predictions on multiple real-world datasets., Comment: 47 pages, 33 figures, camera-ready version, Thirty-ninth International Conference on Machine Learning (ICML 2022)