Search

Natural languages are believed to be (mildly) context-sensitive. Despite underpinning remarkably capable large language models, transformers are unable to model many context-free language tasks. In an attempt to address this limitation in the modeling power of transformer-based language models, we propose augmenting them with a differentiable, stack-based attention mechanism. Our stack-based attention mechanism can be incorporated into any transformer-based language model and adds a level of interpretability to the model. We show that the addition of our stack-based attention mechanism enables the transformer to model some, but not all, deterministic context-free languages., Comment: NAACL 2024 Findings

To answer a question, language models often need to integrate prior knowledge learned during pretraining and new information presented in context. We hypothesize that models perform this integration in a predictable way across different questions and contexts: models will rely more on prior knowledge for questions about entities (e.g., persons, places, etc.) that they are more familiar with due to higher exposure in the training corpus, and be more easily persuaded by some contexts than others. To formalize this problem, we propose two mutual information-based metrics to measure a model's dependency on a context and on its prior about an entity: first, the persuasion score of a given context represents how much a model depends on the context in its decision, and second, the susceptibility score of a given entity represents how much the model can be swayed away from its original answer distribution about an entity. We empirically test our metrics for their validity and reliability. Finally, we explore and find a relationship between the scores and the model's expected familiarity with an entity, and provide two use cases to illustrate their benefits., Comment: Long paper accepted at ACL 2024

We present the fourth data release (DR4) of the SkyMapper Southern Survey (SMSS), the last major step in our hemispheric survey with six optical filters: u, v, g, r, i, z. SMSS DR4 covers 26,000 sq.deg from over 400,000 images acquired by the 1.3m SkyMapper telescope between 2014-03 and 2021-09. The 6-band sky coverage extends from the South Celestial Pole to Dec = +16deg, with some images reaching Dec ~ +28deg. In contrast to previous DRs, we include all good-quality images from the facility taken during that time span, not only those explicitly taken for the public Survey. From the image dataset, we produce a catalogue of nearly 13 billion detections made from ~700 million unique astrophysical objects. The typical 10sigma depths for each field range between 18.5 and 20.5 mag, depending on the filter, but certain sky regions include longer exposures that reach as deep as 22 mag in some filters. As with previous SMSS catalogues, we have cross-matched with a host of other imaging and spectroscopic datasets to facilitate additional science outcomes. SMSS DR4 is now available to the worldwide astronomical community., Comment: 32 pages. Submitted to the Publications of the Astronomical Society of Australia

This paper develops moving frame theory for partial difference equations and for differential-difference equations with one continuous independent variable. In each case, the theory is applied to the invariant calculus of variations and the equivariant formulation of the conservation laws arising from Noether's theorem. The differential-difference theory is not merely an amalgam of the differential and difference theories, but has additional features that reflect the need for the group action to preserve the prolongation structure. Projectable moving frames are introduced; these cause the invariant derivative operator to commute with shifts in the discrete variables. Examples include a Toda-type equation and a method of lines semi-discretization of the nonlinear Schr\"odinger equation.

We show that three-dimensional spherical-shell condensates respond to rotation by forming two aligned triangular Abrikosov-like vortex lattices on each hemispherical surface. The centrifugal force due to rotation causes an elliptical deformation of the spherical shell condensate shape and for faster rotation rates, drives the formation of a central multi-charged vortex-antivortex pair at the poles surrounded by a ring of singly charged vortices in the bulk density. The vortex distributions observed in each hemisphere take a similar form to those found in rotating harmonic plus quartic traps., Comment: Updated with published version, 6 pages, 2 figures, comments welcome

COVID-19 resulted in some of the largest supply chain disruptions in recent history. To mitigate the impact of future disruptions, we propose an integrated hybrid simulation framework to couple nonstationary demand signals from an event like COVID-19 with a model of an end-to-end supply chain. We first create a system dynamics susceptible-infected-recovered (SIR) model, augmenting a classic epidemiological model to create a realistic portrayal of demand patterns for oxygen concentrators (OC). Informed by this granular demand signal, we then create a supply chain discrete event simulation model of OC sourcing, manufacturing, and distribution to test production augmentation policies to satisfy this increased demand. This model utilizes publicly available data, engineering teardowns of OCs, and a supply chain illumination to identify suppliers. Our findings indicate that this coupled approach can use realistic demand during a disruptive event to enable rapid recommendations of policies for increased supply chain resilience with controlled cost.

Recent work has shown that despite their impressive capabilities, text-to-image diffusion models such as DALL-E 2 (Ramesh et al., 2022) can display strange behaviours when a prompt contains a word with multiple possible meanings, often generating images containing both senses of the word (Rassin et al., 2022). In this work we seek to put forward a possible explanation of this phenomenon. Using the similar Stable Diffusion model (Rombach et al., 2022), we first show that when given an input that is the sum of encodings of two distinct words, the model can produce an image containing both concepts represented in the sum. We then demonstrate that the CLIP encoder used to encode prompts (Radford et al., 2021) encodes polysemous words as a superposition of meanings, and that using linear algebraic techniques we can edit these representations to influence the senses represented in the generated images. Combining these two findings, we suggest that the homonym duplication phenomenon described by Rassin et al. (2022) is caused by diffusion models producing images representing both of the meanings that are present in superposition in the encoding of a polysemous word.

Progress in understanding quantum systems has been driven by the exploration of the geometry, topology, and dimensionality of ultracold atomic systems. The NASA Cold Atom Laboratory (CAL) aboard the International Space Station has enabled the study of ultracold atomic bubbles, a terrestrially-inaccessible topology. Proof-of-principle bubble experiments have been performed on CAL with an rf-dressing technique; an alternate technique (dual-species interaction-driven bubbles) has also been proposed. Both techniques can drive discovery in the next decade of fundamental physics research in microgravity., Comment: 17 pages, 2 figures

The ability to generalize compositionally is key to understanding the potentially infinite number of sentences that can be constructed in a human language from only a finite number of words. Investigating whether NLP models possess this ability has been a topic of interest: SCAN (Lake and Baroni, 2018) is one task specifically proposed to test for this property. Previous work has achieved impressive empirical results using a group-equivariant neural network that naturally encodes a useful inductive bias for SCAN (Gordon et al., 2020). Inspired by this, we introduce a novel group-equivariant architecture that incorporates a group-invariant hard alignment mechanism. We find that our network's structure allows it to develop stronger equivariance properties than existing group-equivariant approaches. We additionally find that it outperforms previous group-equivariant networks empirically on the SCAN task. Our results suggest that integrating group-equivariance into a variety of neural architectures is a potentially fruitful avenue of research, and demonstrate the value of careful analysis of the theoretical properties of such architectures., Comment: Accepted at COLING 2022

We demonstrate a high dynamic range Josephson parametric amplifier (JPA) in which the active nonlinear element is implemented using an array of rf-SQUIDs. The device is matched to the 50 $\Omega$ environment with a Klopfenstein-taper impedance transformer and achieves a bandwidth of 250-300 MHz, with input saturation powers up to -95 dBm at 20 dB gain. A 54-qubit Sycamore processor was used to benchmark these devices, providing a calibration for readout power, an estimate of amplifier added noise, and a platform for comparison against standard impedance matched parametric amplifiers with a single dc-SQUID. We find that the high power rf-SQUID array design has no adverse effect on system noise, readout fidelity, or qubit dephasing, and we estimate an upper bound on amplifier added noise at 1.6 times the quantum limit. Lastly, amplifiers with this design show no degradation in readout fidelity due to gain compression, which can occur in multi-tone multiplexed readout with traditional JPAs., Comment: 10 pages, 10 figures

Neural Networks and related Deep Learning methods are currently at the leading edge of technologies used for classifying objects. However, they generally demand large amounts of time and data for model training; and their learned models can sometimes be difficult to interpret. In this paper, we advance FastMapSVM -- an interpretable Machine Learning framework for classifying complex objects -- as an advantageous alternative to Neural Networks for general classification tasks. FastMapSVM extends the applicability of Support-Vector Machines (SVMs) to domains with complex objects by combining the complementary strengths of FastMap and SVMs. FastMap is an efficient linear-time algorithm that maps complex objects to points in a Euclidean space while preserving pairwise domain-specific distances between them. We demonstrate the efficiency and effectiveness of FastMapSVM in the context of classifying seismograms. We show that its performance, in terms of precision, recall, and accuracy, is comparable to that of other state-of-the-art methods. However, compared to other methods, FastMapSVM uses significantly smaller amounts of time and data for model training. It also provides a perspicuous visualization of the objects and the classification boundaries between them. We expect FastMapSVM to be viable for classification tasks in many other real-world domains., Comment: 27 pages, 12 figures

Researchers have noted the importance of equity-based approaches to social and emotional learning (SEL), which emphasize the role of school environment, including adult beliefs, in student well-being. This article builds on this work by examining 129 teachers' perceptions of efficacy in SEL. While participants worked in urban schools, were selected from national fellowship programs, and had similar years of experience and preparation, survey data found that teachers in one program reported higher levels of efficacy in SEL. Interviews and observations with a purposeful sample of these teachers found that despite common challenges with exclusionary discipline practices and limited resources, efficacious teachers described a "social justice learning community," geared for teachers of color, that enhanced their capacities to enact SEL in their schools. Discussion includes the need for critical professional development opportunities in SEL that are race-conscious, context-specific, and asset-based, as well as opportunities for teachers from historically marginalized groups to form specialized learning communities.

Since language models are used to model a wide variety of languages, it is natural to ask whether the neural architectures used for the task have inductive biases towards modeling particular types of languages. Investigation of these biases has proved complicated due to the many variables that appear in the experimental setup. Languages vary in many typological dimensions, and it is difficult to single out one or two to investigate without the others acting as confounders. We propose a novel method for investigating the inductive biases of language models using artificial languages. These languages are constructed to allow us to create parallel corpora across languages that differ only in the typological feature being investigated, such as word order. We then use them to train and test language models. This constitutes a fully controlled causal framework, and demonstrates how grammar engineering can serve as a useful tool for analyzing neural models. Using this method, we find that commonly used neural architectures exhibit different inductive biases: LSTMs display little preference with respect to word ordering, while transformers display a clear preference for some orderings over others. Further, we find that neither the inductive bias of the LSTM nor that of the transformer appears to reflect any tendencies that we see in attested natural languages., Comment: Accepted at ACL 2021

Probes are models devised to investigate the encoding of knowledge -- e.g. syntactic structure -- in contextual representations. Probes are often designed for simplicity, which has led to restrictions on probe design that may not allow for the full exploitation of the structure of encoded information; one such restriction is linearity. We examine the case of a structural probe (Hewitt and Manning, 2019), which aims to investigate the encoding of syntactic structure in contextual representations through learning only linear transformations. By observing that the structural probe learns a metric, we are able to kernelize it and develop a novel non-linear variant with an identical number of parameters. We test on 6 languages and find that the radial-basis function (RBF) kernel, in conjunction with regularization, achieves a statistically significant improvement over the baseline in all languages -- implying that at least part of the syntactic knowledge is encoded non-linearly. We conclude by discussing how the RBF kernel resembles BERT's self-attention layers and speculate that this resemblance leads to the RBF-based probe's stronger performance., Comment: Accepted at NAACL 2021

Numerous accessibility features have been developed and included in consumer operating systems to provide people with a variety of disabilities additional ways to access computing devices. Unfortunately, many users, especially older adults who are more likely to experience ability changes, are not aware of these features or do not know which combination to use. In this paper, we first quantify this problem via a survey with 100 participants, demonstrating that very few people are aware of built-in accessibility features on their phones. These observations led us to investigate accessibility recommendation as a way to increase awareness and adoption. We developed four prototype recommenders that span different accessibility categories, which we used to collect insights from 20 older adults. Our work demonstrates the need to increase awareness of existing accessibility features on mobile devices, and shows that automated recommendation could help people find beneficial accessibility features., Comment: Accepted to Web4All 2021 (W4A '21)

The discovery of topological order has revolutionized the understanding of quantum matter in modern physics and provided the theoretical foundation for many quantum error correcting codes. Realizing topologically ordered states has proven to be extremely challenging in both condensed matter and synthetic quantum systems. Here, we prepare the ground state of the toric code Hamiltonian using an efficient quantum circuit on a superconducting quantum processor. We measure a topological entanglement entropy near the expected value of $\ln2$, and simulate anyon interferometry to extract the braiding statistics of the emergent excitations. Furthermore, we investigate key aspects of the surface code, including logical state injection and the decay of the non-local order parameter. Our results demonstrate the potential for quantum processors to provide key insights into topological quantum matter and quantum error correction., Comment: 6 pages 4 figures, plus supplementary materials

Quantum computing can become scalable through error correction, but logical error rates only decrease with system size when physical errors are sufficiently uncorrelated. During computation, unused high energy levels of the qubits can become excited, creating leakage states that are long-lived and mobile. Particularly for superconducting transmon qubits, this leakage opens a path to errors that are correlated in space and time. Here, we report a reset protocol that returns a qubit to the ground state from all relevant higher level states. We test its performance with the bit-flip stabilizer code, a simplified version of the surface code for quantum error correction. We investigate the accumulation and dynamics of leakage during error correction. Using this protocol, we find lower rates of logical errors and an improved scaling and stability of error suppression with increasing qubit number. This demonstration provides a key step on the path towards scalable quantum computing.

A promising approach to study condensed-matter systems is to simulate them on an engineered quantum platform. However, achieving the accuracy needed to outperform classical methods has been an outstanding challenge. Here, using eighteen superconducting qubits, we provide an experimental blueprint for an accurate condensed-matter simulator and demonstrate how to probe fundamental electronic properties. We benchmark the underlying method by reconstructing the single-particle band-structure of a one-dimensional wire. We demonstrate nearly complete mitigation of decoherence and readout errors and arrive at an accuracy in measuring energy eigenvalues of this wire with an error of ~0.01 rad, whereas typical energy scales are of order 1 rad. Insight into this unprecedented algorithm fidelity is gained by highlighting robust properties of a Fourier transform, including the ability to resolve eigenenergies with a statistical uncertainty of 1e-4 rad. Furthermore, we synthesize magnetic flux and disordered local potentials, two key tenets of a condensed-matter system. When sweeping the magnetic flux, we observe avoided level crossings in the spectrum, a detailed fingerprint of the spatial distribution of local disorder. Combining these methods, we reconstruct electronic properties of the eigenstates where we observe persistent currents and a strong suppression of conductance with added disorder. Our work describes an accurate method for quantum simulation and paves the way to study novel quantum materials with superconducting qubits.

We describe an approach to the high-fidelity measurement of a superconducting qubit using an on-chip microwave photon counter. The protocol relies on the transient response of a dispersively coupled measurement resonator to map the state of the qubit to "bright" and "dark" cavity pointer states that are characterized by a large differential photon occupation. Following this mapping, we photodetect the resonator using the Josephson Photomultipler (JPM), which transitions between classically distinguishable flux states when cavity photon occupation exceeds a certain threshold. Our technique provides access to the binary outcome of projective quantum measurement at the millikelvin stage without the need for quantum-limited preamplification and thresholding at room temperature. We achieve raw single-shot measurement fidelity in excess of 98% across multiple samples using this approach in total measurement times under 500 ns. In addition, we show that the backaction and crosstalk associated with our measurement protocol can be mitigated by exploiting the intrinsic damping of the JPM itself., Comment: 15 pages, 13 Figures, 3 Tables

Direct detection of dark energy or modified gravity may finally be within reach due to ultrasensitive instrumentation such as atom interferometry capable of detecting incredibly small scale accelerations. Forecasts, constraints and measurement bounds can now too perhaps be estimated from accurate numerical simulations of the fifth force and its Laplacian field at solar system scales. The cubic Galileon gravity scalar field model (CGG), which derives from the DGP braneworld model, describes modified gravity incorporating a Vainshtein screening mechanism. The nonlinear derivative interactions in the CGG equation suppress the field near regions of high density, thereby restoring general relativity (GR) while far from such regions, field enhancement is comparable to GR and the equation is dominated by a linear term. This feature of the governing PDE poses some numerical challenges for computation of the scalar potential, force and Laplacian fields even under stationary conditions. Here we present a numerical method based on finite differences for solution of the static CGG scalar field for a 2D axisymmetric Sun-Earth system and a 3D Cartesian Sun-Earth-Moon system. The method relies on gradient descent of an integrated residual based on the normal attractive branch of the CGG equation. The algorithm is shown to be stable, accurate and rapidly convergent toward the global minimum state. We hope this numerical study, which can easily be extended to include smaller bodies such as detection satellites, will prove useful to future measurement of modified gravity force fields at solar system scales., Comment: 30 pages, 15 figures. Submitted to Physical Review D. (Version 2 fixes error in acknowledgments)

Expectancy-value theory highlights the roles of students' expectancies, task values, and perceived costs in their motivation and achievement. While ample research has highlighted the positive associations of expectancy beliefs and task values with academic achievement, research on students' perceived costs is in its infancy. We investigated the temporal interrelations among expectancies, task values and different types of perceived cost, the role of these constructs in biology achievement, and the role of perceived costs as a moderator in the relations of expectancy beliefs to biology achievement. A cross-lagged path analysis of semester-long data from 234 undergraduate biology students pointed to variable relations among expectancies, task values, perceived costs, and biology achievement. For example, while early expectancy beliefs related to later attainment and interest value, early task values and perceived costs did not relate to later expectancy beliefs. Furthermore, early attainment value related to later effort and opportunity cost. Expectancy beliefs and effort and opportunity cost in biology were associated with final biology grade. Finally, effort cost moderated the relationships between expectancy beliefs and students' final grades. These findings provide evidence for the dynamic relations among perceived costs, task values, and expectancy beliefs over a semester and point to the interplay between expectancies and perceived costs in their relation to academic achievement in science. [This paper was published in "Learning and Individual Differences."]

Quantum algorithms offer a dramatic speedup for computational problems in machine learning, material science, and chemistry. However, any near-term realizations of these algorithms will need to be heavily optimized to fit within the finite resources offered by existing noisy quantum hardware. Here, taking advantage of the strong adjustable coupling of gmon qubits, we demonstrate a continuous two-qubit gate set that can provide a 3x reduction in circuit depth as compared to a standard decomposition. We implement two gate families: an iSWAP-like gate to attain an arbitrary swap angle, $\theta$, and a CPHASE gate that generates an arbitrary conditional phase, $\phi$. Using one of each of these gates, we can perform an arbitrary two-qubit gate within the excitation-preserving subspace allowing for a complete implementation of the so-called Fermionic Simulation, or fSim, gate set. We benchmark the fidelity of the iSWAP-like and CPHASE gate families as well as 525 other fSim gates spread evenly across the entire fSim($\theta$, $\phi$) parameter space achieving purity-limited average two-qubit Pauli error of $3.8 \times 10^{-3}$ per fSim gate., Comment: 20 pages, 17 figures

We study linear transformations $T \colon \mathbb{R}[x] \to \mathbb{R}[x]$ of the form $T[x^n]=P_n(x)$ where $\{P_n(x)\}$ is a real orthogonal polynomial system. Such transformations that preserve or shrink the location of the complex zeros of polynomials is a recent object of study, motivated by the Riemann Hypothesis. In particular, we are interested in linear transformations that map polynomials with all real zeros to polynomials with all real zeros. It is well known that any transformation $T \colon \mathbb{C}[x] \rightarrow \mathbb{C}[x]$ has a differential operator representation $T = \sum_{k = 0}^\infty \frac{Q_k(x)}{k!} D^k$. Motivated by the work of Chasse \cite{Chasse-PhD-2011}, Forg\'acs, and Piotrowski \cite{Forgacs-Piotrowski-Hermite-2015}, we seek to understand the behavior of the transformation $T$ by studying the roots of the $Q_k(x)$. We prove four main things. First, we show that the only case where the $Q_k(x)$ are constant and $\{P_n(x)\}$ are an orthogonal system is that when the $P_n$ form a shifted set of generalized probabilist Hermite polynomials. Second, we show that the coefficient polynomials $Q_k(x)$ have real roots when the $P_n(x)$ are the physicist Hermite polynomials or the Laguerre polynomials. Next, we show that in these cases, the roots of successive polynomials strictly interlace, a property that has not yet been studied for coefficient polynomials. We conclude by discussing the Chebyshev and Legendre polynomials, proving a conjecture of Chasse, and presenting several open problems., Comment: 16 pages. This version improves exposition and corrects several typographical errors

Ongoing development of fuel storage and delivery systems for space probes, interplanetary vehicles, satellites and orbital platforms continues to drive interest in propellant management systems that utilize surface tension to retain, channel and control flow in microgravity environments. Although it has been known for decades that capillary flows offer an ideal method of fuel management, there has been little research devoted to the general stability properties of such flows. In this work, we demonstrate theoretically why capillary flows which channel wetting liquids in slender open triangular channels tend to be very stable against disturbances. By utilizing the gradient flow form of the governing fluid interface equation, we first prove that stationary interfaces in the presence of steady flow are asymptotically nonlinearly and exponentially stable in the Lyapunov sense. We then demonstrate that fluid interfaces exhibiting self-similar Washburn dynamics are transiently and asymptotically linearly stable to small perturbations. This second finding relies on a generalized non-modal stability analysis due to the non-normality of the governing disturbance operator. Taken together, these findings reveal the robust nature of transient and steady capillary flows in open grooved channels and likely explains the prevalent use of capillary flow management systems in many emerging technologies ranging from cubesats to point-of-care microfluidic diagnostic systems., Comment: 18 pages, 8 figures, submitted to Phys. Rev. Fluids 12.23.2018