Search

Despite its real-world significance, model performance on tabular data remains underexplored, leaving uncertainty about which model to rely on and which prompt configuration to adopt. To address this gap, we create ToRR, a benchmark for Table Reasoning and Robustness, measuring model performance and robustness on table-related tasks. The benchmark includes 10 datasets that cover different types of table reasoning capabilities across varied domains. ToRR goes beyond model performance rankings, and is designed to reflect whether models can handle tabular data consistently and robustly, across a variety of common table representation formats. We present a leaderboard as well as comprehensive analyses of the results of leading models over ToRR. Our results reveal a striking pattern of brittle model behavior, where even strong models are unable to perform robustly on tabular data tasks. Although no specific table format leads to consistently better performance, we show that testing over multiple formats is crucial for reliably estimating model capabilities. Moreover, we show that the reliability boost from testing multiple prompts can be equivalent to adding more test examples. Overall, our findings show that table understanding and reasoning tasks remain a significant challenge.

Instead of writing a review article on the state of the field, I'm going to instead write a retrospective from the year 2040. I'll tell you how this whole "quantum computing thing" turned out., Comment: I wrote this in between jobs in March 2024 so it is quite out-of-date already. This perspective imagines a possible future, is intentionally opinionated, and is likely mostly wrong. Further caveats inside

While coreference resolution is traditionally used as a component in individual document understanding, in this work we take a more global view and explore what can we learn about a domain from the set of all document-level coreference relations that are present in a large corpus. We derive coreference chains from a corpus of 30 million biomedical abstracts and construct a graph based on the string phrases within these chains, establishing connections between phrases if they co-occur within the same coreference chain. We then use the graph structure and the betweeness centrality measure to distinguish between edges denoting hierarchy, identity and noise, assign directionality to edges denoting hierarchy, and split nodes (strings) that correspond to multiple distinct concepts. The result is a rich, data-driven ontology over concepts in the biomedical domain, parts of which overlaps significantly with human-authored ontologies. We release the coreference chains and resulting ontology under a creative-commons license, along with the code.

Significant research efforts have been made to scale and improve vision-language model (VLM) training approaches. Yet, with an ever-growing number of benchmarks, researchers are tasked with the heavy burden of implementing each protocol, bearing a non-trivial computational cost, and making sense of how all these benchmarks translate into meaningful axes of progress. To facilitate a systematic evaluation of VLM progress, we introduce UniBench: a unified implementation of 50+ VLM benchmarks spanning a comprehensive range of carefully categorized capabilities from object recognition to spatial awareness, counting, and much more. We showcase the utility of UniBench for measuring progress by evaluating nearly 60 publicly available vision-language models, trained on scales of up to 12.8B samples. We find that while scaling training data or model size can boost many vision-language model capabilities, scaling offers little benefit for reasoning or relations. Surprisingly, we also discover today's best VLMs struggle on simple digit recognition and counting tasks, e.g. MNIST, which much simpler networks can solve. Where scale falls short, we find that more precise interventions, such as data quality or tailored-learning objectives offer more promise. For practitioners, we also offer guidance on selecting a suitable VLM for a given application. Finally, we release an easy-to-run UniBench code-base with the full set of 50+ benchmarks and comparisons across 59 models as well as a distilled, representative set of benchmarks that runs in 5 minutes on a single GPU.

Spin-spin exchange interactions between semiconductor spin qubits allow for fast single and two-qubit gates. During exchange, coupling of the qubits to a surrounding phonon bath may cause errors in the resulting gate. Here, the fidelities of exchange operations with semiconductor double quantum dot spin qubits in a Si-SiGe heterostructure coupled to a finite temperature phonon bath are considered. By employing a master equation approach, the isolated effect of each spin-phonon coupling term may be resolved, as well as leakage errors of encoded qubit operations. As the temperature is increased, a crossover is observed from where the primary source of error is due to a phonon induced perturbation of the two electron spin states, to one where the phonon induced coupling to an excited orbital state becomes the dominant error. Additionally, it is shown that a simple trade-off in pulse shape and length can improve robustness to spin-phonon induced errors during gate operations by up to an order of magnitude. Our results suggest that for elevated temperatures within 200-300 mK, exchange gate operations are not currently limited by bulk phonons. This is consistent with recent experiments.

We perform readout of a quantum-dot hybrid qubit coupled to a superconducting resonator through a parametric, longitudinal interaction mechanism. Our experiments are performed with the qubit and resonator frequencies detuned by $\sim$10 GHz, demonstrating that longitudinal coupling can facilitate semiconductor qubit operation in the 'ultra-dispersive' regime of circuit quantum electrodynamics., Comment: Main text and Supplementary Information; 13 pages, 7 figures

Following the recent popularity of Large Language Models (LLMs), several attempts have been made to extend them to the visual domain. From having a visual assistant that could guide us through unfamiliar environments to generative models that produce images using only a high-level text description, the vision-language model (VLM) applications will significantly impact our relationship with technology. However, there are many challenges that need to be addressed to improve the reliability of those models. While language is discrete, vision evolves in a much higher dimensional space in which concepts cannot always be easily discretized. To better understand the mechanics behind mapping vision to language, we present this introduction to VLMs which we hope will help anyone who would like to enter the field. First, we introduce what VLMs are, how they work, and how to train them. Then, we present and discuss approaches to evaluate VLMs. Although this work primarily focuses on mapping images to language, we also discuss extending VLMs to videos.

We theoretically study the dephasing of an Andreev spin qubit (ASQ) due to electric and magnetic noise. Using a tight-binding model, we calculate the Andreev states formed in a Josephson junction where the link is a semiconductor with strong spin-orbit interaction. As a result of both the spin-orbit interaction and induced superconductivity, the local charge and spin of these states varies as a function of externally controllable parameters: the phase difference between the superconducting leads, an applied magnetic field, and filling of the underlying semiconductor. Concomitantly, coupling to fluctuations of the electric or magnetic environment will vary, which informs the rate of dephasing. We qualitatively predict the dependence of dephasing on the nature of the environment, magnetic field, phase difference between the junction, and filling of the semiconductor. Comparing the simulated electric- and magnetic-noise-induced dephasing rate to experiment suggests that the dominant source of noise is magnetic. Moreover, by appropriately tuning these external parameters, we find sweet-spots at which we predict an enhancement in ASQ coherence times., Comment: 12 pages, 12 figures

Josephson junctions constructed from superconductor-quantum dot-superconductor (S-QD-S) heterostructures have been used to realize a variety of voltage-tunable superconducting quantum devices, including qubits and parametric amplifiers. In such devices, the interplay between the charge degree of freedom associated with the quantum dot and its environment must be considered for faithful modeling of circuit dynamics. Here we describe the self-consistent quantization of a capacitively-shunted S-QD-S junction via path-integral formulation. In the effective Hamiltonian, the Josephson potential for the Andreev bound states reproduces earlier results for static phase bias, whereas the charging energy term has new features: (i) the system's capacitance is renormalized by the junction gate voltage, an effect which depends on the strength of the tunneling rates between the dot and its superconducting leads as well, and (ii) an additional charge offset appears for asymmetric junctions. These results are important to understand future experiments and quantum devices incorporating S-QD-S junctions in arbitrary impedance environments.

Model selection for a given target task can be costly, as it may entail extensive annotation of the quality of outputs of different models. We introduce DiffUse, an efficient method to make an informed decision between candidate text generation models based on preference annotations. DiffUse reduces the required amount of annotations, thus saving valuable time and resources in performing evaluation. DiffUse intelligently selects instances by clustering embeddings that represent the semantic differences between model outputs. Thus, it is able to identify a subset of examples that are more informative for preference decisions. Our method is model-agnostic, and can be applied to any text generation model for selecting between models, prompts and configurations. Moreover, we propose a practical iterative approach for dynamically determining how many instances to annotate. In a series of experiments over hundreds of model pairs, we demonstrate that DiffUse can dramatically reduce the required number of annotations -- by up to 75% -- while maintaining high evaluation reliability., Comment: Accepted to ACL (main conference)

Monitored random circuits, consisting of alternating layers of entangling two-qubit gates and projective single-qubit measurements applied to some fraction $p$ of the qubits, have been a topic of recent interest. In particular, the resulting steady state exhibits a phase transition from highly correlated states with "volume-law" entanglement at $pp_{c}$. It is hard to access this transition experimentally, as it cannot be seen at the ensemble level. Naively, to observe it one must repeat the experiment until the set of measurement results repeats itself, with likelihood that is exponentially small in the number of measurements. To overcome this issue, we present a hybrid quantum-classical algorithm which creates a matrix product state (MPS) based "unitary mirror" of the projected circuit. Polynomial-sized tensor networks can represent quantum states with area-law entanglement, and so the unitary mirror can well-approximate the experimental state above $p_{c}$ but fails exponentially below it. The breaking of this mirror can thus pinpoint the critical point. We outline the algorithm and how such results would be obtained. We present a bound on the maximum entanglement entropy of any given state that is well-represented by an MPS, and from the bound suggest how the volume-law phase can be bounded. We consider whether the entanglement could similarly be bounded from below where the MPS fails. Finally, we present numerical results for small qubit numbers and for monitored circuits with random Clifford gates., Comment: 5 pages, 4 figures, supplemental material 8 pages

Understanding how and to what magnitude solid-state qubits couple to metallic wires is crucial to the design of quantum systems such as quantum computers. Here, we investigate the coupling between a multi-level system, or qudit, and a superconducting (SC) resonator's electromagnetic field, focusing on the interaction involving both the transition and diagonal dipole moments of the qudit. Specifically, we explore the effective dynamical (time-dependent) longitudinal coupling that arises when a solid-state qudit is adiabatically modulated at small gate frequencies and amplitudes, in addition to a static dispersive interaction with the SC resonator. For the first time, we derive Hamiltonians describing the longitudinal multi-level interactions in a general dispersive regime, encompassing both dynamical longitudinal and dispersive interactions. These Hamiltonians smoothly transition between their adiabatic values, where the couplings of the n-th level are proportional to the level's energy curvature concerning a qudit gate voltage, and the substantially larger dispersive values, which occur due to a resonant form factor. We provide several examples illustrating the transition from adiabatic to dispersive coupling in different qubit systems, including the charge (1e DQD) qubit, the transmon, the double quantum dot singlet-triplet qubit, and the triple quantum dot exchange-only qubit. In some of these qubits, higher energy levels play a critical role, particularly when their qubit's dipole moment is minimal or zero. For an experimentally relevant scenario involving a spin-charge qubit with magnetic field gradient coupled capacitively to a SC resonator, we showcase the potential of these interactions. They enable close-to-quantum-limited quantum non-demolition (QND) measurements and remote geometric phase gates, demonstrating their practical utility in quantum information processing., Comment: 16 pages, 3 figures

Joint measurements of two-Pauli observables are a powerful tool for both the control and protection of quantum information. By following a simple recipe for measurement choices, single- and two- qubit rotations using two-Pauli parity and single qubit measurements are guaranteed to be unitary whilst requiring only a single ancilla qubit. This language for measurement based quantum computing is shown to be directly applicable to encoded double quantum dot singlet-triplet spin qubits, by measuring spin-parity between dots from neighboring qubits. Along with exchange interaction, a complete, leakage free, measurement based gate set can be shown, up to a known Pauli correction. Both theoretically exact spin-parity measurements and experimentally demonstrated asymmetric spin-parity measurements are shown to be viable for achieving the proposed measurement based scheme, provided some extra leakage mitigating measurement steps. This new method of spin qubit control offers a leakage suppressed, low resource overhead implementation of a measurement-based control that is viable on current spin qubit processor devices.

The recently introduced Quantum Lego framework provides a powerful method for generating complex quantum error correcting codes (QECCs) out of simple ones. We gamify this process and unlock a new avenue for code design and discovery using reinforcement learning (RL). One benefit of RL is that we can specify \textit{arbitrary} properties of the code to be optimized. We train on two such properties, maximizing the code distance, and minimizing the probability of logical error under biased Pauli noise. For the first, we show that the trained agent identifies ways to increase code distance beyond naive concatenation, saturating the linear programming bound for CSS codes on 13 qubits. With a learning objective to minimize the logical error probability under biased Pauli noise, we find the best known CSS code at this task for $\lesssim 20$ qubits. Compared to other (locally deformed) CSS codes, including Surface, XZZX, and 2D Color codes, our $[[17,1,3]]$ code construction actually has \textit{lower} adversarial distance, yet better protects the logical information, highlighting the importance of QECC desiderata. Lastly, we comment on how this RL framework can be used in conjunction with physical quantum devices to tailor a code without explicit characterization of the noise model., Comment: 10 pages + appendices; v2 figure updated and note added