Search

Who ends up in the teacher workforce is greatly influenced by who is admitted into teacher education programs (TEPs). To better understand how the preferences of teacher education faculty might shape admissions of STEM teacher candidates, we surveyed faculty who teach content or methods courses to STEM teacher candidates across five universities. Faculty reported that they most value information collected from individual interviews with applicants and data on the number of STEM courses taken in college and their performance in these courses, and least value data on university admissions tests, high school GPA, and teacher licensure test scores. When we investigate faculty members' revealed preferences through a conjoint analysis, we find that faculty most value applicants who have worked with students from diverse backgrounds and applicants from a marginalized racial or ethnic community, and least value whether they received high grades in math and/or science courses. Finally, we find significant variation in these perceptions across respondents in different faculty roles, who teach different courses, and from different institutions: for example, Arts and Sciences faculty tend to value TEP applicants' performance in college STEM courses relatively more than STEM education faculty, while STEM education faculty tend to value applicants' race and ethnicity relatively more than Arts and Sciences faculty.

Large-scale multimodal representation learning successfully optimizes for zero-shot transfer at test time. Yet the standard pretraining paradigm (contrastive learning on large amounts of image-text data) does not explicitly encourage representations to support few-shot adaptation. In this work, we propose a simple, but carefully designed extension to multimodal pretraining which enables representations to accommodate additional context. Using this objective, we show that vision-language models can be trained to exhibit significantly increased few-shot adaptation: across 21 downstream tasks, we find up to four-fold improvements in test-time sample efficiency, and average few-shot adaptation gains of over 5%, while retaining zero-shot generalization performance across model scales and training durations. In particular, equipped with simple, training-free, metric-based adaptation mechanisms, our representations easily surpass more complex and expensive optimization-based schemes, vastly simplifying generalization to new domains.

Smart contracts form the core of Web3 applications. Contracts mediate the transfer of cryptocurrency, making them irresistible targets for hackers. We introduce ASP, a system aimed at easing the construction of provably secure contracts. The Asp system consists of three closely-linked components: a programming language, a defensive compiler, and a proof checker. The language semantics guarantee that Asp contracts are free of commonly exploited vulnerabilities such as arithmetic overflow and reentrancy. The defensive compiler enforces the semantics and translates Asp to Solidity, the most popular contract language. Deductive proofs establish functional correctness and freedom from critical vulnerabilities such as unauthorized access.

Generalist vision language models (VLMs) have made significant strides in computer vision, but they fall short in specialized fields like healthcare, where expert knowledge is essential. In traditional computer vision tasks, creative or approximate answers may be acceptable, but in healthcare, precision is paramount.Current large multimodal models like Gemini and GPT-4o are insufficient for medical tasks due to their reliance on memorized internet knowledge rather than the nuanced expertise required in healthcare. VLMs are usually trained in three stages: vision pre-training, vision-language pre-training, and instruction fine-tuning (IFT). IFT has been typically applied using a mixture of generic and healthcare data. In contrast, we propose that for medical VLMs, a fourth stage of specialized IFT is necessary, which focuses on medical data and includes information from domain expert models. Domain expert models developed for medical use are crucial because they are specifically trained for certain clinical tasks, e.g. to detect tumors and classify abnormalities through segmentation and classification, which learn fine-grained features of medical data$-$features that are often too intricate for a VLM to capture effectively especially in radiology. This paper introduces a new framework, VILA-M3, for medical VLMs that utilizes domain knowledge via expert models. Through our experiments, we show an improved state-of-the-art (SOTA) performance with an average improvement of ~9% over the prior SOTA model Med-Gemini and ~6% over models trained on the specific tasks. Our approach emphasizes the importance of domain expertise in creating precise, reliable VLMs for medical applications.

Simulating large, strongly interacting fermionic systems remains a major challenge for existing numerical methods. In this work, we present, for the first time, the application of neural quantum states - specifically, hidden fermion determinant states (HFDS) - to simulate the strongly interacting limit of the Fermi-Hubbard model, namely the $t-J$ model, across the entire doping regime. We demonstrate that HFDS achieve energies competitive with matrix product states (MPS) on lattices as large as $8 \times 8$ sites while using several orders of magnitude fewer parameters, suggesting the potential for efficient application to even larger system sizes. This remarkable efficiency enables us to probe low-energy physics across the full doping range, providing new insights into the competition between kinetic and magnetic interactions and the nature of emergent quasiparticles. Starting from the low-doping regime, where magnetic polarons dominate the low energy physics, we track their evolution with increasing doping through analyses of spin and polaron correlation functions. Our findings demonstrate the potential of determinant-based neural quantum states with inherent fermionic sign structure, opening the way for simulating large-scale fermionic systems at any particle filling.

Language model users often issue queries that lack specification, where the context under which a query was issued -- such as the user's identity, the query's intent, and the criteria for a response to be useful -- is not explicit. For instance, a good response to a subjective query like "What book should I read next?" would depend on the user's preferences, and a good response to an open-ended query like "How do antibiotics work against bacteria?" would depend on the user's expertise. This makes evaluation of responses to such queries an ill-posed task, as evaluators may make arbitrary judgments about the response quality. To remedy this, we present contextualized evaluations, a protocol that synthetically constructs context surrounding an underspecified query and provides it during evaluation. We find that the presence of context can 1) alter conclusions drawn from evaluation, even flipping win rates between model pairs, 2) nudge evaluators to make fewer judgments based on surface-level criteria, like style, and 3) provide new insights about model behavior across diverse contexts. Specifically, our procedure uncovers an implicit bias towards WEIRD contexts in models' "default" responses and we find that models are not equally sensitive to following different contexts, even when they are provided in prompts., Comment: Code & data available at https://github.com/allenai/ContextEval

Recent progress with CMOS detector development has opened new parameter space for high cadence time resolved imaging of transients and fast proper motion solar system objects. Using computer simulations for a ground-based 1.23 m telescope, this research note illustrates the gain of a new generation of fast readout low noise qCMOS sensors over CCDs and makes the case for high precision monitoring of asteroid orbits that can potentially shed light on the hypothetical existence of low mass primordial black holes, as well as for other applications requiring high speed imaging., Comment: 3 pages, 1 figure. Accepted by RNAAS

This article presents a generic hybrid numerical method to price a wide range of options on one or several assets, as well as assets with stochastic drift or volatility. In particular for equity and interest rate hybrid with local volatility.

How can we test AI performance? This question seems trivial, but it isn't. Standard benchmarks often have problems such as in-distribution and small-size test sets, oversimplified metrics, unfair comparisons, and short-term outcome pressure. As a consequence, good performance on standard benchmarks does not guarantee success in real-world scenarios. To address these problems, we present Touchstone, a large-scale collaborative segmentation benchmark of 9 types of abdominal organs. This benchmark is based on 5,195 training CT scans from 76 hospitals around the world and 5,903 testing CT scans from 11 additional hospitals. This diverse test set enhances the statistical significance of benchmark results and rigorously evaluates AI algorithms across various out-of-distribution scenarios. We invited 14 inventors of 19 AI algorithms to train their algorithms, while our team, as a third party, independently evaluated these algorithms on three test sets. In addition, we also evaluated pre-existing AI frameworks--which, differing from algorithms, are more flexible and can support different algorithms--including MONAI from NVIDIA, nnU-Net from DKFZ, and numerous other open-source frameworks. We are committed to expanding this benchmark to encourage more innovation of AI algorithms for the medical domain., Comment: Accepted to NeurIPS-2024

This work examines the hypothesis that intravoxel incoherent motion MRI (IVIM) can quantify local cerebral blood flow (qCBF), infarct volume, and define the ischemic penumbra for determination of the perfusion-diffusion mismatch (PWI/DWI) volume in a setting of acute ischemic stroke. Eight experiments were conducted in a pre-clinical middle cerebral artery occlusion (MCAO) model. IVIM and dynamic susceptibility contrast (DSC) imaging were acquired 2.5hr post-MCAO. IVIM was post-processed using software written in-house to produce parametric images of local qCBF, Water Transport Time (WTT), diffusion, and subsequently, PWI/DWI mismatch. These IVIM image parameters were compared with delay-and-dispersion-corrected local-AIF DSC perfusion image parameters including Tmax, qCBF, mean transit time (MTT), and mean diffusivity for DSC PWI/DWI mismatch. Final infarct volume was measured 4hrs post-occlusion. Early (2.5hr post-occlusion) DSC qCBF and IVIM qCBF in the diffusion negative MCA territory correlated strongly (slope=1.00, p=0.01,R2=0.69,Lins CCC=0.71), and both DSC and IVIM qCBF values negatively correlated with final infarct volume (R2=0.78,R2=0.61 respectively). The volume of hypoperfusion measured at 2.5 hours from DSC qCBF and from IVIM qCBF both predicted final infarct volume with good sensitivity and correlation (slope=2.08, R2=0.67, slope=2.50,R2=0.68 respectively). IVIM PWI/DWI ratio was correlated with infarct growth (R2=0.70) and WTT correlated with MTT (slope=0.82,R2=0.60). IVIM qCBF correlated strongly with local-AIF DSC qCBF and IVIM PWI/DWI correlated strongly with infarct growth. Both DSC and IVIM quantitative perfusion image acquired early after occlusion were able to predict final infarct volume, and IVIM simultaneous PWI/DWI ratio predicted infarct growth., Comment: Incomplete analysis

Active metamaterials are engineered structures that possess novel properties that can be changed after the point of manufacture. Their novel properties arise predominantly from their physical structure, as opposed to their chemical composition and can be changed through means such as direct energy addition into wave paths, or physically changing/morphing the structure in response to both a user or environmental input. Active metamaterials are currently of wide interest to the physics community and encompass a range of sub-domains in applied physics (e.g. photonic, microwave, acoustic, mechanical, etc.). They possess the potential to provide solutions that are more suitable to specific applications, or which allow novel properties to be produced which cannot be achieved with passive metamaterials, such as time-varying or gain enhancement effects. They have the potential to help solve some of the important current and future problems faced by the advancement of modern society, such as achieving net-zero, sustainability, healthcare and equality goals. Despite their huge potential, the added complexity of their design and operation, compared to passive metamaterials creates challenges to the advancement of the field, particularly beyond theoretical and lab-based experiments. This roadmap brings together experts in all types of active metamaterials and across a wide range of areas of applied physics. The objective is to provide an overview of the current state of the art and the associated current/future challenges, with the hope that the required advances identified create a roadmap for the future advancement and application of this field.

This paper presents the multiplicity of neutral and charged hadrons produced in 90 GeV$/c$ proton-carbon interactions from a dataset taken by the NA61/SHINE experiment in 2017. Particle identification via dE/dx was performed for the charged hadrons $\pi^\pm$, $K^\pm$, and $p / \bar{p}$; the neutral hadrons $K^0_S$, $\Lambda$, and $\bar{\Lambda}$ were identified via an invariant mass analysis of their decays to charged hadrons. Double-differential multiplicity results as a function of laboratory momentum and polar angle are presented for each particle species; these results provide vital constraints on the predicted neutrino beam flux for current and future long-baseline neutrino oscillation experiments.

Structured light plays an important role in metrology, optical trapping and manipulation, communications, quantum technologies, nonlinear optics and provides a rich playground for addressing new optical phenomena. Here we demonstrate a novel approach for manipulating vector vortex beams carrying longitudinal field components using metamaterials with extreme anisotropy. Implementing vectorial spectroscopy, we show that the propagation of complex beams with inhomogeneous polarisation is strongly affected by the interplay of the metamaterial anisotropy with the transverse and longitudinal field structure of the beam. This phenomenon is especially pronounced in the epsilon-near-zero regime, exclusively realised for light polarised along the metamaterial optical axis, strongly influencing the interaction of longitudinal fields with the metamaterial. The requirements on the balance between the transverse and longitudinal fields to maintain a polarisation singularity at the beam axis allow control of the beam modal content, filtering diffraction effects and tailoring spatial polarisation distribution. The proposed approach offers important capabilities for wavefront shaping as well as local spatial polarisation engineering. The understanding of the interaction of vector beams with metamaterials opens new opportunities for applications in microscopy, information encoding, biochemical sensing and quantum technologies.

With the ubiquity of Large Language Models (LLMs), guardrails have become crucial to detect and defend against toxic content. However, with the increasing pervasiveness of LLMs in multilingual scenarios, their effectiveness in handling multilingual toxic inputs remains unclear. In this work, we introduce a comprehensive multilingual test suite, spanning seven datasets and over ten languages, to benchmark the performance of state-of-the-art guardrails. We also investigates the resilience of guardrails against recent jailbreaking techniques, and assess the impact of in-context safety policies and language resource availability on guardrails' performance. Our findings show that existing guardrails are still ineffective at handling multilingual toxicity and lack robustness against jailbreaking prompts. This work aims to identify the limitations of guardrails and to build a more reliable and trustworthy LLMs in multilingual scenarios.

Melnick 34 (Mk 34) is one of the most massive binary systems known and is one of the brightest X-ray point sources in the 30 Doradus region. We investigated the impact of this massive system on the surrounding interstellar medium (ISM) using the optical spectroscopic capabilities of the narrow-field mode (NFM) of the Multi-Unit Spectroscopic Explorer (MUSE). MUSE-NFM spatially resolved the ISM in the vicinity of Mk 34 with a resolution comparable to that of the HST. The analysis of the [NII]$\lambda$6583 and [SII]$\lambda$6717 emission lines reveals a cone-like structure apparently originating from Mk 34 and extending southeast. Electron density maps and radial velocity measurements of the ISM lines further support an outflow scenario traced by these emissions. While no clear northwestern counterpart to this outflow was observed, we note increased extinction in that direction, towards the R136 cluster. The ISM material along the projected diagonal of the outflow on both sides of Mk 34 shows similar properties in terms of the emission line ratios seen in the Baldwin-Phillips-Terlevich diagram. These results are consistent across two observational epochs. Additionally, we examined the residual maps within a 0.5" radius of Mk 34 after modeling and subtracting the point spread function. The observed variations in the residuals could potentially be linked to Mk 34's known periodic behavior. However, further observations with appropriate cadence are needed to fully monitor the 155 day periodicity of Mk 34's X-ray emissions., Comment: 9 pages, 10 figures. Accepted in A&A

Existing math datasets evaluate the reasoning abilities of large language models (LLMs) by either using the final answer or the intermediate reasoning steps derived from static examples. However, the former approach fails to surface model's uses of shortcuts and wrong reasoning while the later poses challenges in accommodating alternative solutions. In this work, we seek to use symbolic programs as a means for automated evaluation if a model can consistently produce correct final answers across various inputs to the program. We begin by extracting programs for popular math datasets (GSM8K and MATH) using GPT4-o. For those executable programs verified using the original input-output pairs, they are found to encapsulate the proper reasoning required to solve the original text questions. We then prompt GPT4-o to generate new questions using alternative input-output pairs based the extracted program. We apply the resulting datasets to evaluate a collection of LLMs. In our experiments, we observe significant accuracy drops using our proposed evaluation compared with original static examples, suggesting the fragility of math reasoning in state-of-the-art LLMs.

We report the results from a pilot study to search for black holes and other dark companions in binary systems using direct imaging with SHARK-VIS and the iLocater pathfinder "Lili" on the Large Binocular Telescope. Starting from known single-lined spectroscopic binaries, we select systems with high mass functions that could host dark companions and whose spectroscopic orbits indicate a projected orbital separation $\geq 30$ mas. For this first exploration, we selected four systems (HD 137909, HD 104438, HD 117044, and HD 176695). In each case, we identify a luminous companion and measure the flux ratio and angular separation. However, two of the systems (HD 104438 and HD 176695) are not consistent with simple binary systems and are most likely hierarchical triples. The observed companions rule out a massive compact object for HD 137909, HD 117044, and HD 176695. HD 104438 requires further study because the identified star cannot be responsible for the RV orbit and is likely a dwarf tertiary companion. The SHARK-VIS observation was taken near pericenter, and a second image near apocenter is needed to discriminate between a closely separated luminous secondary and a compact object. We show how the combination of RVs and direct imaging can be used to constrain the orbital inclination and companion mass, and discuss the potential of high resolution direct imaging surveys to identify and confirm non-interacting compact object candidates., Comment: 23 pages, 11 figures. Submitted to ApJ

In medical image visualization, path tracing of volumetric medical data like CT scans produces lifelike three-dimensional visualizations. Immersive VR displays can further enhance the understanding of complex anatomies. Going beyond the diagnostic quality of traditional 2D slices, they enable interactive 3D evaluation of anatomies, supporting medical education and planning. Rendering high-quality visualizations in real-time, however, is computationally intensive and impractical for compute-constrained devices like mobile headsets. We propose a novel approach utilizing GS to create an efficient but static intermediate representation of CT scans. We introduce a layered GS representation, incrementally including different anatomical structures while minimizing overlap and extending the GS training to remove inactive Gaussians. We further compress the created model with clustering across layers. Our approach achieves interactive frame rates while preserving anatomical structures, with quality adjustable to the target hardware. Compared to standard GS, our representation retains some of the explorative qualities initially enabled by immersive path tracing. Selective activation and clipping of layers are possible at rendering time, adding a degree of interactivity to otherwise static GS models. This could enable scenarios where high computational demands would otherwise prohibit using path-traced medical volumes.

Video capsule endoscopy has transformed gastrointestinal endoscopy (GIE) diagnostics by offering a non-invasive method for capturing detailed images of the gastrointestinal tract, enabling early disease detection. However, its potential is limited by the sheer volume of images generated during the imaging procedure, which can take anywhere from 6-8 hours and often produce up to 1 million images, necessitating automated analysis. Additionally, the variability of these images, combined with the need for expert annotations and the scarcity of large, high-quality labeled datasets, constrains the effectiveness of current medical image analysis models. To address this, we introduce a novel large gastrointestinal endoscopy dataset, called EndoExtend24, created by merging and re-stratifying the train/test splits of ten existing public and private datasets, ensuring no overlap of patient data across splits. EndoExtend24 includes over 226,000 labeled images, as well as dynamic class mappings, which allow unified training across datasets with differing labeling granularity, supporting up to 123 distinct pathological findings. Further, we propose to leverage domain adaptive pre-training of foundation models in computer vision trained with self-supervision on generic image data, to adapt them to the task of GIE medical diagnosis. Specifically, the EVA-02 model, which is based on the vision transformer architecture and was trained on ImageNet-22k with masked image modeling (using EVA-CLIP as a MIM teacher), is pre-trained on the novel EndoExtend24 dataset to achieve domain adaptation, and finally trained on the Capsule Endoscopy 2024 Challenge dataset. Experimental results demonstrate strong performance with an F1 score of 0.88, an improvement of about 39% over the baseline model's F1 score of 0.49. Additionally, the model achieved a macro AUC score of 0.993 and a balanced accuracy of 89.3%.

We characterize two classical types of conformality of a holomorphic self-map of the unit disk at a boundary point - existence of a finite angular derivative in the sense of Carath\'eodory and the weaker property of angle preservation - in terms of the non-tangential asymptotic behaviour of the hyperbolic distortion of the map. These characterizations are given purely with reference to the intrinsic metric geometry of the unit disk. In particular, we relate the classical Julia-Wolff-Carath\'eodory theorem with the case of equality in the Schwarz-Pick lemma at the boundary. We also provide an operator-theoretic characterization of the existence of a finite angular derivative based on Hilbert space methods. As an application we study the backward dynamics of discrete dynamical systems induced by holomorphic self-maps, and characterize the regularity of the associated pre-models in terms of a Blaschke-type condition involving the hyperbolic distortion along regular backward orbits.

Open domain question answering systems frequently rely on information retrieved from large collections of text (such as the Web) to answer questions. However, such collections of text often contain conflicting information, and indiscriminately depending on this information may result in untruthful and inaccurate answers. To understand the gravity of this problem, we collect a human-annotated dataset, Question Answering with Conflicting Contexts (QACC), and find that as much as 25% of unambiguous, open domain questions can lead to conflicting contexts when retrieved using Google Search. We evaluate and benchmark three powerful Large Language Models (LLMs) with our dataset QACC and demonstrate their limitations in effectively addressing questions with conflicting information. To explore how humans reason through conflicting contexts, we request our annotators to provide explanations for their selections of correct answers. We demonstrate that by finetuning LLMs to explain their answers, we can introduce richer information into their training that guide them through the process of reasoning with conflicting contexts.

Computational text classification is a challenging task, especially for multi-dimensional social constructs. Recently, there has been increasing discussion that synthetic training data could enhance classification by offering examples of how these constructs are represented in texts. In this paper, we systematically examine the potential of theory-driven synthetic training data for improving the measurement of social constructs. In particular, we explore how researchers can transfer established knowledge from measurement instruments in the social sciences, such as survey scales or annotation codebooks, into theory-driven generation of synthetic data. Using two studies on measuring sexism and political topics, we assess the added value of synthetic training data for fine-tuning text classification models. Although the results of the sexism study were less promising, our findings demonstrate that synthetic data can be highly effective in reducing the need for labeled data in political topic classification. With only a minimal drop in performance, synthetic data allows for substituting large amounts of labeled data. Furthermore, theory-driven synthetic data performed markedly better than data generated without conceptual information in mind.

The sector of information and communication technology (ICT) can contribute to the fulfillment of the Paris agreement and the sustainable development goals (SDGs) through the introduction of sustainability strategies. For environmental sustainability, such strategies should contain efficiency, sufficiency, and consistency measures. To propose such, a structural analysis of ICT is undertaken in this manuscript. Thereby, key mechanisms and dynamics behind the usage of ICT and the corresponding energy and resource use are analyzed by describing ICT as a complex system. The system contains data centers, communication networks, smartphone hardware, apps, and the behavior of the users as sub-systems, between which various Morinian interactions are present. Energy and non-energy resources can be seen as inputs of the system, while e-waste is an output. Based on the system description, we propose multiple measures for efficiency, sufficiency and consistency to reduce greenhouse gas emissions and other environmental impacts.

Existing retrieval-based reasoning approaches for large language models (LLMs) heavily rely on the density and quality of the non-parametric knowledge source to provide domain knowledge and explicit reasoning chain. However, inclusive knowledge sources are expensive and sometimes infeasible to build for scientific or corner domains. To tackle the challenges, we introduce Graph Inspired Veracity Extrapolation (GIVE), a novel reasoning framework that integrates the parametric and non-parametric memories to enhance both knowledge retrieval and faithful reasoning processes on very sparse knowledge graphs. By leveraging the external structured knowledge to inspire LLM to model the interconnections among relevant concepts, our method facilitates a more logical and step-wise reasoning approach akin to experts' problem-solving, rather than gold answer retrieval. Specifically, the framework prompts LLMs to decompose the query into crucial concepts and attributes, construct entity groups with relevant entities, and build an augmented reasoning chain by probing potential relationships among node pairs across these entity groups. Our method incorporates both factual and extrapolated linkages to enable comprehensive understanding and response generation. Extensive experiments on reasoning-intense benchmarks on biomedical and commonsense QA demonstrate the effectiveness of our proposed method. Specifically, GIVE enables GPT3.5-turbo to outperform advanced models like GPT4 without any additional training cost, thereby underscoring the efficacy of integrating structured information and internal reasoning ability of LLMs for tackling specialized tasks with limited external resources.

Construction sites are challenging environments for autonomous systems due to their unstructured nature and the presence of dynamic actors, such as workers and machinery. This work presents a comprehensive panoptic scene understanding solution designed to handle the complexities of such environments by integrating 2D panoptic segmentation with 3D LiDAR mapping. Our system generates detailed environmental representations in real-time by combining semantic and geometric data, supported by Kalman Filter-based tracking for dynamic object detection. We introduce a fine-tuning method that adapts large pre-trained panoptic segmentation models for construction site applications using a limited number of domain-specific samples. For this use case, we release a first-of-its-kind dataset of 502 hand-labeled sample images with panoptic annotations from construction sites. In addition, we propose a dynamic panoptic mapping technique that enhances scene understanding in unstructured environments. As a case study, we demonstrate the system's application for autonomous navigation, utilizing real-time RRT* for reactive path planning in dynamic scenarios. The dataset (https://leggedrobotics.github.io/panoptic-scene-understanding.github.io/) and code (https://github.com/leggedrobotics/rsl_panoptic_mapping) for training and deployment are publicly available to support future research., Comment: 9 pages, 7 figures, 4 tables, submitted to 2024 Australasian Conference on Robotics and Automation (ACRA 2024)

While retrieval augmented generation (RAG) has been shown to enhance factuality of large language model (LLM) outputs, LLMs still suffer from hallucination, generating incorrect or irrelevant information. One common detection strategy involves prompting the LLM again to assess whether its response is grounded in the retrieved evidence, but this approach is costly. Alternatively, lightweight natural language inference (NLI) models for efficient grounding verification can be used at inference time. While existing pre-trained NLI models offer potential solutions, their performance remains subpar compared to larger models on realistic RAG inputs. RAG inputs are more complex than most datasets used for training NLI models and have characteristics specific to the underlying knowledge base, requiring adaptation of the NLI models to a specific target domain. Additionally, the lack of labeled instances in the target domain makes supervised domain adaptation, e.g., through fine-tuning, infeasible. To address these challenges, we introduce Automatic Generative Domain Adaptation (Auto-GDA). Our framework enables unsupervised domain adaptation through synthetic data generation. Unlike previous methods that rely on handcrafted filtering and augmentation strategies, Auto-GDA employs an iterative process to continuously improve the quality of generated samples using weak labels from less efficient teacher models and discrete optimization to select the most promising augmented samples. Experimental results demonstrate the effectiveness of our approach, with models fine-tuned on synthetic data using Auto-GDA often surpassing the performance of the teacher model and reaching the performance level of LLMs at 10 % of their computational cost.

Purpose: Segmentation of the breast region in dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) is essential for the automatic measurement of breast density and the quantitative analysis of imaging findings. This study aims to compare various deep learning methods to enhance whole breast segmentation and reduce computational costs as well as environmental effect for future research. Methods: We collected fifty-nine DCE-MRI scans from Stavanger University Hospital and, after preprocessing, analyzed fifty-eight scans. The preprocessing steps involved standardizing imaging protocols and resampling slices to ensure consistent volume across all patients. Using our novel approach, we defined new breast boundaries and generated corresponding segmentation masks. We evaluated seven deep learning models for segmentation namely UNet, UNet++, DenseNet, FCNResNet50, FCNResNet101, DeepLabv3ResNet50, and DeepLabv3ResNet101. To ensure robust model validation, we employed 10-fold cross-validation, dividing the dataset into ten subsets, training on nine, and validating on the remaining one, rotating this process to use all subsets for validation. Results: The models demonstrated significant potential across multiple metrics. UNet++ achieved the highest performance in Dice score, while UNet excelled in validation and generalizability. FCNResNet50, notable for its lower carbon footprint and reasonable inference time, emerged as a robust model following UNet++. In boundary detection, both UNet and UNet++ outperformed other models, with DeepLabv3ResNet also delivering competitive results.

The effectiveness of automatic evaluation of generative models is typically measured by comparing it to human evaluation using correlation metrics. However, metrics like Krippendorff's $\alpha$ and Randolph's $\kappa$, originally designed to measure the reliability of human labeling, make assumptions about human behavior and the labeling process. In this paper, we show how *relying on a single aggregate correlation score* can obscure fundamental differences between human behavior and automatic evaluation methods, including LLM-as-a-Judge. Specifically, we demonstrate that when the proportion of samples with variation or uncertainty in human labels (gathered during human evaluation) is relatively high, machine labels (generated by automatic evaluation methods) may superficially appear to have similar or better correlation with the human majority label compared to human-to-human (HH) correlation. This can create the misleading impression that automatic evaluation is accurate enough to approximate the human majority label. However, as the proportion of samples with consistent human labels increases, the correlation between machine labels and human majority labels declines, falling below HH correlation. Based on these findings, we first propose stratifying results by human label uncertainty to provide a more robust analysis of automatic evaluation performance. Second, recognizing that uncertainty and variation are inherent in perception-based human evaluations, such as those involving attitudes or preferences, we introduce a new metric - *binned Jensen-Shannon Divergence for perception* for such scenarios to better measure the effectiveness of automatic evaluations. Third, we present visualization techniques -- *perception charts*, to compare the strengths and limitations of automatic evaluation and to contextualize correlation measures appropriately

The property of linear discrete-time time-invariant system operators mapping inputs with at most $k-1$ sign changes to outputs with at $k-1$ sign changes is investigated. We show that this property is tractable via the notion of $k$-sign consistency in case of the observability/controllability operator, which as such can also be used as a sufficient condition for the Hankel operator. Our results complement the literature in several aspects: an algebraic characterization, independent of rank and dimension, is provided for variation bounding and diminishing matrices and their computational tractability is discussed. Based on these, we conduct our studies of variation bounding system operators beyond existing studies on order-preserving $k$-variation diminishment. Our results are applied to the open problem of bounding the number of sign changes in a system's impulse response.

For cellular functions like division and polarization, protein pattern formation driven by NTPase cycles is a central spatial control strategy. Operating far from equilibrium, no general theory links microscopic reaction networks and parameters to the pattern type and dynamics. We discover a generic mechanism giving rise to an effective interfacial tension organizing the macroscopic structure of non-equilibrium steady-state patterns. Namely, maintaining protein-density interfaces by cyclic protein attachment and detachment produces curvature-dependent protein redistribution which straightens the interface. We develop a non-equilibrium Neumann angle law and Plateau vertex conditions for interface junctions and mesh patterns, thus introducing the concepts of ``Turing mixtures'' and ``Turing foams''. In contrast to liquid foams and mixtures, these non-equilibrium patterns can select an intrinsic wavelength by interrupting an equilibrium-like coarsening process. Data from in vitro experiments with the E. coli Min protein system verifies the vertex conditions and supports the wavelength dynamics. Our study uncovers interface laws with correspondence to thermodynamic relations that arise from distinct physical processes in active systems. It allows the design of specific pattern morphologies with potential applications as spatial control strategies in synthetic cells., Comment: 11 pages main text, 5 pages Methods, 64 pages Supplementary Information; 12 figures

Calibration of quantum devices is fundamental to successfully deploy quantum algorithms on current available quantum hardware. We present Qibocal, an open-source software library to perform calibration and characterization of superconducting quantum devices within the Qibo framework. Qibocal completes the Qibo middleware framework by providing all necessary tools to easily (re)calibrate self-hosted quantum platforms. After presenting the layout and the features of the library, we give an overview on some of the protocols implemented to perform single and two-qubit gates calibration. Finally, we present applications involving recalibration and monitoring of superconducting platforms., Comment: 12 pages, 9 figures, code available at https://github.com/qiboteam/qibocal

Patchy particles are an intriguing subject of study and indeed a model system in the field of soft matter physics. In recent years, patchy particle models have been applied to describe a wide variety of systems, including colloidal crystals, macromolecular interactions, liquid crystals, and nanoparticle assemblies. Given the importance of the topic, rationalizing and capturing the basic features of these models is crucial to their correct application in specific systems. In this study, we extend the ion-activated attractive patchy particles model previously employed to elucidate the phase behavior of protein solutions in the presence of trivalent salts. Our extension incorporates the effect of repulsion between unoccupied and occupied binding sites, depicted as patches. Furthermore, we examine the influence of model parameters on the liquid-vapor coexistence region within the phase diagram, employing numerical methods. A deeper understanding of this model will facilitate a better comprehension of the effects observed in experiments., Comment: 10 pages, 8 Figures

Advancements in 3D rendering like Gaussian Splatting (GS) allow novel view synthesis and real-time rendering in virtual reality (VR). However, GS-created 3D environments are often difficult to edit. For scene enhancement or to incorporate 3D assets, segmenting Gaussians by class is essential. Existing segmentation approaches are typically limited to certain types of scenes, e.g., ''circular'' scenes, to determine clear object boundaries. However, this method is ineffective when removing large objects in non-''circling'' scenes such as large outdoor scenes. We propose Semantics-Controlled GS (SCGS), a segmentation-driven GS approach, enabling the separation of large scene parts in uncontrolled, natural environments. SCGS allows scene editing and the extraction of scene parts for VR. Additionally, we introduce a challenging outdoor dataset, overcoming the ''circling'' setup. We outperform the state-of-the-art in visual quality on our dataset and in segmentation quality on the 3D-OVS dataset. We conducted an exploratory user study, comparing a 360-video, plain GS, and SCGS in VR with a fixed viewpoint. In our subsequent main study, users were allowed to move freely, evaluating plain GS and SCGS. Our main study results show that participants clearly prefer SCGS over plain GS. We overall present an innovative approach that surpasses the state-of-the-art both technically and in user experience.

Previous studies have found that localized strengthening of the f mode recedes the emergence of active regions on the Sun by one day to three days. To help interpret these observations, we have performed nonlinear simulations of convection with imposed magnetic fields at different depths. We find that the f mode is strengthened when a super-equipartition magnetic field is imposed near the top of the domain. However, neither a magnetic field of equal strength near the bottom of the domain nor an equipartition magnetic field near the top of the domain have a significant effect. Our results suggest that the magnetic precursors of active regions are present near the surface of the Sun for much longer than would be expected if active regions were formed by flux tubes rising from deep within the convection zone. Application to observations should account for the fact that the effects we observe are transient., Comment: 12 pages, 8 figures, 2 tables

LLMs can now act as autonomous agents that interact with digital environments and complete specific objectives (e.g., arranging an online meeting). However, accuracy is still far from satisfactory, partly due to a lack of large-scale, direct demonstrations for digital tasks. Obtaining supervised data from humans is costly, and automatic data collection through exploration or reinforcement learning relies on complex environmental and content setup, resulting in datasets that lack comprehensive coverage of various scenarios. On the other hand, there is abundant knowledge that may indirectly assist task completion, such as online tutorials that were created for human consumption. In this work, we present Synatra, an approach that effectively transforms this indirect knowledge into direct supervision at scale. We define different types of indirect knowledge, and carefully study the available sources to obtain it, methods to encode the structure of direct demonstrations, and finally methods to transform indirect knowledge into direct demonstrations. We use 100k such synthetically-created demonstrations to finetune a 7B CodeLlama, and demonstrate that the resulting agent surpasses all comparably sized models on three web-based task benchmarks Mind2Web, MiniWoB++ and WebArena, as well as surpassing GPT-3.5 on WebArena and Mind2Web. In addition, while synthetic demonstrations prove to be only 3% the cost of human demonstrations (at $0.031 each), we show that the synthetic demonstrations can be more effective than an identical number of human demonstrations collected from limited domains.

Large Language Models (LLMs) have the promise to revolutionize computing broadly, but their complexity and extensive training data also expose significant privacy vulnerabilities. One of the simplest privacy risks associated with LLMs is their susceptibility to membership inference attacks (MIAs), wherein an adversary aims to determine whether a specific data point was part of the model's training set. Although this is a known risk, state of the art methodologies for MIAs rely on training multiple computationally costly shadow models, making risk evaluation prohibitive for large models. Here we adapt a recent line of work which uses quantile regression to mount membership inference attacks; we extend this work by proposing a low-cost MIA that leverages an ensemble of small quantile regression models to determine if a document belongs to the model's training set or not. We demonstrate the effectiveness of this approach on fine-tuned LLMs of varying families (OPT, Pythia, Llama) and across multiple datasets. Across all scenarios we obtain comparable or improved accuracy compared to state of the art shadow model approaches, with as little as 6% of their computation budget. We demonstrate increased effectiveness across multi-epoch trained target models, and architecture miss-specification robustness, that is, we can mount an effective attack against a model using a different tokenizer and architecture, without requiring knowledge on the target model.

We use a novel real-time formulation of the functional renormalization group (FRG) for dynamical systems with reversible mode couplings to study Model H, the conjectured dynamic universality class of the QCD critical point. We emphasize the structural similarities with Model G, conjectured to be the dynamic universality class of the chiral phase transition in the limit of two massless quark flavors. Importantly, our formulation of the real-time FRG preserves all relevant symmetries throughout the FRG flow which, e.g., guarantees the non-renormalization of the reversible mode couplings, as but one exact result. We derive non-perturbative RG flow equations for the kinetic coefficients of both, Model G and H, in parallel, and discuss commonalities and differences in the resulting fixed-point structure and dynamic critical exponents, such as weak-scaling relations which hold in either case versus the characteristic strong scaling of Model G which is absent in Model H., Comment: 8 pages, 1 figure

We investigate the complexity of parameterised holant problems $\textsc{p-Holant}(\mathcal{S})$ for families of signatures~$\mathcal{S}$. The parameterised holant framework was introduced by Curticapean in 2015 as a counter-part to the classical theory of holographic reductions and algorithms and it constitutes an extensive family of coloured and weighted counting constraint satisfaction problems on graph-like structures, encoding as special cases various well-studied counting problems in parameterised and fine-grained complexity theory such as counting edge-colourful $k$-matchings, graph-factors, Eulerian orientations or, subgraphs with weighted degree constraints. We establish an exhaustive complexity trichotomy along the set of signatures $\mathcal{S}$: Depending on $\mathcal{S}$, $\textsc{p-Holant}(\mathcal{S})$ is: (1) solvable in FPT-near-linear time (i.e. $f(k)\cdot \tilde{\mathcal{O}}(|x|)$); (2) solvable in "FPT-matrix-multiplication time" (i.e. $f(k)\cdot {\mathcal{O}}(n^{\omega})$) but not solvable in FPT-near-linear time unless the Triangle Conjecture fails; or (3) #W[1]-complete and no significant improvement over brute force is possible unless ETH fails. This classification reveals a significant and surprising gap in the complexity landscape of parameterised Holants: Not only is every instance either fixed-parameter tractable or #W[1]-complete, but additionally, every FPT instance is solvable in time $f(k)\cdot {\mathcal{O}}(n^{\omega})$. We also establish a complete classification for a natural uncoloured version of parameterised holant problem $\textsc{p-UnColHolant}(\mathcal{S})$, which encodes as special cases the non-coloured analogues of the aforementioned examples. We show that the complexity of $\textsc{p-UnColHolant}(\mathcal{S})$ is different: Depending on $\mathcal{S}$ all instances are either solvable in FPT-near-linear time, or #W[1]-complete.

Jupiter's icy moon Ganymede has a tenuous exosphere produced by sputtering and possibly sublimation of water ice. To date, only atomic hydrogen and oxygen have been directly detected in this exosphere. Here, we present observations of Ganymede's CO$_2$ exosphere obtained with the James Webb Space Telescope. CO$_2$ gas is observed over different terrain types, mainly over those exposed to intense Jovian plasma irradiation, as well as over some bright or dark terrains. Despite warm surface temperatures, the CO$_2$ abundance over equatorial subsolar regions is low. CO$_2$ vapor has the highest abundance over the north polar cap of the leading hemisphere, reaching a surface pressure of 1 pbar. From modeling we show that the local enhancement observed near 12 h local time in this region can be explained by the presence of cold traps enabling CO$_2$ adsorption. However, whether the release mechanism in this high-latitude region is sputtering or sublimation remains unclear. The north polar cap of the leading hemisphere also has unique surface-ice properties, probably linked to the presence of the large atmospheric CO2 excess over this region. These CO2 molecules might have been initially released in the atmosphere after the radiolysis of CO$_2$ precursors, or from the sputtering of CO$_2$ embedded in the H$_2$O ice bedrock. Dark terrains (regiones), more widespread on the north versus south polar regions, possibly harbor CO$_2$ precursors. CO$_2$ molecules would then be redistributed via cold trapping on ice-rich terrains of the polar cap and be diurnally released and redeposited on these terrains. Ganymede's CO$_2$ exosphere highlights the complexity of surface-atmosphere interactions on Jupiter's icy Galilean moons., Comment: 21 pages, 21 figures, Accepted as a Letter in Astronomy and Astrophysics

The ability of message-passing neural networks (MPNNs) to fit complex functions over graphs is limited each iteration of message-passing over a simple makes representations more similar, a phenomenon known as rank collapse, and over-smoothing as a special case. Most approaches to mitigate over-smoothing extend common message-passing schemes, e.g., the graph convolutional network, by utilizing residual connections, gating mechanisms, normalization, or regularization techniques. Our work contrarily proposes to directly tackle the cause of this issue by modifying the message-passing scheme and exchanging different types of messages using multi-relational graphs. We identify the necessary and sufficient condition to ensure linearly independent node representations. As one instantion, we show that operating on multiple directed acyclic graphs always satisfies our condition and propose to obtain these by defining a strict partial ordering of the nodes. We conduct comprehensive experiments that confirm the benefits of operating on multi-relational graphs to achieve more informative node representations.

Large language models (LLMs) have demonstrated remarkable performance across various real-world tasks. However, they often struggle to fully comprehend and effectively utilize their input contexts, resulting in responses that are unfaithful or hallucinated. This difficulty increases for contexts that are long or contain distracting information, which can divert LLMs from fully capturing essential evidence. To address this issue, many works use prompting to help LLMs utilize contextual information more faithfully. For instance, iterative prompting highlights key information in two steps that first ask the LLM to identify important pieces of context and then derive answers accordingly. However, prompting methods are constrained to highlighting key information implicitly in token space, which is often insufficient to fully steer the model's attention. To improve model faithfulness more reliably, we propose AutoPASTA, a method that automatically identifies key contextual information and explicitly highlights it by steering an LLM's attention scores. Like prompting, AutoPASTA is applied at inference time and does not require changing any model parameters. Our experiments on open-book QA demonstrate that AutoPASTA effectively enables models to grasp essential contextual information, leading to substantially improved model faithfulness and performance, e.g., an average improvement of 7.95% for LLAMA3-70B-Instruct. Code will be publicly available at https://github.com/QingruZhang/AutoPASTA ., Comment: 12 pages, 4 figures

Speech language models have recently demonstrated great potential as universal speech processing systems. Such models have the ability to model the rich acoustic information existing in audio signals, beyond spoken content, such as emotion, background noise, etc. Despite this, evaluation benchmarks which evaluate awareness to a wide range of acoustic aspects, are lacking. To help bridge this gap, we introduce SALMon, a novel evaluation suite encompassing background noise, emotion, speaker identity and room impulse response. The proposed benchmarks both evaluate the consistency of the inspected element and how much it matches the spoken text. We follow a modelling based approach, measuring whether a model gives correct samples higher scores than incorrect ones. This approach makes the benchmark fast to compute even for large models. We evaluated several speech language models on SALMon, thus highlighting the strengths and weaknesses of each evaluated method. Code and data are publicly available at https://pages.cs.huji.ac.il/adiyoss-lab/salmon/ .

The osmotic second virial coefficient B2 is an important parameter to describe the interactions and phase behavior of protein solutions, including colloidal systems and macromolecular solutions. Another key parameter to describe the driving force of the nucleation of a new phase is the supersaturation, which is used in the classical nucleation theory framework and is connected with the favorable contribution in the Gibbs free energy in the bulk solution. In this article, we establish a connection between B2 calculated from small angle Xray scattering (SAXS) data and the values of B2 obtained from supersaturation measurements using thermodynamics considerations. The values of the second virial coefficient calculated employing this method agree with those determined via SAXS in the region near the liquid liquid phase separation border for human serum albumin and bovine serum albumin. The general relations adopted are shown to be useful for the estimation of the second virial coefficient B2 for globular proteins, in the proximity of the binodal biphasic coexistent region., Comment: 9 pages, 3 figures

We investigate the phase ordering (pattern formation) of systems of two-dimensional core-shell particles using Monte-Carlo (MC) computer simulations and classical density functional theory (DFT). The particles interact via a pair potential having a hard core and a repulsive square shoulder. Our simulations show that on cooling, the liquid state structure becomes increasingly characterised by long wavelength density modulations, and on further cooling forms a variety of other phases, including clustered, striped and other patterned phases. In DFT, the hard core part of the potential is treated using either fundamental measure theory or a simple local density approximation, whereas the soft shoulder is treated using the random phase approximation. The different DFTs are bench-marked using large-scale grand-canonical-MC and Gibbs-ensemble-MC simulations, demonstrating their predictive capabilities and shortcomings. We find that having the liquid state static structure factor $S(k)$ for wavenumber $k$ is sufficient to identify the Fourier modes governing both the liquid and solid phases. This allows to identify from easier-to-obtain liquid state data the wavenumbers relevant to the periodic phases and to predict roughly where in the phase diagram these patterned phases arise., Comment: 42 pages, 9 figures

Comet C/2017 K2 (Pan-STARRS) provided a rare opportunity to investigate the evolution of coma composition and outgassing patterns over a transitional heliocentric distance (Rh) range where activity drivers in comets are thought to change from "hypervolatile" (CO, CH$_4$, C$_2$H$_6$, and/or CO$_2$)-dominated to H2O-dominated. We performed high-resolution, cross-dispersed, near-infrared spectroscopy of C/2017 K2 with iSHELL at the NASA Infrared Telescope Facility (IRTF) and NIRSPEC at the W. M. Keck 2 Observatory. We report gas rotational temperatures (Trot) and molecular production rates (Q; mol/s) or upper limits for the hypervolatile species CH$_4$, CO, and C$_2$H$_6$, together with less volatile ices (CH$_3$OH, H$_2$O, HCN, C$_2$H$_2$, NH$_3$, and OCS) over a range of pre-perihelion distances, Rh= 3.15 - 2.35 au. We also report (or stringently constrain) abundance ratios (mixing ratios) of the targeted species with respect to CO, C$_2$H$_6$, and (when detected) H$_2$O. All volatiles were enriched relative to water in C/2017 K2 when compared to their mean values among Oort Cloud comets, whereas abundances relative to C2H6 were consistent with their average values from other long-period comets., Comment: Submitted to the Astronomical Journal for publication

46P/Wirtanen is a Jupiter-family comet, probably originating from the Solar System's Kuiper belt, that now resides on a 5.4 year elliptical orbit. During its 2018 apparition, comet 46P passed unusually close to the Earth (within 0.08 au), presenting an outstanding opportunity for close-up observations of its inner coma. Here we present observations of HCN, H$^{13}$CN and HC$^{15}$N emission from 46P using the Atacama Compact Array (ACA). The data were analyzed using the SUBLIME non-LTE radiative transfer code to derive $^{12}$C/$^{13}$C and $^{14}$N/$^{15}$N ratios. The HCN/H$^{13}$CN ratio is found to be consistent with a lack of significant $^{13}$C fractionation, whereas the HCN/HC$^{15}$N ratio of $68\pm27$ (using our most conservative $1\sigma$ uncertainties), indicates a strong enhancement in $^{15}$N compared with the solar and terrestrial values. The observed $^{14}$N/$^{15}$N ratio is also significantly lower than the values of $\sim140$ found in previous comets, implying a strong $^{15}$N enrichment in 46P's HCN. This indicates that the nitrogen in Jupiter-family comets could reach larger isotopic enrichments than previously thought, with implications for the diversity of $^{14}$N/$^{15}$N ratios imprinted into icy bodies at the birth of the Solar System., Comment: Accepted for publication in PSJ, September 2024

We study games in which a leader makes a single commitment, and then multiple followers (each with a different utility function) respond. In particular, we study ambiguous commitment strategies in these games, in which the leader may commit to a set of mixed strategies, and ambiguity-averse followers respond to maximize their worst-case utility over the set of leader strategies. Special cases of this setting have previously been studied when there is a single follower: in these cases, it is known that the leader can increase her utility by making an ambiguous commitment if the follower is restricted to playing a pure strategy, but that no gain can be had from ambiguity if the follower may mix. We confirm that this result continues to hold in the setting of general Stackelberg games. We then develop a theory of ambiguous commitment in games with multiple followers. We begin by considering the case where the leader must make the same commitment against each follower. We establish that -- unlike the case of a single follower -- ambiguous commitment can improve the leader's utility by an unboundedly large factor, even when followers are permitted to respond with mixed strategies and even. We go on to show an advantage for the leader coupling the same commitment across all followers, even when she has the ability to make a separate commitment to each follower. In particular, there exist general sum games in which the leader can enjoy an unboundedly large advantage by coupling her ambiguous commitment across multiple followers rather than committing against each individually. In zero-sum games we show there can be no such coupling advantage. Finally, we give a polynomial time algorithm for computing the optimal leader commitment strategy in the special case in which the leader has 2 actions (and k followers may have m actions), and prove that in the general case, the problem is NP-hard.

Since the entry of kernel theory in the field of quantum machine learning, quantum kernel methods (QKMs) have gained increasing attention with regard to both probing promising applications and delivering intriguing research insights. Two common approaches for computing the underlying Gram matrix have emerged: fidelity quantum kernels (FQKs) and projected quantum kernels (PQKs). Benchmarking these methods is crucial to gain robust insights and to understand their practical utility. In this work, we present a comprehensive large-scale study examining QKMs based on FQKs and PQKs across a manifold of design choices. Our investigation encompasses both classification and regression tasks for five dataset families and 64 datasets, systematically comparing the use of FQKs and PQKs quantum support vector machines and kernel ridge regression. This resulted in over 20,000 models that were trained and optimized using a state-of-the-art hyperparameter search to ensure robust and comprehensive insights. We delve into the importance of hyperparameters on model performance scores and support our findings through rigorous correlation analyses. In this, we also closely inspect two data encoding strategies. Moreover, we provide an in-depth analysis addressing the design freedom of PQKs and explore the underlying principles responsible for learning. Our goal is not to identify the best-performing model for a specific task but to uncover the mechanisms that lead to effective QKMs and reveal universal patterns., Comment: 18 pages main text including 12 figures and 1 table, appendix 14 pages with 19 figures and 1 table; restructure result section and prune appendix

To reduce the need for human annotations, large language models (LLMs) have been proposed as judges of the quality of other candidate models. LLM judges are typically evaluated by measuring the correlation with human judgments on generation tasks such as summarization or machine translation. In contrast, we study LLM judges on mathematical reasoning tasks. These tasks require multi-step reasoning, and the correctness of their solutions is verifiable, enabling a more objective evaluation. We perform a detailed performance analysis and find that the used judges are mostly unable to improve task performance but are able to pick the better model. Our analysis uncovers a strong correlation between judgment performance and the candidate model task performance. We observe that judges tend to choose the model of higher quality even if its answer is incorrect. Further, we show that it is possible to use statistics, such as the task performances of the individual models, to predict judgment performance. In an ablation, we either swap or mask the candidate answers and observe that judges often keep the original judgment, providing evidence that judges incorporate writing style in their judgments. In summary, we find that regularities in the judgments are quantifiable using statistical measures and provide various angles on exploiting them.

There has been substantial recent concern that pricing algorithms might learn to ``collude.'' Supra-competitive prices can emerge as a Nash equilibrium of repeated pricing games, in which sellers play strategies which threaten to punish their competitors who refuse to support high prices, and these strategies can be automatically learned. In fact, a standard economic intuition is that supra-competitive prices emerge from either the use of threats, or a failure of one party to optimize their payoff. Is this intuition correct? Would preventing threats in algorithmic decision-making prevent supra-competitive prices when sellers are optimizing for their own revenue? No. We show that supra-competitive prices can emerge even when both players are using algorithms which do not encode threats, and which optimize for their own revenue. We study sequential pricing games in which a first mover deploys an algorithm and then a second mover optimizes within the resulting environment. We show that if the first mover deploys any algorithm with a no-regret guarantee, and then the second mover even approximately optimizes within this now static environment, monopoly-like prices arise. The result holds for any no-regret learning algorithm deployed by the first mover and for any pricing policy of the second mover that obtains them profit at least as high as a random pricing would -- and hence the result applies even when the second mover is optimizing only within a space of non-responsive pricing distributions which are incapable of encoding threats. In fact, there exists a set of strategies, neither of which explicitly encode threats that form a Nash equilibrium of the simultaneous pricing game in algorithm space, and lead to near monopoly prices. This suggests that the definition of ``algorithmic collusion'' may need to be expanded, to include strategies without explicitly encoded threats.