Search

Sparse autoencoders (SAEs) decompose language model representations into a sparse set of linear latent vectors. Recent works have improved SAEs using language model gradients, but these techniques require many expensive backward passes during training and still cause a significant increase in cross entropy loss when SAE reconstructions are inserted into the model. In this work, we improve on these limitations by taking a fundamentally different approach: we use low-rank adaptation (LoRA) to finetune the language model itself around a previously trained SAE. We analyze our method across SAE sparsity, SAE width, language model size, LoRA rank, and model layer on the Gemma Scope family of SAEs. In these settings, our method reduces the cross entropy loss gap by 30% to 55% when SAEs are inserted during the forward pass. We also find that compared to end-to-end (e2e) SAEs, our approach achieves the same downstream cross entropy loss 3$\times$ to 20$\times$ faster on Gemma-2-2B and 2$\times$ to 10$\times$ faster on Llama-3.2-1B. We further show that our technique improves downstream metrics and can adapt multiple SAEs at once. Our results demonstrate that improving model interpretability is not limited to post-hoc SAE training; Pareto improvements can also be achieved by directly optimizing the model itself., Comment: Code available at https://github.com/matchten/LoRA-Models-for-SAEs

Machine unlearning seeks to systematically remove specified data from a trained model, effectively achieving a state as though the data had never been encountered during training. While metrics such as Unlearning Accuracy (UA) and Membership Inference Attack (MIA) provide a baseline for assessing unlearning performance, they fall short of evaluating the completeness and reliability of forgetting. This is because the ground truth labels remain potential candidates within the scope of uncertainty quantification, leaving gaps in the evaluation of true forgetting. In this paper, we identify critical limitations in existing unlearning metrics and propose enhanced evaluation metrics inspired by conformal prediction. Our metrics can effectively capture the extent to which ground truth labels are excluded from the prediction set. Furthermore, we observe that many existing machine unlearning methods do not achieve satisfactory forgetting performance when evaluated with our new metrics. To address this, we propose an unlearning framework that integrates conformal prediction insights into Carlini & Wagner adversarial attack loss. Extensive experiments on the image classification task demonstrate that our enhanced metrics offer deeper insights into unlearning effectiveness, and that our unlearning framework significantly improves the forgetting quality of unlearning methods.

Precision Medicine (PM) transforms the traditional "one-drug-fits-all" paradigm by customising treatments based on individual characteristics, and is an emerging topic for HCI research on digital health. A key element of PM, the Polygenic Risk Score (PRS), uses genetic data to predict an individual's disease risk. Despite its potential, PRS faces barriers to adoption, such as data inclusivity, psychological impact, and public trust. We conducted a mixed-methods study to explore how people perceive PRS, formed of surveys (n=254) and interviews (n=11) with UK-based participants. The interviews were supplemented by interactive storyboards with the ContraVision technique to provoke deeper reflection and discussion. We identified ten key barriers and five themes to PRS adoption and proposed design implications for a responsible PRS framework. To address the complexities of PRS and enhance broader PM practices, we introduce the term Human-Precision Medicine Interaction (HPMI), which integrates, adapts, and extends HCI approaches to better meet these challenges., Comment: This paper is conditionally accepted for publication at the SIGCHI Conference on Human Factors in Computing Systems (CHI 2025)

In this work, we study the propagation and spin oscillations of neutrinos in their scattering by a supermassive black hole (SMBH) surrounded by a realistic accretion disk. We use a semi-analytical model of a thick accretion disk which can co-rotate and counter-rotate with respect to BH. Moreover, we assume that a disk contains only a toroidal magnetic magnetic field of moderate strength. Spin precession of neutrinos, which are supposed to be Dirac particles, is caused by the interaction of the neutrino magnetic moment with the magnetic field in the disk. We consider the incoming flux of neutrinos having an arbitrary angle with respect to the BH spin since the recent results of the Event Horizon Telescope indicate that the BH spin in the galactic center is not always perpendicular to the galactic plane. For our study, we consider a large number of incoming test neutrinos. We briefly discuss our results and their applications in the observations of astrophysical neutrinos., Comment: 34 pages, 20 figures

We consider the homogeneous mean-field Bose gas at positive temperature. We show that spontaneous $U(1)$ symmetry breaking occurs if and only if the system displays Bose-Einstein condensation in the sense that the one-particle density matrix of the Gibbs state has a macroscopic eigenvalue.

We study the problem of piecewise stationary bandits without prior knowledge of the underlying non-stationarity. We propose the first $\textit{feasible}$ black-box algorithm applicable to most common parametric bandit variants. Our procedure, termed Detection Augmented Bandit (DAB), is modular, accepting any stationary bandit algorithm as input and augmenting it with a change detector. DAB achieves optimal regret in the piecewise stationary setting under mild assumptions. Specifically, we prove that DAB attains the order-optimal regret bound of $\tilde{\mathcal{O}}(\sqrt{N_T T})$, where $N_T$ denotes the number of changes over the horizon $T$, if its input stationary bandit algorithm has order-optimal stationary regret guarantees. Applying DAB to different parametric bandit settings, we recover recent state-of-the-art results. Notably, for self-concordant bandits, DAB achieves optimal dynamic regret, while previous works obtain suboptimal bounds and require knowledge on the non-stationarity. In simulations on piecewise stationary environments, DAB outperforms existing approaches across varying number of changes. Interestingly, despite being theoretically designed for piecewise stationary environments, DAB is also effective in simulations in drifting environments, outperforming existing methods designed specifically for this scenario., Comment: 13 pages, 5 figures

In-Context Reinforcement Learning (ICRL) represents a promising paradigm for developing generalist agents that learn at inference time through trial-and-error interactions, analogous to how large language models adapt contextually, but with a focus on reward maximization. However, the scalability of ICRL beyond toy tasks and single-domain settings remains an open challenge. In this work, we present the first steps toward scaling ICRL by introducing a fixed, cross-domain model capable of learning behaviors through in-context reinforcement learning. Our results demonstrate that Algorithm Distillation, a framework designed to facilitate ICRL, offers a compelling and competitive alternative to expert distillation to construct versatile action models. These findings highlight the potential of ICRL as a scalable approach for generalist decision-making systems. Code to be released at https://github.com/dunnolab/vintix, Comment: Preprint. In review

Large language model-based (LLM-based) agents have become common in settings that include non-cooperative parties. In such settings, agents' decision-making needs to conceal information from their adversaries, reveal information to their cooperators, and infer information to identify the other agents' characteristics. To investigate whether LLMs have these information control and decision-making capabilities, we make LLM agents play the language-based hidden-identity game, The Chameleon. In the game, a group of non-chameleon agents who do not know each other aim to identify the chameleon agent without revealing a secret. The game requires the aforementioned information control capabilities both as a chameleon and a non-chameleon. The empirical results show that while non-chameleon LLM agents identify the chameleon, they fail to conceal the secret from the chameleon, and their winning probability is far from the levels of even trivial strategies. To formally explain this behavior, we give a theoretical analysis for a spectrum of strategies, from concealing to revealing, and provide bounds on the non-chameleons' winning probability. Based on the empirical results and theoretical analysis of different strategies, we deduce that LLM-based non-chameleon agents reveal excessive information to agents of unknown identities. Our results point to a weakness of contemporary LLMs, including GPT-4, GPT-4o, Gemini 1.5, and Claude 3.5 Sonnet, in strategic interactions.

The maximum element of the vector output by the Softmax function approaches zero as the input vector size increases. Transformer-based language models rely on Softmax to compute attention scores, causing the attention distribution to flatten as the context size grows. This reduces the model's ability to prioritize key information effectively and potentially limits its length generalization. To address this problem, we propose Scalable-Softmax (SSMax), which replaces Softmax in scenarios where the input vector size varies. SSMax can be seamlessly integrated into existing Transformer-based architectures. Experimental results in language modeling show that models using SSMax not only achieve faster loss reduction during pretraining but also significantly improve performance in long contexts and key information retrieval. Furthermore, an analysis of attention scores reveals that SSMax enables the model to focus attention on key information even in long contexts. Additionally, although models that use SSMax from the beginning of pretraining achieve better length generalization, those that have already started pretraining can still gain some of this ability by replacing Softmax in the attention layers with SSMax, either during or after pretraining., Comment: 11 pages, 8 figures

We consider the homogeneous mean-field Bose gas at temperatures proportional to the critical temperature of its Bose--Einstein condensation phase transition. Our main result is a trace-norm approximation of the grand canonical Gibbs state in terms of a reference state, which is given by a convex combination of products of coherent states and Gibbs states associated with certain temperature-dependent Bogoliubov Hamiltonians. The convex combination is expressed as an integral over a Gibbs distribution of a one-mode $\Phi^4$-theory describing the condensate. We interpret this result as a justification of Bogoliubov theory at positive temperature. Further results derived from the norm approximation include various limiting distributions for the number of particles in the condensate, as well as precise formulas for the one- and two-particle density matrices of the Gibbs state. Key ingredients of our proof, which are of independent interest, include two novel abstract correlation inequalities. The proof of one of them is based on an application of an infinite-dimensional version of Stahl's theorem., Comment: 100 pages

We introduce a prescription for estimating the flux of the 7.7 micron and 11.3 micron\ polycyclic aromatic hydrocarbon (PAH) features from broadband JWST/MIRI images. Probing PAH flux with MIRI imaging data has advantages in field of view, spatial resolution, and sensitivity compared with MIRI spectral maps, but comparisons with spectra are needed to calibrate these flux estimations over a wide variety of environments. For 267 MIRI/MRS spectra from independent regions in the four luminous infrared galaxies (LIRGs) in the Great Observatories All-sky LIRG Survey (GOALS) early release science program, we derive synthetic filter photometry and directly compare estimated PAH fluxes to those measured from detailed spectral fits. We find that for probing PAH 7.7 micron, the best combination of filters is F560W, F770W, and either F1500W or F2100W, and the best for PAH 11.3 micron is F560W, F1000W, F1130W, and F1500W. The prescription with these combinations yields predicted flux densities that typically agree with values from spectral decomposition within ~7% and ~5% for PAH 7.7 and 11.3 micron, respectively., Comment: Submitted to ApJ

Due to labor shortages in specialty crop industries, a need for robotic automation to increase agricultural efficiency and productivity has arisen. Previous manipulation systems perform well in harvesting in uncluttered and structured environments. High tunnel environments are more compact and cluttered in nature, requiring a rethinking of the large form factor systems and grippers. We propose a novel codesigned framework incorporating a global detection camera and a local eye-in-hand camera that demonstrates precise localization of small fruits via closed-loop visual feedback and reliable error handling. Field experiments in high tunnels show our system can reach an average of 85.0\% of cherry tomato fruit in 10.98s on average.

Test-time scaling is a promising new approach to language modeling that uses extra test-time compute to improve performance. Recently, OpenAI's o1 model showed this capability but did not publicly share its methodology, leading to many replication efforts. We seek the simplest approach to achieve test-time scaling and strong reasoning performance. First, we curate a small dataset s1K of 1,000 questions paired with reasoning traces relying on three criteria we validate through ablations: difficulty, diversity, and quality. Second, we develop budget forcing to control test-time compute by forcefully terminating the model's thinking process or lengthening it by appending "Wait" multiple times to the model's generation when it tries to end. This can lead the model to double-check its answer, often fixing incorrect reasoning steps. After supervised finetuning the Qwen2.5-32B-Instruct language model on s1K and equipping it with budget forcing, our model s1 exceeds o1-preview on competition math questions by up to 27% (MATH and AIME24). Further, scaling s1 with budget forcing allows extrapolating beyond its performance without test-time intervention: from 50% to 57% on AIME24. Our model, data, and code are open-source at https://github.com/simplescaling/s1., Comment: 46 pages (9 main), 10 figures, 14 tables

Efficient real-world deployments of large language models (LLMs) rely on Key-Value (KV) caching for processing and generating long outputs, reducing the need for repetitive computation. For large contexts, Key-Value caches can take up tens of gigabytes of device memory, as they store vector representations for each token and layer. Recent work has shown that the cached vectors can be compressed through quantization, pruning or merging, but these techniques often compromise quality towards higher compression rates. In this work, we aim to improve Key & Value compression by exploiting two observations: 1) the inherent dependencies between keys and values across different layers, and 2) high-compression mechanisms for internal network states. We propose AQUA-KV, an adaptive quantization for Key-Value caches that relies on compact adapters to exploit existing dependencies between Keys and Values, and aims to "optimally" compress the information that cannot be predicted. AQUA-KV significantly improves compression rates, while maintaining high accuracy on state-of-the-art LLM families. On Llama 3.2 LLMs, we achieve near-lossless inference at 2-2.5 bits per value with under $1\%$ relative error in perplexity and LongBench scores. AQUA-KV is one-shot, simple, and efficient: it can be calibrated on a single GPU within 1-6 hours, even for 70B models., Comment: Preprint, under review

In contrast to problems of interference in (exogenous) treatments, models of interference in unit-specific (endogenous) outcomes do not usually produce a reduced-form representation where outcomes depend on other units' treatment status only at a short network distance, or only through a known exposure mapping. This remains true if the structural mechanism depends on outcomes of peers only at a short network distance, or through a known exposure mapping. In this paper, we first define causal estimands that are identified and estimable from a single experiment on the network under minimal assumptions on the structure of interference, and which represent average partial causal responses which generally vary with other global features of the realized assignment. Under a fixed-population, design-based approach, we show unbiasedness, consistency and asymptotic normality for inverse-probability weighting (IPW) estimators for those causal parameters from a randomized experiment on a single network. We also analyze more closely the case of marginal interventions in a model of equilibrium with smooth response functions where we can recover LATE-type weighted averages of derivatives of those response functions. Under additional structural assumptions, these "agnostic" causal estimands can be combined to recover model parameters, but also retain their less restrictive causal interpretation.

Fine-tuning large language models (LLMs) on devices is attracting increasing interest. Recent works have fused low-rank adaptation (LoRA) techniques with federated fine-tuning to mitigate challenges associated with device model sizes and data scarcity. Still, the heterogeneity of computational resources remains a critical bottleneck: while higher-rank modules generally enhance performance, varying device capabilities constrain LoRA's feasible rank range. Existing approaches attempting to resolve this issue either lack analytical justification or impose additional computational overhead, leaving a wide gap for an efficient and theoretically-grounded solution. To address these challenges, we propose federated sketching LoRA (FSLoRA), which leverages a sketching mechanism to enable devices to selectively update submatrices of global LoRA modules maintained by the server. By adjusting the sketching ratios, which determine the ranks of the submatrices on the devices, FSLoRA flexibly adapts to device-specific communication and computational constraints. We provide a rigorous convergence analysis of FSLoRA that characterizes how the sketching ratios affect the convergence rate. Through comprehensive experiments on multiple datasets and LLM models, we demonstrate FSLoRA's superior performance compared to various baselines., Comment: 23 pages

Traversing rough terrain requires dynamic bipeds to stabilize themselves through foot placement without stepping in unsafe areas. Planning these footsteps online is challenging given non-convexity of the safe terrain, and imperfect perception and state estimation. This paper addresses these challenges with a full-stack perception and control system for achieving underactuated walking on discontinuous terrain. First, we develop model-predictive footstep control (MPFC), a single mixed-integer quadratic program which assumes a convex polygon terrain decomposition to optimize over discrete foothold choice, footstep position, ankle torque, template dynamics, and footstep timing at over 100 Hz. We then propose a novel approach for generating convex polygon terrain decompositions online. Our perception stack decouples safe-terrain classification from fitting planar polygons, generating a temporally consistent terrain segmentation in real time using a single CPU thread. We demonstrate the performance of our perception and control stack through outdoor experiments with the underactuated biped Cassie, achieving state of the art perceptive bipedal walking on discontinuous terrain. Supplemental Video: https://youtu.be/eCOD1bMi638

Willems' fundamental lemma has recently received an impressive amount of attention from the (data-driven) control community. In this paper, we formulate a version of this celebrated result based on frequency-domain data. In doing so, we bridge the gap between recent developments in data-driven analysis and control, and the readily-available techniques and extensive expertise for non-parametric frequency-domain identification in academia and industry. In addition, we generalize our results to allow multiple frequency-domain data sets to be carefully combined to form a sufficiently rich data set. Building on these results, we propose a data-driven predictive control scheme based on measured frequency-domain data of the plant. This novel scheme provides a frequency-domain counterpart of the well-known data-enabled predictive control scheme DeePC based on time-domain data. We prove that, under appropriate conditions, the new frequency-domain data-driven predictive control (FreePC) scheme is equivalent to the corresponding DeePC scheme, and we demonstrate the benefits of FreePC and the use of frequency-domain data in a numerical case study. These benefits include the ability to collect data in closed loop with a pre-stabilizing controller, dealing with noisy data, without increasing computational complexity, and intuitively visualizing the uncertainty in the frequency-domain data. In addition, we further showcase the potential of our frequency-domain Willems' fundamental lemma in applications to data-driven simulation, and the linear-quadratic regulator (LQR) problem. Finally, we show that our results can be used to evaluate the transfer function of the system at a desired frequency based on a finite amount of frequency-domain data.

The emergence of Machine Learning systems everywhere raises new challenges, such as dealing with interactions or competition between multiple learners. In that goal, we study multi-agent sequential decision-making by considering principal-agent interactions in a tree structure. In this problem, the reward of a player is influenced by the actions of her children, who are all self-interested and non-cooperative. Our main finding is that it is possible to steer all the players towards the globally optimal set of actions by simply allowing single-step contracts between them. A contract is established between a principal and one of her agents: the principal actually offers the proposed payment if the agent picks the recommended action. The analysis poses specific challenges due to the intricate interactions between the nodes of the tree. Within a bandit setup, we propose algorithmic solutions for the players to end up being no-regret with respect to the optimal pair of actions and contracts. In the long run, allowing contracts makes the players act as if they were collaborating together, although they remain non-cooperative.

The exponential local-global principle, or Skolem conjecture, says: Suppose that \(b\) is a positive integer, and that the sequence \((u_{n})_{n = -\infty}^{\infty}\) is such that every term is in \(\mathbb{Z}[1/b]\), the linear recurrence \(u_{n + d} = a_{1}u_{n + d - 1} + \cdots + a_{d}u_{n}\) holds for all integers \(n\), and every root of \(x^{d} - a_{1}x^{d - 1} - a_{2}x^{d - 2} - \cdots - a_{d}\) is nonzero and simple; then there is no zero term \(u_{n}\) if and only if, for some integer \(m\) that is larger than \(1\) and relatively prime to \(b\), every term \(u_{n}\) is not in \(m\mathbb{Z}[1/b]\). Particular cases of the conjecture are known, but the general conjecture is open. This paper proves some apparently new quadratic and degenerate cubic cases of the exponential local-global principle via power residue symbols. This work was presented at the Stellenbosch Number Theory Conference 2025 in January 2025 at Stellenbosch University; much of the work was also presented at the 67th Annual Congress of the South African Mathematical Society in December 2024 at the University of Pretoria., Comment: 7 pages, 0 figures

Rotating synthetic aperture (RSA) imaging system captures images of the target scene at different rotation angles by rotating a rectangular aperture. Deblurring acquired RSA images plays a critical role in reconstructing a latent sharp image underlying the scene. In the past decade, the emergence of blind convolution technology has revolutionised this field by its ability to model complex features from acquired images. Most of the existing methods attempt to solve the above ill-posed inverse problem through maximising a posterior. Despite this progress, researchers have paid limited attention to exploring low-dimensional manifold structures of the latent image within a high-dimensional ambient-space. Here, we propose a novel method to process RSA images using manifold fitting and penalisation in the content of multi-frame blind convolution. We develop fast algorithms for implementing the proposed procedure. Simulation studies demonstrate that manifold-based deconvolution can outperform conventional deconvolution algorithms in the sense that it can generate a sharper estimate of the latent image in terms of estimating pixel intensities and preserving structural details., Comment: 39 pages, 9 figures

Language models have recently been shown capable of performing regression tasks wherein numeric predictions are represented as decoded strings. In this work, we provide theoretical grounds for this capability and furthermore investigate the utility of causal auto-regressive sequence models when they are applied to any feature representation. We find that, despite being trained in the usual way - for next-token prediction via cross-entropy loss - decoding-based regression is as performant as traditional approaches for tabular regression tasks, while being flexible enough to capture arbitrary distributions, such as in the task of density estimation., Comment: Google DeepMind Technical Report, 25 pages. Code can be found at https://github.com/google-research/optformer/tree/main/optformer/decoding_regression

We investigate the dust mass build-up and star formation efficiency of two galaxies at $z>12$, GHZ2 and GS-z14-0, by combining ALMA and JWST observations with an analytical model that assumes dust at thermal equilibrium. We obtained $3\sigma$ constraints on dust mass of $\log M_{\rm dust}/M_{\odot}<5.0$ and $<5.3$, respectively. These constraints are in tension with a high dust condensation efficiency in stellar ejecta but are consistent with models with a short metal accretion timescale at $z>12$. Given the young stellar ages of these galaxies ($t_{\rm age}\sim10\,{\rm Myrs}$), dust grain growth via accretion may still be ineffective at this stage, though it likely works efficiently to produce significant dust in galaxies at $z\sim7$. The star formation efficiencies, defined as the SFR divided by molecular gas mass, reach $\sim10\,{\rm Gyr}^{-1}$ in a 10\,Myr timescale, aligning with the expected redshift evolution of `starburst' galaxies with efficiencies that are $\sim0.5$--$1\,{\rm dex}$ higher than those in main-sequence galaxies. This starburst phase seems to be common in UV-bright galaxies at $z>12$ and is likely associated with the unique conditions of the early phases of galaxy formation, such as bursty star formation and/or negligible feedback from super-Eddington accretion. Direct observations of molecular gas tracers like [C\,{\sc ii}] will be crucial to further understanding the nature of bright galaxies at $z>12$.

Inspired by a chessboard puzzle of Dudeney, the general position problem in graph theory asks for the largest sets $S$ of vertices in a graph such that no three elements of $S$ lie on a common shortest path. The number of vertices in such a largest set is the general position number of the graph. This paper provides a survey of this rapidly growing problem, which now has an extensive literature. We cover exact results for various graph classes and the behaviour of the general position number under various graph products and operations. We also discuss interesting variations of the general position problem, for example variants corresponding to different graph convexities, as well as dynamic, fractional, colouring and game versions of the problem.

In this paper, we propose a novel loop closure detection algorithm that uses graph attention neural networks to encode semantic graphs to perform place recognition and then use semantic registration to estimate the 6 DoF relative pose constraint. Our place recognition algorithm has two key modules, namely, a semantic graph encoder module and a graph comparison module. The semantic graph encoder employs graph attention networks to efficiently encode spatial, semantic and geometric information from the semantic graph of the input point cloud. We then use self-attention mechanism in both node-embedding and graph-embedding steps to create distinctive graph vectors. The graph vectors of the current scan and a keyframe scan are then compared in the graph comparison module to identify a possible loop closure. Specifically, employing the difference of the two graph vectors showed a significant improvement in performance, as shown in ablation studies. Lastly, we implemented a semantic registration algorithm that takes in loop closure candidate scans and estimates the relative 6 DoF pose constraint for the LiDAR SLAM system. Extensive evaluation on public datasets shows that our model is more accurate and robust, achieving 13% improvement in maximum F1 score on the SemanticKITTI dataset, when compared to the baseline semantic graph algorithm. For the benefit of the community, we open-source the complete implementation of our proposed algorithm and custom implementation of semantic registration at https://github.com/crepuscularlight/SemanticLoopClosure

It is widely accepted that the Bayesian ideal observer (IO) should be used to guide the objective assessment and optimization of medical imaging systems. The IO employs complete task-specific information to compute test statistics for making inference decisions and performs optimally in signal detection tasks. However, the IO test statistic typically depends non-linearly on the image data and cannot be analytically determined. The ideal linear observer, known as the Hotelling observer (HO), can sometimes be used as a surrogate for the IO. However, when image data are high dimensional, HO computation can be difficult. Efficient channels that can extract task-relevant features have been investigated to reduce the dimensionality of image data to approximate IO and HO performance. This work proposes a novel method for generating efficient channels by use of the gradient of a Lagrangian-based loss function that was designed to learn the HO. The generated channels are referred to as the Lagrangian-gradient (L-grad) channels. Numerical studies are conducted that consider binary signal detection tasks involving various backgrounds and signals. It is demonstrated that channelized HO (CHO) using L-grad channels can produce significantly better signal detection performance compared to the CHO using PLS channels. Moreover, it is shown that the proposed L-grad method can achieve significantly lower computation time compared to the PLS method., Comment: SPIE Medical Imaging 2025

Background: Software engineering requires both technical skills and creative problem-solving. Blind and low-vision software professionals (BLVSPs) encounter numerous workplace challenges, including inaccessible tools and collaboration hurdles with sighted colleagues. Objective: This study explores the innovative strategies employed by BLVSPs to overcome these accessibility barriers, focusing on their custom solutions and the importance of supportive communities. Methodology: We conducted semi-structured interviews with 30 BLVSPs and used reflexive thematic analysis to identify key themes. Results: Findings reveal that BLVSPs are motivated to develop creative and adaptive solutions, highlighting the vital role of collaborative communities in fostering shared problem-solving. Conclusion: For BLVSPs, creative problem-solving is essential for navigating inaccessible work environments, in contrast to sighted peers, who pursue optimization. This study enhances understanding of how BLVSPs navigate accessibility challenges through innovation., Comment: Pre-print of accepted paper for CHASE 2025 (18th International Conference on Cooperative and Human Aspects of Software Engineering)

Motivated by the differential basis theorem of Kolchin and the difference-differential basis theorem of Cohn, in this paper we present a basis theorem for polynomial rings equipped with commuting generalised Hasse-Schmidt operators (in the sense of Moosa and Scanlon). We recover Kolchin and Cohn's results as special cases of our main theorem., Comment: 21 pages

In this work, we present and evaluate SELMA, a Speech-Enabled Language Model for virtual Assistant interactions that integrates audio and text as inputs to a Large Language Model (LLM). SELMA is designed to handle three primary and two auxiliary tasks related to interactions with virtual assistants simultaneously within a single end-to-end model. We employ low-rank adaptation modules for parameter-efficient training of both the audio encoder and the LLM. Additionally, we implement a feature pooling strategy enabling the system to recognize global patterns and improve accuracy on tasks less reliant on individual sequence elements. Experimental results on Voice Trigger (VT) detection, Device-Directed Speech Detection (DDSD), and Automatic Speech Recognition (ASR), demonstrate that our approach both simplifies the typical input processing pipeline of virtual assistants significantly and also improves performance compared to dedicated models for each individual task. SELMA yields relative Equal-Error Rate improvements of 64% on the VT detection task, and 22% on DDSD, while also achieving word error rates close to the baseline., Comment: Accepted at ICASSP 2025

Tables serve as a fundamental format for representing structured relational data. While current language models (LMs) excel at many text-based tasks, they still face challenges in table understanding due to the complex characteristics of tabular data, such as their structured nature. In this paper, we aim to enhance LMs for improved table understanding. We identify four key challenges: 1) difficulty in locating target data, 2) deficiency in table semantics, 3) numerical inaccuracies in textual reasoning, and 4) semantic inflexibility in symbolic reasoning. To address these issues, we propose TableMaster, a recipe and comprehensive framework that integrates multiple solutions to overcome these obstacles. TableMaster first extracts relevant table content and verbalizes it with enriched semantic context. Additionally, we introduce adaptive reasoning, a flexible approach that dynamically adjusts between textual and symbolic reasoning, tailoring the reasoning process to each query. Extensive analyses and experiments demonstrate our findings and the effectiveness of TableMaster. On the WikiTQ dataset, TableMaster achieves an accuracy of 78.13% using GPT-4o-mini, surpassing existing baselines.

Artificial Kitaev chains based on arrays of quantum dots are promising platforms for realizing Majorana Bound States (MBSs). In a two-site Kitaev chain, it is possible to find these non-Abelian zero-energy excitations at certain points in parameter space (sweet spots). These states, commonly referred to as Poor man's Majorana bound states (PMMs), are challenging to find and stabilize experimentally. In this work, we investigate the evolution of the sweet spots as we increase the number of sites of the Kitaev chain. To this end, we use the Bogoliubov-de Gennes representation to study the excitations of the system, and the scattering matrix and Green functions formalisms to calculate the zero-bias conductance. Our results show that the sweet spots evolve into a region that grows bigger and becomes gradually more protected as the number of sites $N$ increases. Due to the protection of the MBSs, we refer to this region as a topological island. We obtain similar results by considering a realistic spinful model with finite magnetic fields in a chain of normal-superconducting quantum dots. For long chains, $N \geq 20$, we show the emergence of strictly zero-energy plateaus robust against disorder. Finally, we demonstrate that the topological islands can be observed by performing conductance measurements via a quantum dot side-coupled to the Kitaev chain. Our work shows that the fine-tuning required to create and detect PMMs in a 2-site Kitaev chain is significantly relaxed as the length of the chain increases and details how PMMs evolve into MBSs. Our results are consistent with experimental reports for 2 and 3-site chains.

Recent advancements in data-driven weather forecasting models have delivered deterministic models that outperform the leading operational forecast systems based on traditional, physics-based models. However, these data-driven models are typically trained with a mean squared error loss function, which causes smoothing of fine scales through a "double penalty" effect. We develop a simple, parameter-free modification to this loss function that avoids this problem by separating the loss attributable to decorrelation from the loss attributable to spectral amplitude errors. Fine-tuning the GraphCast model with this new loss function results in sharp deterministic weather forecasts, an increase of the model's effective resolution from 1,250km to 160km, improvements to ensemble spread, and improvements to predictions of tropical cyclone strength and surface wind extremes.

We develop a unified theory for fault-tolerant quantum computation in quantum low-density parity-check (qLDPC) and topological codes. We show that there exist hidden simplicial complex structures encoding the topological data for all qLDPC and CSS codes obtained from product construction by generalizing the Freedman-Hastings code-to-manifold mapping. This is achieved by building manifolds corresponding to high-dimensional topological expanders from the Tanner graphs of the skeleton classical or quantum codes, which further form a product manifold and an associated thickened product code defined on its triangulation with only a constant qubit overhead. This suggests that qLDPC or more generally CSS codes obtained from product constructions are topological, and hence can admit cohomology operations such as cup products, physically corresponding to higher symmetries in the underlying topological quantum field theory. When applying this mapping to a 3D hypergraph product code obtained from the product of 3 copies of good classical expander codes, we obtain the first non-Clifford logical CCZ gates via constant depth circuits on a code with constant stabilizer weight $w=O(1)$, constant rate $K=\Theta(N)$, and polynomial distance $D=\Omega(N^{1/3})$. When applied to 3D homological product codes consisting of the product of a pair of good quantum and classical LDPC codes, we can further improve the distance to $D=\Omega(\sqrt{N})$ exceeding the $N^{1/3}$ distance barrier implied by the Bravyi-K\"onig bound for conventional topological codes. Our work suggests that it is feasible to apply native logical non-Clifford gates on qLDPC codes or directly inject high-fidelity magic states as resources (`magic state fountain') without the distillation process. For the homological product construction, the fountain can inject $\Theta(\sqrt{N})$ magic states in parallel in a single round., Comment: 34 pages, 15 figures

We introduce a new class of generative diffusion models that, unlike conventional denoising diffusion models, achieve a time-homogeneous structure for both the noising and denoising processes, allowing the number of steps to adaptively adjust based on the noise level. This is accomplished by conditioning the forward process using Doob's $h$-transform, which terminates the process at a suitable sampling distribution at a random time. The model is particularly well suited for generating data with lower intrinsic dimensions, as the termination criterion simplifies to a first-hitting rule. A key feature of the model is its adaptability to the target data, enabling a variety of downstream tasks using a pre-trained unconditional generative model. These tasks include natural conditioning through appropriate initialization of the denoising process and classification of noisy data.

Numerous machine learning and industrial problems can be modeled as the minimization of a sum of $N$ so-called clipped convex functions (SCC), i.e. each term of the sum stems as the pointwise minimum between a constant and a convex function. In this work, we extend this framework to capture more problems of interest. Specifically, we allow each term of the sum to be a pointwise minimum of an arbitrary number of convex functions, called components, turning the objective into a sum of pointwise minima of convex functions (SMC). Problem (SCC) is NP-hard, highlighting an appeal for scalable local heuristics. In this spirit, one can express (SMC) objectives as the difference between two convex functions to leverage the possibility to apply (DC) algorithms to compute critical points of the problem. Our approach relies on a bi-convex reformulation of the problem. From there, we derive a family of local methods, dubbed as relaxed alternating minimization (r-AM) methods, that include classical alternating minimization (AM) as a special case. We prove that every accumulation point of r-AM is critical. In addition, we show the empirical superiority of r-AM, compared to traditional AM and (DC) approaches, on piecewise-linear regression and restricted facility location problems. Under mild assumptions, (SCC) can be cast as a mixed-integer convex program (MICP) using perspective functions. This approach can be generalized to (SMC) but introduces many copies of the primal variable. In contrast, we suggest a compact big-M based (MICP) equivalent formulation of (SMC), free of these extra variables. Finally, we showcase practical examples where solving our (MICP), restricted to a neighbourhood of a given candidate (i.e. output iterate of a local method), will either certify the candidate's optimality on that neighbourhood or providing a new point, strictly better, to restart the local method.

We prove that there are at most 13 real quadratic fields that admit a ternary universal quadratic lattice, thus establishing a strong version of Kitaoka's Conjecture for quadratic fields. More generally, we obtain explicit upper bounds on the discriminants of real quadratic fields with a quadratic lattice of rank at most 7 that represents all totally positive multiples of a fixed integer., Comment: 27 pages; comments are welcome!

In this project, we propose a Variational Inference algorithm to approximate posterior distributions. Building on prior methods, we develop the Gradient-Steered Stein Variational Gradient Descent (G-SVGD) approach. This method introduces a novel loss function that combines a weighted gradient and the Evidence Lower Bound (ELBO) to enhance convergence speed and accuracy. The learning rate is determined through a suboptimal minimization of this loss function within a gradient descent framework. The G-SVGD method is compared against the standard Stein Variational Gradient Descent (SVGD) approach, employing the ADAM optimizer for learning rate adaptation, as well as the Markov Chain Monte Carlo (MCMC) method. We assess performance in two wave prospection models representing low-contrast and high-contrast subsurface scenarios. To achieve robust numerical approximations in the forward model solver, a five-point operator is employed, while the adjoint method improves accuracy in computing gradients of the log posterior. Our findings demonstrate that G-SVGD accelerates convergence and offers improved performance in scenarios where gradient evaluation is computationally expensive. The abstract highlights the algorithm's applicability to wave prospection models and its potential for broader applications in Bayesian inference. Finally, we discuss the benefits and limitations of G-SVGD, emphasizing its contribution to advancing computational efficiency in uncertainty quantification.

Given two compact metric spaces $X$ and $Y$, a Lipschitz continuous cost function $c$ on $X \times Y$ and two probabilities $\mu \in\mathcal{P}(X),\,\nu\in\mathcal{P}(Y)$, we propose to study the Monge-Kantorovich problem and its duality from a zero temperature limit of a convex pressure function. We consider the entropy defined by $H(\pi) = -D_{KL}(\pi|\mu\times \nu)$, where $D_{KL}$ is the Kullback-Leibler divergence, and then the pressure defined by the variational principle \[P(\beta A) = \sup_{\pi \in \Pi(\mu,\nu)} \left[ \smallint \beta A\,d\pi + H(\pi)\right],\]where $\beta>0$ and $A=-c$. We will show that it admits a dual formulation and when $\beta \to+\infty$ we recover the solution for the usual Monge-Kantorovich problem and its Kantorovich duality. Such approach is similar to one which is well known in Thermodynamic Formalism and Ergodic Optimization, where $\beta$ is interpreted as the inverse of the temperature ($\beta = \frac{1}{T}$) and $\beta\to+\infty$ is interpreted as a zero temperature limit.

The concept of power can be explored at several scales: from physical action and process effectuation, all the way to complex social dynamics. A spectrum-wide analysis of power requires attention to the fundamental principles that constrain these processes. In the social realm, the acquisition and maintenance of power is intertwined with both social interactions and cognitive processing capacity: socially-facilitated empowerment grants agents more information-processing capacities and opportunities, either by relying on others to bring about desired policies or ultimately outcomes, and/or by enjoying more information-processing possibilities as a result of relying on others for the reproduction of (material) tasks. The effects of social empowerment thus imply an increased ability to harness computation toward desired ends, thereby augmenting the evolution of a specific state space. Empowered individuals attract the attention of others, who contribute to increasing the scale of their access to various policies effectuating these state spaces. The presented argument posits that social power, in the context of active inference, is a function of several variables. As a result of its power-amplifying effects, this extended computational ability also buffers against possible vulnerabilities. We propose that individuals wield power not only by associating with others possessing desirable policies, but also by enhancing their ability to intake and compute information effectively. This dual mechanism is argued to create a cyclical, reinforcing pattern wherein the empowered are able to incrementally expand the scope of policies and state spaces available to them while minimizing risk-exposure.

Tools of quantum and statistical field theories have ben successfully ported to turbulence. Here, we review the key results of turbulence field theory. Thermalized spectrally-truncated Euler equation is described by \textit{equilibrium field theory}, in which the equipartitioned Fourier modes generate zero energy flux. In contrast, modelling of hydrodynamic turbulence (HDT), which has small viscosity, requires \textit{nonequilibrium field theory}. In HDT, the viscosity is renormalized using field theory that leads to wavenumber-dependent viscosity and energy spectrum. Field theory calculations also yields nonzero energy flux for HDT. These field theory computations have been generalized to other systems, e.g., passive scalar and magnetohydrodynamics. In this review, we cover these aspects, along with a brief coverage of weak turbulence and intermittency., Comment: 51 pages

Accurate constitutive models and corresponding mechanical property values for the meninges are important for predicting mechanical damage to brain tissue due to traumatic brain injury. The meninges are often oversimplified in current finite element (FE) head models due to their complex anatomy and spatially-variant mechanical behavior. This study performed a systematic review (SR) on the mechanical properties of each individual layer of the meninges to obtain benchmark data for FE modeling and to identify gaps in the current literature. Relevant studies were filtered through three stages: a broad initial search filter, a large language model classifier, and manual verification by a human reviewer. Out of over 25,000 studies initially considered, this review ultimately included 47 studies on the dura mater, 8 on the arachnoid mater, and 7 on the pia mater, representing the largest and most comprehensive SR on the mechanical properties of the meninges. Each layer was found to exhibit nonlinear rate dependence that varies with species, age, location, and orientation. This study revealed that the elastic modulus of pia mater most often used in simplified linear elastic FE models is likely underestimated by an order of magnitude and fails to consider directional dependence. Future studies investigating the mechanical properties of the meninges should focus on a wider range of loading rates as well as age effects for the arachnoid mater and pia mater, as these features are relatively understudied and expected to affect the fidelity of FE predictions., Comment: 56 pages including supplemental. One graphical abstract, 6 main text figures, 2 supplemental figures

Piezo1 ion channels are stretch-activated ion channels involved in a variety of important physiological and pathophysiological processes, as for example cardiovascular development and homeostasis. Since its discovery, it is known that this type of ion channel desensitize when exposed to stretch. However, recent experiments on Piezo1 ion channels have uncovered that their stretch response is qualitatively different when exposed to positive electrochemical driving forces, where the desensitization is reset. In this work, we propose a novel voltage-modulated mathematical model of Piezo1 based on a continuous-time Markov chain. We show that our Piezo1 model is able to quantitatively reproduce a wide range of experimental observations. Furthermore, we integrate our new ion channel model into the Mahajan-Shiferaw ventricular cardiomyocyte model to study the effect of electromechanical pacing at the cellular scale. This integrated cell model is able to qualitatively reproduce some aspects of the experimental observations regarding the rate-dependence of electromechanical pacing protocols. Our studies suggest that the Piezo1 ion channel is an important component that significantly contributes to the electromechanical coupled response of cardiomyocytes.

Multivariate Time Series Imputation (MTSI) is crucial for many applications, such as healthcare monitoring and traffic management, where incomplete data can compromise decision-making. Existing state-of-the-art methods, like Denoising Diffusion Probabilistic Models (DDPMs), achieve high imputation accuracy; however, they suffer from significant computational costs and are notably time-consuming due to their iterative nature. In this work, we propose CoSTI, an innovative adaptation of Consistency Models (CMs) for the MTSI domain. CoSTI employs Consistency Training to achieve comparable imputation quality to DDPMs while drastically reducing inference times, making it more suitable for real-time applications. We evaluate CoSTI across multiple datasets and missing data scenarios, demonstrating up to a 98% reduction in imputation time with performance on par with diffusion-based models. This work bridges the gap between efficiency and accuracy in generative imputation tasks, providing a scalable solution for handling missing data in critical spatio-temporal systems., Comment: 20 pages, 5 figures, 13 tables

We prove absence of ground states in the infrared-divergent spin boson model at large coupling. Our key argument reduces the proof to verifying long range order in the dual one-dimensional continuum Ising model, i.e., to showing that the respective two point function is lower bounded by a strictly positive constant. We can then use known results from percolation theory to establish long range order at large coupling. Combined with the known existence of ground states at small coupling, our result proves that the spin boson model undergoes a phase transition with respect to the coupling strength. We also present an expansion for the vacuum overlap of the spin boson ground state in terms of the Ising $n$-point functions, which implies that the phase transition is unique, i.e., that there is a critical coupling constant below which a ground state exists and above which none can exist., Comment: 17 pages

The Quick Ultra-Violet Kilonova surveyor (QUVIK), a two-band UV space telescope approved for funding as a Czech national science and technology mission, will focus on detecting early UV light of kilonovae (Werner et al., 2024). In addition, it will study the UV emission of stars and stellar systems (Krti\v{c}ka et al., 2024) as well as the intense and variable emission of active galactic nuclei (AGN) or galactic nuclei activated by tidal disruption events (Zaja\v{c}ek et al., 2024). In this contribution, we describe the role of this small ($\sim 30$-cm diameter) UV telescope for studying bright, nearby AGN. With its NUV and FUV bands, the telescope will perform high-cadence ($\sim$ 0.1-1 day) two-band photometric monitoring of nearby AGN ($z<1$), which will allow us to probe accretion disk sizes/temperature profiles via photometric reverberation mapping. Thanks to its versatility, QUVIK will be able to perform a moderately fast repointing ($<20$ min) to target candidates for tidal disruption events (TDEs). Early detection of the UV emission following a TDE optical flare, in combination with the subsequent two-band UV monitoring performed simultaneously with other observatories, will enable us to infer the time delay (or its lack of) between the optical, UV, and X-ray emission. In combination with theoretical models, it will be possible to shed more light on the origin of the UV/optical emission of TDEs. Furthermore, the two-band monitoring of nuclear transients will be beneficial in distinguishing between TDEs (nearly constant blue colour) and supernovae (progressive reddening)., Comment: 28 pages, 9 figures, submitted to the JATIS special issue "Ultraviolet Science & Instrumentation: On the Way to Habitable Worlds Observatory and Beyond"

Liquid Argon Time Projection Chambers (LArTPCs) are widely used in particle physics experiments. They use light and charge released in events to reconstruct and analyze them. Light information collected by the Photon Detection System (PDS) is used mainly to determine the initial time stamp, $t_0$, and to support determining the total energy of the event. For these purposes, it is important to simulate and predict the propagation of light inside the detector. There are several approaches to this, and this work discusses some of the options.

Numerous powerful large language models (LLMs) are now available for use as writing support tools, idea generators, and beyond. Although these LLMs are marketed as helpful creative assistants, several works have shown that using an LLM as a creative partner results in a narrower set of creative outputs. However, these studies only consider the effects of interacting with a single LLM, begging the question of whether such narrowed creativity stems from using a particular LLM -- which arguably has a limited range of outputs -- or from using LLMs in general as creative assistants. To study this question, we elicit creative responses from humans and a broad set of LLMs using standardized creativity tests and compare the population-level diversity of responses. We find that LLM responses are much more similar to other LLM responses than human responses are to each other, even after controlling for response structure and other key variables. This finding of significant homogeneity in creative outputs across the LLMs we evaluate adds a new dimension to the ongoing conversation about creativity and LLMs. If today's LLMs behave similarly, using them as a creative partners -- regardless of the model used -- may drive all users towards a limited set of "creative" outputs.

E-processes enable hypothesis testing with ongoing data collection while maintaining Type I error control. However, when testing multiple hypotheses simultaneously, current $e$-value based multiple testing methods such as e-BH are not invariant to the order in which data are gathered for the different $e$-processes. This can lead to undesirable situations, e.g., where a hypothesis rejected at time $t$ is no longer rejected at time $t+1$ after choosing to gather more data for one or more $e$-processes unrelated to that hypothesis. We argue that multiple testing methods should always work with suprema of $e$-processes. We provide an example to illustrate that e-BH does not control this FDR at level $\alpha$ when applied to suprema of $e$-processes. We show that adjusters can be used to ensure FDR-sup control with e-BH under arbitrary dependence.

In resonance fluorescence excitation experiments, light emitted from solid-state quantum emitters is typically filtered to eliminate the laser photons, ensuring that only red-shifted Stokes photons are detected. Theoretical analyses of the fluorescence intensity correlation often model emitters as two-level systems, focusing on light emitted exclusively from the purely electronic transition (the Zero-Phonon Line), or rely on statistical approaches based on conditional probabilities that do not account for quantum coherences. Here, we propose a general model to characterize the correlation of either Zero-Phonon Line photons or Stokes-shifted photons. This model successfully reproduces the experimental correlation of Stokes-shifted photons emitted from two interacting molecules and predicts that this correlation is affected by quantum coherence. Besides, we analyze the role of quantum coherence in light emission from two uncorrelated emitters, which helps to clarify the discrepancy between theory and experiments regarding the value of the correlation of photons emitted from this system at zero delay time., Comment: Main text: 6 pages, 3 figures. Supplemental Material: 9 pages, 5 figures

In this paper we begin the study of well-failed graphs, that is, graphs in which every maximal failed zero forcing set is a maximum failed zero forcing set, or equivalently, in which every minimal fort is a minimum fort. We characterize trees that are well-failed. Along the way, we prove that the set of vertices in a graph that are not in any minimal fort is identical to the set of vertices that are in no minimal zero forcing set, which allows us to characterize vertices in a tree that are in no minimal fort.