Search

This memorial article for Mark Sapir provides a brief overview of his life and career. Among his many contributions we highlight two of his most celebrated achievements: his groundbreaking solutions to Burnside-type problems for semigroups and his innovative construction of S-machines. Additionally, reflections from his colleagues and friends offer a heartfelt tribute, blending professional insights with personal memories., Comment: With contributions by Victor Guba, Olga Kharlampovich, Simon M. Goberstein, Stuart W. Margolis, Lev Shneerson, John Meakin, Ilya Kapovich, Eugene Plotkin, Michael Mihalik, Samuel Corson, and Arman Darbinyan

Modern power grids are evolving to become more interconnected, include more electric vehicles (EVs), and utilize more renewable energy sources (RES). Increased interconnectivity provides an opportunity to manage EVs and RES by using price signaling to shift EV loads towards periods of high RES output. This work uses ERCOT's 2035 RES installation plans and projections for Texas's EV fleet to examine and compare how both open-loop control and model predictive control (MPC) schemes can leverage time varying rates for EV charging to utilize excess RES supply that may otherwise be underutilized in a highly weather-dependent grid. The results show that while open-loop control increases RES usage, MPC increases RES usage even further by responding to RES outputs that differ from forecasts due to the inherent uncertainty of weather predictions. If MPC is used with time steps that are too frequent, however, difficulties arise; EV owners may find it too onerous to keep up with changing price structures, and frequent over-corrections to charging profiles can lead to a ``rebound peak" phenomenon. Therefore, control schemes should balance maximizing RES usage with ensuring customer participation.

Large language models (LLMs) have demonstrated remarkable capabilities, but their adoption is limited by high computational costs during inference. While increasing parameter counts enhances accuracy, it also widens the gap between state-of-the-art capabilities and practical deployability. We present Puzzle, a framework to accelerate LLM inference on specific hardware while preserving their capabilities. Through an innovative application of neural architecture search (NAS) at an unprecedented scale, Puzzle systematically optimizes models with tens of billions of parameters under hardware constraints. Our approach utilizes blockwise local knowledge distillation (BLD) for parallel architecture exploration and employs mixed-integer programming for precise constraint optimization. We demonstrate the real-world impact of our framework through Llama-3.1-Nemotron-51B-Instruct (Nemotron-51B), a publicly available model derived from Llama-3.1-70B-Instruct. Nemotron-51B achieves a 2.17x inference throughput speedup, fitting on a single NVIDIA H100 GPU while preserving 98.4% of the original model's capabilities. Nemotron-51B currently stands as the most accurate language model capable of inference on a single GPU with large batch sizes. Remarkably, this transformation required just 45B training tokens, compared to over 15T tokens used for the 70B model it was derived from. This establishes a new paradigm where powerful models can be optimized for efficient deployment with only negligible compromise of their capabilities, demonstrating that inference performance, not parameter count alone, should guide model selection. With the release of Nemotron-51B and the presentation of the Puzzle framework, we provide practitioners immediate access to state-of-the-art language modeling capabilities at significantly reduced computational costs.

We explore the behaviour emerging from learning agents repeatedly interacting strategically for a wide range of learning dynamics that includes projected gradient, replicator and log-barrier dynamics. Going beyond the better-understood classes of potential games and zero-sum games, we consider the setting of a general repeated game with finite recall, for different forms of monitoring. We obtain a Folk Theorem-like result and characterise the set of payoff vectors that can be obtained by these dynamics, discovering a wide range of possibilities for the emergence of algorithmic collusion., Comment: arXiv admin note: text overlap with arXiv:2210.08857 by other authors

Minimizer schemes, or just minimizers, are a very important computational primitive in sampling and sketching biological strings. Assuming a fixed alphabet of size $\sigma$, a minimizer is defined by two integers $k,w\ge2$ and a total order $\rho$ on strings of length $k$ (also called $k$-mers). A string is processed by a sliding window algorithm that chooses, in each window of length $w+k-1$, its minimal $k$-mer with respect to $\rho$. A key characteristic of the minimizer is the expected density of chosen $k$-mers among all $k$-mers in a random infinite $\sigma$-ary string. Random minimizers, in which the order $\rho$ is chosen uniformly at random, are often used in applications. However, little is known about their expected density $\mathcal{DR}_\sigma(k,w)$ besides the fact that it is close to $\frac{2}{w+1}$ unless $w\gg k$. We first show that $\mathcal{DR}_\sigma(k,w)$ can be computed in $O(k\sigma^{k+w})$ time. Then we attend to the case $w\le k$ and present a formula that allows one to compute $\mathcal{DR}_\sigma(k,w)$ in just $O(w \log w)$ time. Further, we describe the behaviour of $\mathcal{DR}_\sigma(k,w)$ in this case, establishing the connection between $\mathcal{DR}_\sigma(k,w)$, $\mathcal{DR}_\sigma(k+1,w)$, and $\mathcal{DR}_\sigma(k,w+1)$. In particular, we show that $\mathcal{DR}_\sigma(k,w)<\frac{2}{w+1}$ (by a tiny margin) unless $w$ is small. We conclude with some partial results and conjectures for the case $w>k$., Comment: Accepted to SOFSEM 2025

Perpetual voting studies fair collective decision-making in settings where many decisions are to be made, and is a natural framework for settings such as parliaments and the running of blockchain Decentralized Autonomous Organizations (DAOs). We focus our attention on the binary case (YES/NO decisions) and \textit{individual} guarantees for each of the participating agents. We introduce a novel notion, inspired by the popular maxi-min-share (MMS) for fair allocation. The agent expects to get as many decisions as if they were to optimally partition the decisions among the agents, with an adversary deciding which of the agents decides on what bundle. We show an online algorithm that guarantees the MMS notion for $n=3$ agents, an offline algorithm for $n=4$ agents, and show that no online algorithm can guarantee the $MMS^{adapt}$ for $n\geq 7$ agents. We also show that the Maximum Nash Welfare (MNW) outcome can only guarantee $O(\frac{1}{n})$ of the MMS notion in the worst case.

We analyze the Bass and SI models for the spreading of innovations and epidemics, respectively, on homogeneous complete networks, circular networks, and heterogeneous complete networks with two homogeneous groups. We allow the network parameters to be time dependent, which is a prerequisite for the analysis of optimal strategies on networks. Using a novel top-down analysis of the master equations, we present a simple proof for the monotone convergence of these models to their respective infinite-population limits. This leads to explicit expressions for the expected adoption or infection level in the Bass and SI models, respectively, on infinite homogeneous complete and circular networks, and on heterogeneous complete networks with two homogeneous groups with time-dependent parameters., Comment: 20 pages, 3 figures

We present a novel approach to automating the identification of risk factors for diseases from medical literature, leveraging pre-trained models in the bio-medical domain, while tuning them for the specific task. Faced with the challenges of the diverse and unstructured nature of medical articles, our study introduces a multi-step system to first identify relevant articles, then classify them based on the presence of risk factor discussions and, finally, extract specific risk factor information for a disease through a question-answering model. Our contributions include the development of a comprehensive pipeline for the automated extraction of risk factors and the compilation of several datasets, which can serve as valuable resources for further research in this area. These datasets encompass a wide range of diseases, as well as their associated risk factors, meticulously identified and validated through a fine-grained evaluation scheme. We conducted both automatic and thorough manual evaluation, demonstrating encouraging results. We also highlight the importance of improving models and expanding dataset comprehensiveness to keep pace with the rapidly evolving field of medical research., Comment: BioNLP@ACL2024, 12 pages

In this paper, we present and study the \emph{Hamming distance oracle problem}. In this problem, the task is to preprocess two strings $S$ and $T$ of lengths $n$ and $m$, respectively, to obtain a data-structure that is able to answer queries regarding the Hamming distance between a substring of $S$ and a substring of $T$. For a constant size alphabet strings, we show that for every $x\le nm$ there is a data structure with $\tilde{O}(nm/x)$ preprocess time and $O(x)$ query time. We also provide a combinatorial conditional lower bound, showing that for every $\varepsilon > 0$ and $x \le nm$ there is no data structure with query time $O(x)$ and preprocess time $O((\frac{nm}{x})^{1-\varepsilon})$ unless combinatorial fast matrix multiplication is possible. For strings over general alphabet, we present a data structure with $\tilde{O}(nm/\sqrt{x})$ preprocess time and $O(x)$ query time for every $x \le nm$.

The recently introduced Consistency models pose an efficient alternative to diffusion algorithms, enabling rapid and good quality image synthesis. These methods overcome the slowness of diffusion models by directly mapping noise to data, while maintaining a (relatively) simpler training. Consistency models enable a fast one- or few-step generation, but they typically fall somewhat short in sample quality when compared to their diffusion origins. In this work we propose a novel and highly effective technique for post-processing Consistency-based generated images, enhancing their perceptual quality. Our approach utilizes a joint classifier-discriminator model, in which both portions are trained adversarially. While the classifier aims to grade an image based on its assignment to a designated class, the discriminator portion of the very same network leverages the softmax values to assess the proximity of the input image to the targeted data manifold, thereby serving as an Energy-based Model. By employing example-specific projected gradient iterations under the guidance of this joint machine, we refine synthesized images and achieve an improved FID scores on the ImageNet 64x64 dataset for both Consistency-Training and Consistency-Distillation techniques.

A $\lambda$-cover of a string $S$ is a set of strings $\{C_i\}_1^\lambda$ such that every index in $S$ is contained in an occurrence of at least one string $C_i$. The existence of a $1$-cover defines a well-known class of quasi-periodic strings. Quasi-periodicity can be decided in linear time, and all $1$-covers of a string can be reported in linear time plus the size of the output. Since in general it is NP-complete to decide whether a string has a $\lambda$-cover, the natural next step is the development of efficient algorithms for $2$-covers. Radoszewski and Straszy\'nski [ESA 2020] analysed the particular case where the strings in a $2$-cover must be of the same length. They provided an algorithm that reports all such $2$-covers of $S$ in time near-linear in $|S|$ and in the size of the output. In this work, we consider $2$-covers in full generality. Since every length-$n$ string has $\Omega(n^2)$ trivial $2$-covers (every prefix and suffix of total length at least $n$ constitute such a $2$-cover), we state the reporting problem as follows: given a string $S$ and a number $m$, report all $2$-covers $\{C_1,C_2\}$ of $S$ with length $|C_1|+|C_2|$ upper bounded by $m$. We present an $\tilde{O}(n + Output)$ time algorithm solving this problem, with Output being the size of the output. This algorithm admits a simpler modification that finds a $2$-cover of minimum length. We also provide an $\tilde{O}(n)$ time construction of a $2$-cover oracle which, given two substrings $C_1,C_2$ of $S$, reports in poly-logarithmic time whether $\{C_1,C_2\}$ is a $2$-cover of $S$., Comment: 31 pages

A decision maker repeatedly chooses one of a finite set of actions. In each period, the decision maker's payoff depends on fixed basic payoff of the chosen action and the frequency with which the action has been chosen in the past. We analyze optimal strategies associated with three types of evaluations of infinite payoffs: discounted present value, the limit inferior, and the limit superior of the partial averages. We show that when the first two are the evaluation schemes, a stationary strategy can always achieve the best possible outcome. However, for the latter evaluation scheme, a stationary strategy can achieve the best outcome only if all actions that are chosen with strictly positive frequency by an optimal stationary strategy have the same basic payoff.

Hairpin completion, derived from the hairpin formation observed in DNA biochemistry, is an operation applied to strings, particularly useful in DNA computing. Conceptually, a right hairpin completion operation transforms a string $S$ into $S\cdot S'$ where $S'$ is the reverse complement of a prefix of $S$. Similarly, a left hairpin completion operation transforms a string $S$ into $S'\cdot S$ where $S'$ is the reverse complement of a suffix of $S$. The hairpin completion distance from $S$ to $T$ is the minimum number of hairpin completion operations needed to transform $S$ into $T$. Recently Boneh et al. showed an $O(n^2)$ time algorithm for finding the hairpin completion distance between two strings of length at most $n$. In this paper we show that for any $\varepsilon>0$ there is no $O(n^{2-\varepsilon})$-time algorithm for the hairpin completion distance problem unless the Strong Exponential Time Hypothesis (SETH) is false. Thus, under SETH, the time complexity of the hairpin completion distance problem is quadratic, up to sub-polynomial factors., Comment: To be published in CPM 2024

Contextual biasing enables speech recognizers to transcribe important phrases in the speaker's context, such as contact names, even if they are rare in, or absent from, the training data. Attention-based biasing is a leading approach which allows for full end-to-end cotraining of the recognizer and biasing system and requires no separate inference-time components. Such biasers typically consist of a context encoder; followed by a context filter which narrows down the context to apply, improving per-step inference time; and, finally, context application via cross attention. Though much work has gone into optimizing per-frame performance, the context encoder is at least as important: recognition cannot begin before context encoding ends. Here, we show the lightweight phrase selection pass can be moved before context encoding, resulting in a speedup of up to 16.1 times and enabling biasing to scale to 20K phrases with a maximum pre-decoding delay under 33ms. With the addition of phrase- and wordpiece-level cross-entropy losses, our technique also achieves up to a 37.5% relative WER reduction over the baseline without the losses and lightweight phrase selection pass., Comment: 9 pages, 3 figures, accepted by NAACL 2024 - Industry Track

We are given a bounded Borel-measurable real-valued function on a product of countably many Polish spaces, and a product probability measure. We are interested in points in the product space that can be used to approximate the expected value of this function. We define two notions. A point is called a weak $\epsilon$-approximation, where $\epsilon \geq 0$, if the Dirac measure on this point, except in finitely many coordinates where another measure can be taken, gives an expected value that is $\epsilon$-close to the original expected value. A point is called a strong $\epsilon$-approximation if the same holds under the restriction that in those finitely many coordinates the measure is equal to the original one. We prove that both the set of weak 0-approximation points and the set of strong $\epsilon$-approximation points, for any $\epsilon>0$, have measure 1 under the original measure. Finally, we provide two applications: (i) in Game Theory on the minmax guarantee levels of the players in games with infinitely many players, and (ii) in Decision Theory on the set of feasible expected payoffs in infinite duration problems.

We investigate the role of topology on the lattice determination of the $\mathrm{SU}(3)$ strong coupling renormalized via gradient flow. To deal with the topological freezing of standard local algorithms, the definition of the coupling is usually projected onto the zero topological sector. However, it is not obvious that this definition is not biased by the loss of ergodicity. We instead avoid the topological freezing using a novel algorithm, the Parallel Tempering on Boundary Conditions. The comparison with a standard algorithm shows that, even in the case where the latter is severely frozen, one obtains the same projected coupling. Moreover, we show that the two definitions of the coupling, projected and non-projected, lead to the same flow of the renormalization scale. Our results imply that projecting the coupling does not affect the determination of the dynamically-generated scale of the theory $\Lambda$, as obtained through the step-scaling method., Comment: v1: 30 pages, 9 figures. v2: 33 pages, 9 figures, revised and expanded text in Sections 1 and 2, expanded discussion about topological quantities, results and conclusions unchanged, matches accepted version on EPJC

In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio. The family includes two new models: (1) an updated Gemini 1.5 Pro, which exceeds the February version on the great majority of capabilities and benchmarks; (2) Gemini 1.5 Flash, a more lightweight variant designed for efficiency with minimal regression in quality. Gemini 1.5 models achieve near-perfect recall on long-context retrieval tasks across modalities, improve the state-of-the-art in long-document QA, long-video QA and long-context ASR, and match or surpass Gemini 1.0 Ultra's state-of-the-art performance across a broad set of benchmarks. Studying the limits of Gemini 1.5's long-context ability, we find continued improvement in next-token prediction and near-perfect retrieval (>99%) up to at least 10M tokens, a generational leap over existing models such as Claude 3.0 (200k) and GPT-4 Turbo (128k). Finally, we highlight real-world use cases, such as Gemini 1.5 collaborating with professionals on completing their tasks achieving 26 to 75% time savings across 10 different job categories, as well as surprising new capabilities of large language models at the frontier; when given a grammar manual for Kalamang, a language with fewer than 200 speakers worldwide, the model learns to translate English to Kalamang at a similar level to a person who learned from the same content.

Recall that a group $G$ is said to be $\frac{3}{2}$-generated if every non-trivial element $g\in G$ has a co-generator in $G$ (i.e., an element which together with $g$ generates $G$). Thompson's group $V$ was proved to be $\frac{3}{2}$-generated by Donoven and Harper in 2019. It was the first example of an infinite finitely presented non-cyclic $\frac{3}{2}$-generated group. In 2022, Bleak, Harper and Skipper proved that Thompson's group $T$ is also $\frac{3}{2}$-generated. Since the abelianization of Thompson's group $F$ is $\mathbb{Z}$, it cannot be $\frac{3}{2}$-generated. However, we recently proved that Thompson's group $F$ is "almost" $\frac{3}{2}$-generated in the sense that every element of $F$ whose image in the abelianization forms part of a generating pair of $\mathbb{Z}^2$ is part of a generating pair of $F$. A natural generalization of $\frac{3}{2}$-generation is the notion of spread. Recall that the spread of a group $G$ is the supremum over all integers $k$ such that every $k$ non-trivial elements of $G$ have a common co-generator in $G$. The uniform spread of a group $G$ is the supremum over all integers $k$ for which there exists a conjugacy class $C\subseteq G$ such that every $k$ non-trivial elements of $G$ have a common co-generator which belongs to $C$. In this paper we study modified versions of these notions for Thompson's group $F$., Comment: 21 pages, 1 figure. arXiv admin note: text overlap with arXiv:2210.03564

This work introduces a novel approach to price rainbow options, a type of path-independent multi-asset derivatives, with quantum computers. Leveraging the Iterative Quantum Amplitude Estimation method, we present an end-to-end quantum circuit implementation, emphasizing efficiency by delaying the transition to price space. Moreover, we analyze two different amplitude loading techniques for handling exponential functions. Experiments on the IBM QASM simulator validate our quantum pricing model, contributing to the evolving field of quantum finance., Comment: 9 pages, 5 figures

We present a proposal for calculating the running of the coupling constant of the $\mathrm{SU}(3)$ pure-gauge theory, which combines the Twisted Gradient Flow (TGF) renormalization scheme with Parallel Tempering on Boundary Conditions (PTBC). The TGF is a gradient flow-based renormalization scheme formulated in an asymmetric lattice with twisted boundary conditions. Combined with step scaling, it has been successfully used to calculate the $\mathrm{SU}(3)$ $\Lambda$ parameter. As with all gradient flow-based schemes, the coupling constant is highly correlated with the topological charge and affected by topology freezing, an issue addressed by projecting the determination of the coupling onto the zero topological sector. As an alternative to the zero-charge projection, we combine TGF with PTBC, by replicating multiple copies of the same lattice, interpolating between periodic and open boundary conditions in a parallel-tempered manner. We present a first exploration of these ideas by analyzing specific ensembles of $\mathrm{SU}(3)$ lattices with and without PTBC., Comment: 11 pages, 5 figures, 40th International Symposium on Lattice Field Theory, LATTICE2023 Fermilab, Batavia, Illinois, USA, from July 31 to August 4, 2023. V2: text unchanged, some previous figures were not correct and have been updated

Instructional coaching, informed by observation tools that measure teachers' practices, has been effective in improving teaching quality in early learning programs. However, existing measurement tools limit teachers' abilities to implement this type of instructional coaching at scale. To address this challenge, a team at SRI Education, along with partners at Substantial and the Bill and Melinda Gates Foundation, created a Target Product Profile (TPP) that describes the ideal features and capabilities of observation tools to support instructional coaching with prekindergarten teachers. This TPP is designed to spur investments, improvements, and innovations in existing and future observation tools for instructional coaches that measure important teaching practices and provide useful feedback on ways to improve classroom quality. [This resource was developed with Substantial.]

We introduce the natural notion of a matching frame in a $2$-dimensional string. A matching frame in a $2$-dimensional $n\times m$ string $M$, is a rectangle such that the strings written on the horizontal sides of the rectangle are identical, and so are the strings written on the vertical sides of the rectangle. Formally, a matching frame in $M$ is a tuple $(u,d,\ell,r)$ such that $M[u][\ell ..r] = M[d][\ell ..r]$ and $M[u..d][\ell] = M[u..d][r]$. In this paper, we present an algorithm for finding the maximum perimeter matching frame in a matrix $M$ in $\tilde{O}(n^{2.5})$ time (assuming $n \ge m)$. Additionally, for every constant $\epsilon> 0$ we present a near-linear $(1-\epsilon)$-approximation algorithm for the maximum perimeter of a matching frame. In the development of the aforementioned algorithms, we introduce inventive technical elements and uncover distinctive structural properties that we believe will captivate the curiosity of the community.

We present a joint Speech and Language Model (SLM), a multitask, multilingual, and dual-modal model that takes advantage of pretrained foundational speech and language models. SLM freezes the pretrained foundation models to maximally preserves their capabilities, and only trains a simple adapter with just 1\% (156M) of the foundation models' parameters. This adaptation not only leads SLM to achieve strong performance on conventional tasks such as speech recognition (ASR) and speech translation (AST), but also introduces the novel capability of zero-shot instruction-following for more diverse tasks: given a speech input and a text instruction, SLM is able to perform unseen generation tasks including contextual biasing ASR using real-time context, dialog generation, speech continuation, and question answering, etc. Our approach demonstrates that the representational gap between pretrained speech and language models might be narrower than one would expect, and can be bridged by a simple adaptation mechanism. As a result, SLM is not only efficient to train, but also inherits strong capabilities already acquired in foundation models of different modalities.

Contextual biasing refers to the problem of biasing the automatic speech recognition (ASR) systems towards rare entities that are relevant to the specific user or application scenarios. We propose algorithms for contextual biasing based on the Knuth-Morris-Pratt algorithm for pattern matching. During beam search, we boost the score of a token extension if it extends matching into a set of biasing phrases. Our method simulates the classical approaches often implemented in the weighted finite state transducer (WFST) framework, but avoids the FST language altogether, with careful considerations on memory footprint and efficiency on tensor processing units (TPUs) by vectorization. Without introducing additional model parameters, our method achieves significant word error rate (WER) reductions on biasing test sets by itself, and yields further performance gain when combined with a model-based biasing method.

Quantum sensing is an ever-evolving research field describing the use of a quantum phenomenon to perform measurement of a physical quantity. Amongst different types of quantum sensors, atomic vapor-based quantum effects are extensively used to measure quantities such as time, velocity, acceleration, and electric and magnetic fields. Here, we propose and demonstrate remote quantum sensing using a chip-scale atomic vapor cell. Specifically, we remotely interrogate mm-scale micromachined vapor cells, and measure the ambient Earth's magnetic field at a standoff distance of ~10 meters and a sensitivity of ~1 pT/Hz^0.5 . Simultaneously we are able measure the distance between micro-cell and the interrogating system by means of time-of-flight measurements, thus correlating between position and magnetic field. Consequently, we provide a novel toolset to measure and map arbitrary, remote, and hard to access magnetic field in unshielded environments with high sensitivity and spatial resolution, paving the way to a variety of novel applications in diverse fields such as medicine, communication, defense, space-exploration, and quantum technologies.

We give an $\tilde O(n^2)$ time algorithm for computing the exact Dynamic Time Warping distance between two strings whose run-length encoding is of size at most $n$. This matches (up to log factors) the known (conditional) lower bound, and should be compared with the previous fastest $O(n^3)$ time exact algorithm and the $\tilde O(n^2)$ time approximation algorithm.