Search

Linear inverse problems are ubiquitous in various science and engineering disciplines. Of particular importance in the past few decades, is the incorporation of sparsity based priors, in particular $\ell_1$ priors, into linear inverse problems, which led to the flowering of fields of compressive sensing (CS) and sparsity based signal processing. More recently, methods based on a Compound Gaussian (CG) prior have been investigated and demonstrate improved results over CS in practice. This paper is the first attempt to identify and elucidate the fundamental structures underlying the success of CG methods by studying CG in the context of a broader framework of generalized-sparsity-based-inference. After defining our notion of generalized sparsity we introduce a weak null space property and proceed to generalize two well-known methods in CS, basis pursuit and iteratively reweighted least squares (IRLS). We show how a subset of CG-induced regularizers fits into this framework.

This paper presents the first gossip algorithms that are robust to adversarial corruptions. Gossip algorithms distribute information in a scalable and efficient way by having random pairs of nodes exchange small messages. Value aggregation problems are of particular interest in this setting as they occur frequently in practice and many elegant algorithms have been proposed for computing aggregates and statistics such as averages and quantiles. An important and well-studied advantage of gossip algorithms is their robustness to message delays, network churn, and unreliable message transmissions. These crucial robustness guarantees however only hold if all nodes follow the protocol and no messages are corrupted. In this paper, we remedy this by providing a framework to model both adversarial participants and message corruptions in gossip-style communications by allowing an adversary to control a small fraction of the nodes or corrupt messages arbitrarily. Despite this very powerful and general corruption model, we show that one can design robust gossip algorithms for many important aggregation problems. Our algorithms guarantee that almost all nodes converge to an approximately correct answer with optimal efficiency and essentially as fast as without corruptions. The design of adversarially-robust gossip algorithms poses completely new challenges. Despite this, our algorithms remain very simple variations of known non-robust algorithms with often only subtle changes to avoid non-compliant nodes gaining too much influence over outcomes. While our algorithms remain simple, their analysis is much more complex and often requires a completely different approach than the non-adversarial setting.

In this paper we introduce a new way of measuring the robustness of Erd\H{o}s--Ko--Rado (EKR) Theorems on permutation groups. EKR-type results can be viewed as results about the independence numbers of certain corresponding graphs, namely the derangement graphs, and random subgraphs of these graphs have been used to measure the robustness of these extremal results. In the context of permutation groups, the derangement graphs are Cayley graphs on the permutation group in question. We propose studying extremal properties of subgraphs of derangement graphs, that are themselves Cayley graphs of the group, to measure robustness. We present a variety of results about the robustness of the EKR property of various permutation groups using this new measure., Comment: 29 pages

We describe a concrete device roadmap towards a fault-tolerant quantum computing architecture based on noise-resilient, topologically protected Majorana-based qubits. Our roadmap encompasses four generations of devices: a single-qubit device that enables a measurement-based qubit benchmarking protocol; a two-qubit device that uses measurement-based braiding to perform single-qubit Clifford operations; an eight-qubit device that can be used to show an improvement of a two-qubit operation when performed on logical qubits rather than directly on physical qubits; and a topological qubit array supporting lattice surgery demonstrations on two logical qubits. Devices that enable this path require a superconductor-semiconductor heterostructure that supports a topological phase, quantum dots and coupling between those quantum dots that can create the appropriate loops for interferometric measurements, and a microwave readout system that can perform fast, low-error single-shot measurements. We describe the key design components of these qubit devices, along with the associated protocols for demonstrations of single-qubit benchmarking, Clifford gate execution, quantum error detection, and quantum error correction, which differ greatly from those in more conventional qubits. Finally, we comment on implications and advantages of this architecture for utility-scale quantum computation., Comment: 11+6 pages, 8+5 figures

We present PromptArtisan, a groundbreaking approach to multi-instruction image editing that achieves remarkable results in a single pass, eliminating the need for time-consuming iterative refinement. Our method empowers users to provide multiple editing instructions, each associated with a specific mask within the image. This flexibility allows for complex edits involving mask intersections or overlaps, enabling the realization of intricate and nuanced image transformations. PromptArtisan leverages a pre-trained InstructPix2Pix model in conjunction with a novel Complete Attention Control Mechanism (CACM). This mechanism ensures precise adherence to user instructions, granting fine-grained control over the editing process. Furthermore, our approach is zero-shot, requiring no additional training, and boasts improved processing complexity compared to traditional iterative methods. By seamlessly integrating multi-instruction capabilities, single-pass efficiency, and complete attention control, PromptArtisan unlocks new possibilities for creative and efficient image editing workflows, catering to both novice and expert users alike., Comment: Accepted in ICASSP 2025

Retrieval-Augmented Generation (RAG) has emerged as a prominent method for incorporating domain knowledge into Large Language Models (LLMs). While RAG enhances response relevance by incorporating retrieved domain knowledge in the context, retrieval errors can still lead to hallucinations and incorrect answers. To recover from retriever failures, domain knowledge is injected by fine-tuning the model to generate the correct response, even in the case of retrieval errors. However, we observe that without systematic knowledge augmentation, fine-tuned LLMs may memorize new information but still fail to extract relevant domain knowledge, leading to poor performance. In this work, we present a novel framework that significantly enhances the fine-tuning process by augmenting the training data in two ways -- context augmentation and knowledge paraphrasing. In context augmentation, we create multiple training samples for a given QA pair by varying the relevance of the retrieved information, teaching the model when to ignore and when to rely on retrieved content. In knowledge paraphrasing, we fine-tune with multiple answers to the same question, enabling LLMs to better internalize specialized knowledge. To mitigate catastrophic forgetting due to fine-tuning, we add a domain-specific identifier to a question and also utilize a replay buffer containing general QA pairs. Experimental results demonstrate the efficacy of our method over existing techniques, achieving up to 10\% relative gain in token-level recall while preserving the LLM's generalization capabilities., Comment: 22 pages, 14 tables, to be published in NAACL 2025

Fine-tuning Large Language Models (LLMs) on specific datasets is a common practice to improve performance on target tasks. However, this performance gain often leads to overfitting, where the model becomes too specialized in either the task or the characteristics of the training data, resulting in a loss of generalization. This paper introduces Selective Self-to-Supervised Fine-Tuning (S3FT), a fine-tuning approach that achieves better performance than the standard supervised fine-tuning (SFT) while improving generalization. S3FT leverages the existence of multiple valid responses to a query. By utilizing the model's correct responses, S3FT reduces model specialization during the fine-tuning stage. S3FT first identifies the correct model responses from the training set by deploying an appropriate judge. Then, it fine-tunes the model using the correct model responses and the gold response (or its paraphrase) for the remaining samples. The effectiveness of S3FT is demonstrated through experiments on mathematical reasoning, Python programming and reading comprehension tasks. The results show that standard SFT can lead to an average performance drop of up to $4.4$ on multiple benchmarks, such as MMLU and TruthfulQA. In contrast, S3FT reduces this drop by half, i.e. $2.5$, indicating better generalization capabilities than SFT while performing significantly better on the fine-tuning tasks., Comment: 10 pages, Accepted to NAACL Findings 2025. arXiv admin note: text overlap with arXiv:2409.04787

A family $\mathcal{F}$ of spanning trees of the complete graph on $n$ vertices $K_n$ is \emph{$t$-intersecting} if any two members have a forest on $t$ edges in common. We prove an Erd\H{o}s--Ko--Rado result for $t$-intersecting families of spanning trees of $K_n$. In particular, we show there exists a constant $C > 0$ such that for all $n \geq C (\log n) t$ the largest $t$-intersecting families are the families consisting of all trees that contain a fixed set of $t$ disjoint edges (as well as the stars on $n$ vertices for $t = 1$). The proof uses the spread approximation technique in conjunction with the Lopsided Lov\'asz Local Lemma., Comment: 18 pages

This article is a write-up of pedagogical lectures delivered at the Asia Pacific Center for Theoretical Physics online winter school held in January 2021, and also as part of the Quantum Information, Quantum Field Theory and Gravity program held at ICTS, Bengaluru, in August 2024. The topics covered include a brief derivation of Hawking radiation from the perspective of correlation functions, a description of entropy paradoxes in the eternal black hole, and details of the associated entanglement island computations in two-dimensional models., Comment: To be published in a collected volume on The Black Hole Information Paradox edited by Ali Akil and Cosimo Bambi

This work identifies the first privacy-aware stratified sampling scheme that minimizes the variance for general private mean estimation under the Laplace, Discrete Laplace (DLap) and Truncated-Uniform-Laplace (TuLap) mechanisms within the framework of differential privacy (DP). We view stratified sampling as a subsampling operation, which amplifies the privacy guarantee; however, to have the same final privacy guarantee for each group, different nominal privacy budgets need to be used depending on the subsampling rate. Ignoring the effect of DP, traditional stratified sampling strategies risk significant variance inflation. We phrase our optimal survey design as an optimization problem, where we determine the optimal subsampling sizes for each group with the goal of minimizing the variance of the resulting estimator. We establish strong convexity of the variance objective, propose an efficient algorithm to identify the integer-optimal design, and offer insights on the structure of the optimal design.

The discovery of superconductivity under high pressure in Ruddlesden-Popper phases of bulk nickelates has sparked great interest in stabilizing ambient pressure superconductivity in thin-film form using epitaxial strain. Recently, signs of superconductivity have been observed in compressively strained bilayer nickelate thin films with an onset temperature exceeding 40 K, albeit with broad and two-step-like transitions. Here, we report intrinsic superconductivity and normal-state transport properties in compressively strained La$_2$PrNi$_2$O$_7$ thin films, achieved through a combination of isovalent Pr substitution, growth optimization, and precision ozone annealing. The superconducting onset occurs above 48 K, with zero resistance reached above 30 K, and the critical current density at 1.4 K is 100-fold larger than previous reports. The normal-state resistivity exhibits quadratic temperature dependence indicative of Fermi liquid behaviour, and other phenomenological similarities to transport in overdoped cuprates suggest parallels in their emergent properties., Comment: 18 pages, 4 figures

This paper presents an algorithmic framework for solving unconstrained stochastic optimization problems using only stochastic function evaluations. We employ central finite-difference based gradient estimation methods to approximate the gradients and dynamically control the accuracy of these approximations by adjusting the sample sizes used in stochastic realizations. We analyze the theoretical properties of the proposed framework on nonconvex functions. Our analysis yields sublinear convergence results to the neighborhood of the solution, and establishes the optimal worst-case iteration complexity ($\mathcal{O}(\epsilon^{-1})$) and sample complexity ($\mathcal{O}(\epsilon^{-2})$) for each gradient estimation method to achieve an $\epsilon$-accurate solution. Finally, we demonstrate the performance of the proposed framework and the quality of the gradient estimation methods through numerical experiments on nonlinear least squares problems.

Cryo-electron microscopy (cryo-EM) is an experimental technique for protein structure determination that images an ensemble of macromolecules in near-physiological contexts. While recent advances enable the reconstruction of dynamic conformations of a single biomolecular complex, current methods do not adequately model samples with mixed conformational and compositional heterogeneity. In particular, datasets containing mixtures of multiple proteins require the joint inference of structure, pose, compositional class, and conformational states for 3D reconstruction. Here, we present Hydra, an approach that models both conformational and compositional heterogeneity fully ab initio by parameterizing structures as arising from one of K neural fields. We employ a new likelihood-based loss function and demonstrate the effectiveness of our approach on synthetic datasets composed of mixtures of proteins with large degrees of conformational variability. We additionally demonstrate Hydra on an experimental dataset of a cellular lysate containing a mixture of different protein complexes. Hydra expands the expressivity of heterogeneous reconstruction methods and thus broadens the scope of cryo-EM to increasingly complex samples.

We present HadaCore, a modified Fast Walsh-Hadamard Transform (FWHT) algorithm optimized for the Tensor Cores present in modern GPU hardware. HadaCore follows the recursive structure of the original FWHT algorithm, achieving the same asymptotic runtime complexity but leveraging a hardware-aware work decomposition that benefits from Tensor Core acceleration. This reduces bottlenecks from compute and data exchange. On Nvidia A100 and H100 GPUs, HadaCore achieves speedups of 1.1-1.4x and 1.0-1.3x, with a peak gain of 3.5x and 3.6x respectively, when compared to the existing state-of-the-art implementation of the original algorithm. We also show that when using FP16 or BF16, our implementation is numerically accurate, enabling comparable accuracy on MMLU benchmarks when used in an end-to-end Llama3 inference run with quantized (FP8) attention.

To build effective therapeutics, biologists iteratively mutate antibody sequences to improve binding and stability. Proposed mutations can be informed by previous measurements or by learning from large antibody databases to predict only typical antibodies. Unfortunately, the space of typical antibodies is enormous to search, and experiments often fail to find suitable antibodies on a budget. We introduce Clone-informed Bayesian Optimization (CloneBO), a Bayesian optimization procedure that efficiently optimizes antibodies in the lab by teaching a generative model how our immune system optimizes antibodies. Our immune system makes antibodies by iteratively evolving specific portions of their sequences to bind their target strongly and stably, resulting in a set of related, evolving sequences known as a clonal family. We train a large language model, CloneLM, on hundreds of thousands of clonal families and use it to design sequences with mutations that are most likely to optimize an antibody within the human immune system. We propose to guide our designs to fit previous measurements with a twisted sequential Monte Carlo procedure. We show that CloneBO optimizes antibodies substantially more efficiently than previous methods in realistic in silico experiments and designs stronger and more stable binders in in vitro wet lab experiments., Comment: Code available at https://github.com/AlanNawzadAmin/CloneBO

We report on the successful design and fabrication of optical modules using a 50 micron pitch polymer waveguide interface, integrated for low loss, high density optical data transfer with very low space requirements on a Si photonics die. This prototype module meets JEDEC reliability standards and promises to increase the number of optical fibers that can be connected at the edge of a chip, a measure known as beachfront density, by six times compared to state of the art technology. Scalability of the polymer waveguide to less than 20 micron pitch stands to improve the bandwidth density upwards of 10 Tbps/mm.

Antibody therapies have been employed to address some of today's most challenging diseases, but must meet many criteria during drug development before reaching a patient. Humanization is a sequence optimization strategy that addresses one critical risk called immunogenicity - a patient's immune response to the drug - by making an antibody more "human-like" in the absence of a predictive lab-based test for immunogenicity. However, existing humanization strategies generally yield very few humanized candidates, which may have degraded biophysical properties or decreased drug efficacy. Here, we re-frame humanization as a conditional generative modeling task, where humanizing mutations are sampled from a language model trained on human antibody data. We describe a sampling process that incorporates models of therapeutic attributes, such as antigen binding affinity, to obtain candidate sequences that have both reduced immunogenicity risk and maintained or improved therapeutic properties, allowing this algorithm to be readily embedded into an iterative antibody optimization campaign. We demonstrate in silico and in lab validation that in real therapeutic programs our generative humanization method produces diverse sets of antibodies that are both (1) highly-human and (2) have favorable therapeutic properties, such as improved binding to target antigens.

Prefix circuits are fundamental components in digital adders, widely used in digital systems due to their efficiency in calculating carry signals. Synthesizing prefix circuits with minimized area and delay is crucial for enhancing the performance of modern computing systems. Recently, large language models (LLMs) have demonstrated a surprising ability to perform text generation tasks. We propose PrefixLLM, that leverages LLMs for prefix circuit synthesis. PrefixLLM transforms the prefix circuit synthesis task into a structured text generation problem, termed the Structured Prefix Circuit Representation (SPCR), and introduces an iterative framework to automatically and accurately generate valid SPCRs. We further present a design space exploration (DSE) framework that uses LLMs to iteratively search for area and delay optimized prefix circuits. Compared to state-of-the-art, PrefixLLM can reduce the area by 3.70% under the same delay constraint. This work highlights the use of LLMs in the synthesis of arithmetic circuits, which can be transformed into the structured text generation.

Large Language Models (LLMs) have achieved remarkable advancements, but their monolithic nature presents challenges in terms of scalability, cost, and customization. This paper introduces the Composition of Experts (CoE), a modular compound AI system leveraging multiple expert LLMs. CoE leverages a router to dynamically select the most appropriate expert for a given input, enabling efficient utilization of resources and improved performance. We formulate the general problem of training a CoE and discuss inherent complexities associated with it. We propose a two-step routing approach to address these complexities that first uses a router to classify the input into distinct categories followed by a category-to-expert mapping to obtain desired experts. CoE offers a flexible and cost-effective solution to build compound AI systems. Our empirical evaluation demonstrates the effectiveness of CoE in achieving superior performance with reduced computational overhead. Given that CoE comprises of many expert LLMs it has unique system requirements for cost-effective serving. We present an efficient implementation of CoE leveraging SambaNova SN40L RDUs unique three-tiered memory architecture. CoEs obtained using open weight LLMs Qwen/Qwen2-7B-Instruct, google/gemma-2-9b-it, google/gemma-2-27b-it, meta-llama/Llama-3.1-70B-Instruct and Qwen/Qwen2-72B-Instruct achieve a score of $59.4$ with merely $31$ billion average active parameters on Arena-Hard and a score of $9.06$ with $54$ billion average active parameters on MT-Bench.

Challenge to control the quantum states of matter via light have been at the forefront of modern research on driven quantum matter. We explore the imprinting effects of structured light on superconductors, demonstrating how the quantum numbers of light-specifically spin angular momentum, orbital angular momentum, and radial order-can be transferred to the superconducting order parameter and control vortex dynamics. Using Laguerre-Gaussian beams, we show that by tuning the quantum numbers and the amplitude of the electric field, it is possible to manipulate a variety of vortex behaviors, including breathing vortex pairs, braiding vortex pairs, vortex droplets, supervortices, and swirling 2D vortex rings. More complex structure of vortex-clusters, such as vortex-flake structures, and standing wave motions, also emerge under specific quantum numbers. These results demonstrate the ability to control SC vortex motion and phase structures through structured light, offering potential applications in quantum fluids and optical control of superconducting states. Our findings present a diagram that links light's quantum numbers to the resulting SC vortex behaviors, highlighting the capacity of light to transfer its symmetry onto superconducting condensates. We point that this approach represents the extension of the printing to quantum printing by light in a coherent state of electrons.

Optimal camera placement plays a crucial role in applications such as surveillance, environmental monitoring, and infrastructure inspection. Even highly abstracted versions of this problem are NP-hard due to the high-dimensional continuous domain of camera configurations (i.e., positions and orientations) and difficulties in efficiently and accurately calculating camera coverage. In this paper, we present a novel framework for optimal camera placement that uses integer programming and adaptive sampling strategies to maximize coverage, given a limited camera budget. We develop a modified maximum k-coverage formulation and two adaptive sampling strategies, Explore and Exploit (E&E) and Target Uncovered Spaces (TUS), that iteratively add new camera configurations to the candidate set in order to improve the solution. E&E focuses on local search around camera configurations chosen in previous iterations, whereas TUS focuses specifically on covering regions that were previously uncovered. We first conduct theoretical analysis to provide bounds on the probability of finding an optimal solution and expected sampling needs, while ensuring monotonic improvements in coverage. Then, we conduct a detailed numerical analysis over different environments. Results show that E&E achieves coverage improvements of 3.3-16.0% over all baseline random sampling approaches, while maintaining manageable computational times. Meanwhile, TUS performs well in open environments and with tight camera budgets, achieving gains of 6.9-9.1% in such conditions. Compared to the baseline, our approach achieves similar coverage using only 30-70% of the sampling budget, demonstrating its computational efficiency. Through a case study, we obtain insights into optimal camera placement decisions for a typical indoor surveillance application., Comment: 49 pages, 16 figures

This white paper, developed through close collaboration between IBM Research and UIUC researchers within the IIDAI Institute, envisions transforming hybrid cloud systems to meet the growing complexity of AI workloads through innovative, full-stack co-design approaches, emphasizing usability, manageability, affordability, adaptability, efficiency, and scalability. By integrating cutting-edge technologies such as generative and agentic AI, cross-layer automation and optimization, unified control plane, and composable and adaptive system architecture, the proposed framework addresses critical challenges in energy efficiency, performance, and cost-effectiveness. Incorporating quantum computing as it matures will enable quantum-accelerated simulations for materials science, climate modeling, and other high-impact domains. Collaborative efforts between academia and industry are central to this vision, driving advancements in foundation models for material design and climate solutions, scalable multimodal data processing, and enhanced physics-based AI emulators for applications like weather forecasting and carbon sequestration. Research priorities include advancing AI agentic systems, LLM as an Abstraction (LLMaaA), AI model optimization and unified abstractions across heterogeneous infrastructure, end-to-end edge-cloud transformation, efficient programming model, middleware and platform, secure infrastructure, application-adaptive cloud systems, and new quantum-classical collaborative workflows. These ideas and solutions encompass both theoretical and practical research questions, requiring coordinated input and support from the research community. This joint initiative aims to establish hybrid clouds as secure, efficient, and sustainable platforms, fostering breakthroughs in AI-driven applications and scientific discovery across academia, industry, and society., Comment: 70 pages, 27 figures

In this paper, we propose algorithms that exploit negative curvature for solving noisy nonlinear nonconvex unconstrained optimization problems. We consider both deterministic and stochastic inexact settings, and develop two-step algorithms that combine directions of negative curvature and descent directions to update the iterates. Under reasonable assumptions, we prove second-order convergence results and derive complexity guarantees for both settings. To tackle large-scale problems, we develop a practical variant that utilizes the conjugate gradient method with negative curvature detection and early stopping to compute a step, a simple adaptive step size scheme, and a strategy for selecting the sample sizes of the gradient and Hessian approximations as the optimization progresses. Numerical results on two machine learning problems showcase the efficacy and efficiency of the practical method., Comment: 39 pages, 6 figures

Objective: To assess the antioxidant, wound healing, anti-ulcer, and anti-inflammatory properties of Jatropha maheshwari. Methods: Jatropha maheshwari was collected from Kanyakumari district and authenticated. Ethanol was used for continuous hot percolation extraction of the plant. Antioxidant activity was evaluated using DPPH and ABTS assays. The wound healing potential was assessed through a wound scratch assay using 3T3-L1 murine fibroblast cell lines. Anti-ulcer activity was measured using the acid-neutralizing capacity (ANC) and H+/K+-ATPase inhibitory activity methods. Anti-inflammatory effects were determined through dose-dependent studies of the ethanol extract. Results: Antioxidant Activity: The crude extract exhibited strong antioxidant capacity with percentage inhibition values of 88% (DPPH) and 75.7% (ABTS). Wound Healing Activity: The wound closure rate in treated 3T3-L1 cell lines reached 97.88%, indicating potent wound healing properties. Anti-Ulcer Activity: The extract demonstrated an 80.1% inhibition compared to the control when tested alongside omeprazole. ANC per gram of antacid was recorded at 20 and 26 for oral doses of 10 and 500 {\mu}g/ml, respectively. Anti-Inflammatory Activity: Dose-dependent inhibition percentages of 30.1% (10 {\mu}g/ml) and 96.5% (500 {\mu}g/ml) were observed (p<0.001) relative to inflammation control. Additional percentages of 26.3% (10 {\mu}g/ml) and 42.6% (500 {\mu}g/ml) further confirmed the anti-inflammatory activity. Conclusion: Jatropha maheshwari demonstrates significant antioxidant, wound healing, anti-ulcer, and anti-inflammatory properties. Further research is warranted to elucidate the mechanisms underlying these pharmacological effects.

Token generation speed is critical to power the next wave of AI inference applications. GPUs significantly underperform during token generation due to synchronization overheads at kernel boundaries, utilizing only 21% of their peak memory bandwidth. While recent dataflow architectures mitigate these overheads by enabling aggressive fusion of decoder layers into a single kernel, they too leave performance on the table due to synchronization penalties at layer boundaries. This paper presents kernel looping, a specialized global optimization technique which exploits an optimization opportunity brought by combining the unique layer-level fusion possible in modern dataflow architectures with the repeated layer structure found in language models. Kernel looping eliminates synchronization costs between consecutive calls to the same kernel by transforming these calls into a single call to a modified kernel containing a pipelined outer loop. We evaluate kernel looping on the SambaNova SN40L Reconfigurable Dataflow Unit (RDU), a commercial dataflow accelerator for AI. Experiments demonstrate that kernel looping speeds up the decode phase of a wide array of powerful open-source models by up to 2.2$\times$ on SN40L. Kernel looping allows scaling of decode performance over multiple SN40L sockets, achieving speedups of up to 2.5$\times$. Finally, kernel looping enables SN40L to achieve over 90% of peak performance on 8 and 16 sockets and achieve a speedup of up to 3.7$\times$ over DGX H100. Kernel looping, as well as the models evaluated in this paper, are deployed in production in a commercial AI inference cloud.

The importance of Reinforcement Learning from Human Feedback (RLHF) in aligning large language models (LLMs) with human values cannot be overstated. RLHF is a three-stage process that includes supervised fine-tuning (SFT), reward learning, and policy learning. Although there are several offline and online approaches to aligning LLMs, they often suffer from distribution shift issues. These issues arise from the inability to accurately capture the distributional interdependence between the reward learning and policy learning stages. Consequently, this has led to various approximated approaches, but the theoretical insights and motivations remain largely limited to tabular settings, which do not hold in practice. This gap between theoretical insights and practical implementations is critical. It is challenging to address this gap as it requires analyzing the performance of AI alignment algorithms in neural network-parameterized settings. Although bi-level formulations have shown promise in addressing distribution shift issues, they suffer from the hyper-gradient problem, and current approaches lack efficient algorithms to solve this. In this work, we tackle these challenges employing the bi-level formulation laid out in Kwon et al. (2024) along with the assumption \emph{Weak Gradient Domination} to demonstrate convergence in an RLHF setup, obtaining a sample complexity of $\epsilon^{-\frac{7}{2}}$ . Our key contributions are twofold: (i) We propose a bi-level formulation for AI alignment in parameterized settings and introduce a first-order approach to solve this problem. (ii) We analyze the theoretical convergence rates of the proposed algorithm and derive state-of-the-art bounds. To the best of our knowledge, this is the first work to establish convergence rate bounds and global optimality for the RLHF framework in neural network-parameterized settings.

Medical task-oriented dialogue systems can assist doctors by collecting patient medical history, aiding in diagnosis, or guiding treatment selection, thereby reducing doctor burnout and expanding access to medical services. However, doctor-patient dialogue datasets are not readily available, primarily due to privacy regulations. Moreover, existing datasets lack comprehensive annotations involving medical slots and their different attributes, such as symptoms and their onset, progression, and severity. These comprehensive annotations are crucial for accurate diagnosis. Finally, most existing datasets are non-English, limiting their utility for the larger research community. In response, we introduce MediTOD, a new dataset of doctor-patient dialogues in English for the medical history-taking task. Collaborating with doctors, we devise a questionnaire-based labeling scheme tailored to the medical domain. Then, medical professionals create the dataset with high-quality comprehensive annotations, capturing medical slots and their attributes. We establish benchmarks in supervised and few-shot settings on MediTOD for natural language understanding, policy learning, and natural language generation subtasks, evaluating models from both TOD and biomedical domains. We make MediTOD publicly available for future research., Comment: EMNLP2024 Camera Ready Version

We experimentally determine the bounds of the magnetic-field-induced superconducting and magnetic phases near the crystalline $b$ axis of uranium ditelluride (UTe$_2$). By measuring the magnetoresistance as a function of rotation angle and field strength in magnetic fields as large as 41.5 T, we have studied these boundaries in three dimensions of magnetic field direction. The phase boundaries in all cases obey crystallographic symmetries and no additional symmetries, evidence against any symmetry-breaking quadrupolar or higher magnetic order. We find that the upper critical field of the zero-field superconducting state is well-described by an anisotropic mass model. In contrast, the angular boundaries of the $b$-axis-oriented field-reentrant superconducting phase are nearly constant as a function of field up to the metamagnetic transition, with anisotropy between the $ab$ and $bc$ planes that is comparable to the angular anisotropy of the metamagnetic transition itself. We discuss the relationship between the observed superconducting boundaries and the underlying $\mathbf{d}$ vector that represents the spin-triplet order parameter. Additionally, we report an unexplained normal-state feature in resistance and track its evolution as a function of field strength and angle., Comment: 17 pages, 16 figures

Software Development Waste (SDW) is defined as any resource-consuming activity that does not add value to the client or the organization developing the software. SDW impacts the overall efficiency and productivity of a software project as the scale and size of the project grows. Although engineering leaders usually put in effort to minimize waste, the lack of definitive measures to track and manage SDW is a cause of concern. To address this gap, we propose five measures, namely Stale Forks, Project Diversification Index, PR Rejection Rate, Backlog Inversion Index, and Feature Fulfillment Rate to potentially identify unused artifacts, building the wrong feature/product, mismanagement of backlog types of SDW. We apply these measures on ten open-source projects and share our observations to apply them in practice for managing SDW., Comment: 9 pages, This manuscript is a pre-publication version of the paper that was published at IEEE SEAA 2024

Hyperparameters are a critical factor in reliably training well-performing reinforcement learning (RL) agents. Unfortunately, developing and evaluating automated approaches for tuning such hyperparameters is both costly and time-consuming. As a result, such approaches are often only evaluated on a single domain or algorithm, making comparisons difficult and limiting insights into their generalizability. We propose ARLBench, a benchmark for hyperparameter optimization (HPO) in RL that allows comparisons of diverse HPO approaches while being highly efficient in evaluation. To enable research into HPO in RL, even in settings with low compute resources, we select a representative subset of HPO tasks spanning a variety of algorithm and environment combinations. This selection allows for generating a performance profile of an automated RL (AutoRL) method using only a fraction of the compute previously necessary, enabling a broader range of researchers to work on HPO in RL. With the extensive and large-scale dataset on hyperparameter landscapes that our selection is based on, ARLBench is an efficient, flexible, and future-oriented foundation for research on AutoRL. Both the benchmark and the dataset are available at https://github.com/automl/arlbench., Comment: Accepted at the 17th European Workshop on Reinforcement Learning

The strange metal is a mysterious non-Fermi liquid which shows linear-in-$T$ resistivity behavior at finite temperatures, and, as found in recent experiment, vanishingly small shot noise in the linear-in-$T$ regime. Here, we investigate the shot noise of a strange metal based on a phenomenological model of marginal Fermi liquid (MFL), where fermions couple to some collective boson mode, leading to $T$-linear scattering rate at finite $T$. It is found that in the diffusive regime where the MFL scattering length is small compared to the system size, the shot noise vanishes, and the thermal noise becomes a temperature- and voltage-independent constant. Introducing additional impurity scattering increases the shot noise, and is probably consistent with the current experiment., Comment: 18 pages with 5 figures

Pattern recognition-based myoelectric control is traditionally trained with static or ramp contractions, but this fails to capture the dynamic nature of real-world movements. This study investigated the benefits of training classifiers with continuous dynamic data, encompassing transitions between various movement classes. We employed both conventional (LDA) and deep learning (LSTM) classifiers, comparing their performance when trained with ramp data, continuous dynamic data, and continuous dynamic data augmented with a self-supervised learning technique (VICReg). An online Fitts' Law test with $20$ participants evaluated the usability and effectiveness of each classifier. Results demonstrate that temporal models, particularly LSTMs trained with continuous dynamic data, significantly outperformed traditional approaches. Furthermore, VICReg pre-training led to additional improvements in online performance and user experience. Qualitative feedback highlighted the importance of smooth, jitter-free control and consistent performance across movement classes. These findings underscore the potential of continuous dynamic data and self-supervised learning for advancing sEMG-PR-based myoelectric control, paving the way for more intuitive and user-friendly prosthetic devices., Comment: Submitted to IEEE Transactions on Neural Systems and Rehabilitation Engineering

Despite continued efforts to improve classification accuracy, it has been reported that offline accuracy is a poor indicator of the usability of pattern recognition-based myoelectric control. One potential source of this disparity is the existence of transitions between contraction classes that happen during regular use and are reported to be problematic for pattern recognition systems. Nevertheless, these transitions are often ignored or undefined during both the training and testing processes. In this work, we propose a set of metrics for analyzing the transitions that occur during the voluntary changes between contraction classes during continuous control. These metrics quantify the common types of errors that occur during transitions and compare them to existing metrics that apply only to the steady-state portions of the data. We then use these metrics to analyze transition characteristics of 6 commonly used classifiers on a novel dataset that includes continuous transitions between all combinations of seven different contraction classes. Results show that a linear discriminant classifier consistently outperforms other conventional classifiers during both transitions and steady-state conditions, despite having an almost identical offline performance. Results also show that, although offline training metrics correlate with steady-state performance, they do not correlate with transition performance. These insights suggest that the proposed set of metrics could provide a shift in perspective on the way pattern recognition systems are evaluated and provide a more representative picture of a classifier's performance, potentially narrowing the gap between offline performance and online usability., Comment: Published in Biomedical Signal Processing and Control