Search

We present an empirical study investigating how specific properties of preference datasets, such as mixed-quality or noisy data, affect the performance of Preference Optimization (PO) algorithms. Our experiments, conducted in MuJoCo environments, reveal several scenarios where state-of-the-art PO methods experience significant drops in performance. To address this issue, we introduce a novel PO framework based on mirror descent, which can recover existing methods like Direct Preference Optimization (DPO) and Odds-Ratio Preference Optimization (ORPO) for specific choices of the mirror map. Within this framework, we employ evolutionary strategies to discover new loss functions capable of handling the identified problematic scenarios. These new loss functions lead to significant performance improvements over DPO and ORPO across several tasks. Additionally, we demonstrate the generalization capability of our approach by applying the discovered loss functions to fine-tuning large language models using mixed-quality data, where they outperform ORPO.

Neural networks are traditionally trained under the assumption that data come from a stationary distribution. However, settings which violate this assumption are becoming more popular; examples include supervised learning under distributional shifts, reinforcement learning, continual learning and non-stationary contextual bandits. In this work we introduce a novel learning approach that automatically models and adapts to non-stationarity, via an Ornstein-Uhlenbeck process with an adaptive drift parameter. The adaptive drift tends to draw the parameters towards the initialisation distribution, so the approach can be understood as a form of soft parameter reset. We show empirically that our approach performs well in non-stationary supervised and off-policy reinforcement learning settings.

How to optimally persuade an agent who has a private type? When elicitation is feasible, this amounts to a fairly standard principal-agent-style mechanism design problem, where the persuader employs a mechanism to first elicit the agent's type and then plays the corresponding persuasion strategy based on the agent's report. The optimal mechanism design problem in this setting is relatively well-understood in the literature, with incentive compatible (IC) mechanisms known to be optimal and computationally tractable. In this paper, we study this problem given a credible agent, i.e., if the agent claims they are of a certain type in response to the mechanism's elicitation, then they will act optimally with respect to the claimed type, even if they are actually not of that type. We present several interesting findings in this new setting that differ significantly from results in the non-credible setting. In terms of the structure of optimal mechanisms, we show that not only may IC mechanisms fail to be optimal, but all mechanisms following the standard `eliciting-then-persuading' mechanism design structure may be suboptimal. To achieve optimality requires two additional instruments -- pre-signaling and non-binding elicitation -- which naturally result in multi-stage mechanisms. We characterize optimal mechanisms under these design choices. Based on our characterization, we provide a polynomial-time algorithm for computing optimal multi-stage mechanisms. We also discover that in scenarios that allow for it, partial information elicitation can be employed to improve the principal's payoff even further. Though, surprisingly, an unbounded number of rounds of information exchange between the principal and the agent may be necessary to achieve optimality.

We study fair allocation of indivisible chores to agents under budget constraints, where each chore has an objective size and disutility. This model captures scenarios where a set of chores need to be divided among agents with limited time, and each chore has a specific time needed for completion. We propose a budget-constrained model for allocating indivisible chores, and systematically explore the differences between goods and chores in this setting. We establish the existence of an EFX allocation. We then show that EF2 allocations are polynomial-time computable in general; for many restricted settings, we strengthen this result to EF1. For divisible chores, we develop an efficient algorithm for computing an EF allocation., Comment: Appears in the 17th International Symposium on Algorithmic Game Theory (SAGT), 2024

Supervised fine-tuning (SFT) and alignment of large language models (LLMs) are key steps in providing a good user experience. However, the concept of an appropriate alignment is inherently application-dependent, and current methods often rely on heuristic choices to drive the optimization. In this work, we formulate SFT and alignment as a constrained optimization problem, where the LLM is trained on a task while being required to meet application-specific requirements, without resorting to heuristics. To solve this, we propose Lagrange Large Language Models (L3Ms), which employ logarithmic barriers to enforce the constraints. This approach allows for the customization of L3Ms across diverse applications while avoiding heuristic-driven processes. We demonstrate experimentally the versatility and efficacy of L3Ms in achieving tailored alignments for various applications.

AI systems crucially rely on human ratings, but these ratings are often aggregated, obscuring the inherent diversity of perspectives in real-world phenomenon. This is particularly concerning when evaluating the safety of generative AI, where perceptions and associated harms can vary significantly across socio-cultural contexts. While recent research has studied the impact of demographic differences on annotating text, there is limited understanding of how these subjective variations affect multimodal safety in generative AI. To address this, we conduct a large-scale study employing highly-parallel safety ratings of about 1000 text-to-image (T2I) generations from a demographically diverse rater pool of 630 raters balanced across 30 intersectional groups across age, gender, and ethnicity. Our study shows that (1) there are significant differences across demographic groups (including intersectional groups) on how severe they assess the harm to be, and that these differences vary across different types of safety violations, (2) the diverse rater pool captures annotation patterns that are substantially different from expert raters trained on specific set of safety policies, and (3) the differences we observe in T2I safety are distinct from previously documented group level differences in text-based safety tasks. To further understand these varying perspectives, we conduct a qualitative analysis of the open-ended explanations provided by raters. This analysis reveals core differences into the reasons why different groups perceive harms in T2I generations. Our findings underscore the critical need for incorporating diverse perspectives into safety evaluation of generative AI ensuring these systems are truly inclusive and reflect the values of all users., Comment: 20 pages, 7 figures

The focus of this work is the study of Parikh matrices with emphasis on two concrete problems. In the first part of our presentation we show that a conjecture by Dick at al. in 2021 only stands in the case of ternary alphabets, while providing counterexamples for larger alphabets. In particular, we show that the only type of distinguishability in the case of 3-letter alphabets is the trivial one. The second part of the paper builds on the notion of Parikh matrices for projections of words, discussed initially in this work, and answers, once more in the case of a ternary alphabet, a question posed by Atanasiu et al. in 2022 with regards to the minimal Hamming distance in between words sharing a congruency class., Comment: Submitted for journal publication

We study a fair division model where indivisible items arrive sequentially, and must be allocated immediately and irrevocably. Previous work on online fair division has shown impossibility results in achieving approximate envy-freeness under these constraints. In contrast, we consider an informed setting where the algorithm has complete knowledge of future items, and aim to ensure that the cumulative allocation at each round satisfies approximate envy-freeness -- which we define as temporal envy-freeness up to one item (TEF1). We focus on settings where items can be exclusively goods or exclusively chores. For goods, while TEF1 allocations may not always exist, we identify several special cases where they do -- two agents, two item types, generalized binary valuations, unimodal preferences -- and provide polynomial-time algorithms for these cases. We also prove that determining the existence of a TEF1 allocation is NP-hard. For chores, we establish analogous results for the special cases, but present a slightly weaker intractability result. We also establish the incompatibility between TEF1 and Pareto-optimality, with the implication that it is intractable to find a TEF1 allocation that maximizes any $p$-mean welfare, even for two agents.

We propose SymDiff, a novel method for constructing equivariant diffusion models using the recently introduced framework of stochastic symmetrisation. SymDiff resembles a learned data augmentation that is deployed at sampling time, and is lightweight, computationally efficient, and easy to implement on top of arbitrary off-the-shelf models. Notably, in contrast to previous work, SymDiff typically does not require any neural network components that are intrinsically equivariant, avoiding the need for complex parameterizations and the use of higher-order geometric features. Instead, our method can leverage highly scalable modern architectures as drop-in replacements for these more constrained alternatives. We show that this additional flexibility yields significant empirical benefit on $\mathrm{E}(3)$-equivariant molecular generation. To the best of our knowledge, this is the first application of symmetrisation to generative modelling, suggesting its potential in this domain more generally.

Algorithmic efficiency is essential to reducing energy use and time taken for computational problems. Optimizing efficiency is important for tasks involving multiple resources, for example in stochastic calculations where the size of the random ensemble competes with the time-step. We define the complexity order of an algorithm needing multiple resources as the exponent of inverse total error with respect to the total resources used. The optimum order c is predicted for cases involving competing, factorable resources. As an illustration, this is applied to computing averages in a stochastic differential equation, with numerical examples that agree with earlier literature, as well as with the more general analytic theory that is obtained here.

Intelligent reflecting surface (IRS) and movable antenna (MA) technologies have been proposed to enhance wireless communications by creating favorable channel conditions. This paper investigates the joint beamforming and antenna position design for an MA-enabled IRS (MA-IRS)-aided multi-user multiple-input single-output (MU-MISO) communication system, where the MA-IRS is deployed to aid the communication between the MA-enabled base station (BS) and user equipment (UE). In contrast to conventional fixed position antenna (FPA)-enabled IRS (FPA-IRS), the MA-IRS enhances the wireless channel by controlling the positions of the reflecting elements. To verify the system's effectiveness and optimize its performance, we formulate a sum-rate maximization problem with a minimum rate threshold constraint for the MU-MISO communication. To tackle the non-convex problem, a product Riemannian manifold optimization (PRMO) method is proposed for the joint design of the beamforming and MA positions. Specifically, a product Riemannian manifold space (PRMS) is constructed and the corresponding Riemannian gradient is derived for updating the variables, and the Riemannian exact penalty (REP) method and a Riemannian Broyden-Fletcher-Goldfarb-Shanno (RBFGS) algorithm is derived to obtain a feasible solution over the PRMS. Simulation results demonstrate that compared with the conventional FPA-IRS-aided MU-MISO communication, the reflecting elements of the MA-IRS can move to the positions with higher channel gain, thus enhancing the system performance. Furthermore, it is shown that integrating MA with IRS leads to higher performance gains compared to integrating MA with BS., Comment: 13 pages, 11 figures

We study Dirac fermions in the presence of a space-dependent chiral gauge field and thermodynamic gradients, establishing a connection to the inverse spin Hall effect. The chiral gauge field induces a chiral magnetic field, resulting in a surface Fermi arc state and a chiral Landau level state which, although is delocalized in the bulk, we show to be more robust against impurities. By applying chemical potential and temperature gradients, we achieve nonzero charge currents, with each gradient leading to distinct Fermi level dependencies, both of which have been observed in a recent experiment. Unlike the conventional mixed axial-gravitational anomaly, our currents require a noncollinear chiral magnetic field and thermodynamic gradient. We further derive low-energy transport formulas and demonstrate the importance of carefully treating the ultraviolet cutoff for understanding our lattice calculations., Comment: 7 pages, 3 figures

Future electronics require aggressive scaling of channel material thickness while maintaining device performance. Two-dimensional (2D) semiconductors are promising candidates, but despite over two decades of research, experimental performance still lags theoretical expectations. Here, we develop an oxygen-free approach to push the electrical transport of 2D field-effect transistors toward the theoretical phonon-limited intrinsic mobility. We achieve record carrier mobilities of 91 (132) cm2V-1s-1 for mono- (bi-) layer MoS2 transistors on SiO2 substrate. Statistics from over 60 devices confirm that oxygen-free fabrication enhances key figures of merit by more than an order of magnitude. While previous studies suggest that 2D transition metal dichalcogenides such as MoS2 and WS2 are stable in air, we show that short-term ambient exposure can degrade their device performance through irreversible oxygen chemisorption. This study emphasizes the criticality of avoiding oxygen exposure, offering guidance for device manufacturing for fundamental research and practical applications of 2D materials.

This paper proposes a novel framework for audio deepfake detection with two main objectives: i) attaining the highest possible accuracy on available fake data, and ii) effectively performing continuous learning on new fake data in a few-shot learning manner. Specifically, we conduct a large audio deepfake collection using various deep audio generation methods. The data is further enhanced with additional augmentation methods to increase variations amidst compressions, far-field recordings, noise, and other distortions. We then adopt the Audio Spectrogram Transformer for the audio deepfake detection model. Accordingly, the proposed method achieves promising performance on various benchmark datasets. Furthermore, we present a continuous learning plugin module to update the trained model most effectively with the fewest possible labeled data points of the new fake type. The proposed method outperforms the conventional direct fine-tuning approach with much fewer labeled data points., Comment: Submitted to INTERSPEECH 2024

One method of scaling up quantum systems is to adopt a modular approach. In the ion trap architecture, an efficient photonic interface between independent linear ion traps would allow for such expansion. To this end, an optical cavity with a small mode volume can be utilised to enhance the photon emission probability from the ion. Miniature fibre-based Fabry-Perot cavities have been integrated into three-dimensional Paul traps that hold a single ion, whereas an efficient interface between an optical cavity and a linear trap that can keep multiple ions has remained elusive. This presents a barrier for combining the benefits of the motional coupling in a chain of ions with optical interface between ion traps. In this paper, we show that simple electrically conductive shielding of the fibres could provide substantial advantage in mitigating the adverse effects of stray charges and motional heating by dielectrics. We also reveal that the conductive shields are not compatible with the conventional radio frequency (rf) drive in ion traps but using two rf signals with opposite phases can solve this issue. Furthermore the role played by the symmetry of the electrodes when incorporating an element that disrupts the translational symmetry of a linear trap is elucidated analytically. As a result it is realized that two-dimensional implementation of a linear ion trap such as a surface trap is inherently not suitable for integrating a shielded miniature optical cavity due to the lack of geometrical symmetry. Based on the insights obtained through the analysis, we identify essential components and a design strategy that should be incorporated in a linear ion trap for successful integration of a miniature optical cavity.

The miniaturization of ion trap and the precise placement of its electrodes are necessary for the integration of a microcavity to facilitate efficient ion-cavity coupling. We present a miniature monolithic ion trap made of gold-coated fused silica with high numerical aperture access. A laser writing method referred to as selective laser etching is employed to extract a trap structure from a block of fused silica. The fully monolithic structure eliminates the need for any post-fabrication alignment. Trenches are integrated into this structure such that the various electrodes on the monolithic device remain electrically isolated following their metalization via evaporative coating. We give details of the trap design and production, along with the demonstration of successful trapping of ions and characerization of the trap.

We investigate a model of sequential decision-making where a single alternative is chosen at each round. We focus on two objectives-utilitarian welfare (Util) and egalitarian welfare (Egal)-and consider the computational complexity of the associated maximization problems, as well as their compatibility with strategyproofness and proportionality. We observe that maximizing Util is easy, but the corresponding decision problem for Egal is NP-complete even in restricted cases. We complement this hardness result for Egal with parameterized complexity analysis and an approximation algorithm. Additionally, we show that, while a mechanism that outputs a Util outcome is strategyproof, all deterministic mechanisms for computing Egal outcomes fail a very weak variant of strategyproofness, called non-obvious manipulability (NOM). However, we show that when agents have non-empty approval sets at each timestep, choosing an Egal-maximizing outcome while breaking ties lexicographically satisfies NOM. Regarding proportionality, we prove that a proportional (PROP) outcome can be computed efficiently, but finding an outcome that maximizes Util while guaranteeing PROP is NP-hard. We also derive upper and lower bounds on the price of proportionality with respect to Util and Egal., Comment: Appears in the 27th European Conference on Artificial Intelligence (ECAI), 2024

We study two-stage committee elections where voters have dynamic preferences over candidates; at each stage, a committee is chosen under a given voting rule. We are interested in identifying a winning committee for the second stage that overlaps as much as possible with the first-stage committee. We show a full complexity dichotomy for the class of Thiele rules: this problem is tractable for Approval Voting (AV) and hard for all other Thiele rules (including, in particular, Proportional Approval Voting and the Chamberlin-Courant rule). We extend this dichotomy to the greedy variants of Thiele rules. We also explore this problem from a parameterized complexity perspective for several natural parameters. We complement the theory with experimental analysis: e.g., we investigate the average number of changes in the committee as a function of changes in voters' preferences and the role of ties., Comment: Appears in the 27th European Conference on Artificial Intelligence (ECAI), 2024

By representing any constraint-based causal learning algorithm via a placeholder property, we decompose the correctness condition into a part relating the distribution and the true causal graph, and a part that depends solely on the distribution. This provides a general framework to obtain correctness conditions for causal learning, and has the following implications. We provide exact correctness conditions for the PC algorithm, which are then related to correctness conditions of some other existing causal discovery algorithms. We show that the sparsest Markov representation condition is the weakest correctness condition resulting from existing notions of minimality for maximal ancestral graphs and directed acyclic graphs. We also reason that additional knowledge than just Pearl-minimality is necessary for causal learning beyond faithfulness.

This research addresses the issue of missing structured data in dental records by extracting diagnostic information from unstructured text. The updated periodontology classification system's complexity has increased incomplete or missing structured diagnoses. To tackle this, we use advanced AI and NLP methods, leveraging GPT-4 to generate synthetic notes for fine-tuning a RoBERTa model. This significantly enhances the model's ability to understand medical and dental language. We evaluated the model using 120 randomly selected clinical notes from two datasets, demonstrating its improved diagnostic extraction accuracy. The results showed high accuracy in diagnosing periodontal status, stage, and grade, with Site 1 scoring 0.99 and Site 2 scoring 0.98. In the subtype category, Site 2 achieved perfect scores, outperforming Site 1. This method enhances extraction accuracy and broadens its use across dental contexts. The study underscores AI and NLP's transformative impact on healthcare delivery and management. Integrating AI and NLP technologies enhances documentation and simplifies administrative tasks by precisely extracting complex clinical information. This approach effectively addresses challenges in dental diagnostics. Using synthetic training data from LLMs optimizes the training process, improving accuracy and efficiency in identifying periodontal diagnoses from clinical notes. This innovative method holds promise for broader healthcare applications, potentially improving patient care quality., Comment: 11 pages, 2 tables, 3 figures, under review

Generative models have the potential to accelerate key steps in the discovery of novel molecular therapeutics and materials. Diffusion models have recently emerged as a powerful approach, excelling at unconditional sample generation and, with data-driven guidance, conditional generation within their training domain. Reliably sampling from high-value regions beyond the training data, however, remains an open challenge -- with current methods predominantly focusing on modifying the diffusion process itself. In this paper, we develop context-guided diffusion (CGD), a simple plug-and-play method that leverages unlabeled data and smoothness constraints to improve the out-of-distribution generalization of guided diffusion models. We demonstrate that this approach leads to substantial performance gains across various settings, including continuous, discrete, and graph-structured diffusion processes with applications across drug discovery, materials science, and protein design., Comment: Published in the Proceedings of the 41st International Conference on Machine Learning (ICML 2024)

Knowledge management (KM) involves collecting, organizing, storing, and disseminating information to improve decision-making, innovation, and performance. Implementing KM at scale has become essential for organizations to effectively leverage vast accessible data. This paper is a compilation of concepts that emerged from KM workshops hosted by EMBL-EBI, attended by SMEs and industry. We provide guidance on envisioning, executing, evaluating, and evolving knowledge management platforms. We emphasize essential considerations such as setting knowledge domain boundaries and measuring success, as well as the importance of making knowledge accessible for downstream applications and non-computational users and highlights necessary personal and organizational skills for success. We stress the importance of collaboration and the need for convergence on shared principles and commitment to provide or seek resources to advance KM. The community is invited to join the journey of KM and contribute to the advancement of the field by applying and improving on the guidelines described., Comment: 18 pages, 1 figure

Often times in imitation learning (IL), the environment we collect expert demonstrations in and the environment we want to deploy our learned policy in aren't exactly the same (e.g. demonstrations collected in simulation but deployment in the real world). Compared to policy-centric approaches to IL like behavioural cloning, reward-centric approaches like inverse reinforcement learning (IRL) often better replicate expert behaviour in new environments. This transfer is usually performed by optimising the recovered reward under the dynamics of the target environment. However, (a) we find that modern deep IL algorithms frequently recover rewards which induce policies far weaker than the expert, even in the same environment the demonstrations were collected in. Furthermore, (b) these rewards are often quite poorly shaped, necessitating extensive environment interaction to optimise effectively. We provide simple and scalable fixes to both of these concerns. For (a), we find that reward model ensembles combined with a slightly different training objective significantly improves re-training and transfer performance. For (b), we propose a novel evolution-strategies based method EvIL to optimise for a reward-shaping term that speeds up re-training in the target environment, closing a gap left open by the classical theory of IRL. On a suite of continuous control tasks, we are able to re-train policies in target (and source) environments more interaction-efficiently than prior work., Comment: 17 pages, 8 figures, ICML 2024

A new neutron SIMulation program based on the versatile GEANT4 toolkit, neuSIM4, has been developed to describe interactions of neutrons in the NE213 liquid scintillator from 0.1 to 3000 MeV. neuSIM4 is designed to accommodate complicated modern detector geometry setups with multiple scintillator detectors, each of which can be outfitted with more than one photo-multiplier. To address a broad spectrum of neutron energies, two new neutron interaction physics models, KSCIN and NxQMD, have been implemented in GEANT4. For neutrons with energy below 110 MeV, we incorporate a total of eleven neutron induced reaction channels on hydrogen and carbon nuclei, including nine carbon inelastic reaction channels, into KSCIN. Beyond 110 MeV, we implement a neutron induced reaction model, NxQMD, in GEANT4. We use its results as reference to evaluate other neutron-interaction physics models in GEANT4. We find that results from an existing cascade physics model (INCL) in GEANT4 agree very well with the results from NxQMD, and results from both codes agree with new and existing light response data. To connect KSCIN to NxQMD or INCL, we introduce a transition region where the contribution of neuSIM4 linearly decreases with corresponding increased contributions from NxQMD or INCL. To demonstrate the application of the new code, we simulate the light response and performance of a 2 x 2 m2 neutron detector wall array consisting of 25 2m-long scintillation bars. We are able to compare the predicted light response functions to the shape of the experimental response functions and calculate the efficiency of the neutron detector array for neutron energies up to 200 MeV. These simulation results will be pivotal for understanding the performance of modern neutron arrays with intricate geometries, especially in the measurements of neutron energy spectra in heavy-ion reactions.

When working with 3D facial data, improving fidelity and avoiding the uncanny valley effect is critically dependent on accurate 3D facial performance capture. Because such methods are expensive and due to the widespread availability of 2D videos, recent methods have focused on how to perform monocular 3D face tracking. However, these methods often fall short in capturing precise facial movements due to limitations in their network architecture, training, and evaluation processes. Addressing these challenges, we propose a novel face tracker, FlowFace, that introduces an innovative 2D alignment network for dense per-vertex alignment. Unlike prior work, FlowFace is trained on high-quality 3D scan annotations rather than weak supervision or synthetic data. Our 3D model fitting module jointly fits a 3D face model from one or many observations, integrating existing neutral shape priors for enhanced identity and expression disentanglement and per-vertex deformations for detailed facial feature reconstruction. Additionally, we propose a novel metric and benchmark for assessing tracking accuracy. Our method exhibits superior performance on both custom and publicly available benchmarks. We further validate the effectiveness of our tracker by generating high-quality 3D data from 2D videos, which leads to performance gains on downstream tasks., Comment: 22 pages, 25 figures, to be published in CVPR 2024

We introduce RecurrentGemma, a family of open language models which uses Google's novel Griffin architecture. Griffin combines linear recurrences with local attention to achieve excellent performance on language. It has a fixed-sized state, which reduces memory use and enables efficient inference on long sequences. We provide two sizes of models, containing 2B and 9B parameters, and provide pre-trained and instruction tuned variants for both. Our models achieve comparable performance to similarly-sized Gemma baselines despite being trained on fewer tokens.

Pretraining transformers are generally time-consuming. Fully quantized training (FQT) is a promising approach to speed up pretraining. However, most FQT methods adopt a quantize-compute-dequantize procedure, which often leads to suboptimal speedup and significant performance degradation when used in transformers due to the high memory access overheads and low-precision computations. In this work, we propose Jetfire, an efficient and accurate INT8 training method specific to transformers. Our method features an INT8 data flow to optimize memory access and a per-block quantization method to maintain the accuracy of pretrained transformers. Extensive experiments demonstrate that our INT8 FQT method achieves comparable accuracy to the FP16 training baseline and outperforms the existing INT8 training works for transformers. Moreover, for a standard transformer block, our method offers an end-to-end training speedup of 1.42x and a 1.49x memory reduction compared to the FP16 baseline. Our code is open sourced at https://github.com/thu-ml/Jetfire-INT8Training., Comment: 15 pages, 8 figures, 11 tables

Two-dimensional semiconducting transition metal dichalcogenides (TMDs) are promising for optoelectronic applications due to their strongly bound excitons. While bright excitons have been thoroughly scrutinized, dark excitons are much less investigated as they are not observable with far-field spectroscopy. However, with their non-zero momenta, dark excitons are significant for applications requiring long-range transport or coupling to external fields. We access such dark excitons in WS2 monolayer using transient photoemission electron microscopy with sub-diffraction limited spatial resolution (75 nm) and exceptionally high temporal resolution (13 fs). Image time series of TMD flakes are recorded at several different fluences. We directly observe the ultrafast formation of dark K-Q excitons in monolayer WS2 occurring within 14-50 fs and follow their subsequent picosecond decay. We distinguish exciton dynamics between the interior and edges of the monolayer TMD and conclude that the long-term evolution of dark excitations is defect-mediated while intervalley scattering is not affected., Comment: 25 pages, 3 figures, 1 supplementary

Recent successes suggest that parameter-efficient fine-tuning of foundation models as the state-of-the-art method for transfer learning in vision, replacing the rich literature of alternatives such as meta-learning. In trying to harness the best of both worlds, meta-tuning introduces a subsequent optimization stage of foundation models but has so far only shown limited success and crucially tends to underperform on out-of-distribution (OOD) tasks. In this paper, we introduce Sparse MetA-Tuning (SMAT), a method inspired by sparse mixture-of-experts approaches and trained to isolate subsets of pre-trained parameters automatically for meta-tuning on each task. SMAT successfully overcomes OOD sensitivity and delivers on the promise of enhancing the transfer abilities of vision foundation models beyond parameter-efficient fine-tuning. We establish new state-of-the-art results on a challenging combination of Meta-Dataset augmented with additional OOD tasks in both zero-shot and gradient-based adaptation settings. In addition, we provide a thorough analysis of the superiority of learned over hand-designed sparsity patterns for sparse expert methods and the pivotal importance of the sparsity level in balancing between in-distribution and out-of-distribution generalization. Our code is publicly available., Comment: The Forty-first International Conference on Machine Learning, 2024

Due to the rapid generation and dissemination of information, large language models (LLMs) quickly run out of date despite enormous development costs. To address the crucial need to keep models updated, online learning has emerged as a critical tool when utilizing LLMs for real-world applications. However, given the ever-expanding corpus of unseen documents and the large parameter space of modern LLMs, efficient adaptation is essential. To address these challenges, we propose Memory of Amortized Contexts (MAC), an efficient and effective online adaptation framework for LLMs with strong knowledge retention. We propose a feature extraction and memory-augmentation approach to compress and extract information from new documents into compact modulations stored in a memory bank. When answering questions, our model attends to and extracts relevant knowledge from this memory bank. To learn informative modulations in an efficient manner, we utilize amortization-based meta-learning, which substitutes an otherwise required optimization process with a single forward pass of the encoder. Subsequently, we learn to choose from and aggregate selected documents into a single modulation by conditioning on the question, allowing us to adapt a frozen language model during test time without requiring further gradient updates. Our experiment demonstrates the superiority of MAC in multiple aspects, including online adaptation performance, time, and memory efficiency. In addition, we show how MAC can be combined with and improve the performance of popular alternatives such as retrieval augmented generations (RAGs). Code is available at: https://github.com/jihoontack/MAC., Comment: Published as a conference proceeding for NeurIPS 2024

We consider the problem of online fine tuning the parameters of a language model at test time, also known as dynamic evaluation. While it is generally known that this approach improves the overall predictive performance, especially when considering distributional shift between training and evaluation data, we here emphasize the perspective that online adaptation turns parameters into temporally changing states and provides a form of context-length extension with memory in weights, more in line with the concept of memory in neuroscience. We pay particular attention to the speed of adaptation (in terms of sample efficiency),sensitivity to the overall distributional drift, and the computational overhead for performing gradient computations and parameter updates. Our empirical study provides insights on when online adaptation is particularly interesting. We highlight that with online adaptation the conceptual distinction between in-context learning and fine tuning blurs: both are methods to condition the model on previously observed tokens.

House allocation refers to the problem where $m$ houses are to be allocated to $n$ agents so that each agent receives one house. Since an envy-free house allocation does not always exist, we consider finding such an allocation in the presence of subsidy. We show that computing an envy-free allocation with minimum subsidy is NP-hard in general, but can be done efficiently if $m$ differs from $n$ by an additive constant or if the agents have identical utilities.

Recurrent neural networks (RNNs) have fast inference and scale efficiently on long sequences, but they are difficult to train and hard to scale. We propose Hawk, an RNN with gated linear recurrences, and Griffin, a hybrid model that mixes gated linear recurrences with local attention. Hawk exceeds the reported performance of Mamba on downstream tasks, while Griffin matches the performance of Llama-2 despite being trained on over 6 times fewer tokens. We also show that Griffin can extrapolate on sequences significantly longer than those seen during training. Our models match the hardware efficiency of Transformers during training, and during inference they have lower latency and significantly higher throughput. We scale Griffin up to 14B parameters, and explain how to shard our models for efficient distributed training., Comment: 25 pages, 11 figures

By relaxing conditions for natural structure learning algorithms, a family of constraint-based algorithms containing all exact structure learning algorithms under the faithfulness assumption, we define localised natural structure learning algorithms (LoNS). We also provide a set of necessary and sufficient assumptions for consistency of LoNS, which can be thought of as a strict relaxation of the restricted faithfulness assumption. We provide a practical LoNS algorithm that runs in exponential time, which is then compared with related existing structure learning algorithms, namely PC/SGS and the relatively recent sparsest permutation algorithm. Simulation studies are also provided., Comment: UAI2024

Offline reinforcement learning aims to enable agents to be trained from pre-collected datasets, however, this comes with the added challenge of estimating the value of behavior not covered in the dataset. Model-based methods offer a solution by allowing agents to collect additional synthetic data via rollouts in a learned dynamics model. The prevailing theoretical understanding is that this can then be viewed as online reinforcement learning in an approximate dynamics model, and any remaining gap is therefore assumed to be due to the imperfect dynamics model. Surprisingly, however, we find that if the learned dynamics model is replaced by the true error-free dynamics, existing model-based methods completely fail. This reveals a major misconception. Our subsequent investigation finds that the general procedure used in model-based algorithms results in the existence of a set of edge-of-reach states which trigger pathological value overestimation and collapse in Bellman-based algorithms. We term this the edge-of-reach problem. Based on this, we fill some gaps in existing theory and also explain how prior model-based methods are inadvertently addressing the true underlying edge-of-reach problem. Finally, we propose Reach-Aware Value Learning (RAVL), a simple and robust method that directly addresses the edge-of-reach problem and achieves strong performance across both proprioceptive and pixel-based benchmarks. Code open-sourced at: https://github.com/anyasims/edge-of-reach., Comment: Code open-sourced at: https://github.com/anyasims/edge-of-reach

In the current landscape of deep learning research, there is a predominant emphasis on achieving high predictive accuracy in supervised tasks involving large image and language datasets. However, a broader perspective reveals a multitude of overlooked metrics, tasks, and data types, such as uncertainty, active and continual learning, and scientific data, that demand attention. Bayesian deep learning (BDL) constitutes a promising avenue, offering advantages across these diverse settings. This paper posits that BDL can elevate the capabilities of deep learning. It revisits the strengths of BDL, acknowledges existing challenges, and highlights some exciting research avenues aimed at addressing these obstacles. Looking ahead, the discussion focuses on possible ways to combine large-scale foundation models with BDL to unlock their full potential., Comment: Proceedings of the 41st International Conference on Machine Learning, Vienna, Austria. PMLR 235, 2024

Sequential Bayesian inference over predictive functions is a natural framework for continual learning from streams of data. However, applying it to neural networks has proved challenging in practice. Addressing the drawbacks of existing techniques, we propose an optimization objective derived by formulating continual learning as sequential function-space variational inference. In contrast to existing methods that regularize neural network parameters directly, this objective allows parameters to vary widely during training, enabling better adaptation to new tasks. Compared to objectives that directly regularize neural network predictions, the proposed objective allows for more flexible variational distributions and more effective regularization. We demonstrate that, across a range of task sequences, neural networks trained via sequential function-space variational inference achieve better predictive accuracy than networks trained with related methods while depending less on maintaining a set of representative points from previous tasks., Comment: Published in Proceedings of the 39th International Conference on Machine Learning (ICML 2022)

Reliable predictive uncertainty estimation plays an important role in enabling the deployment of neural networks to safety-critical settings. A popular approach for estimating the predictive uncertainty of neural networks is to define a prior distribution over the network parameters, infer an approximate posterior distribution, and use it to make stochastic predictions. However, explicit inference over neural network parameters makes it difficult to incorporate meaningful prior information about the data-generating process into the model. In this paper, we pursue an alternative approach. Recognizing that the primary object of interest in most settings is the distribution over functions induced by the posterior distribution over neural network parameters, we frame Bayesian inference in neural networks explicitly as inferring a posterior distribution over functions and propose a scalable function-space variational inference method that allows incorporating prior information and results in reliable predictive uncertainty estimates. We show that the proposed method leads to state-of-the-art uncertainty estimation and predictive performance on a range of prediction tasks and demonstrate that it performs well on a challenging safety-critical medical diagnosis task in which reliable uncertainty estimation is essential., Comment: Published in Advances in Neural Information Processing Systems 35 (NeurIPS 2022)

Interrogating the evolution of biological changes at early stages of life requires longitudinal profiling of molecules, such as DNA methylation, which can be challenging with children. We introduce a probabilistic and longitudinal machine learning framework based on multi-mean Gaussian processes (GPs), accounting for individual and gene correlations across time. This method provides future predictions of DNA methylation status at different individual ages while accounting for uncertainty. Our model is trained on a birth cohort of children with methylation profiled at ages 0-4, and we demonstrated that the status of methylation sites for each child can be accurately predicted at ages 5-7. We show that methylation profiles predicted by multi-mean GPs can be used to estimate other phenotypes, such as epigenetic age, and enable comparison to other health measures of interest. This approach encourages epigenetic studies to move towards longitudinal design for investigating epigenetic changes during development, ageing and disease progression., Comment: 18 pages, 12 figures, 3 tables

This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consists of Ultra, Pro, and Nano sizes, suitable for applications ranging from complex reasoning tasks to on-device memory-constrained use-cases. Evaluation on a broad range of benchmarks shows that our most-capable Gemini Ultra model advances the state of the art in 30 of 32 of these benchmarks - notably being the first model to achieve human-expert performance on the well-studied exam benchmark MMLU, and improving the state of the art in every one of the 20 multimodal benchmarks we examined. We believe that the new capabilities of the Gemini family in cross-modal reasoning and language understanding will enable a wide variety of use cases. We discuss our approach toward post-training and deploying Gemini models responsibly to users through services including Gemini, Gemini Advanced, Google AI Studio, and Cloud Vertex AI.

Graph burning is a natural discrete graph algorithm inspired by the spread of social contagion. Despite its simplicity, some open problems remain steadfastly unsolved, notably the burning number conjecture, which says that every connected graph of order $m^2$ has burning number at most $m$. Earlier, we showed that the conjecture also holds for a path forest, which is disconnected, provided each of its paths is sufficiently long. However, finding the least sufficient length for this to hold turns out to be nontrivial. In this note, we present our initial findings and conjectures that associate the problem to some naturally impossibly burnable path forests. It is noteworthy that our problem can be reformulated as a topic concerning sumset partition of integers., Comment: Accepted and published by DMTCS

X-ray circular dichroism, arising from the contrast in X-ray absorption between opposite photon helicities, serves as a spectroscopic tool to measure the magnetization of ferromagnetic materials and identify the handedness of chiral crystals. Antiferromagnets with crystallographic chirality typically lack X-ray magnetic circular dichroism because of time-reversal symmetry, yet exhibit weak X-ray natural circular dichroism. Here, we report the observation of giant natural circular dichroism in the Ni $L_3$-edge X-ray absorption of Ni$_3$TeO$_6$, a polar and chiral antiferromagnet with effective time-reversal symmetry. To unravel this intriguing phenomenon, we propose a phenomenological model that classifies the movement of photons in a chiral crystal within the same symmetry class as that of a magnetic field. The coupling of X-ray polarization with the induced magnetization yields giant X-ray natural circular dichroism, revealing the altermagnetism of Ni$_3$TeO$_6$. Our findings provide evidence for the interplay between magnetism and crystal chirality in natural optical activity. Additionally, we establish the first example of a new class of magnetic materials exhibiting circular dichroism with time-reversal symmetry., Comment: Accepted by Advanced Materials (2024.2.16) Revised title: Giant X-ray circular dichroism in a time-reversal invariant altermagnet Revised drafts: Main 14 pages, 4 figures, and SI 20 pages, 8 figures