Search

The impact of hydration on the reorientational dynamics of the deep eutectic solvent (DES) ethaline, a 2:1 molar mixture of ethylene glycol and choline chloride was studied. Its overall response was explored by means of shear mechanical rheology. To achieve component-selective insights into the dynamics of this material, isotope-edited deuteron and oxygen spin-lattice and spin-spin relaxometry as well as stimulated-echo spectroscopy were applied and yielded motional correlation times from above room temperature down to the highly vis-cous regime. For all temperatures the cholinium anion was found to reorient two times faster than ethylene glycol, while the water and the ethylene glycol molecules display very similar mobilities. While hydration enhances the component dynamics with respect to that of dry ethaline, the present findings reveal that it does not detectably increase the heterogeneity of the solvent. Merely, the timescale similarity that is found for the hydrogen bond donor and the water molecules over a particularly wide temperature range impressively attests to the stability of the native solvent structure in the "water-in-DES" regime.

Maternal mortality remains a significant global public health challenge. One promising approach to reducing maternal deaths occurring during facility-based childbirth is through early warning systems, which require the consistent monitoring of mothers' vital signs after giving birth. Wireless vital sign monitoring devices offer a labor-efficient solution for continuous monitoring, but their scarcity raises the critical question of how to allocate them most effectively. We devise an allocation algorithm for this problem by modeling it as a variant of the popular Restless Multi-Armed Bandit (RMAB) paradigm. In doing so, we identify and address novel, previously unstudied constraints unique to this domain, which render previous approaches for RMABs unsuitable and significantly increase the complexity of the learning and planning problem. To overcome these challenges, we adopt the popular Proximal Policy Optimization (PPO) algorithm from reinforcement learning to learn an allocation policy by training a policy and value function network. We demonstrate in simulations that our approach outperforms the best heuristic baseline by up to a factor of $4$.

In multi-task reinforcement learning, agents train on a fixed set of tasks and have to generalise to new ones. Recent work has shown that increased exploration improves this generalisation, but it remains unclear why exactly that is. In this paper, we introduce the concept of reachability in multi-task reinforcement learning and show that an initial exploration phase increases the number of reachable tasks the agent is trained on. This, and not the increased exploration, is responsible for the improved generalisation, even to unreachable tasks. Inspired by this, we propose a novel method Explore-Go that implements such an exploration phase at the beginning of each episode. Explore-Go only modifies the way experience is collected and can be used with most existing on-policy or off-policy reinforcement learning algorithms. We demonstrate the effectiveness of our method when combined with some popular algorithms and show an increase in generalisation performance across several environments., Comment: arXiv admin note: text overlap with arXiv:2406.08069

Focusing on the bipartite Stable Marriage problem, we investigate different robustness measures related to stable matchings. We analyze the computational complexity of computing them and analyze their behavior in extensive experiments on synthetic instances. For instance, we examine whether a stable matching is guaranteed to remain stable if a given number of adversarial swaps in the agent's preferences are performed and the probability of stability when applying swaps uniformly at random. Our results reveal that stable matchings in our synthetic data are highly unrobust to adversarial swaps, whereas the average-case view presents a more nuanced and informative picture.

LLMs are increasingly used to design reward functions based on human preferences in Reinforcement Learning (RL). We focus on LLM-designed rewards for Restless Multi-Armed Bandits, a framework for allocating limited resources among agents. In applications such as public health, this approach empowers grassroots health workers to tailor automated allocation decisions to community needs. In the presence of multiple agents, altering the reward function based on human preferences can impact subpopulations very differently, leading to complex tradeoffs and a multi-objective resource allocation problem. We are the first to present a principled method termed Social Choice Language Model for dealing with these tradeoffs for LLM-designed rewards for multiagent planners in general and restless bandits in particular. The novel part of our model is a transparent and configurable selection component, called an adjudicator, external to the LLM that controls complex tradeoffs via a user-selected social welfare function. Our experiments demonstrate that our model reliably selects more effective, aligned, and balanced reward functions compared to purely LLM-based approaches.

We study two-stage committee elections where voters have dynamic preferences over candidates; at each stage, a committee is chosen under a given voting rule. We are interested in identifying a winning committee for the second stage that overlaps as much as possible with the first-stage committee. We show a full complexity dichotomy for the class of Thiele rules: this problem is tractable for Approval Voting (AV) and hard for all other Thiele rules (including, in particular, Proportional Approval Voting and the Chamberlin-Courant rule). We extend this dichotomy to the greedy variants of Thiele rules. We also explore this problem from a parameterized complexity perspective for several natural parameters. We complement the theory with experimental analysis: e.g., we investigate the average number of changes in the committee as a function of changes in voters' preferences and the role of ties., Comment: Appears in the 27th European Conference on Artificial Intelligence (ECAI), 2024

Traditional game-theoretic research for security applications primarily focuses on the allocation of external protection resources to defend targets. This work puts forward the study of a new class of games centered around strategically arranging targets to protect them against a constrained adversary, with motivations from varied domains such as peacekeeping resource transit and cybersecurity. Specifically, we introduce Escape Sensing Games (ESGs). In ESGs, a blue player manages the order in which targets pass through a channel, while her opponent tries to capture the targets using a set of sensors that need some time to recharge after each activation. We present a thorough computational study of ESGs. Among others, we show that it is NP-hard to compute best responses and equilibria. Nevertheless, we propose a variety of effective (heuristic) algorithms whose quality we demonstrate in extensive computational experiments.

Novel imaging and neurostimulation techniques open doors for advancements in closed-loop control of activity in biological neural networks. This would allow for applications in the investigation of activity propagation, and for diagnosis and treatment of pathological behaviour. Due to the partially observable characteristics of activity propagation, through networks in which edges can not be observed, and the dynamic nature of neuronal systems, there is a need for adaptive, generalisable control. In this paper, we introduce an environment that procedurally generates neuronal networks with different topologies to investigate this generalisation problem. Additionally, an existing transformer-based architecture is adjusted to evaluate the generalisation performance of a deep RL agent in the presented partially observable environment. The agent demonstrates the capability to generalise control from a limited number of training networks to unseen test networks.

One of the remaining challenges in reinforcement learning is to develop agents that can generalise to novel scenarios they might encounter once deployed. This challenge is often framed in a multi-task setting where agents train on a fixed set of tasks and have to generalise to new tasks. Recent work has shown that in this setting increased exploration during training can be leveraged to increase the generalisation performance of the agent. This makes sense when the states encountered during testing can actually be explored during training. In this paper, we provide intuition why exploration can also benefit generalisation to states that cannot be explicitly encountered during training. Additionally, we propose a novel method Explore-Go that exploits this intuition by increasing the number of states on which the agent trains. Explore-Go effectively increases the starting state distribution of the agent and as a result can be used in conjunction with most existing on-policy or off-policy reinforcement learning algorithms. We show empirically that our method can increase generalisation performance in an illustrative environment and on the Procgen benchmark.

Automated voice calls are an effective method of delivering maternal and child health information to mothers in underserved communities. One method to fight dwindling listenership is through an intervention in which health workers make live service calls. Previous work has shown that we can use AI to identify beneficiaries whose listenership gets the greatest boost from an intervention. It has also been demonstrated that listening to the automated voice calls consistently leads to improved health outcomes for the beneficiaries of the program. These two observations combined suggest the positive effect of AI-based intervention scheduling on behavioral and health outcomes. This study analyzes the relationship between the two. Specifically, we are interested in mothers' health knowledge in the post-natal period, measured through survey questions. We present evidence that improved listenership through AI-scheduled interventions leads to a better understanding of key health issues during pregnancy and infancy. This improved understanding has the potential to benefit the health outcomes of mothers and their babies., Comment: Accepted at Autonomous Agents for Social Good (AASG) workshop at AAMAS'24

Traditional mathematical programming solvers require long computational times to solve constrained minimization problems of complex and large-scale physical systems. Therefore, these problems are often transformed into unconstrained ones, and solved with computationally efficient optimization approaches based on first-order information, such as the gradient descent method. However, for unconstrained problems, balancing the minimization of the objective function with the reduction of constraint violations is challenging. We consider the class of time-dependent minimization problems with increasing (possibly) nonlinear and non-convex objective function and non-decreasing (possibly) nonlinear and non-convex inequality constraints. To efficiently solve them, we propose a penalty-based guardrail algorithm (PGA). This algorithm adapts a standard penalty-based method by dynamically updating the right-hand side of the constraints with a guardrail variable which adds a margin to prevent violations. We evaluate PGA on two novel application domains: a simplified model of a district heating system and an optimization model derived from learned deep neural networks. Our method significantly outperforms mathematical programming solvers and the standard penalty-based method, and achieves better performance and faster convergence than a state-of-the-art algorithm (IPDD) within a specified time limit.

We introduce a new class of two dimensional gravity models using ideas motivated by the Teleparallel Equivalent of General Relativity. This leads to a rather natural formulation of a theory that has close links with Jackiw-Teitelboim gravity. After introducing the theory and discussing its vacuum solutions, we present the Hamiltonian analysis. This implies the presence of a single dynamical degree of freedom, which is in sharp contrast to General Relativity, where there are no degrees of freedom in two spacetime dimensions. Our approach can be extended to various other lower-dimensional gravity theories and thus could be of wider interest., Comment: 22 pages; v2 minor changes

We study azimuthal geodesics in closed Friedmann-Lem\^aitre-Robertson-Walker (FLRW) cosmological models and give the angular distance travelled by a test particle moving along such a geodesic during one cycle of expansion and re-collapse of the universe. We extend previous results regarding the path followed by light rays to the two-fluid case, also including a cosmological constant, as well as to massive test particles., Comment: 19 pages, 9 figures

The dielectric response of liquids reflects both, reorientation of single molecular dipoles and collective modes, i.e., dipolar cross-correlations. A recent theory predicts the latter to produce an additional slow peak in the dielectric loss spectrum. Following this idea we argue that in supercooled liquids the high-frequency power law exponent of the dielectric loss $\beta$ should be correlated with the degree of dipolar order, i.e., the Kirkwood correlation factor $g_K$. This notion is confirmed for 25 supercooled liquids. While our findings support recent theoretical work the results are shown to violate the earlier Kivelson-Madden theory., Comment: To appear in Phys. Rev. Lett

Cosmological models can be studied effectively using dynamical systems techniques. Starting from Brown's formulation of the variational principle for relativistic fluids, we introduce new types of couplings involving a perfect fluid, a scalar field, and boundary terms. We describe three different coupling models, one of which turns out to be particularly relevant for cosmology. Its behaviour is similar to that of models in which dark matter decays into dark energy. In particular, for a constant coupling, the model mimics well-known dynamical dark energy models while the non-constant couplings offer a rich dynamical structure, unseen before. We are able to achieve this richness whilst working in a two-dimensional phase space. This is a significant advantage which allows us to provide a clear physical interpretation of the key features and draw analogies with previously studied models., Comment: 28 pages, 10 figures, 5 tables; v2 minor changes, updated references

When resources are scarce, an allocation policy is needed to decide who receives a resource. This problem occurs, for instance, when allocating scarce medical resources and is often solved using modern ML methods. This paper introduces methods to evaluate index-based allocation policies -- that allocate a fixed number of resources to those who need them the most -- by using data from a randomized control trial. Such policies create dependencies between agents, which render the assumptions behind standard statistical tests invalid and limit the effectiveness of estimators. Addressing these challenges, we translate and extend recent ideas from the statistics literature to present an efficient estimator and methods for computing asymptotically correct confidence intervals. This enables us to effectively draw valid statistical conclusions, a critical gap in previous work. Our extensive experiments validate our methodology in practical settings, while also showcasing its statistical power. We conclude by proposing and empirically verifying extensions of our methodology that enable us to reevaluate a past randomized control trial to evaluate different ML allocation policies in the context of a mHealth program, drawing previously invisible conclusions.

We analyze how numerical experiments regarding elections were conducted within the computational social choice literature (focusing on papers published in the IJCAI, AAAI, and AAMAS conferences). We analyze the sizes of the studied elections and the methods used for generating preference data, thereby making previously hidden standards and practices explicit. In particular, we survey a number of statistical cultures for generating elections and their commonly used parameters.

The trace-free Einstein equations contain one equation less than the complete field equations. In a static and spherically symmetric spacetime, the number of field equations is thus reduced to two. The equation of pressure isotropy of general relativity, however, is preserved thus showing that any known perfect fluid spacetime is a suitable candidate for the trace-free scenario. The extra freedom in imposing two constraints may now be exploited to include polytopes, something that is difficult in general relativity. The point here is that using any known exact solution one can find a polytropic star for various values of the polytropic index. One arrives at Tolman-Oppenheimer-Volkoff type equations and can study their solutions explicitly. Two examples of well-known stellar distributions that generate polytropes with physically reasonable behaviour are discussed. These models are regular, exhibit a sound speed that is never superluminal and are adiabatically stable in the sense of Chandrasekhar. We investigate a compactness measure confirming that our results are consistent with some observational data., Comment: 14 pages, 2 figures

In reinforcement learning (RL), different reward functions can define the same optimal policy but result in drastically different learning performance. For some, the agent gets stuck with a suboptimal behavior, and for others, it solves the task efficiently. Choosing a good reward function is hence an extremely important yet challenging problem. In this paper, we explore an alternative approach for using rewards for learning. We introduce \textit{max-reward RL}, where an agent optimizes the maximum rather than the cumulative reward. Unlike earlier works, our approach works for deterministic and stochastic environments and can be easily combined with state-of-the-art RL algorithms. In the experiments, we study the performance of max-reward RL algorithms in two goal-reaching environments from Gymnasium-Robotics and demonstrate its benefits over standard RL. The code is available at https://github.com/veviurko/To-the-Max.

This white paper outlines a long-term scientific vision for the development of digital-democracy technology. We contend that if digital democracy is to meet the ambition of enabling a participatory renewal in our societies, then a comprehensive multi-methods research effort is required that could, over the years, support its development in a democratically principled, empirically and computationally informed way. The paper is co-authored by an international and interdisciplinary team of researchers and arose from the Lorentz Center Workshop on ``Algorithmic Technology for Democracy'' (Leiden, October 2022).

The Mallows model is a popular distribution for ranked data. We empirically and theoretically analyze how the properties of rankings sampled from the Mallows model change when increasing the number of alternatives. We find that real-world data behaves differently than the Mallows model, yet is in line with its recent variant proposed by Boehmer et al. [2021]. As part of our study, we issue several warnings about using the model., Comment: 33 pages, 9 figures

Modern cosmology is closely linked to our understanding of radial null geodesics as these model the propagation of light signals through an expanding universe. Azimuthal geodesics, on the other hand, are perhaps best known for their relevance within closed cosmological models. Such models typically have a finite lifetime: the universe expands up to a maximum size after which it recollapses during the so-called big crunch. An azimuthal geodesic starting at the beginning of the universe will travel a finite angular distance during the expansion and recollapse. It is well-known that this angle is $2\pi$ for a matter-dominated universe and $\pi$ for a radiation-dominated solution. Here we derive the simple formula $$\Delta \varphi = \frac{2\pi}{1+3w}$$ for an arbitrary linear equation of state parameter $w$. To the best of our knowledge this result has not been reported elsewhere and fills a small gap in the literature., Comment: 7 pages

We study ways of evaluating the performance of losing projects in participatory budgeting (PB) elections by seeking actions that would have led to their victory. We focus on lowering the projects' costs, obtaining additional approvals for them, and asking supporters to refrain from approving other projects: The larger a change is needed, the less successful is the given project. We seek efficient algorithms for computing our measures and we analyze and compare them experimentally. We focus on the greedyAV, Phragm\'en, and Equal-Shares PB rules.

We provide the first large-scale data collection of real-world approval-based committee elections. These elections have been conducted on the Polkadot blockchain as part of their Nominated Proof-of-Stake mechanism and contain around one thousand candidates and tens of thousands of (weighted) voters each. We conduct an in-depth study of application-relevant questions, including a quantitative and qualitative analysis of the outcomes returned by different voting rules. Besides considering proportionality measures that are standard in the multiwinner voting literature, we pay particular attention to less-studied measures of overrepresentation, as these are closely related to the security of the Polkadot network. We also analyze how different design decisions such as the committee size affect the examined measures., Comment: Accepted to AAAI'24

Agents trained with DQN rely on an observation at each timestep to decide what action to take next. However, in real world applications observations can change or be missing entirely. Examples of this could be a light bulb breaking down, or the wallpaper in a certain room changing. While these situations change the actual observation, the underlying optimal policy does not change. Because of this we want our agent to continue taking actions until it receives a (recognized) observation again. To achieve this we introduce a combination of a neural network architecture that uses hidden representations of the observations and a novel n-step loss function. Our implementation is able to withstand location based blindness stretches longer than the ones it was trained on, and therefore shows robustness to temporary blindness. For access to our implementation, please email Nathan, Marije, or Pau.

Physical ageing is the generic term for irreversible processes in glassy materials resulting from molecular rearrangements. One formalism for describing such ageing processes involves the concept of material time, which may be thought of as time measured on a clock whose rate changes as the glass ages. Experimental determination of material time has so far not been realized, however. Here, we show how dynamic light-scattering measurements provide a way forward. We determined the material time for an ageing sample of the glass-former 1-phenyl-1-propanol after temperature jumps close to the glass transition from the time-autocorrelation function of the intensity fluctuations probed by multispeckle dynamic light scattering. These fluctuations are shown to be stationary and reversible when regarded as a function of the material time. The glass-forming colloidal synthetic clay Laponite and a chemically ageing curing epoxy are also shown to display material-time-reversible scattered-light intensity fluctuations, and simulations of an ageing binary system monitoring the potential energy confirm material-time reversibility. In addition to demonstrating direct measurements of the material time, our findings identify a fundamental property of ageing in quite different contexts that presents a challenge to the current theories of ageing.

The glass formation and the dipolar reorientational motions in deep eutectic solvents (DESs) are frequently overlooked, despite their crucial role in defining the room-temperature physiochemical properties. To understand the effects of these dynamics on the ionic conductivity and their relation to the mechanical properties of the DES, we conducted broadband dielectric and rheological spectroscopy over a wide temperature range on three well-established carboxylic-acid-based natural DESs: oxaline, maline, and the eutectic mixture of choline chloride with phenylacetic acid (phenylaceline). In all three DESs, we observe signs of glassy freezing in the temperature dependence of their dipolar reorientational and structural dynamics, as well as varying degrees of motional decoupling between the different observed dynamics: Maline and oxaline display a breaking of the Walden rule near the glass-transition temperature, while the relation between the dc conductivity and dipolar relaxation time in both maline and phenylaceline is best described by a power law. The glass-forming properties of the investigated systems not only govern the orientational dipolar motions and rheological properties, which are of interest from a fundamental point of view, but they also affect the dc conductivity, even at room temperature, which is of high technical relevance., Comment: 11 pages, 6 figures + Supplemental Material (3 pages, 4 figures)

Since 2018, the potential for a high-energy neutrino telescope, named the Pacific Ocean Neutrino Experiment (P-ONE), has been thoroughly examined by two pathfinder missions, STRAW and STRAW-b, short for short for Strings for Absorption Length in Water. The P-ONE project seeks to install a neutrino detector with a one cubic kilometer volume in the Cascadia Basin's deep marine surroundings, situated near the western shores of Vancouver Island, Canada. To assess the environmental conditions and feasibility of constructing a neutrino detector of that scale, the pathfinder missions, STRAW and STRAW-b, have been deployed at a depth of 2.7 km within the designated site for P-ONE and were connected to the NEPTUNE observatory, operated by Ocean Networks Canada (ONC). While STRAW focused on analyzing the optical properties of water in the Cascadia Basin, \ac{strawb} employed cameras and spectrometers to investigate the characteristics of bioluminescence in the deep-sea environment. This report introduces the STRAW-b concept, covering its scientific objectives and the instrumentation used. Furthermore, it discusses the design considerations implemented to guarantee a secure and dependable deployment process of STRAW-b. Additionally, it showcases the data collected by battery-powered loggers, which monitored the mechanical stress on the equipment throughout the deployment. The report also offers an overview of STRAW-b's operation, with a specific emphasis on the notable advancements achieved in the data acquisition (DAQ) system and its successful integration with the server infrastructure of ONC., Comment: 20 pages, 11 figures, 2 tables

In this paper, we address the problem of real-time motion planning for multiple robotic manipulators that operate in close proximity. We build upon the concept of dynamic fabrics and extend them to multi-robot systems, referred to as Multi-Robot Dynamic Fabrics (MRDF). This geometric method enables a very high planning frequency for high-dimensional systems at the expense of being reactive and prone to deadlocks. To detect and resolve deadlocks, we propose Rollout Fabrics where MRDF are forward simulated in a decentralized manner. We validate the methods in simulated close-proximity pick-and-place scenarios with multiple manipulators, showing high success rates and real-time performance., Comment: 6 pages + 1 page references, 2 tables, 4 figures, preprint version to accepted paper to IEEE International Symposium on Multi-Robot & Multi-Agent Systems, Boston, 2023

Inclusive e$^+$e$^-$ production has been studied with HADES in $\pi^-$ + p, $\pi^-$ + C and $\pi^- + \mathrm{CH}_2$ reactions, using the GSI pion beam at $\sqrt{s_{\pi p}}$ = 1.49 GeV. Invariant mass and transverse momentum distributions have been measured and reveal contributions from Dalitz decays of $\pi^0$, $\eta$ mesons and baryon resonances. The transverse momentum distributions are very sensitive to the underlying kinematics of the various processes. The baryon contribution exhibits a deviation up to a factor seven from the QED reference expected for the dielectron decay of a hypothetical point-like baryon with the production cross section constrained from the inverse $\gamma$ n$\rightarrow \pi^-$ p reaction. The enhancement is attributed to a strong four-momentum squared dependence of the time-like electromagnetic transition form factors as suggested by Vector Meson Dominance (VMD). Two versions of the VMD, that differ in the photon-baryon coupling, have been applied in simulations and compared to data. VMD1 (or two-component VMD) assumes a coupling via the $\rho$ meson and a direct coupling of the photon, while in VMD2 (or strict VMD) the coupling is only mediated via the $\rho$ meson. The VMD2 model, frequently used in transport calculations for dilepton decays, is found to overestimate the measured dielectron yields, while a good description of the data can be obtained with the VMD1 model assuming no phase difference between the two amplitudes. Similar descriptions have also been obtained using a time-like baryon transition form factor model where the pion cloud plays the major role., Comment: (HADES collaboration)

A high-resolution, temperature oscillation-based probe of physical aging in complex systems is introduced. The Fourier analysis of the measured responses allows one to extract a high-order aging-related complex susceptibility that is not accessible via traditional temperature-jump and -ramp procedures. To demonstrate the potential of this oscillatory approach, we analyze the periodic time evolution of glycerol's structural relaxation using shear rheology as a vehicle. Thereby, we access up to the sixth harmonics and detect aging fingerprints within a resolution range of three orders of magnitude for temperature amplitudes of up to 4 K. The high-order aging coefficients and susceptibilities obtained for glycerol are described well by the Tool-Narayanaswami-Moynihan formalism., Comment: 5 pages, 4 figures

It is well known that the Einstein-Hilbert action in two dimensions is topological and yields an identically vanishing Einstein tensor. Consequently one is faced with difficulties when formulating a non-trivial gravity model. We present a new, intrinsically two-dimensional, approach to this problem based on the Einstein action. This yields a well defined variational approach which results in new field equations that break diffeomorphism invariance. Our proposed approach does not require the introduction of additional scalar fields, nor the use of conformal transformations. However, we can show how including conformal counter terms leads to equivalent results. In doing so, we can provide an explanation for why previous approaches worked. Solutions to the field equations are briefly discussed., Comment: 8 pages. Final version accepted for publication

Predict and optimize is an increasingly popular decision-making paradigm that employs machine learning to predict unknown parameters of optimization problems. Instead of minimizing the prediction error of the parameters, it trains predictive models using task performance as a loss function. The key challenge to train such models is the computation of the Jacobian of the solution of the optimization problem with respect to its parameters. For linear problems, this Jacobian is known to be zero or undefined; hence, approximations are usually employed. For non-linear convex problems, however, it is common to use the exact Jacobian. This paper demonstrates that the zero-gradient problem appears in the non-linear case as well -- the Jacobian can have a sizeable null space, thereby causing the training process to get stuck in suboptimal points. Through formal proofs, this paper shows that smoothing the feasible set resolves this problem. Combining this insight with known techniques from the literature, such as quadratic programming approximation and projection distance regularization, a novel method to approximate the Jacobian is derived. In simulation experiments, the proposed method increases the performance in the non-linear case and at least matches the existing state-of-the-art methods for linear problems.

We consider the problem of subset selection where one is given multiple rankings of items and the goal is to select the highest ``quality'' subset. Score functions from the multiwinner voting literature have been used to aggregate rankings into quality scores for subsets. We study this setting of subset selection problems when, in addition, rankings may contain systemic or unconscious biases toward a group of items. For a general model of input rankings and biases, we show that requiring the selected subset to satisfy group fairness constraints can improve the quality of the selection with respect to unbiased rankings. Importantly, we show that for fairness constraints to be effective, different multiwinner score functions may require a drastically different number of rankings: While for some functions, fairness constraints need an exponential number of rankings to recover a close-to-optimal solution, for others, this dependency is only polynomial. This result relies on a novel notion of ``smoothness'' of submodular functions in this setting that quantifies how well a function can ``correctly'' assess the quality of items in the presence of bias. The results in this paper can be used to guide the choice of multiwinner score functions for the subset selection setting considered here; we additionally provide a tool to empirically enable this., Comment: The conference version of this paper appears in ICML 2023

In contrast to classical reinforcement learning, distributional reinforcement learning algorithms aim to learn the distribution of returns rather than their expected value. Since the nature of the return distribution is generally unknown a priori or arbitrarily complex, a common approach finds approximations within a set of representable, parametric distributions. Typically, this involves a projection of the unconstrained distribution onto the set of simplified distributions. We argue that this projection step entails a strong inductive bias when coupled with neural networks and gradient descent, thereby profoundly impacting the generalization behavior of learned models. In order to facilitate reliable uncertainty estimation through diversity, this work studies the combination of several different projections and representations in a distributional ensemble. We establish theoretical properties of such projection ensembles and derive an algorithm that uses ensemble disagreement, measured by the average $1$-Wasserstein distance, as a bonus for deep exploration. We evaluate our algorithm on the behavior suite benchmark and find that diverse projection ensembles lead to significant performance improvements over existing methods on a wide variety of tasks with the most pronounced gains in directed exploration problems., Comment: 21 pages, 7 figures, submitted to NeurIPS 2023

In reinforcement learning (RL), key components of many algorithms are the exploration strategy and replay buffer. These strategies regulate what environment data is collected and trained on and have been extensively studied in the RL literature. In this paper, we investigate the impact of these components in the context of generalisation in multi-task RL. We investigate the hypothesis that collecting and training on more diverse data from the training environments will improve zero-shot generalisation to new tasks. We motivate mathematically and show empirically that generalisation to tasks that are "reachable'' during training is improved by increasing the diversity of transitions in the replay buffer. Furthermore, we show empirically that this same strategy also shows improvement for generalisation to similar but "unreachable'' tasks which could be due to improved generalisation of the learned latent representations., Comment: 15 pages, 8 figures