Search

As robots become more common for both able-bodied individuals and those living with a disability, it is increasingly important that lay people be able to drive multi-degree-of-freedom platforms with low-dimensional controllers. One approach is to use state-conditioned action mapping methods to learn mappings between low-dimensional controllers and high DOF manipulators -- prior research suggests these mappings can simplify the teleoperation experience for users. Recent works suggest that neural networks predicting a local linear function are superior to the typical end-to-end multi-layer perceptrons because they allow users to more easily undo actions, providing more control over the system. However, local linear models assume actions exist on a linear subspace and may not capture nuanced actions in training data. We observe that the benefit of these mappings is being an odd function concerning user actions, and propose end-to-end nonlinear action maps which achieve this property. Unfortunately, our experiments show that such modifications offer minimal advantages over previous solutions. We find that nonlinear odd functions behave linearly for most of the control space, suggesting architecture structure improvements are not the primary factor in data-driven teleoperation. Our results suggest other avenues, such as data augmentation techniques and analysis of human behavior, are necessary for action maps to become practical in real-world applications, such as in assistive robotics to improve the quality of life of people living with w disability., Comment: 13 Pages, 7 Figures, presented at Collaborative AI and Modeling of Humans AAAI Bridge Program Submission

In recent times, Reinforcement learning (RL) has been widely applied to many challenging tasks. However, in order to perform well, it requires access to a good reward function which is often sparse or manually engineered with scope for error. Introducing human prior knowledge is often seen as a possible solution to the above-mentioned problem, such as imitation learning, learning from preference, and inverse reinforcement learning. Learning from feedback is another framework that enables an RL agent to learn from binary evaluative signals describing the teacher's (positive or negative) evaluation of the agent's action. However, these methods often make the assumption that evaluative teacher feedback is perfect, which is a restrictive assumption. In practice, such feedback can be noisy due to limited teacher expertise or other exacerbating factors like cognitive load, availability, distraction, etc. In this work, we propose the CANDERE-COACH algorithm, which is capable of learning from noisy feedback by a nonoptimal teacher. We propose a noise-filtering mechanism to de-noise online feedback data, thereby enabling the RL agent to successfully learn with up to 40% of the teacher feedback being incorrect. Experiments on three common domains demonstrate the effectiveness of the proposed approach.

Traditionally, Reinforcement Learning (RL) problems are aimed at optimization of the behavior of an agent. This paper proposes a novel take on RL, which is used to learn the policy of another agent, to allow real-time recognition of that agent's goals. Goal Recognition (GR) has traditionally been framed as a planning problem where one must recognize an agent's objectives based on its observed actions. Recent approaches have shown how reinforcement learning can be used as part of the GR pipeline, but are limited to recognizing predefined goals and lack scalability in domains with a large goal space. This paper formulates a novel problem, "Online Dynamic Goal Recognition" (ODGR), as a first step to address these limitations. Contributions include introducing the concept of dynamic goals into the standard GR problem definition, revisiting common approaches by reformulating them using ODGR, and demonstrating the feasibility of solving ODGR in a navigation domain using transfer learning. These novel formulations open the door for future extensions of existing transfer learning-based GR methods, which will be robust to changing and expansive real-time environments., Comment: 8 pages, 1 figure, RLC workshop, WAHT workshop

Optimizing warehouse layouts is crucial due to its significant impact on efficiency and productivity. We present an AI-driven framework for automated warehouse layout generation. This framework employs constrained beam search to derive optimal layouts within given spatial parameters, adhering to all functional requirements. The feasibility of the generated layouts is verified based on criteria such as item accessibility, required minimum clearances, and aisle connectivity. A scoring function is then used to evaluate the feasible layouts considering the number of storage locations, access points, and accessibility costs. We demonstrate our method's ability to produce feasible, optimal layouts for a variety of warehouse dimensions and shapes, diverse door placements, and interconnections. This approach, currently being prepared for deployment, will enable human designers to rapidly explore and confirm options, facilitating the selection of the most appropriate layout for their use-case.

Galaxy cluster abundance measurements provide a classic test of cosmology. They are most sensitive to the evolved amplitude of fluctuations, usually expressed as $S_8 = \sigma_8\sqrt{\Omega_m/0.3}$. Thus, abundance constraints exhibit a strong degeneracy between $\sigma_8$ and $\Omega_{\rm m}$, as do other similar low-redshift tests such as cosmic shear. The mass distribution in the infall region around galaxy clusters, where material is being accreted from the surrounding field, also exhibits a cosmological dependence, but in this case it is nearly orthogonal to the $S_8$ direction in the $\Omega_m$--$\sigma_8$ plane, making it highly complementary to halo abundance or cosmic shear studies. We explore how weak lensing measurements of the infall region might be used to complement abundance studies, considering three different tests. The splashback radius is a prominent feature of the infall region; we show that detection of this feature in lensing data from the Euclid survey could independently constrain $\Omega_{\rm m}$ and $\sigma_8$ to $\pm 0.05$. Another feature, the depletion radius where the bias reaches a minimum, also shows cosmological dependence, though it is challenging to observe in practice. The strongest constraints come from direct measurements of the shear profile in the infall region at $2$--$4\,r_{200{\rm c}}$. Combining the latter with abundance constraints such as those reported from SRG$/$eROSITA should reduce the area of the error contours by an estimated factor of $1.2$ using a sample of clusters observed by the UNIONS survey, or a factor of $3$ using clusters observed by the Euclid Wide survey over a broader range of redshift., Comment: 14 pages, 10 figures

It is well known that there are persistent statistical tensions with the standard model in the low Q2 HERA deep inelastic scattering neutral current data characterized by a turn-over of F2(x, Q2) at low x and low Q2. One important experimental signature that sheds light on this low Q2 region is the determination of the longitudinal structure function FL(x, Q2). This paper describes a novel method to determine FL based on an extrapolation of the reduced NC cross section at fixed s and Q to the minimum value of x given by Q2/s. At this kinematic point, the reduced cross section equals 2xF1 = F2 - FL so that a determination of both this value and the value of F2, determines FL. Since the polarization of the exchanged photon is transverse at this kinematic point, we expect FL to be small because its dominate gluon component is strongly suppressed. Surprisingly, we find FL at low Q2 to be much larger than expectation and observe that both FL and F2 at x = Q2/s show several properties consistent with the dipole picture. We discuss the statistical as well as chief systematic errors of our method and we tabulate our determinations of F2, 2xF1 and FL in the Appendix., Comment: 42 pages, 13 figures

We introduce a new family of video prediction models designed to support downstream control tasks. We call these models Video Occupancy models (VOCs). VOCs operate in a compact latent space, thus avoiding the need to make predictions about individual pixels. Unlike prior latent-space world models, VOCs directly predict the discounted distribution of future states in a single step, thus avoiding the need for multistep roll-outs. We show that both properties are beneficial when building predictive models of video for use in downstream control. Code is available at \href{https://github.com/manantomar/video-occupancy-models}{\texttt{github.com/manantomar/video-occupancy-models}}.

Cosmological parameters such as $\Omega_{\rm{M}}$ and $\sigma_{8}$ can be measured indirectly using various methods, including galaxy cluster abundance and cosmic shear. These measurements constrain the composite parameter $S_{8}$, leading to degeneracy between $\Omega_{\rm{M}}$ and $\sigma_{8}$. However, some structural properties of galaxy clusters also correlate with cosmological parameters, due to their dependence on a cluster's accretion history. In this work, we focus on the splashback radius, an observable cluster feature that represents a boundary between a cluster and the surrounding Universe. Using a suite of cosmological simulations with a range of values for $\Omega_{\rm{M}}$ and $\sigma_{8}$, we show that the position of the splashback radius around cluster-mass halos is greater in cosmologies with smaller values of $\Omega_{\rm{M}}$ or larger values of $\sigma_{8}$. This variation breaks the degeneracy between $\Omega_{\rm{M}}$ and $\sigma_{8}$ that comes from measurements of the $S_{8}$ parameter. We also show that this variation is, in principle, measurable in observations. As the splashback radius can be determined from the same weak lensing analysis already used to estimate $S_{8}$, this new approach can tighten low-redshift constraints on cosmological parameters, either using existing data, or using upcoming data such as that from Euclid and LSST., Comment: 12 pages, 7 figures, 1 table, accepted for publication in ApJ

Numerous metrics exist to quantify the dynamical state of galaxy clusters, both observationally and within simulations. Many of these correlate strongly with one another, but it is not clear whether all of these measures probe the same intrinsic properties. In this work, we use two different statistical approaches -- principal component analysis (PCA) and uniform manifold approximation and projection (UMAP) -- to investigate which dynamical properties of a cluster are in fact the best descriptors of its dynamical state. We use measurements taken directly from The Three Hundred suite of galaxy cluster simulations, as well as morphological properties calculated using mock X-ray and SZ maps of the same simulated clusters. We find that four descriptions of dynamical state naturally arise, and although correlations exist between these, a given cluster can be "dynamically relaxed" according to all, none, or some of these four descriptions. These results demonstrate that it is highly important for future observational and theoretical studies to consider in which sense clusters are dynamically relaxed. Cluster dynamical states are complex and multi-dimensional, and so it is not meaningful to classify them simply as "relaxed" and "unrelaxed" based on a single linear scale., Comment: 18 pages, 7 figures, 3 tables, accepted for publication in MNRAS

For autonomous agents to successfully integrate into human-centered environments, agents should be able to learn from and adapt to humans in their native settings. Preference-based reinforcement learning (PbRL) is a promising approach that learns reward functions from human preferences. This enables RL agents to adapt their behavior based on human desires. However, humans live in a world full of diverse information, most of which is not relevant to completing a particular task. It becomes essential that agents learn to focus on the subset of task-relevant environment features. Unfortunately, prior work has largely ignored this aspect; primarily focusing on improving PbRL algorithms in standard RL environments that are carefully constructed to contain only task-relevant features. This can result in algorithms that may not effectively transfer to a more noisy real-world setting. To that end, this work proposes R2N (Robust-to-Noise), the first PbRL algorithm that leverages principles of dynamic sparse training to learn robust reward models that can focus on task-relevant features. We study the effectiveness of R2N in the Extremely Noisy Environment setting, an RL problem setting where up to 95% of the state features are irrelevant distractions. In experiments with a simulated teacher, we demonstrate that R2N can adapt the sparse connectivity of its neural networks to focus on task-relevant features, enabling R2N to significantly outperform several state-of-the-art PbRL algorithms in multiple locomotion and control environments.

The current standard model of cosmology successfully describes a variety of measurements, but the nature of its main ingredients, dark matter and dark energy, remains unknown. Euclid is a medium-class mission in the Cosmic Vision 2015-2025 programme of the European Space Agency (ESA) that will provide high-resolution optical imaging, as well as near-infrared imaging and spectroscopy, over about 14,000 deg^2 of extragalactic sky. In addition to accurate weak lensing and clustering measurements that probe structure formation over half of the age of the Universe, its primary probes for cosmology, these exquisite data will enable a wide range of science. This paper provides a high-level overview of the mission, summarising the survey characteristics, the various data-processing steps, and data products. We also highlight the main science objectives and expected performance., Comment: Accepted for publication in the A&A special issue`Euclid on Sky'

To create useful reinforcement learning (RL) agents, step zero is to design a suitable reward function that captures the nuances of the task. However, reward engineering can be a difficult and time-consuming process. Instead, human-in-the-loop (HitL) RL allows agents to learn reward functions from human feedback. Despite recent successes, many of the HitL RL methods still require numerous human interactions to learn successful reward functions. To improve the feedback efficiency of HitL RL methods (i.e., require less feedback), this paper introduces Sub-optimal Data Pre-training, SDP, an approach that leverages reward-free, sub-optimal data to improve scalar- and preference-based HitL RL algorithms. In SDP, we start by pseudo-labeling all low-quality data with rewards of zero. Through this process, we obtain free reward labels to pre-train our reward model. This pre-training phase provides the reward model a head start in learning, whereby it can identify that low-quality transitions should have a low reward, all without any actual feedback. Through extensive experiments with a simulated teacher, we demonstrate that SDP can significantly improve or achieve competitive performance with state-of-the-art (SOTA) HitL RL algorithms across nine robotic manipulation and locomotion tasks.

Gaze-based interactions offer a potential way for users to naturally engage with mixed reality (XR) interfaces. Black-box machine learning models enabled higher accuracy for gaze-based interactions. However, due to the black-box nature of the model, users might not be able to understand and effectively adapt their gaze behaviour to achieve high quality interaction. We posit that explainable AI (XAI) techniques can facilitate understanding of and interaction with gaze-based model-driven system in XR. To study this, we built a real-time, multi-level XAI interface for gaze-based interaction using a deep learning model, and evaluated it during a visual search task in XR. A between-subjects study revealed that participants who interacted with XAI made more accurate selections compared to those who did not use the XAI system (i.e., F1 score increase of 10.8%). Additionally, participants who used the XAI system adapted their gaze behavior over time to make more effective selections. These findings suggest that XAI can potentially be used to assist users in more effective collaboration with model-driven interactions in XR.

This paper introduces the problem of learning to place logic blocks in Field-Programmable Gate Arrays (FPGAs) and a learning-based method. In contrast to previous search-based placement algorithms, we instead employ Reinforcement Learning (RL) with the goal of minimizing wirelength. In addition to our preliminary learning results, we also evaluated a novel decomposition to address the nature of large search space when placing many blocks on a chipboard. Empirical experiments evaluate the effectiveness of the learning and decomposition paradigms on FPGA placement tasks., Comment: accepted by ISEDA2024

In reinforcement learning (RL), an agent learns to perform a task by interacting with an environment and receiving feedback (a numerical reward) for its actions. However, the assumption that rewards are always observable is often not applicable in real-world problems. For example, the agent may need to ask a human to supervise its actions or activate a monitoring system to receive feedback. There may even be a period of time before rewards become observable, or a period of time after which rewards are no longer given. In other words, there are cases where the environment generates rewards in response to the agent's actions but the agent cannot observe them. In this paper, we formalize a novel but general RL framework - Monitored MDPs - where the agent cannot always observe rewards. We discuss the theoretical and practical consequences of this setting, show challenges raised even in toy environments, and propose algorithms to begin to tackle this novel setting. This paper introduces a powerful new formalism that encompasses both new and existing problems and lays the foundation for future research., Comment: AAMAS 2024, Main Track

One of the final frontiers in the development of complex human - AI collaborative systems is the ability of AI agents to comprehend the natural language and perform tasks accordingly. However, training efficient Reinforcement Learning (RL) agents grounded in natural language has been a long-standing challenge due to the complexity and ambiguity of the language and sparsity of the rewards, among other factors. Several advances in reinforcement learning, curriculum learning, continual learning, language models have independently contributed to effective training of grounded agents in various environments. Leveraging these developments, we present a novel algorithm, Grounded Language Instruction through DEmonstration in RL (GLIDE-RL) that introduces a teacher-instructor-student curriculum learning framework for training an RL agent capable of following natural language instructions that can generalize to previously unseen language instructions. In this multi-agent framework, the teacher and the student agents learn simultaneously based on the student's current skill level. We further demonstrate the necessity for training the student agent with not just one, but multiple teacher agents. Experiments on a complex sparse reward environment validates the effectiveness of our proposed approach., Comment: 12 pages, 6 figures, to be presented at AAMAS 2024

Fine-tuning Large Language Models (LLMs) adapts a trained model to specific downstream tasks, significantly improving task-specific performance. Supervised Fine-Tuning (SFT) is a common approach, where an LLM is trained to produce desired answers. However, LLMs trained with SFT sometimes make simple mistakes and result in hallucinations on reasoning tasks such as question-answering. Without external feedback, it is difficult for SFT to learn a good mapping between the question and the desired answer, especially with a small dataset. This paper introduces an alternative to SFT called Natural Language Feedback for Finetuning LLMs (LaFFi). LaFFi has LLMs directly predict the feedback they will receive from an annotator. We find that requiring such reflection can significantly improve the accuracy in in-domain question-answering tasks, providing a promising direction for the application of natural language feedback in the realm of SFT LLMs. Additional ablation studies show that the portion of human-annotated data in the annotated datasets affects the fine-tuning performance., Comment: Paper accepted in Human-Centric Representation Learning workshop at AAAI 2024 (https://hcrl-workshop.github.io/2024/)

The visual world provides an abundance of information, but many input pixels received by agents often contain distracting stimuli. Autonomous agents need the ability to distinguish useful information from task-irrelevant perceptions, enabling them to generalize to unseen environments with new distractions. Existing works approach this problem using data augmentation or large auxiliary networks with additional loss functions. We introduce MaDi, a novel algorithm that learns to mask distractions by the reward signal only. In MaDi, the conventional actor-critic structure of deep reinforcement learning agents is complemented by a small third sibling, the Masker. This lightweight neural network generates a mask to determine what the actor and critic will receive, such that they can focus on learning the task. The masks are created dynamically, depending on the current input. We run experiments on the DeepMind Control Generalization Benchmark, the Distracting Control Suite, and a real UR5 Robotic Arm. Our algorithm improves the agent's focus with useful masks, while its efficient Masker network only adds 0.2% more parameters to the original structure, in contrast to previous work. MaDi consistently achieves generalization results better than or competitive to state-of-the-art methods., Comment: Accepted as full-paper (oral) at AAMAS 2024. Code is available at https://github.com/bramgrooten/mask-distractions and see our 40-second video at https://youtu.be/2oImF0h1k48

While there has been significant progress in curriculum learning and continuous learning for training agents to generalize across a wide variety of environments in the context of single-agent reinforcement learning, it is unclear if these algorithms would still be valid in a multi-agent setting. In a competitive setting, a learning agent can be trained by making it compete with a curriculum of increasingly skilled opponents. However, a general intelligent agent should also be able to learn to act around other agents and cooperate with them to achieve common goals. When cooperating with other agents, the learning agent must (a) learn how to perform the task (or subtask), and (b) increase the overall team reward. In this paper, we aim to answer the question of what kind of cooperative teammate, and a curriculum of teammates should a learning agent be trained with to achieve these two objectives. Our results on the game Overcooked show that a pre-trained teammate who is less skilled is the best teammate for overall team reward but the worst for the learning of the agent. Moreover, somewhat surprisingly, a curriculum of teammates with decreasing skill levels performs better than other types of curricula., Comment: 9 pages, 5 figures. Presented at Agent Learning in Open-Endedness Workshop at Neural Information Processing Systems (NeurIPS 2023)

Full automation is often not achievable or desirable in critical systems with high-stakes decisions. Instead, human-AI teams can achieve better results. To research, develop, evaluate, and validate algorithms suited for such teaming, lightweight experimentation platforms that enable interactions between humans and multiple AI agents are necessary. However, there are limited examples of such platforms for defense environments. To address this gap, we present the Cogment human-machine teaming experimentation platform, which implements human-machine teaming (HMT) use cases that features heterogeneous multi-agent systems and can involve learning AI agents, static AI agents, and humans. It is built on the Cogment platform and has been used for academic research, including work presented at the ALA workshop at AAMAS this year [1]. With this platform, we hope to facilitate further research on human-machine teaming in critical systems and defense environments., Comment: 9 pages, 6 figures Presented at Conference on Artificial Intelligence for Defense (CAID) 2023

Dark matter haloes grow at a rate that depends on the value of the cosmological parameters $\sigma_8$ and $\Omega_{\rm m}$ through the initial power spectrum and the linear growth factor. While halo abundance is routinely used to constrain these parameters, through cluster abundance studies, the halo growth rate is not. In recent work, we proposed constraining the cosmological parameters using observational estimates of the overall dynamical "age" of clusters, expressed, for instance, by their half-mass assembly redshift $z_{50}$. Here we explore the prospects for using the instantaneous growth rate, as estimated from the halo merger rate, from the average growth rate over the last dynamical time, or from the fraction of systems with recent episodes of major growth. We show that the merger rate is mainly sensitive to the amplitude of fluctuations $\sigma_8$, while the rates of recent growth provide constraints in the $\Omega_{\rm m}$-$\sigma_8$ plane that are almost orthogonal to those provided by abundance studies. Data collected for forthcoming cluster abundance studies, or studies of the galaxy merger rate in current and future galaxy surveys, may thus provide additional constraints on the cosmological parameters complementary to those already derived from halo abundance., Comment: 16 pages, 12 figures, 1 table and two figures in appendix

The massive proliferation of social media data represents a transformative moment in conflict studies. This data can provide unique insights into the spread and use of weaponry, but the scale and types of data are problematic for traditional open-source intelligence. This paper presents preliminary, transdisciplinary work using computer vision to identify specific weapon systems and the insignias of the armed groups using them. There is potential to not only track how weapons are distributed through networks of armed units but also to track which types of weapons are being used by the different types of state and non-state military actors in Ukraine. Such a system could ultimately be used to understand conflicts in real-time, including where humanitarian and medical aid is most needed. We believe that using AI to help automate such processes should be a high-priority goal for our community, with near-term real-world payoffs.

Dwarf galaxies are important tracers of small-scale cosmological structure, yet much of our knowledge about these systems comes from the limited sample of dwarf galaxies within the Local Group. To make a comprehensive inventory of dwarf populations in the local Universe, we require effective methods for deriving distance estimates for large numbers of faint, low surface brightness objects. Here we test the surface brightness fluctuation (SBF) method, traditionally applied to brighter early-type galaxies, on a sample of 20 nearby dwarf galaxies detected in the COSMOS field. These objects are partially resolved in HST ACS images, and have confirmed redshift distances in the range 17-130 Mpc. We discuss the many model choices required in applying the SBF method, and explore how these affect the final distance estimates. Amongst other variations on the method, when applying the SBF method, we alter the standard equation to include a term accounting for the power spectrum of the background, greatly improving our results. For the most robust modelling choices, we find a roughly Gaussian SBF signal that correlates linearly with distance out to distances of 50-100 Mpc, but with only a fraction of the power expected. At larger distances, there is excess power relative to that predicted, probably from undetected point sources. Overall, obtaining accurate SBF distances to faint, irregular galaxies remains challenging, but may yet prove possible with the inclusion of more information about galaxy properties and point source populations, and the use of more advanced techniques., Comment: 18 pages, 26 figures, accepted by MNRAS

Many modern datasets, such as those in ecology and geology, are composed of samples with spatial structure and dependence. With such data violating the usual independent and identically distributed (IID) assumption in machine learning and classical statistics, it is unclear a priori how one should measure the performance and generalization of models. Several authors have empirically investigated cross-validation (CV) methods in this setting, reaching mixed conclusions. We provide a class of unbiased estimation methods for general quadratic errors, correlated Gaussian response, and arbitrary prediction function $g$, for a noise-elevated version of the error. Our approach generalizes the coupled bootstrap (CB) from the normal means problem to general normal data, allowing correlation both within and between the training and test sets. CB relies on creating bootstrap samples that are intelligently decoupled, in the sense of being statistically independent. Specifically, the key to CB lies in generating two independent "views" of our data and using them as stand-ins for the usual independent training and test samples. Beginning with Mallows' $C_p$, we generalize the estimator to develop our generalized $C_p$ estimators (GC). We show at under only a moment condition on $g$, this noise-elevated error estimate converges smoothly to the noiseless error estimate. We show that when Stein's unbiased risk estimator (SURE) applies, GC converges to SURE as in the normal means problem. Further, we use these same tools to analyze CV and provide some theoretical analysis to help understand when CV will provide good estimates of error. Simulations align with our theoretical results, demonstrating the effectiveness of GC and illustrating the behavior of CV methods. Lastly, we apply our estimator to a model selection task on geothermal data in Nevada., Comment: 28 pages, 13 figures

For a group G and positive interger m, Gm denotes the subgroup generated by the elements gm where g runs through G. The subgroups not of the form Gm are called nonpower subgroups. We extend the classification of groups with few nonpower subgroups from groups with at most 9 nonpower subgroups to groups with at most 13 nonpower subgroups., Comment: 16 pages, 0 figures

A growing body of evidence suggests that the central density of cuspy dark matter subhaloes is conserved in minor mergers. However, empirical models of subhalo evolution, calibrated from simulations, often assume a drop in the central density. Since empirical models of subhaloes are used in galaxy-galaxy lensing studies and dark matter annihilation calculations, we explore the consequences of assuming different subhalo models. We find that dark matter annihilation calculations are very sensitive to the assumed subhalo mass profile, and different models can give more than a magnitude difference in the J-factor and boost factor in individual haloes. On the other hand, the shear and convergence profiles used in galaxy-galaxy lensing are sensitive to the initial profile assumed (e.g., NFW versus Einato) but are otherwise well-approximated by a simple model in which the original profile is sharply truncated. We conclude that since the innermost parts of haloes are difficult to resolve in simulations, it is important to have a theoretical understanding of how subhaloes evolve to make accurate predictions of the dark matter annihilation signal., Comment: 13 figures. To be submitted to JCAP

This paper introduces a local search method for improving an existing program with respect to a measurable objective. Program Optimization with Locally Improving Search (POLIS) exploits the structure of a program, defined by its lines. POLIS improves a single line of the program while keeping the remaining lines fixed, using existing brute-force synthesis algorithms, and continues iterating until it is unable to improve the program's performance. POLIS was evaluated with a 27-person user study, where participants wrote programs attempting to maximize the score of two single-agent games: Lunar Lander and Highway. POLIS was able to substantially improve the participants' programs with respect to the game scores. A proof-of-concept demonstration on existing Stack Overflow code measures applicability in real-world problems. These results suggest that POLIS could be used as a helpful programming assistant for programming problems with measurable objectives., Comment: International Joint Conference on Artificial Intelligence (IJCAI) 2023

Conversational implicatures are pragmatic inferences that require listeners to deduce the intended meaning conveyed by a speaker from their explicit utterances. Although such inferential reasoning is fundamental to human communication, recent research indicates that large language models struggle to comprehend these implicatures as effectively as the average human. This paper demonstrates that by incorporating Grice's Four Maxims into the model through chain-of-thought prompting, we can significantly enhance its performance, surpassing even the average human performance on this task.

Informational parsimony provides a useful inductive bias for learning representations that achieve better generalization by being robust to noise and spurious correlations. We propose \textit{information gating} as a way to learn parsimonious representations that identify the minimal information required for a task. When gating information, we can learn to reveal as little information as possible so that a task remains solvable, or hide as little information as possible so that a task becomes unsolvable. We gate information using a differentiable parameterization of the signal-to-noise ratio, which can be applied to arbitrary values in a network, e.g., erasing pixels at the input layer or activations in some intermediate layer. When gating at the input layer, our models learn which visual cues matter for a given task. When gating intermediate layers, our models learn which activations are needed for subsequent stages of computation. We call our approach \textit{InfoGating}. We apply InfoGating to various objectives such as multi-step forward and inverse dynamics models, Q-learning, and behavior cloning, highlighting how InfoGating can naturally help in discarding information not relevant for control. Results show that learning to identify and use minimal information can improve generalization in downstream tasks. Policies based on InfoGating are considerably more robust to irrelevant visual features, leading to improved pretraining and finetuning of RL models., Comment: NeurIPS 2023