Search

Reinforcement learning (RL) is a powerful machine learning technique that has been successfully applied to a wide variety of problems. However, it can be unpredictable and produce suboptimal results in complicated learning environments. This is especially true when multiple agents learn simultaneously, which creates a complex system that is often analytically intractable. Our work considers the fundamental framework of Q-learning in Public Goods Games, where RL individuals must work together to achieve a common goal. This setting allows us to study the tragedy of the commons and free rider effects in AI cooperation, an emerging field with potential to resolve challenging obstacles to the wider application of artificial intelligence. While this social dilemma has been mainly investigated through traditional and evolutionary game theory, our approach bridges the gap between these two by studying agents with an intermediate level of intelligence. Specifically, we consider the influence of learning parameters on cooperation levels in simulations and a limiting system of differential equations, as well as the effect of evolutionary pressures on exploration rate in both of these models. We find selection for higher and lower levels of exploration, as well as attracting values, and a condition that separates these in a restricted class of games. Our work enhances the theoretical understanding of evolutionary Q-learning, and extends our knowledge of the evolution of machine behavior in social dilemmas.

Using medium-band imaging from the newly released Merian Survey, we conduct a nonparametric morphological analysis of H$\alpha$ emission maps and stellar continua for a sample of galaxies with $8\lesssim\log (M_\star/M_\odot) < 10.3$ at $0.064

We present the Merian Survey, an optical imaging survey optimized for studying the physical properties of bright star-forming dwarf galaxies. Merian is carried out with two medium-band filters ($N708$ and $N540$, centered at $708$ and $540$ nm), custom-built for the Dark Energy Camera (DECam) on the Blanco telescope. Merian covers $\sim 750\,\mathrm{deg}^2$ of equatorial fields, overlapping with the Hyper Suprime-Cam Subaru Strategic Program (HSC-SSP) wide, deep, and ultra-deep fields. When combined with the HSC-SSP imaging data ($grizy$), the new Merian DECam medium-band imaging allows for photometric redshift measurements via the detection of H$\rm\alpha$ and [OIII] line emission flux excess in the $N708$ and $N540$ filters, respectively, at $0.06

We reconsider four-dimensional scalar field theory in presence of Robin boundary conditions on two parallel plates. These boundary conditions are directly imposed in the path integral definition of the theory via auxiliary fields living on the plates. We discuss how this leads to boundary corrections to the standard energy momentum tensor operator. Via a dimensional reduction to an effective three-dimensional boundary theory, we compute the Casimir energy in terms of the plate separation and the two Robin parameters, as well as the scalar field propagator in the presence of the plates. Coincidentally, the boundary contribution vanishes in the expectation value for the vacuum energy, thereby giving results in full accordance with other energy expressions in the literature for the same setup. We also discuss for which values of the Robin parameters this energy is real-valued., Comment: 10 pages, 1 .pdf figure

Multi-armed bandit models have proven to be useful in modeling many real world problems in the areas of control and sequential decision making with partial information. However, in many scenarios, such as those prevalent in healthcare and operations management, the decision maker's expected reward will decrease if an action is selected too frequently while it may recover if they abstain from selecting this action. This scenario is further complicated when choosing a particular action also expends a random amount of a limited resource where the distribution is also initially unknown to the decision maker. In this paper we study a class of models that address this setting that we call reducing or gaining unknown efficacy bandits with stochastic knapsack constraints (ROGUEwK). We propose a combination upper confidence bound (UCB) and lower confidence bound (LCB) approximation algorithm for optimizing this model. Our algorithm chooses which action to play at each time point by solving a linear program (LP) with the UCB for the average rewards and LCB for the average costs as inputs. We show that the regret of our algorithm is sub-linear as a function of time and total constraint budget when compared to a dynamic oracle. We validate the performance of our algorithm against existing state of the art non-stationary and knapsack bandit approaches in a simulation study and show that our methods are able to on average achieve a 13% improvement in terms of total reward., Comment: 8 pages, 3 figures

Cold quasars are a rare population of luminous, unobscured quasars associated with host galaxies that have a high star formation rate. We aimed to study the host galaxies of sixty four of these cold quasars in order to probe how the supermassive black holes and host galaxies were coevolving. We compiled data from the XXL survey and crossmatched with the VHS, WISE, and HerMES surveys to obtain multiwavelength photometry spanning the Xray to the infrared and including optical spectroscopy. From the data, we calculated the supermassive black hole masses using broad emission from the magnesium II and hydrogen beta lines. We compared this with the stellar mass of the entire galaxy and find that the black holes are significantly more massive than would be predicted by local relations, indicating that the majority of black hole growth precedes the bulk of the the stellar mass formation. In addition to this, we created a spectral energy distribution for each galaxy to calculate the star formation rate. We compared the star formation rate with the black hole accretion rate and find that the stellar mass is rapidly increasing at a relative rate faster than the black hole growth, supporting the picture where the black hole grows first., Comment: 9 pages, 4 figures

A key challenge in medical decision making is learning treatment policies for patients with limited observational data. This challenge is particularly evident in personalized healthcare decision-making, where models need to take into account the intricate relationships between patient characteristics, treatment options, and health outcomes. To address this, we introduce prescriptive networks (PNNs), shallow 0-1 neural networks trained with mixed integer programming that can be used with counterfactual estimation to optimize policies in medium data settings. These models offer greater interpretability than deep neural networks and can encode more complex policies than common models such as decision trees. We show that PNNs can outperform existing methods in both synthetic data experiments and in a case study of assigning treatments for postpartum hypertension. In particular, PNNs are shown to produce policies that could reduce peak blood pressure by 5.47 mm Hg (p=0.02) over existing clinical practice, and by 2 mm Hg (p=0.01) over the next best prescriptive modeling technique. Moreover PNNs were more likely than all other models to correctly identify clinically significant features while existing models relied on potentially dangerous features such as patient insurance information and race that could lead to bias in treatment.

Correlation functions are important probes for the behavior of quantum field theories. Already at tree-level, the Refined Gribov Zwanziger (RGZ) effective action for Yang-Mills theories provides a good approximation for the gluon propagator, as compared to that calculated by nonperturbative methods such as Lattice Field Theory and Dyson-Schwinger Equations. However, the study of higher correlation functions of the RGZ theory is still at its beginning. In this work we evaluate the ghost-antighost-gluon vertex function in Landau gauge at one-loop level, in $d=4$ space-time dimensions for the gauge groups SU(2) and SU(3). More precisely, we extend the analysis conducted in [1] for the soft-gluon limit to an arbitrary kinematic configuration. We introduce renormalization group effects by means of a toy model for the running coupling and investigate the impact of such a model in the ultraviolet tails of our results. We find that RGZ results match fairly closely those from lattice simulations, Schwinger-Dyson equations and the Curci-Ferrari model for three different kinematic configurations. This is compatible with RGZ being a feasible theory for the strong interaction in the infrared regime., Comment: 24 pages, 12 figures

How cooperation evolves and particularly maintains at a large scale remains an open problem for improving humanity across domains ranging from climate change to pandemic response. To shed light on how behavioral norms can resolve the social dilemma of cooperation, here we present a formal mathematical model of individuals' decision making under general social norms, encompassing a variety of concerns and motivations an individual may have beyond simply maximizing their own payoffs. Using the canonical game of the Prisoner's Dilemma, we compare four different norms: compassion, universalizability, reciprocity, and equity, to determine which social forces can facilitate the evolution of cooperation, if any. We analyze our model through a variety of limiting cases, including weak selection, low mutation, and large population sizes. This is complemented by computer simulations of population dynamics via a Fisher process, which confirm our theoretical results. We find that the first two norms lead to the emergence of cooperation in a wide range of games, but the latter two do not on their own. Due to its generality, our framework can be used to investigate many more norms, as well as how norms themselves emerge and evolve. Our work complements recent work on fair-minded learning dynamics and provides a useful bottom-up perspective into understanding the impact of top-down social norms on collective cooperative intelligence.