Search

Reinforcement learning (RL) is a powerful technique for training intelligent agents, but understanding why these agents make specific decisions can be quite challenging. This lack of transparency in RL models has been a long-standing problem, making it difficult for users to grasp the reasons behind an agent's behaviour. Various approaches have been explored to address this problem, with one promising avenue being reward decomposition (RD). RD is appealing as it sidesteps some of the concerns associated with other methods that attempt to rationalize an agent's behaviour in a post-hoc manner. RD works by exposing various facets of the rewards that contribute to the agent's objectives during training. However, RD alone has limitations as it primarily offers insights based on sub-rewards and does not delve into the intricate cause-and-effect relationships that occur within an RL agent's neural model. In this paper, we present an extension of RD that goes beyond sub-rewards to provide more informative explanations. Our approach is centred on a causal learning framework that leverages information-theoretic measures for explanation objectives that encourage three crucial properties of causal factors: causal sufficiency, sparseness, and orthogonality. These properties help us distill the cause-and-effect relationships between the agent's states and actions or rewards, allowing for a deeper understanding of its decision-making processes. Our framework is designed to generate local explanations and can be applied to a wide range of RL tasks with multiple reward channels. Through a series of experiments, we demonstrate that our approach offers more meaningful and insightful explanations for the agent's action selections., Comment: https://lukaswill.github.io/; Accepted as oral by CLeaR 2024

Explaining the behaviour of intelligent agents learned by reinforcement learning (RL) to humans is challenging yet crucial due to their incomprehensible proprioceptive states, variational intermediate goals, and resultant unpredictability. Moreover, one-step explanations for RL agents can be ambiguous as they fail to account for the agent's future behaviour at each transition, adding to the complexity of explaining robot actions. By leveraging abstracted actions that map to task-specific primitives, we avoid explanations on the movement level. To further improve the transparency and explainability of robotic systems, we propose an explainable Q-Map learning framework that combines reward decomposition (RD) with abstracted action spaces, allowing for non-ambiguous and high-level explanations based on object properties in the task. We demonstrate the effectiveness of our framework through quantitative and qualitative analysis of two robotic scenarios, showcasing visual and textual explanations, from output artefacts of RD explanations, that are easy for humans to comprehend. Additionally, we demonstrate the versatility of integrating these artefacts with large language models (LLMs) for reasoning and interactive querying., Comment: https://lukaswill.github.io

Neural fields are neural networks which map coordinates to a desired signal. When a neural field should jointly model multiple signals, and not memorize only one, it needs to be conditioned on a latent code which describes the signal at hand. Despite being an important aspect, there has been little research on conditioning strategies for neural fields. In this work, we explore the use of neural fields as decoders for 2D semantic segmentation. For this task, we compare three conditioning methods, simple concatenation of the latent code, Feature Wise Linear Modulation (FiLM), and Cross-Attention, in conjunction with latent codes which either describe the full image or only a local region of the image. Our results show a considerable difference in performance between the examined conditioning strategies. Furthermore, we show that conditioning via Cross-Attention achieves the best results and is competitive with a CNN-based decoder for semantic segmentation., Comment: 13 pages, 4 figures, submitted to ICANN 2023

The first population of X-ray binaries (XRBs) is expected to affect the thermal and ionization states of the gas in the early Universe. Although these X-ray sources are predicted to have important implications for high-redshift observable signals, such as the hydrogen 21-cm signal from cosmic dawn and the cosmic X-ray background, their properties are poorly explored, leaving theoretical models largely uninformed. In this paper we model a population of X-ray binaries arising from zero metallicity stars. We explore how their properties depend on the adopted initial mass function (IMF) of primordial stars, finding a strong effect on their number and X-ray production efficiency. We also present scaling relations between XRBs and their X-ray emission with the local star formation rate, which can be used in sub-grid models in numerical simulations to improve the X-ray feedback prescriptions. Specifically, we find that the uniformity and strength of the X-ray feedback in the intergalactic medium is strongly dependant on the IMF. Bottom-heavy IMFs result in a smoother distribution of XRBs, but have a luminosity orders of magnitude lower than more top-heavy IMFs. Top-heavy IMFs lead to more spatially uneven, albeit strong, X-ray emission. An intermediate IMF has a strong X-ray feedback while sustaining an even emission across the intergalactic medium. These differences in X-ray feedback could be probed in the future with measurements of the cosmic dawn 21-cm line of neutral hydrogen, which offers us a new way of constraining population III IMF., Comment: Accepted for publication in MNRAS, 17 pages, 9 figures

The ultra-faint dwarf galaxy Reticulum~II was enriched by a single rare and prolific r-process event. The r-process content of Reticulum~II thus provides a unique opportunity to study metal mixing in a relic first galaxy. Using multi-object high-resolution spectroscopy with VLT/GIRAFFE and Magellan/M2FS, we identify 32 clear spectroscopic member stars and measure abundances of Mg, Ca, Fe, and Ba where possible. We find $72^{+10}_{-12}$% of the stars are r-process-enhanced, with a mean $\left\langle\mbox{[Ba/H]}\right\rangle=-1.68~\pm~0.07$ and unresolved intrinsic dispersion $\sigma_{\rm [Ba/H]} < 0.20$. The homogeneous r-process abundances imply that Ret~II's metals are well-mixed by the time the r-enhanced stars form, which simulations have shown requires at least 100 Myr of metal mixing in between bursts of star formation to homogenize. This is the first direct evidence of bursty star formation in an ultra-faint dwarf galaxy. The homogeneous dilution prefers a prompt and high-yield r-process site, such as collapsar disk winds or prompt neutron star mergers. We also find evidence from [Ba/H] and [Mg/Ca] that the r-enhanced stars in Ret~II formed in the absence of substantial pristine gas accretion, perhaps indicating that ${\approx}70$% of Ret~II stars formed after reionization., Comment: 37 pages, 12 figures, 10 tables, accepted to AJ

Galaxies are thought to reside inside of large gravitationally bound structures of dark matter, so-called haloes. While the smallest of these haloes host no or only a few stars, the biggest host entire clusters of galaxies. Over cosmic history, haloes often collided and merged, forming bigger and bigger structures. Merger trees, i.e., catalogues of haloes evolving and connections between them as they grow and merge, have become a vital tool in describing and understanding the history of cosmological objects such as our Galaxy. Semi-analytical models, built on top of such merger trees, are a common approach for theoretical studies in cosmology. The semi-analytical nature of such models is especially beneficial when the dynamic range in spatial and time scales that need to be considered becomes too large for numerical simulations. Ancient Stars and Local Observables by Tracing Halos (A-SLOTH) is such a semi-analytical model and it is designed to simulate star formation in the early Universe in a fast and accessible way. It uses merger trees, either from numerical simulations or generated by statistical algorithms to describe the history of galaxies. The processes of baryonic physics, in particular gas cooling, star formation and stellar feedback are described with approximations and statistical models. The range of applications for this model is extensive and we, therefore, make it available to the scientific community., Comment: Published in JOSS. Git repo is available at https://gitlab.com/thartwig/asloth

Physically realistic models of stellar spectra are needed in a variety of astronomical studies, from the analysis of fundamental stellar parameters, to studies of exoplanets and stellar populations in galaxies. Here we present a new version of the widely-used radiative transfer code Turbospectrum, which we update with the capacity to perform spectrum synthesis for lines of multiple chemical elements in Non-Local Thermodynamic Equilibrium (NLTE). We use the code in the analysis of metallicites and abundances of the Gaia FGK benchmark stars, using one-dimensional MARCS atmospheric models and the averages of 3D radiation-hydrodynamics simulations of stellar surface convection. We show that the new more physically realistic models offer a better description of the observed data and make the program and the associated microphysics data publicly available, including grids of NLTE departure coefficients for H, O, Na, Mg, Si, Ca, Ti, Mn, Fe, Co, Ni, Sr, and Ba.

The semi-analytical model A-SLOTH (Ancient Stars and Local Observables by Tracing Halos) is the first public code that connects the formation of the first stars and galaxies to observables. After several successful projects with this model, we publish the source code and describe the public version in this paper. The model is based on dark matter merger trees that can either be generated based on Extended Press-Schechter theory or that can be imported from dark matter simulations. On top of these merger trees, A-SLOTH applies analytical recipes for baryonic physics to model the formation of both metal-free and metal-poor stars and the transition between them with unprecedented precision and fidelity. A-SLOTH samples individual stars and includes radiative, chemical, and mechanical feedback. It is calibrated based on six observables, such as the optical depth to Thomson scattering, the stellar mass of the Milky Way and its satellite galaxies, the number of extremely-metal poor stars, and the cosmic star formation rate density at high redshift. A-SLOTH has versatile applications with moderate computational requirements. It can be used to constrain the properties of the first stars and high-z galaxies based on local observables, predicts properties of the oldest and most metal-poor stars in the Milky Way, can serve as a subgrid model for larger cosmological simulations, and predicts next-generation observables of the early Universe, such as supernova rates or gravitational wave events., Comment: Published in ApJ, 35 pages, video summary can be watched at https://www.youtube.com/watch?v=M_JRT2AsOqg and git repository is available at https://gitlab.com/thartwig/asloth

The chemical composition of the Sun is requested in the context of various studies in astrophysics, among them in the calculation of the standard solar models (SSMs), which describe the evolution of the Sun from the pre-main-sequence to its present age. In this work, we provide a critical re-analysis of the solar chemical abundances and corresponding SSMs. For the photospheric values, we employ new high-quality solar observational data collected with the IAG facility, state-of-the art non-equilibrium modelling, new oscillator strengths, and different atmospheric models, including the MARCS model, but also averages based on Stagger and CO5BOLD 3D radiation-hydrodynamics simulations of stellar convection. We perform new calculations of oscillator strengths for transitions in O I and N I. For O I - the critical element for the interior models - calculations are carried out using several independent methods. We find unprecedented agreement between the new estimates of transition probabilities, thus supporting our revised solar oxygen abundance. We also provide new estimates of the noble gas Ne abundance. We investigate our results in comparison with the previous estimates. We discuss the consistency of our photospheric measurements with meteoritic values taking into account systematic and correlated errors. Finally, we provide revised chemical abundances, leading to a new value of the solar photospheric present-day metallicity $Z/X = 0.0225$, and employ them in the calculations of the SSM. We find that the puzzling mismatch between the helioseismic constraints on the solar interior structure and the model is resolved with the new chemical composition., Comment: accepted for publication in A&A

Properties of the first generation of stars (Pop III), such as their initial mass function (IMF), are poorly constrained by observations and have yet to converge between simulations. The cosmological 21-cm signal of neutral hydrogen is predicted to be sensitive to Lyman-band photons produced by these stars, thus providing a unique way to probe the first stellar population. In this paper, we investigate the impacts of the Pop III IMF on the cosmic dawn 21-cm signal via the Wouthuysen-Field effect, Lyman-Werner feedback, Ly-alpha heating, and CMB heating. We calculate the emission spectra of star-forming halos for different IMFs by integrating over individual metal-free stellar spectra, computed from a set of stellar evolution histories and stellar atmospheres, and taking into account variability of the spectra with stellar age. Through this study, we therefore relax two common assumptions: that the zero-age main sequence emission rate of a Pop III star is representative of its lifetime mean emission rate, and that Pop III emission can be treated as instantaneous. Exploring a bottom-heavy, a top-heavy, and intermediate IMFs, we show that variations in the 21-cm signal are driven by stars lighter than 20 solar masses. For the explored models we find maximum relative differences of 59% in the cosmic dawn global 21-cm signal, and 131% between power spectra. Although this impact is modest, precise modelling of the first stars and their evolution is necessary for accurate prediction and interpretation of the 21-cm signal., Comment: 20 pages, 16 figures. Accepted for publication in MNRAS, updated to accepted version

We study the stellar mass-to-halo mass relation at $z=0$ in 30 Milky Way-like systems down to the ultra-faint ($M_* < 10^5 M_\odot$) regime using the semi-analytic model A-SLOTH. A new model allows us to follow star formation and the stochastic stellar feedback from individually sampled Pop II stars. Our fiducial model produces consistent results with the stellar mass-to-halo mass relation derived from abundance matching and the observed cumulative stellar mass function above the observational completeness. We find a plateau in the stellar mass-to-halo mass relation in the ultra-faint regime. The stellar mass of this plateau tells us how many stars formed before supernovae occur and regulate further star formation, which is determined by the Pop~II star formation efficiency. We also find that the number of luminous satellites increases rapidly as $M_*$ decreases until $M_* \approx 10^4 M_\odot$. Finally, we find that the relative streaming velocity between baryons and dark matter at high redshift is important in determining the number of ultra-faint dwarf galaxies at $z=0$. The new model in A-SLOTH provides a framework to study the stellar properties and the formation history of metal-poor stars in Milky Way and its satellites., Comment: 18 pages, 11 figures, 3 tables

We present detailed abundance measurements for 45 globular clusters (GCs) in galaxies in (and, in one case, beyond) the Local Group. The measurements are based on new high-resolution integrated-light spectra of GCs in NGC 185, NGC 205, M31, M33, and NGC 2403, combined with reanalysis of previous observations of GCs in the Fornax dSph, WLM, NGC 147, NGC 6822, and the Milky Way. The GCs cover the range -2.8 < [Fe/H] < -0.1 and we determined abundances for Fe, Na, Mg, Si, Ca, Sc, Ti, Cr, Mn, Ni, Cu, Zn, Zr, Ba, and Eu. Corrections for non local thermodynamic equilibrium effects are included for Na, Mg, Ca, Ti, Mn, Fe, Ni, and Ba. For several of the galaxies, our measurements provide the first quantitative constraints on the detailed composition of their metal-poor stellar populations. Overall, the GCs in different galaxies exhibit remarkably uniform abundance patterns of the alpha-, iron-peak, and neutron-capture elements, with a dispersion of less than 0.1 dex in [alpha/Fe] for the full sample. There is a hint that GCs in dwarf galaxies are slightly less alpha-enhanced (by about 0.04 dex on average) than those in larger galaxies. One GC in M33 (HM33-B) resembles the most metal-rich GCs in the Fornax dSph (Fornax 4) and NGC 6822 (SC7) by having alpha-element abundances closer to scaled-solar values, possibly hinting at an accretion origin. We find that the alpha-element abundances strongly correlate with those of Na, Sc, Ni, and Zn. Several GCs with [Fe/H]<-1.5 are deficient in Mg compared to other alpha-elements. We find no GCs with strongly enhanced r-process abundances as reported for metal-poor stars in some ultra-faint dwarfs and the Magellanic Clouds. The similarity of the abundance patterns for metal-poor GCs in different environments points to similar early enrichment histories and only allow for minor variations in the initial mass function., Comment: 34 pages + 6 appendices. Accepted for publication in Astronomy & Astrophysics

Mapping Cosmic Dawn with 21-cm tomography offers an exciting new window into the era of primordial star formation. However, self-consistent implementation of both the process of star formation and the related 21-cm signal is challenging, due to the multi-scale nature of the problem. In this study, we develop a flexible semi-analytical model to follow the formation of the first stars and the process of gradual transition from primordial to metal-enriched star formation. For this transition we use different in scenarios with varying time-delays (or recovery times) between the first supernovae and the formation of the second generation of stars. We use recovery times between 10 and 100\,Myr and find that these delays have a strong impact on the redshift at which the transition to metal-enriched star formation occurs. We then explore the effect of this transition on the 21-cm signal and find that the recovery time has a distinctive imprint in the signal. Together with an improved understanding of how this time-delay relates to the properties of Population~III stars, future 21-cm observations can give independent constraints on the earliest epoch of star formation., Comment: 17 pages, 11 Figures, submitted to MNRAS, comments very welcome

The lack of observations of abundance patterns originating in pair-instability supernovae has been a long-standing problem in relation to the first stars. This class of supernovae is expected to have an abundance pattern with a strong odd-even effect, making it substantially different from present-day supernovae. In this study, we use a cosmological radiation hydrodynamics simulation to model such supernovae and the subsequent formation of the second generation of stars. We incorporate streaming velocities for the first time. There are 14 star-forming minihalos in our $1\,\mathrm{Mpc}\,h^{-1}$ box, leading to 14 supernovae occurring before redshift $z=19.5$, where we start reducing the complexity of the simulation. Following the explosions, extremely metal-poor stars form in 10 halos via internal and external enrichment, which makes it the most common outcome. Only one halo does not recollapse during the simulations. This result is at tension with the current (lack of) observations of metal-poor stars with pair instability supernova abundance patterns, suggesting that these very massive stars might be rare even in the early Universe. The results from this simulation also give us insights into what drives different modes of recollapse and what determines the mixing behavior of metals after very energetic supernovae., Comment: 21 pages, 17 figures, accepted for publication in ApJ