Search

Exploration is a crucial skill for self-improvement and open-ended problem-solving. However, it remains uncertain whether large language models can effectively explore the state-space. Existing evaluations predominantly focus on the trade-off between exploration and exploitation, often assessed in multi-armed bandit problems. In contrast, this work isolates exploration as the sole objective, tasking the agent with delivering information that enhances future returns. For the evaluation, we propose to decompose missing rewards into exploration and exploitation components by measuring the optimal achievable return for the states already explored. Our experiments with various LLMs reveal that most models struggle to sufficiently explore the state-space and that weak exploration is insufficient. We observe a positive correlation between model size and exploration performance, with larger models demonstrating superior capabilities. Furthermore, we show that our decomposition provides insights into differences in behaviors driven by agent instructions during prompt engineering, offering a valuable tool for refining LLM performance in exploratory tasks.

We analyze the sensitivity of non-radial fluid oscillation modes and tidal deformations in neutron stars to high-order nuclear empirical parameters (NEP). In particular, we study the impact of the curvature and skewness of the symmetry energy $K_{\rm sym}$, $Q_{\rm sym}$, and the skewness of the binding energy in symmetric nuclear matter $Q_{\rm sat}$. As we are interested in the possibility of gravitational wave detection by future interferometers, we consider that the tidal interaction is the driving force for the quadrupolar non-radial fluid oscillations. We have also studied the correlations between those quantities, which will be useful to understand the strong physics of gravitational wave phenomena. Our main results show that $K_{\rm sym}$ impacts the frequencies of the fundamental mode mainly for low-mass neutron stars. The NEP $Q_{\rm sym}$ and $Q_{\rm sat}$ affect the fundamental modes of intermediate and heavy neutron stars, respectively. In the case of the first pressure mode, $K_{\rm sym}$ shows a small effect, while $Q_{\rm sat}$ shows a considerable decrease in this oscillation mode independent of the neutron star mass. Similarly, for tidal deformability, the NEP $Q_{\rm sat}$ and $Q_{\rm sym}$ show a bigger impact than $K_{\rm sym}$. Given the impact of the NEP on gravitational wave phenomena and the currently large uncertainties of these parameters, the prospect of higher sensitivity in future gravitational wave detectors promise a possible new tool to constrain high-order NEP.

Providing reliable data on the properties of atomic nuclei and infinite nuclear matter to astrophysical applications remains extremely challenging, especially when treating both properties coherently within the same framework. Methods based on energy density functionals (EDFs) enable manageable calculations of nuclear structure throughout the entire nuclear chart and of the properties of infinite nuclear matter across a wide range of densities and asymmetries. To address these challenges, we present BSkG4, the latest Brussels-Skyrme-on-a-Grid model. It is based on an EDF of the extended Skyrme type with terms that are both momentum and density-dependent, and refines the treatment of $^1S_0$ nucleon pairing gaps in asymmetric nuclear matter as inspired by more advanced many-body calculations. The newest model maintains the accuracy of earlier BSkGs for known atomic masses, radii and fission barriers with rms deviations of 0.633 MeV w.r.t. 2457 atomic masses, 0.0246 fm w.r.t. 810 charge radii, and 0.36 MeV w.r.t 45 primary fission barriers of actinides. It also improves some specific pairing-related properties, such as the $^1S_0$ pairing gaps in asymmetric nuclear matter, neutron separation energies, $Q_\beta$ values, and moments of inertia of finite nuclei. This improvement is particularly relevant for describing the $r$-process nucleosynthesis as well as various astrophysical phenomena related to the rotational evolution of neutron stars, their oscillations, and their cooling.

Reinforcement learning (RL) has seen significant success across various domains, but its adoption is often limited by the black-box nature of neural network policies, making them difficult to interpret. In contrast, symbolic policies allow representing decision-making strategies in a compact and interpretable way. However, learning symbolic policies directly within on-policy methods remains challenging. In this paper, we introduce SYMPOL, a novel method for SYMbolic tree-based on-POLicy RL. SYMPOL employs a tree-based model integrated with a policy gradient method, enabling the agent to learn and adapt its actions while maintaining a high level of interpretability. We evaluate SYMPOL on a set of benchmark RL tasks, demonstrating its superiority over alternative tree-based RL approaches in terms of performance and interpretability. To the best of our knowledge, this is the first method, that allows a gradient-based end-to-end learning of interpretable, axis-aligned decision trees within existing on-policy RL algorithms. Therefore, SYMPOL can become the foundation for a new class of interpretable RL based on decision trees. Our implementation is available under: https://github.com/s-marton/SYMPOL

Weather regimes describe the large-scale atmospheric circulation in the mid-latitudes in terms of a few circulation states that modulate regional surface weather. Subseasonal forecasts of prevailing weather regimes have proven skillful and valuable to energy applications. Previous studies have mainly focused on the mean surface weather associated with a regime. However, we show in this paper that variability of surface weather within a regime cannot be ignored. These intra-regime variations, caused by different `subflavors' of the same regime, can be captured by continuous regime indices and allow a refined application of weather regimes. Here we discuss wintertime temperature and wind speed regime anomalies for four selected countries, and provide guidance on the operational use and interpretation of regime forecasts. In an accompanying supplementary dataset we provide similar analysis for all European countries, seasons and key energy variables, useful as an applied reference., Comment: 13 pages, 4 figures, supplement in SI.pdf. Submitted to Geophysical Research Letters

Accurate volumetric image registration is highly relevant for clinical routines and computer-aided medical diagnosis. Recently, researchers have begun to use transformers in learning-based methods for medical image registration, and have achieved remarkable success. Due to the strong global modeling capability, Transformers are considered a better option than convolutional neural networks (CNNs) for registration. However, they use bulky models with huge parameter sets, which require high computation edge devices for deployment as portable devices or in hospitals. Transformers also need a large amount of training data to produce significant results, and it is often challenging to collect suitable annotated data. Although existing CNN-based image registration can offer rich local information, their global modeling capability is poor for handling long-distance information interaction and limits registration performance. In this work, we propose a CNN-based registration method with an enhanced receptive field, a low number of parameters, and significant results on a limited training dataset. For this, we propose a residual U-Net with embedded parallel dilated-convolutional blocks to enhance the receptive field. The proposed method is evaluated on inter-patient and atlas-based datasets. We show that the performance of the proposed method is comparable and slightly better than transformer-based methods by using only $\SI{1.5}{\percent}$ of its number of parameters.

Reliable forecasts of quasi-stationary, recurrent, and persistent large-scale atmospheric circulation patterns (weather regimes) are crucial for various socio-economic sectors. Despite steady progress, probabilistic weather regime predictions still exhibit biases in the exact timing and amplitude of weather regimes. This study thus aims at advancing probabilistic weather regime predictions in the North Atlantic-European region through ensemble post-processing. Here, we focus on the representation of seven year-round weather regimes in the sub-seasonal to seasonal reforecasts of the European Centre for Medium-Range Weather Forecasts. The manifestation of each of the seven regimes can be expressed by a continuous weather regime index, representing the projection of the instantaneous 500-hPa geopotential height anomalies (Z500A) onto the respective mean regime pattern. We apply a two-step ensemble post-processing involving first univariate ensemble model output statistics and second ensemble copula coupling, which restores the multivariate dependency structure. Compared to current forecast calibration practices, which rely on correcting the Z500 field by the lead time dependent mean bias, our approach extends the forecast skill horizon for daily/instantaneous regime forecasts moderately by 1.2 days to 14.5 days. Additionally, to our knowledge our study is the first to systematically evaluate the multivariate aspects of forecast quality for weather regime forecasts. Our method outperforms current practices in the multivariate aspect, as measured by the energy and variogram score. Still our study shows, that even with advanced post-processing weather regime prediction becomes difficult beyond 14 days, which likely points towards intrinsic limits of predictability for daily/instantaneous regime forecasts. The proposed method can easily be applied to operational weather regime forecasts., Comment: submitted to Quarterly Journal of the Royal Meteorological Society (QJRMS)

The accurate prediction of intense precipitation events is one of the main objectives of operational weather services. This task is even more relevant nowadays, with the rapid progression of global warming which intensifies these events. Numerical weather prediction models have improved continuously over time, providing uncertainty estimation with dynamical ensembles. However, direct precipitation forecasting is still challenging. Greater availability of machine learning tools paves the way to a hybrid forecasting approach, with the optimal combination of physical models, event statistics, and user-oriented post-processing. Here we describe a specific chain, based on a random forest pipeline, specialised in recognizing favourable synoptic conditions leading to precipitation extremes and subsequently classifying extremes into predefined types. The application focuses on Northern and Central Italy, taken as a testbed region, but is seamlessly extensible to other regions and timescales. The system is called MaLCoX (Machine Learning model predicting Conditions for eXtreme precipitation) and is running daily at the Italian regional weather service of ARPAE Emilia-Romagna. MalCoX has been trained with the ARCIS gridded high-resolution precipitation dataset as the target truth, using the last 20 years of the ECMWF re-forecast dataset as input predictors. We show that, with a long enough training period, the optimal blend of larger-scale information with direct model output improves the probabilistic forecast accuracy of extremes in the medium range. In addition, with specific methods, we provide a useful diagnostic to convey to forecasters the underlying physical storyline which makes a meteorological event extreme., Comment: 18 pages, 13 figures

We develop a compressible liquid-drop model to describe the crust of neutron stars for which the role of the nuclear clusters, the neutron gas, and the electrons are clearly identified. The novelty relies on the contribution of the neutron gas, which is qualitatively adjusted to reproduce 'ab initio' predictions in dilute neutron matter. We relate the properties of dilute neutron matter to the ones of neutron stars crust and we compare the mean-field approximation to an improved approach that better describes dilute neutron matter. The latter is quite sensitive to the unitary limit, a universal feature of Fermi systems having a large value of the scattering length and a small interaction range. While the impact of the accurate description of dilute neutron matter is important in uniform matter (up to 30\% corrections with respect to a mean-field calculations), we find a reduction of this impact in the context of the crust of neutron stars due to the additional matter components (nuclear clusters and electrons). In agreement with our previous works, dilute neutron matter is however a necessary ingredient for accurate predictions of the properties of the crust of neutron stars.

We report on the magnetoelectric dynamics in the linear magnetoelectric antiferromagnet TbPO$_4$ studied by broadband dielectric spectroscopy. For the phase transition into the magnetoelectric antiferromagnetic phase at $T_{N} \approx 2.3$ K, a finite magnetic field $H$ induces critical behavior in the quasi-static permittivity $\varepsilon'$. Plotting the corresponding anomaly as function of $T/T_N(H)$, we observe the scaling behavior $\Delta \varepsilon' \propto H^2$, a clear fingerprint of linear magnetoelectric antiferromagnets. Above the phase transition, we find a critical slowing down of the ferroic fluctuations in finite magnetic field. This behaviour can be understood via a magnetic-field-induced relaxational response that resembles the soft-mode behaviour in canonical ferroelectrics and multiferroics.

We present BSkG3, the latest entry in the Brussels-Skyrme-on-a-grid series of large-scale models of nuclear structure based on an energy density functional. Compared to its predecessors, the new model offers a more realistic description of nucleonic matter at the extreme densities relevant to neutron stars. This achievement is made possible by incorporating a constraint on the infinite nuclear matter properties at high densities in the parameter adjustment, ensuring in this way that the predictions of BSkG3 for the nuclear Equation of State are compatible with the observational evidence for heavy pulsars with $M > 2 M_{\odot}$. Instead of the usual phenomenological pairing terms, we also employ a more microscopically founded treatment of nucleon pairing, resulting in extrapolations to high densities that are in line with the predictions of advanced many-body methods and are hence more suited to the study of superfluidity in neutron stars. By adopting an extended form of the Skyrme functional, we are able to reconcile the description of matter at high densities and at saturation density: the new model further refines the description of atomic nuclei offered by its predecessors. A qualitative improvement is our inclusion of ground state reflection asymmetry, in addition to the spontaneous breaking of rotational, axial, and time-reversal symmetry. Quantitatively, the model offers lowered root-mean-square deviations on 2457 masses (0.631 MeV), 810 charge radii (0.0237 fm) and an unmatched accuracy with respect to 45 primary fission barriers of actinide nuclei (0.33 MeV). Reconciling the complexity of neutron stars with those of atomic nuclei establishes BSkG3 as a tool of choice for applications to nuclear structure, the nuclear equation of state and nuclear astrophysics in general.

A number of studies have investigated the large-scale drivers and upstream-precursors of extreme weather events, making it clear that the earliest warning signs of extreme events can be remote in both time and space from the impacted region. Integrating and leveraging our understanding of dynamical precursors provides a new perspective on ensemble forecasting for extreme events, focused on building story-lines of possible event evolution. This then acts as a tool for raising awareness of the conditions conducive to high-impact weather, and providing early warning of their possible development. However, operational applications of this developing knowledge-base is limited so far, perhaps for want of a clear framework for doing so. Here, we present such a framework, supported by open software tools, designed for identifying large-scale precursors of categorical weather events in an automated fashion, and for reducing them to scalar indices suitable for statistical prediction, forecast interpretation, and model validation. We demonstrate this framework by systematically analysing the precursor circulations of daily rainfall extremes across 18 regional- to national-scale European domains. We discuss the precursor rainfall dynamics for three disparate regions, and show our findings are consistent with, and extend, previous work. We provide an estimate of the predictive utility of these precursors across Europe based on logistic regression, and show that large-scale precursors can usefully predict heavy rainfall between two and six days ahead, depending on region and season. We further show how for more continental-scale applications the regionally-specific precursors can be synthesised into a minimal set of indices that drive heavy precipitation. We then provide comments and guidance for generalisation and application of our demonstrated approach to new variables, timescales and regions., Comment: 3 figure SI, 22 manuscript pages, 10 figures, submitted to QJRMS

Deep reinforcement learning algorithms typically act on the same set of actions. However, this is not sufficient for a wide range of real-world applications where different subsets are available at each step. In this thesis, we consider the problem of interval restrictions as they occur in pathfinding with dynamic obstacles. When actions that lead to collisions are avoided, the continuous action space is split into variable parts. Recent research learns with strong assumptions on the number of intervals, is limited to convex subsets, and the available actions are learned from the observations. Therefore, we propose two approaches that are independent of the state of the environment by extending parameterized reinforcement learning and ConstraintNet to handle an arbitrary number of intervals. We demonstrate their performance in an obstacle avoidance task and compare the methods to penalties, projection, replacement, as well as discrete and continuous masking from the literature. The results suggest that discrete masking of action-values is the only effective method when constraints did not emerge during training. When restrictions are learned, the decision between projection, masking, and our ConstraintNet modification seems to depend on the task at hand. We compare the results with varying complexity and give directions for future work., Comment: Master Thesis

Renewable generation from wind and solar power is strongly weather-dependent. To plan future sustainable energy systems that are robust to this variability, a better understanding of why and when periods of low wind and solar power output occur is valuable. We call such periods of low wind and solar power output `Dunkelflauten', the German word for dark wind lulls. In this article, we analyse the meteorological conditions during Dunkelflauten in Germany by applying the concept of weather regimes. Weather regimes are quasi-stationary, recurrent, and persistent large-scale circulation patterns which explain multi-day atmospheric variability (5-15 days). We use a regime definition that allows us to distinguish four different types of blocked regimes, characterised by high pressure situations in the North Atlantic-European region. We find that in Germany, Dunkelflauten mainly occur in winter when the solar power output is anyway low and when the wind power output drops for several consecutive days. A high-pressure system over Germany, associated with the European Blocking regime, is responsible for most of the Dunkelflauten. Dunkelflauten during the Greenland Blocking regime are associated with colder temperatures than usual, causing higher electricity demand and presenting a particular challenge as space heating demand electrifies in future. Furthermore, we show that Dunkelflauten occur predominantly when a weather regime is well-established and persists longer than usual. Our study provides novel insight on the occurrence and meteorological characteristics of Dunkelflauten, which is essential for planning resilient energy systems and supporting grid operators to prepare for potential shortages in supply., Comment: 20pages, 11figures, submitted to "Meteorological Applications" by Royal Meteorological Society (https://rmets.onlinelibrary.wiley.com/journal/14698080)

Accurate estimates of the binding energy of nuclei far from stability that cannot be produced in the laboratory are crucial to our understanding of nuclear processes in astrophysical scenarios. Models based on energy density functionals have shown that they are capable of reproducing all known masses with root-mean-square error better than 800 keV, while retaining a firm microscopic foundation. However, it was recently pointed out in [M. Hukkanen et al., arXiv:2210.10674] that the recent BSkG1 model fails to account for a contribution to the binding energy that is specific to odd-odd nuclei, and which can be studied by using appropriate mass difference formulas. We analyse here the (lacking) performance of three recent microscopic mass models with respect to such formulas and examine possibilities to remedy this deficiency in the future., Comment: 6 pages, 2 figures; Contribution to the proceedings of INPC 2022, Cape Town, South Africa

We report here the current developments on the Brussels-Skyrme-on-a-Grid (BSkG) atomic mass models. In comparison with our previous models, BSkG3 improves the infinite nuclear matter (INM) properties which opens its applications to neutron stars. The results presented here show that BSkG3 preserve the excellent agreement with experimental nuclear masses and radii, together with fission barriers of actinides obtained by BSkG1 and BSkG2, while the nuclear matter properties are considerably improved., Comment: 3 pages, 2 figures

The mixed-valent iridate Ba3InIr2O9 has been discussed as a promising candidate for quantum spin-liquid behavior. The compound exhibits Ir$^{4.5+}$ ions in face-sharing IrO6 octahedra forming Ir2O9 dimers with three t2g holes per dimer. Our results establish Ba3InIr2O9 as a cluster Mott insulator. Strong intra-dimer hopping delocalizes the three t2g holes in quasi-molecular dimer states while inter-dimer charge fluctuations are suppressed by Coulomb repulsion. The magnetism of Ba3InIr2O9 emerges from spin-orbit entangled quasi-molecular moments with yet unexplored interactions, opening up a new route to unconventional magnetic properties of 5d compounds. Using single-crystal x-ray diffraction we find the monoclinic space group C2/c already at room temperature. Dielectric spectroscopy shows insulating behavior. Resonant inelastic x-ray scattering (RIXS) reveals a rich excitation spectrum below 1.5 eV with a sinusoidal dynamical structure factor that unambiguously demonstrates the quasi-molecular character of the electronic states. Below 0.3 eV, we observe a series of excitations. According to exact diagonalization calculations, such low-energy excitations reflect the proximity of Ba3InIr2O9 to a hopping-induced phase transition based on the condensation of a quasi-molecular spin-orbit exciton. The dimer ground state roughly hosts two holes in a bonding j=1/2 orbital and the third hole in a bonding j=3/2 orbital., Comment: 14 pages, 10 figures, V2: published version, minor changes

We have extended the compressible liquid-drop model (CLDM) with a density-dependent surface term (eCLDM), which allows for a unified description of both the nuclear ground state energies and the incompressibility modulus in finite nuclei $K_A$. We analyse the role of the nuclear empirical parameters, e.g., $K_{sat}$, $Q_{sat}$, $L_{sym}$ and $K_{sym}$, which contribute to the bulk properties, as well as the role of the finite size contributions. For the bulk properties, the density and isospin dependencies of the nuclear incompressibility in infinite matter are characterized by introducing new empirical parameters, and two new constraints for the value of $K_{sym}$ are suggested. For finite nuclei, we employ a Bayesian approach coupled to a Markov-Chain Monte-Carlo (MCMC) exploration of the parameter space to confront the model predictions of $K_A$ in Zr, Sn and Pb isotopes to the experimental data. We show that $Q_{sat}\approx -950\pm 200$~MeV describes the experimental measurements of $K_A$ in these isotopes. This value is different from the ones deduced from phenomenological nuclear energy density functionals, suggesting a possible explanation of their difficulty to accurately describe Zr, Sn and Pb data all together. In addition we explore the impact of a fictitious measurement of the Giant Monopole Resonance energy in $^{132}$Sn. We show that this measurement, provided it is accurate enough, will allow to better determine $K_{sym}$ and $K_{\tau}$. Finally we explore the properties of the sound speed around saturation density and show the important role of finite size terms in finite nuclei since they reduce the sound speed to approximately half compared to nuclear matter., Comment: 21 pages, 12 figures