Search

Centralized training for decentralized execution paradigm emerged as the state-of-the-art approach to epsilon-optimally solving decentralized partially observable Markov decision processes. However, scalability remains a significant issue. This paper presents a novel and more scalable alternative, namely sequential-move centralized training for decentralized execution. This paradigm further pushes the applicability of Bellman's principle of optimality, raising three new properties. First, it allows a central planner to reason upon sufficient sequential-move statistics instead of prior simultaneous-move ones. Next, it proves that epsilon-optimal value functions are piecewise linear and convex in sufficient sequential-move statistics. Finally, it drops the complexity of the backup operators from double exponential to polynomial at the expense of longer planning horizons. Besides, it makes it easy to use single-agent methods, e.g., SARSA algorithm enhanced with these findings applies while still preserving convergence guarantees. Experiments on two- as well as many-agent domains from the literature against epsilon-optimal simultaneous-move solvers confirm the superiority of the novel approach. This paradigm opens the door for efficient planning and reinforcement learning methods for multi-agent systems.

In the Vision-and-Language Navigation in Continuous Environments (VLN-CE) task, the human user guides an autonomous agent to reach a target goal via a series of low-level actions following a textual instruction in natural language. However, most existing methods do not address the likely case where users may make mistakes when providing such instruction (e.g. "turn left" instead of "turn right"). In this work, we address a novel task of Interactive VLN in Continuous Environments (IVLN-CE), which allows the agent to interact with the user during the VLN-CE navigation to verify any doubts regarding the instruction errors. We propose an Interactive Instruction Error Detector and Localizer (I2EDL) that triggers the user-agent interaction upon the detection of instruction errors during the navigation. We leverage a pre-trained module to detect instruction errors and pinpoint them in the instruction by cross-referencing the textual input and past observations. In such way, the agent is able to query the user for a timely correction, without demanding the user's cognitive load, as we locate the probable errors to a precise part of the instruction. We evaluate the proposed I2EDL on a dataset of instructions containing errors, and further devise a novel metric, the Success weighted by Interaction Number (SIN), to reflect both the navigation performance and the interaction effectiveness. We show how the proposed method can ask focused requests for corrections to the user, which in turn increases the navigation success, while minimizing the interactions., Comment: Accepted at IEEE RO-MAN 2024

Vision-and-Language Navigation in Continuous Environments (VLN-CE) is one of the most intuitive yet challenging embodied AI tasks. Agents are tasked to navigate towards a target goal by executing a set of low-level actions, following a series of natural language instructions. All VLN-CE methods in the literature assume that language instructions are exact. However, in practice, instructions given by humans can contain errors when describing a spatial environment due to inaccurate memory or confusion. Current VLN-CE benchmarks do not address this scenario, making the state-of-the-art methods in VLN-CE fragile in the presence of erroneous instructions from human users. For the first time, we propose a novel benchmark dataset that introduces various types of instruction errors considering potential human causes. This benchmark provides valuable insight into the robustness of VLN systems in continuous environments. We observe a noticeable performance drop (up to -25%) in Success Rate when evaluating the state-of-the-art VLN-CE methods on our benchmark. Moreover, we formally define the task of Instruction Error Detection and Localization, and establish an evaluation protocol on top of our benchmark dataset. We also propose an effective method, based on a cross-modal transformer architecture, that achieves the best performance in error detection and localization, compared to baselines. Surprisingly, our proposed method has revealed errors in the validation set of the two commonly used datasets for VLN-CE, i.e., R2R-CE and RxR-CE, demonstrating the utility of our technique in other tasks. Code and dataset will be made available upon acceptance at https://intelligolabs.github.io/R2RIE-CE, Comment: 3 figures, 8 pages

Partially Observable Markov Decision Processes (POMDPs) are a powerful framework for planning under uncertainty. They allow to model state uncertainty as a belief probability distribution. Approximate solvers based on Monte Carlo sampling show great success to relax the computational demand and perform online planning. However, scaling to complex realistic domains with many actions and long planning horizons is still a major challenge, and a key point to achieve good performance is guiding the action-selection process with domain-dependent policy heuristics which are tailored for the specific application domain. We propose to learn high-quality heuristics from POMDP traces of executions generated by any solver. We convert the belief-action pairs to a logical semantics, and exploit data- and time-efficient Inductive Logic Programming (ILP) to generate interpretable belief-based policy specifications, which are then used as online heuristics. We evaluate thoroughly our methodology on two notoriously challenging POMDP problems, involving large action spaces and long planning horizons, namely, rocksample and pocman. Considering different state-of-the-art online POMDP solvers, including POMCP, DESPOT and AdaOPS, we show that learned heuristics expressed in Answer Set Programming (ASP) yield performance superior to neural networks and similar to optimal handcrafted task-specific heuristics within lower computational time. Moreover, they well generalize to more challenging scenarios not experienced in the training phase (e.g., increasing rocks and grid size in rocksample, incrementing the size of the map and the aggressivity of ghosts in pocman).

Multi-agent planning and reinforcement learning can be challenging when agents cannot see the state of the world or communicate with each other due to communication costs, latency, or noise. Partially Observable Stochastic Games (POSGs) provide a mathematical framework for modelling such scenarios. This paper aims to improve the efficiency of planning and reinforcement learning algorithms for POSGs by identifying the underlying structure of optimal state-value functions. The approach involves reformulating the original game from the perspective of a trusted third party who plans on behalf of the agents simultaneously. From this viewpoint, the original POSGs can be viewed as Markov games where states are occupancy states, \ie posterior probability distributions over the hidden states of the world and the stream of actions and observations that agents have experienced so far. This study mainly proves that the optimal state-value function is a convex function of occupancy states expressed on an appropriate basis in all zero-sum, common-payoff, and Stackelberg POSGs., Comment: Currently under review at JAIR

Detailed measurements of the spectral structure of cosmic-ray electrons and positrons from 10.6 GeV to 7.5 TeV are presented from over 7 years of observations with the CALorimetric Electron Telescope (CALET) on the International Space Station. Because of the excellent energy resolution (a few percent above 10 GeV) and the outstanding e/p separation (10$^5$), CALET provides optimal performance for a detailed search of structures in the energy spectrum. The analysis uses data up to the end of 2022, and the statistics of observed electron candidates has increased more than 3 times since the last publication in 2018. By adopting an updated boosted decision tree analysis, a sufficient proton rejection power up to 7.5 TeV is achieved, with a residual proton contamination less than 10%. The observed energy spectrum becomes gradually harder in the lower energy region from around 30 GeV, consistently with AMS-02, but from 300 to 600 GeV it is considerably softer than the spectra measured by DAMPE and Fermi-LAT. At high energies, the spectrum presents a sharp break around 1 TeV, with a spectral index change from -3.15 to -3.91, and a broken power law fitting the data in the energy range from 30 GeV to 4.8 TeV better than a single power law with 6.9 sigma significance, which is compatible with the DAMPE results. The break is consistent with the expected effects of radiation loss during the propagation from distant sources (except the highest energy bin). We have fitted the spectrum with a model consistent with the positron flux measured by AMS-02 below 1 TeV and interpreted the electron + positron spectrum with possible contributions from pulsars and nearby sources. Above 4.8 TeV, a possible contribution from known nearby supernova remnants, including Vela, is addressed by an event-by-event analysis providing a higher proton-rejection power than a purely statistical analysis., Comment: main text: 7 pages, 4 figures; supplemental material: 10 pages, 8 figures, 1 table

Hera represents the European Space Agency's inaugural planetary defense space mission and plays a pivotal role in the Asteroid Impact and Deflection Assessment international collaboration with NASA DART mission that performed the first asteroid deflection experiment using the kinetic impactor techniques. With the primary objective of conducting a detailed post-impact survey of the Didymos binary asteroid following the DART impact on its small moon called Dimorphos, Hera aims to comprehensively assess and characterize the feasibility of the kinetic impactor technique in asteroid deflection while conducting an in-depth investigation of the asteroid binary, including its physical and compositional properties as well as the effect of the impact on the surface and shape of Dimorphos. In this work, we describe the Hera radio science experiment, which will allow us to precisely estimate critical parameters, including the mass, which is required to determine the momentum enhancement resulting from the DART impact, mass distribution, rotational states, relative orbits, and dynamics of the asteroids Didymos and Dimorphos. Through a multi-arc covariance analysis, we present the achievable accuracy for these parameters, which consider the full expected asteroid phase and are based on ground radiometric, Hera optical images, and Hera to CubeSats InterSatellite Link radiometric measurements. The expected formal uncertainties for Didymos and Dimorphos GM are better than 0.01% and 0.1%, respectively, while their J2 formal uncertainties are better than 0.1% and 10%, respectively. [...]

We present the observation of a charge-sign dependent solar modulation of galactic cosmic rays (GCRs) with the CALorimetric Electron Telescope onboard the International Space Station over 6 yr, corresponding to the positive polarity of the solar magnetic field. The observed variation of proton count rate is consistent with the neutron monitor count rate, validating our methods for determining the proton count rate. It is observed by the CALorimetric Electron Telescope that both GCR electron and proton count rates at the same average rigidity vary in anticorrelation with the tilt angle of the heliospheric current sheet, while the amplitude of the variation is significantly larger in the electron count rate than in the proton count rate. We show that this observed charge-sign dependence is reproduced by a numerical ``drift model'' of the GCR transport in the heliosphere. This is a clear signature of the drift effect on the long-term solar modulation observed with a single detector., Comment: main text: 6 pages, 3 figures, supplemental material: 8 pages, 3 figures, 1 table

We present the results of a direct measurement of the cosmic-ray helium spectrum with the CALET instrument in operation on the International Space Station since 2015. The observation period covered by this analysis spans from October 13, 2015 to April 30, 2022 (2392 days). The very wide dynamic range of CALET allowed to collect helium data over a large energy interval, from ~40 GeV to ~250 TeV, for the first time with a single instrument in Low Earth Orbit. The measured spectrum shows evidence of a deviation of the flux from a single power-law by more than 8$\sigma$ with a progressive spectral hardening from a few hundred GeV to a few tens of TeV. This result is consistent with the data reported by space instruments including PAMELA, AMS-02, DAMPE and balloon instruments including CREAM. At higher energy we report the onset of a softening of the helium spectrum around 30 TeV (total kinetic energy). Though affected by large uncertainties in the highest energy bins, the observation of a flux reduction turns out to be consistent with the most recent results of DAMPE. A Double Broken Power Law (DBPL) is found to fit simultaneously both spectral features: the hardening (at lower energy) and the softening (at higher energy). A measurement of the proton to helium flux ratio in the energy range from 60 GeV/n to about 60 TeV/n is also presented, using the CALET proton flux recently updated with higher statistics.

Partially Observable Monte Carlo Planning (POMCP) is an efficient solver for Partially Observable Markov Decision Processes (POMDPs). It allows scaling to large state spaces by computing an approximation of the optimal policy locally and online, using a Monte Carlo Tree Search based strategy. However, POMCP suffers from sparse reward function, namely, rewards achieved only when the final goal is reached, particularly in environments with large state spaces and long horizons. Recently, logic specifications have been integrated into POMCP to guide exploration and to satisfy safety requirements. However, such policy-related rules require manual definition by domain experts, especially in real-world scenarios. In this paper, we use inductive logic programming to learn logic specifications from traces of POMCP executions, i.e., sets of belief-action pairs generated by the planner. Specifically, we learn rules expressed in the paradigm of answer set programming. We then integrate them inside POMCP to provide soft policy bias toward promising actions. In the context of two benchmark scenarios, rocksample and battery, we show that the integration of learned rules from small task instances can improve performance with fewer Monte Carlo simulations and in larger task instances. We make our modified version of POMCP publicly available at https://github.com/GiuMaz/pomcp_clingo.git., Comment: To appear in the Proceedings of 22nd International Conference on Autonomous Agents and Multiagent Systems (AAMAS) 2023

We propose a solution for Active Visual Search of objects in an environment, whose 2D floor map is the only known information. Our solution has three key features that make it more plausible and robust to detector failures compared to state-of-the-art methods: (i) it is unsupervised as it does not need any training sessions. (ii) During the exploration, a probability distribution on the 2D floor map is updated according to an intuitive mechanism, while an improved belief update increases the effectiveness of the agent's exploration. (iii) We incorporate the awareness that an object detector may fail into the aforementioned probability modelling by exploiting the success statistics of a specific detector. Our solution is dubbed POMP-BE-PD (Pomcp-based Online Motion Planning with Belief by Exploration and Probabilistic Detection). It uses the current pose of an agent and an RGB-D observation to learn an optimal search policy, exploiting a POMDP solved by a Monte-Carlo planning approach. On the Active Vision Database benchmark, we increase the average success rate over all the environments by a significant 35% while decreasing the average path length by 4% with respect to competing methods. Thus, our results are state-of-the-art, even without using any training procedure., Comment: 12 pages,8 figures. Submitted for review at IEEE Transactions on Pattern Analysis and Machine Intelligence. arXiv admin note: text overlap with arXiv:2009.08140

We present the measurement of the energy dependence of the boron flux in cosmic rays and its ratio to the carbon flux \textcolor{black}{in an energy interval from 8.4 GeV$/n$ to 3.8 TeV$/n$} based on the data collected by the CALorimetric Electron Telescope (CALET) during $\sim 6.4$ years of operation on the International Space Station. An update of the energy spectrum of carbon is also presented with an increase in statistics over our previous measurement. The observed boron flux shows a spectral hardening at the same transition energy $E_0 \sim 200$ GeV$/n$ of the C spectrum, though B and C fluxes have different energy dependences. The spectral index of the B spectrum is found to be $\gamma = -3.047\pm0.024$ in the interval $25 < E < 200$ GeV$/n$. The B spectrum hardens by $\Delta \gamma_B=0.25\pm0.12$, while the best fit value for the spectral variation of C is $\Delta \gamma_C=0.19\pm0.03$. The B/C flux ratio is compatible with a hardening of $0.09\pm0.05$, though a single power-law energy dependence cannot be ruled out given the current statistical uncertainties. A break in the B/C ratio energy dependence would support the recent AMS-02 observations that secondary cosmic rays exhibit a stronger hardening than primary ones. We also perform a fit to the B/C ratio with a leaky-box model of the cosmic-ray propagation in the Galaxy in order to probe a possible residual value $\lambda_0$ of the mean escape path length $\lambda$ at high energy. We find that our B/C data are compatible with a non-zero value of $\lambda_0$, which can be interpreted as the column density of matter that cosmic rays cross within the acceleration region., Comment: main text: 7 pages, 3 figures; supplemental material: 13 pages, 8 figures, 3 tables

A precise measurement of the cosmic-ray proton spectrum with the Calorimetric Electron Telescope (CALET) is presented in the energy interval from 50 GeV to 60 TeV, and the observation of a softening of the spectrum above 10 TeV is reported. The analysis is based on the data collected during $\sim$6.2 years of smooth operations aboard the International Space Station and covers a broader energy range with respect to the previous proton flux measurement by CALET, with an increase of the available statistics by a factor of $\sim$2.2. Above a few hundred GeV we confirm our previous observation of a progressive spectral hardening with a higher significance (more than 20 sigma). In the multi-TeV region we observe a second spectral feature with a softening around 10 TeV and a spectral index change from =2.6 to -2.9 consistently, within the errors, with the shape of the spectrum reported by DAMPE. We apply a simultaneous fit of the proton differential spectrum which well reproduces the gradual change of the spectral index encompassing the lower energy power-law regime and the two spectral features observed at higher energies., Comment: main text: 8 pages, 5 figures, supplemental material: 9 pages, 7 figures, 1 table, marked as a PRL Editor's Suggestion

The measurement of cosmic-ray individual spectra provides unique information regarding the origin and propagation of astro-particles. Due to the limited acceptance of current space experiments, protons and nuclei around the "knee" region ($\sim1\ PeV$) can only be observed by ground based experiments. Thanks to an innovative design, the High Energy cosmic-Radiation Detection (HERD) facility will allow direct observation up to this energy region: the instrument is mainly based on a 3D segmented, isotropic and homogeneous calorimeter which properly measures the energy of particles coming from each direction and it will be made of about 7500 LYSO cubic crystals. The read-out of the scintillation light is done with two independent systems: the first one based on wave-length shifting fibers coupled to Intensified scientific CMOS cameras, the second one is made of two photo-diodes with different active areas connected to a custom front-end electronics. This photo-diode system is designed to achieve a huge dynamic range, larger than $10^7$, while having a small power consumption, few mW per channel. Thanks to a good signal-to-noise ratio, the capability of a proper calibration, by using signals of both non-interacting and showering particles, is also guaranteed. In this paper, the current design and the performance obtained by several tests of the photo-diode read-out system are discussed.

The multi-TeV energy region of the cosmic-ray spectra has been recently explored by direct detection experiments that used calorimetric techniques to measure the energy of the cosmic particles. Interesting spectral features have been observed in both all-electron and nuclei spectra. However, the interpretation of the results is compromised by the disagreements between the data obtained from the various experiments, that are not reconcilable with the quoted experimental uncertainties. Understanding the reason for the discrepancy among the measurements is of fundamental importance in view of the forthcoming high-energy cosmic-ray experiments planned for space, as well as for the correct interpretation of the available results. The purpose of this work is to investigate the possibility that a systematic effect may derive from the non-proportionality of the light response of inorganic crystals, typically used in high-energy calorimetry due to their excellent energy-resolution performance. The main reason for the non-proportionality of the crystals is that scintillation light yield depends on ionisation density. Experimental data obtained with ion beams were used to characterize the light response of various scintillator materials. The obtained luminous efficiencies were used as input of a Monte Carlo simulation to perform a comparative study of the effect of the light-yield non-proportionality on the detection of high-energy electromagnetic and hadronic showers. The result of this study indicates that, if the calorimeter response is calibrated by using the energy deposit of minimum ionizing particles, the measured shower energy might be affected by a significant systematic shift, at the level of few percent, whose sign and magnitude depend specifically on the type of scintillator material used., Comment: to be published in JINST

The CALorimetric Electron Telescope (CALET) on the International Space Station (ISS) consists of a high-energy cosmic ray CALorimeter (CAL) and a lower-energy CALET Gamma ray Burst Monitor (CGBM). CAL is sensitive to electrons up to 20 TeV, cosmic ray nuclei from Z = 1 through Z $\sim$ 40, and gamma rays over the range 1 GeV - 10 TeV. CGBM observes gamma rays from 7 keV to 20 MeV. The combined CAL-CGBM instrument has conducted a search for gamma ray bursts (GRBs) since Oct. 2015. We report here on the results of a search for X-ray/gamma ray counterparts to gravitational wave events reported during the LIGO/Virgo observing run O3. No events have been detected that pass all acceptance criteria. We describe the components, performance, and triggering algorithms of the CGBM - the two Hard X-ray Monitors (HXM) consisting of LaBr$_{3}$(Ce) scintillators sensitive to 7 keV to 1 MeV gamma rays and a Soft Gamma ray Monitor (SGM) BGO scintillator sensitive to 40 keV to 20 MeV - and the high-energy CAL consisting of a CHarge-Detection module (CHD), IMaging Calorimeter (IMC), and fully active Total Absorption Calorimeter (TASC). The analysis procedure is described and upper limits to the time-averaged fluxes are presented., Comment: 27 pages, 10 figures. Accepted for publication in ApJ