Search

In this work we investigate the use of hierarchical multiagent reinforcement learning methods for the computation of policies to resolve congestion problems in the air traffic management domain. To address cases where the demand of airspace use exceeds capacity, we consider agents representing flights, who need to decide on ground delays at the pre-tactical stage of operations, towards executing their trajectories while adhering to airspace capacity constraints. Hierarchical reinforcement learning manages to handle real-world problems with high complexity, by partitioning the task into hierarchies of states and/or actions. This provides an efficient way of exploring the state–action space and constructing an advantageous decision-making mechanism. We first establish a general framework of hierarchical multiagent reinforcement learning, and then, we further formulate four alternative schemes of abstractions, on states, actions, or both. To quantitatively assess the quality of solutions of the proposed approaches and show the potential of the hierarchical methods in resolving the demand–capacity balance problem, we provide experimental results on real-world evaluation cases, where we measure the average delay per flight and the number of flights with delays.

In this work we investigate the use of a reinforcement learning (RL) framework for the autonomous navigation of a group of mini-robots in a multi-agent collaborative environment. Each mini-robot is...

In this article, we introduce the “functionally weighted neural network,” a new addition to the rich collection of artificial neural networks. Instead of a finite number of discrete nodes, we consider an infinite number of continuously distributed nodes. The weights assume a functional form, and the sum over the nodes becomes an integral. The gain is a significant reduction in the number of adjustable parameters, accompanied by an enhanced generalization performance. To quantitatively assess the quality of this new network, we have performed numerical experiments on a number of benchmark datasets. Comparison with state-of-the-art techniques reveals the advantages of the proposed method and emphasizes its modeling potential.

In this work we present an advanced deep multiagent reinforcement learning scheme for autonomous driving of multiple vehicles in traffic networks with road segments and unsignaled intersections. The key aspect of the proposed method lies on the definition of the possible routes one vehicle can follow as agents, called route-agents, that allows the transfer learning and the reuse of their learned policies. An informative state space is constructed by introducing a collision matrix that “displays” a predictive traffic map of the intersections that the vehicles traverse. Also, an efficient reward function is designed that aims at providing optimal driving policies in order to navigate vehicles safely and rapidly to their destinations, avoiding collisions. A combined scheme of a Deep Q-Network (DQN) and a feedforward neural network along with the Double Q-leaning technique is employed that offers more stable solutions. Several experiments were made in simulated road traffic maps that empirically illustrate the efficiency of the proposed deep multiagent framework.

One of the primary goals of Artificial Intelligence research is to develop machines with human-like intelligence, perception and reasoning. In this direction teaching apprentice agents by observing demonstrations delivered by experts is a framework of imitation learning that can provide improved solutions and it is possible to significantly outperform the demonstrator. Inverse reinforcement learning (IRL) is a paradigm relying on Markov Decision Processes (MDPs) that has a twofold target: to learn optimum policies of autonomous agents for solving complex tasks from successful demonstrations, and also to discover the unknown reward function that could explain the expert behavior. In this article we are addressing the trajectory prediction problem in the aviation domain by using an IRL approach. The proposed learning scheme provides an imitation process where the algorithm tries to imitate demonstrated trajectories, exploiting raw trajectory data enriched with contextual features and learn an efficient reward model that is learned during imitation and has generalization capabilities to unknown cases. We show several experimental results using real trajectory data from the Spanish FIR that confirms the effectiveness of our approach in automatically predicting trajectories.

Class imbalance is an inherent characteristic of multi-label data that hinders most multi-label learning methods. One efficient and flexible strategy to deal with this problem is to employ sampling techniques before training a multi-label learning model. Although existing multi-label sampling approaches alleviate the global imbalance of multi-label datasets, it is actually the imbalance level within the local neighbourhood of minority class examples that plays a key role in performance degradation. To address this issue, we propose a novel measure to assess the local label imbalance of multi-label datasets, as well as two multi-label sampling approaches based on the local label imbalance, namely MLSOL and MLUL. By considering all informative labels, MLSOL creates more diverse and better labeled synthetic instances for difficult examples, while MLUL eliminates instances that are harmful to their local region. Experimental results on 13 multi-label datasets demonstrate the effectiveness of the proposed measure and sampling approaches for a variety of evaluation metrics, particularly in the case of an ensemble of classifiers trained on repeated samples of the original data., Comment: arXiv admin note: text overlap with arXiv:1905.00609

This paper investigates the use of reinforcement learning for the path planning of an autonomous triangular marine platform in unknown environments under various environmental disturbances. The marine platform is over-actuated, i.e. it has more control inputs than degrees of freedom. The proposed approach uses a high-level online least-squared policy iteration scheme for value function approximation in order to estimate sub-optimal policy. The chosen action is considered as the desired input to a fast and efficient low-level velocity controller. We evaluate our approach in a simulated environment, including the dynamic model of the platform, the dynamics and limitations of the actuators, and the presence of wind, wave, and sea current disturbances. Simulation results are presented that demonstrate the performance of the proposed approach. Despite the model dynamics, the actuation dynamics and constrains, and the environmental disturbances, the presented results are promising.

In this paper we propose a reinforcement learning (RL) framework for the autonomous navigation of a pair of mini-robots that are driven by inertial forces. The inertial forces are provided by two vibration motors on each mini-robot which are controlled by a simple and efficient low-level speed controller. The action of the RL agent is the direction of the velocity of each mini-robot, and it based on the position of each mini-robot, the distance between the mini-robots, and the sign of the distance gradient. Each mini-robot is considered as a moving obstacle to the other that must by avoided. We have introduced a suitable reward function that results into an efficient collaborative RL approach. A simulation environment is created using the ROS framework, that include the dynamic model of the mini-robot and of the vibration motors. Several application scenarios are simulated, and the presented results demonstrate the performance of the proposed framework.

In this work we investigate the use of hierarchical collaborative reinforcement learning methods (H-CMARL) for the computation of joint policies to resolve congestion problems in the Air Traffic Management (ATM) domain. In particular, to address cases where the demand of airspace use exceeds capacity, we consider agents representing flights, who need to decide jointly on ground delays at the pre-tactical stage of operations, towards executing their trajectories while adhering to airspace capacity constraints. In doing so, agents collaborate, applying collaborative multi-agent reinforcement learning methods. Specifically, starting from a multiagent Markov Decision Process problem formulation, we introduce a flat and a hierarchical collaborative multiagent reinforcement learning method at two levels (the ground and an abstract one). To quantitatively assess the quality of solutions of the proposed approaches and show the potential of the hierarchical method in resolving the demand-capacity balance problems, we provide experimental results on real-world evaluation cases, where we measure the average delay of flights and the number of flights with delays.

An ensemble inference mechanism is proposed on the Angry Birds domain. It is based on an efficient tree structure for encoding and representing game screenshots, where it exploits its enhanced modeling capability. This has the advantage to establish an informative feature space and modify the task of game playing to a regression analysis problem. To this direction, we assume that each type of object material and bird pair has its own Bayesian linear regression model. In this way, a multi-model regression framework is designed that simultaneously calculates the conditional expectations of several objects and makes a target decision through an ensemble of regression models. Learning procedure is performed according to an online estimation strategy for the model parameters. We provide comparative experimental results on several game levels that empirically illustrate the efficiency of the proposed methodology., Comment: Angry Birds AI Symposium, ECAI 2014

The expansion of TRIAC to TRIACII code will be described. Both codes have been developed for recognition and parameters measurements of particles’ tracks from images of Solid State Nuclear Track Detectors. While the first program considers the tracks as circles, TRIACII code, using image analysis tools, counts the number of tracks and depending on the current working mode classifies them according to their radii (Mode I- circular tracks) or their axis (Mode II- elliptical tracks), their mean intensity value (brightness) and their orientation. Hough transform techniques are used for the estimation of tracks’ number and their parameters which are able to give results even for overlapping tracks. The new program has been used for radon’s progeny behavior and alpha particles’ energy discrimination.

A computer program named TRIAC written in MATLAB has been developed for track recognition and track parameters measurements from images of Solid State Nuclear Track Detectors CR39. The program using image analysis tools counts the number of tracks and classifies the tracks according to their radii. Comparison of manual scanning counts with those output by the automatic system are presented for detectors exposed to a radon rich environment. The system was also tested to differentiate tracks recorded by alpha particles of different energies