Search

This paper tackles the intricate challenge of autonomous navigation for Unmanned Aerial Vehicles (UAVs) through dynamically changing environments. We focus on a sophisticated Deep Reinforcement Learning (DRL) approach using the Soft Actor-Critic (SAC) algorithm, optimized for UAV path planning within a continuous action space. This methodology leverages environmental image data to enhance the precision of flight maneuvers and effective obstacle avoidance. Our approach, validated through extensive simulations in Gazebo and field tests, demonstrates the algorithm’s efficacy in enabling UAVs to adeptly navigate obstacles using depth maps. The study further explores the robustness of the SAC algorithm by comparing it with traditional DRL methods, emphasizing its superior performance in real-world applications. This research contributes significantly to advancing UAV technology, particularly in autonomous motion planning, by integrating cutting-edge machine learning techniques. The video link is: https://www.youtube.com/watch?v=Nd_aMzejNXY. [ABSTRACT FROM AUTHOR]

In recent years, how to use renewable energy to reduce the energy cost of internet data center (IDC) has been an urgent problem to be solved. More and more solutions are beginning to consider machine learning, but many of the existing methods need to take advantage of some future information, which is difficult to obtain in the actual operation process. In this paper, we focus on reducing the energy cost of IDC by controlling the energy flow of renewable energy without any future information. we propose an efficient energy dynamic control algorithm based on the theory of reinforcement learning, which approximates the optimal solution by learning the feedback of historical control decisions. For the purpose of avoiding overestimation, improving the convergence ability of the algorithm, we use the double Q -method to further optimize. The extensive experimental results show that our algorithm can on average save the energy cost by 18.3% and reduce the rate of grid intervention by 26.2% compared with other algorithms, and thus has good application prospects. [ABSTRACT FROM AUTHOR]

In reinforcement learning (RL), the exploration/exploitation (E/E) dilemma is a very crucial issue, which can be described as searching between the exploration of the environment to find more profitable actions, and the exploitation of the best empirical actions for the current state. We focus on the single trajectory RL problem where an agent is interacting with a partially unknown MDP over single trajectories, and try to deal with the E/E in this setting. Given the reward function, we try to find a good E/E strategy to address the MDPs under some MDP distribution. This is achieved by selecting the best strategy in mean over a potential MDP distribution from a large set of candidate strategies, which is done by exploiting single trajectories drawn from plenty of MDPs. In this paper, we mainly make the following contributions: (1) We discuss the strategy-selector algorithm based on formula set and polynomial function. (2) We provide the theoretical and experimental regret analysis of the learned strategy under an given MDP distribution. (3) We compare these methods with the 'state-of-the-art' Bayesian RL method experimentally. [ABSTRACT FROM AUTHOR]

Reinforcement learning (RL) has been successfully used in many fields. With the increasing complexity of environments and tasks, it is difficult for a single learning algorithm to cope with complicated problems with high performance. This paper proposes a new multiple learning architecture, "Aggregated Multiple Reinforcement Learning System (AMRLS)", which aggregates different RL algorithms in each learning step to make more appropriate sequential decisions than those made by individual learning algorithms. This architecture was tested on a Cart-Pole system. The presented simulation results confirm our prediction and reveal that aggregation not only provides robustness and fault tolerance ability, but also produces more smooth learning curves and needs fewer learning steps than individual learning algorithms. [ABSTRACT FROM AUTHOR]

Graphical avatars have gained popularity in many application domains such as three-dimensional (3D) animation movies and animated simulations for product design. However, the methods to edit avatars' behaviors in the 3D graphical environment remained to be a challenging research topic. Since the hand-crafted methods are time-consuming and inefficient, the automatic actions of the avatars are required. To achieve the autonomous behaviors of the avatars, artificial intelligence should be used in this research area. In this paper, we present a novel approach to construct a system of automatic avatars in the 3D graphical environments based on the machine learning techniques. Specific framework is created for controlling the behaviors of avatars, such as classifying the difference among the environments and using hierarchical structure to describe these actions. Because of the requirement of simulating the interactions between avatars and environments after the classification of the environment, Reinforcement Learning is used to compute the policy to control the avatar intelligently in the 3D environment for the solution of the problem of different situations. Thus, our approach has solved problems such as where the levels of the missions will be defined and how the learning algorithm will be used to control the avatars. In this paper, our method to achieve these goals will be presented. The main contributions of this paper are presenting a hierarchical structure to control avatars automatically, developing a method for avatars to recognize environment and presenting an approach for making the policy of avatars' actions intelligently. [ABSTRACT FROM AUTHOR]

Traditional image enhancement algorithms do not account for the subjective evaluation of human operators. Every observer has a different opinion of an ideally enhanced image. Automated Techniques for obtaining a subjectively ideal image enhancement are desirable, but currently do not exist. In this paper, we demonstrate that Reinforcement Learning is a potential method for solving this problem. We have developed an agent that uses the Q-learning algorithm. The agent modifies the contrast of an image with a simple linear point transformation based on the histogram of the image and feedback it receives from human observers. The results of several testing sessions have indicated that the agent performs well within a limited number of iterations. [ABSTRACT FROM AUTHOR]

AlphaGo and its successors AlphaGo Zero and AlphaZero made international headlines with their incredible successes in game playing, which have been touted as further evidence of the immense potential of artificial intelligence, and in particular, machine learning. AlphaGo defeated the reigning human world champion Go player Lee Sedol 4 games to 1, in March 2016 in Seoul, Korea, an achievement that surpassed previous computer game-playing program milestones by IBM's Deep Blue in chess and by IBM's Watson in the U.S. TV game show Jeopardy. AlphaGo then followed this up by defeating the world's number one Go player Ke Jie 3-0 at the Future of Go Summit in Wuzhen, China in May 2017. Then, in December 2017, AlphaZero stunned the chess world by dominating the top computer chess program Stockfish (which has a far higher rating than any human) in a 100-game match by winning 28 games and losing none (72 draws) after training from scratch for just four hours! The deep neural networks of AlphaGo, AlphaZero, and all their incarnations are trained using a technique called Monte Carlo tree search (MCTS), whose roots can be traced back to an adaptive multistage sampling (AMS) simulation-based algorithm for Markov decision processes (MDPs) published in Operations Research back in 2005 [Chang, HS, MC Fu, J Hu and SI Marcus (2005). An adaptive sampling algorithm for solving Markov decision processes. Operations Research, 53, 126–139.] (and introduced even earlier in 2002). After reviewing the history and background of AlphaGo through AlphaZero, the origins of MCTS are traced back to simulation-based algorithms for MDPs, and its role in training the neural networks that essentially carry out the value/policy function approximation used in approximate dynamic programming, reinforcement learning, and neuro-dynamic programming is discussed, including some recently proposed enhancements building on statistical ranking & selection research in the operations research simulation community. [ABSTRACT FROM AUTHOR]

Reinforcement learning is one of the more attractive machine learning technologies, due to its unsupervised learning structure and ability to continually learn even as the operating environment changes. Additionally, by applying reinforcement learning to multiple cooperative software agents (a multi-agent system) not only allows each individual agent to learn from its own experience, but also opens up the opportunity for the individual agents to learn from the other agents in the system, thus accelerating the rate of learning. This research presents the novel use of fuzzy state aggregation, as the means of function approximation, combined with the fastest policy hill climbing methods of Win or Lose Fast (WoLF) and policy-dynamics based WoLF (PD-WoLF). The combination of fast policy hill climbing and fuzzy state aggregation function approximation is tested in two stochastic environments: Tileworld and the simulated robot soccer domain, RoboCup. The Tileworld results demonstrate that a single agent using the combination of FSA and PHC learns quicker and performs better than combined fuzzy state aggregation and Q-learning reinforcement learning alone. Results from the multi-agent RoboCup domain again illustrate that the policy hill climbing algorithms perform better than Q-learning alone in a multi-agent environment. The learning is further enhanced by allowing the agents to share their experience through a weighted strategy sharing. [ABSTRACT FROM AUTHOR]

The eligibility trace is the most important mechanism used so far in reinforcement learning to handle delayed reward. Here, we introduce a new kind of eligibility trace, the goal-directed trace, and show that it results in more reliable learning than the conventional trace. In addition, we also propose a new efficient algorithm for solving the goal-directed reinforcement learning problem. [ABSTRACT FROM AUTHOR]

Emergence of game strategy in multiagent systems is studied. Symbolic and subsymbolic (neural network) approaches are compared. Symbolic approach is represented by a backtrack algorithm with specified search depth, whereas the subsymbolic approach is represented by feedforward neural networks that are adapted by reinforcement temporal difference TD(λ) technique. As a test game, we use simplified checkers. The problem is studied in the framework of multiagent system, where each agent is endowed with a neural network used for a classification of checkers positions. Three different strategies are used. The first strategy corresponds to a single agent that repeatedly plays games against MinMax version of a backtrack search method. The second strategy corresponds to single agents that are repeatedly playing a megatournament, where each agent plays two different games with all other agents, one game with white pieces and the other game with black pieces. After finishing each game, both agents modify their neural networks by reinforcement learning. The third strategy is an evolutionary modification of the second one. When a megatournament is finished, each agent is evaluated by a fitness, which reflects its success in the given megatournament (more successful agents have greater fitness). It is demonstrated that all these approaches lead to a population of agents very successfully playing checkers against a backtrack algorithm with the search depth 3. [ABSTRACT FROM AUTHOR]