1. A Memory-Efficient Markov Decision Process Computation Framework Using BDD-based Sampling Representation
- Author
-
Jiang Hu, Sunil P. Khatri, Frank Liu, and He Zhou
- Subjects
0303 health sciences ,0209 industrial biotechnology ,Theoretical computer science ,Computer science ,Binary decision diagram ,Bayesian probability ,Parallel algorithm ,Sampling (statistics) ,02 engineering and technology ,Solver ,03 medical and health sciences ,020901 industrial engineering & automation ,Theory of computation ,Reinforcement learning ,Markov decision process ,Representation (mathematics) ,030304 developmental biology - Abstract
Although Markov Decision Process (MDP) has wide applications in autonomous systems as a core model in Reinforcement Learning, a key bottleneck is the large memory utilization of the state transition probability matrices. This is particularly problematic for computational platforms with limited memory, or for Bayesian MDP, which requires dozens of such matrices. To mitigate this difficulty, we propose a highly memory-efficient representation for probability matrices using Binary Decision Diagram (BDD) based sampling, and develop a corresponding (Bayesian/classical) MDP solver on a CPU-GPU platform. Simulation results indicate our approach reduces memory by one and two orders of magnitude for Bayesian/classical MDP, respectively.CCS CONCEPTS• Theory of computation $\rightarrow$ Data compression; Parallel algorithms; Sequential decision making; • Mathematics of computing $\rightarrow$Bayesian computation
- Published
- 2019
- Full Text
- View/download PDF