Shifting Deep Reinforcement Learning Algorithm Toward Training Directly in Transient Real-World Environment: A Case Study in Powertrain Control

Deep reinforcement learning (DRL) excels at playing a wide variety of simulated games and allows for a generic learning process that does not consider a specific knowledge of the task. However, due to the fact that a large prohibitively number of interactions with the environment are required and that the initial policy behavior is almost random, such an algorithm cannot be trained directly in a real-world environment while satisfying given safety constraints. In this article, a control framework based on DRL that shifts toward training directly in the transient real-world environment is proposed. This research is working on the assumption that some demonstration knowledge that operates under previous controllers and an abstract of the agent environment dynamics are available. By encoding this prior knowledge into a sophisticated learning architecture, a warm-starting DRL algorithm with a safe exploration guarantee can be anticipated. Taking the boost control problem for a variable geometry turbocharger equipped diesel engine as an example, the proposed algorithm improves the initial performance by 74.6% and the learning efficiency by an order of magnitude in contrast to its vanilla counterpart. Compared with other existing DRL-based powertrain control methods, the proposed algorithm can realize the “model-free” concept in the strict sense, making it attractive for future DRL-based powertrain control algorithms to build on.