Search

• Expert demonstration data of hybrid electric vehicle energy management are represented by domain adaptive meta-learning. • By embedding expert demonstration model, an apprenticeship-reinforcement learning framework is established to improve fuel economy. • It is verified under three hybrid vehicle models with different powertrain architectures and several driving cycles. • Compared to traditional reinforcement learning, the apprenticeship-reinforcement learning can effectively improve the fuel emergency and CO 2 emissions. Deep reinforcement learning (DRL) is a potential solution to develop efficient energy management strategies (EMS) for hybrid electric vehicles (HEV) that can adapt to the changing topology of electrified powertrains and the uncertainty of various driving scenarios. However, traditional DRL has many disadvantages, such as low efficiency and poor stability. This study proposes an apprenticeship-reinforcement learning (A-RL) framework based on expert demonstration (ED) model embedding to improve DRL. First, the demonstration data, calculated by dynamic programming (DP), were collected, and domain adaptive meta-learning (DAML) was used to train the ED model with the adaptive capability of working conditions. Then combined apprenticeship learning (AL) with DRL, and the ED model was used to guide the DRL to output action. The method was validated on three HEV models, and the results show that the training convergence rate increases significantly under the framework. The average increase that the apprenticeship-deep deterministic policy gradient (A-DDPG) based method applied to three HEVs achieved was 34.9 %. Apprenticeship-twin delayed twin delayed deep deterministic policy gradient (A-TD3) achieved 23 % acceleration in the power-split HEV. Because A-DDPG's EMS is more forward-looking and can mimic ED to some extent, the frequency of engine operation in the high-efficiency range has increased. Therefore, A-DDPG can improve the fuel economy of the series hybrid electric bus (HEB) by 0.2–2.7 %, and improvements averaged to about 9.6 % in the series–parallel HEV while maintaining the final SOC. This study aims to improve the sampling efficiency and optimal performance of EMS-based DRL and provide a basis for the design and development of vehicle energy saving and emission reduction. [ABSTRACT FROM AUTHOR]

• A co-simulation platform for Hybrid Electric Vehicles is developed for virtual assessment of Energy Management, performance, CO 2 & pollutant emissions. • CO 2 & pollutant emissions models are based on convolutional neural network for accurate transient state prediction. • A continuous deep reinforcement learning based EMS of HEV is proposed. • DDQL is trained with random generated driving cycles based on Markov Chain approach from real recordings. • Cumulative d error for predicted pollutants emissions is below 8.5% while HEV simulation is running up to 10 times faster than real time in a standard laptop. The Energy Management Strategy (EMS) in an HEV is the key for improving fuel economy and simultaneously reducing pollutant emissions. This paper presents a methodology for developing hybrid models that enable EMS testing as well as the evaluation of fuel consumption, CO 2 and pollutant emissions (CO, NO x and THC). In this context, pollutant emissions are hard to quantify with static models such as the well-known map-based approach which is mainly due to the pronounced impact of transient effects. The novelty of this paper primarily comes from the characterization of pollutant emissions through Convolutional Neural Networks (CNN), providing high accuracy for both, instantaneous and cumulative values. The input parameters are classical Internal Combustion Engine (ICE) measurements such as engine speed, air mass flow, torque and exhaust temperature. The proposed CNNs are reduced to a minimum for low complexity and fast computability. These models are developed with experimental data from chassis dyno testing of a conventional turbo-charged gasoline engine vehicle. The pollutant emission models are used in conjunction with physical models of the remaining powertrain allowing for real time simulations of the complete HEV vehicle. The Double Deep-Q learning algorithm is proposed for the EMS and compared to the Dynamic programming (DP) solution. The introduced methodology is developed in a co-simulation framework between MATLAB-Simulink and AMESIM. The resulting model runs between 8 and 10 times faster than real time in an off-the-shelf PC. This provides the capability for developing models suitable for HIL (hardware-in-the-loop) and SIL (software-in-the-loop) applications. The final error in predicted CO 2 remains below 2.5% while the final cumulative error for pollutants is below 8.5% in the case of CO and HC emissions. [ABSTRACT FROM AUTHOR]