Search

• Expert demonstration data of hybrid electric vehicle energy management are represented by domain adaptive meta-learning. • By embedding expert demonstration model, an apprenticeship-reinforcement learning framework is established to improve fuel economy. • It is verified under three hybrid vehicle models with different powertrain architectures and several driving cycles. • Compared to traditional reinforcement learning, the apprenticeship-reinforcement learning can effectively improve the fuel emergency and CO 2 emissions. Deep reinforcement learning (DRL) is a potential solution to develop efficient energy management strategies (EMS) for hybrid electric vehicles (HEV) that can adapt to the changing topology of electrified powertrains and the uncertainty of various driving scenarios. However, traditional DRL has many disadvantages, such as low efficiency and poor stability. This study proposes an apprenticeship-reinforcement learning (A-RL) framework based on expert demonstration (ED) model embedding to improve DRL. First, the demonstration data, calculated by dynamic programming (DP), were collected, and domain adaptive meta-learning (DAML) was used to train the ED model with the adaptive capability of working conditions. Then combined apprenticeship learning (AL) with DRL, and the ED model was used to guide the DRL to output action. The method was validated on three HEV models, and the results show that the training convergence rate increases significantly under the framework. The average increase that the apprenticeship-deep deterministic policy gradient (A-DDPG) based method applied to three HEVs achieved was 34.9 %. Apprenticeship-twin delayed twin delayed deep deterministic policy gradient (A-TD3) achieved 23 % acceleration in the power-split HEV. Because A-DDPG's EMS is more forward-looking and can mimic ED to some extent, the frequency of engine operation in the high-efficiency range has increased. Therefore, A-DDPG can improve the fuel economy of the series hybrid electric bus (HEB) by 0.2–2.7 %, and improvements averaged to about 9.6 % in the series–parallel HEV while maintaining the final SOC. This study aims to improve the sampling efficiency and optimal performance of EMS-based DRL and provide a basis for the design and development of vehicle energy saving and emission reduction. [ABSTRACT FROM AUTHOR]

• Model-free decentralized coordination control of electric vehicle aggregator and energy hub entities is proposed. • Novel multi-agent DRL framework for continuous control is utilized. • LSTM and safety networks are introduced to stabilize the training process. • Developed algorithm exhibits effective and fast control against uncertainties. • Developed method outperforms 3 advanced benchmark algorithms. Energy hub (EH) is an independent entity that benefits to the efficiency, flexibility, and reliability of integrated energy systems (IESs). On the other hand, the rapid emerging of electric vehicles (EVs) drives the EV aggregator (EVAGG) as another independent entity to facilitate the electricity exchange with the grid. However, due to privacy consideration for different owners, it is challenging to investigate the optimal coordinated strategies for such interconnected entities only by exchanging the information of electrical energy. Besides, the existence of parameter uncertainties (load demands, EVs' charging behaviors, wind power and photovoltaic generation), continuous decision space, dynamic energy flows, and non-convex multi-objective function is difficult to solve. To this end, this paper proposes a novel model-free multi-agent deep reinforcement learning (MADRL) -based decentralized coordination model to minimize the energy costs of EH entities and maximize profits of EVAGGs. First, a long short-term memory (LSTM) module is used to capture the future trend of uncertainties. Then, the coordination problem is formulated as Markov games and solved by the attention enabled MADRL algorithm, where the EH or EVAGG entity is modeled as an adaptive agent. An attention mechanism makes each agent only focus on state information related to the reward. The proposed MADRL adopts the forms of offline centralized training to learn the optimal coordinated control strategy, and decentralized execution to enable agents' online decisions to only require local measurements. A safety network is employed to cope with equality constraints (demand–supply balance). Simulation results illustrate that the proposed method achieves similar results compared to the traditional model-based method with perfect knowledge of system models, and the computation performance is at least two orders of magnitudes shorter than the traditional method. The testing results of the proposed method are better than those of the Concurrent and other MADRL method, with 10.79%/3.06% lower energy cost and 17.11%/6.82% higher profits of aggregator. Besides, the electric equality constraint of the proposed method is only 0.25 MW averaged per day, which is a small and acceptable violation. [ABSTRACT FROM AUTHOR]

• A co-simulation platform for Hybrid Electric Vehicles is developed for virtual assessment of Energy Management, performance, CO 2 & pollutant emissions. • CO 2 & pollutant emissions models are based on convolutional neural network for accurate transient state prediction. • A continuous deep reinforcement learning based EMS of HEV is proposed. • DDQL is trained with random generated driving cycles based on Markov Chain approach from real recordings. • Cumulative d error for predicted pollutants emissions is below 8.5% while HEV simulation is running up to 10 times faster than real time in a standard laptop. The Energy Management Strategy (EMS) in an HEV is the key for improving fuel economy and simultaneously reducing pollutant emissions. This paper presents a methodology for developing hybrid models that enable EMS testing as well as the evaluation of fuel consumption, CO 2 and pollutant emissions (CO, NO x and THC). In this context, pollutant emissions are hard to quantify with static models such as the well-known map-based approach which is mainly due to the pronounced impact of transient effects. The novelty of this paper primarily comes from the characterization of pollutant emissions through Convolutional Neural Networks (CNN), providing high accuracy for both, instantaneous and cumulative values. The input parameters are classical Internal Combustion Engine (ICE) measurements such as engine speed, air mass flow, torque and exhaust temperature. The proposed CNNs are reduced to a minimum for low complexity and fast computability. These models are developed with experimental data from chassis dyno testing of a conventional turbo-charged gasoline engine vehicle. The pollutant emission models are used in conjunction with physical models of the remaining powertrain allowing for real time simulations of the complete HEV vehicle. The Double Deep-Q learning algorithm is proposed for the EMS and compared to the Dynamic programming (DP) solution. The introduced methodology is developed in a co-simulation framework between MATLAB-Simulink and AMESIM. The resulting model runs between 8 and 10 times faster than real time in an off-the-shelf PC. This provides the capability for developing models suitable for HIL (hardware-in-the-loop) and SIL (software-in-the-loop) applications. The final error in predicted CO 2 remains below 2.5% while the final cumulative error for pollutants is below 8.5% in the case of CO and HC emissions. [ABSTRACT FROM AUTHOR]