Multi-step reinforcement learning for model-free predictive energy management of an electrified off-highway vehicle

Quan Zhou, Ji Li, Bin Shuai, Huw Williams, Yinglong He, Ziyang Li, Hongming Xu*, Fuwu Yan

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

21 Citations (Scopus)
191 Downloads (Pure)

Abstract

The energy management system of an electrified vehicle is one of the most important supervisory control systems which manages the use of on-board energy resources. This paper researches a ‘model-free’ predictive energy management system for a connected electrified off-highway vehicle. A new reinforcement learning algorithm with the capability of ‘multi-step’ learning is proposed to enable the all-life-long online optimisation of the energy management control policy. Three multi-step learning strategies (Sum-to-Terminal, Average-to-Neighbour Recurrent-to-Terminal) are researched for the first time. Hardware-in-the-loop tests are carried out to examine the control functionality for real application of the proposed ‘model-free’ method. The results show that the proposed method can continuously improve the vehicle's energy efficiency during the real-time hardware-in-the-loop test, which increased from the initial level of 34% to 44% after 5 h’ 35-step learning. Compared with a well-designed model-based predictive energy management control policy, the model-free predictive energy management method can increase the prediction horizon length by 71% (from 35 to 65 steps with 1 s interval in real-time computation) and can save energy by at least 7.8% for the same driving conditions.

Original languageEnglish
Article number113755
Number of pages12
JournalApplied Energy
Volume255
Early online date28 Aug 2019
DOIs
Publication statusPublished - 1 Dec 2019

Keywords

  • Energy management
  • Hybrid electric vehicle
  • Markov decision problem
  • Model-free predictive control
  • Multi-step reinforcement learning

ASJC Scopus subject areas

  • Building and Construction
  • Energy(all)
  • Mechanical Engineering
  • Management, Monitoring, Policy and Law

Fingerprint

Dive into the research topics of 'Multi-step reinforcement learning for model-free predictive energy management of an electrified off-highway vehicle'. Together they form a unique fingerprint.

Cite this