Projects per year
Abstract
This paper investigates a model-free supervisory control methodology with double Q-learning for the hybrid vehicle in charge-sustaining scenarios. It aims to improve the vehicle’s energy efficiency continuously while maintaining the battery’s state-of-charge in real-world driving. Two new heuristic action execution policies, the max-value-based policy and the random policy, are proposed for the double Q-learning method to reduce overestimation of the merit-function values for each action in power-split control of the vehicle. Experimental studies based on software-in-the-loop (offline learning) and hardware-in-the-loop (online learning) platforms are carried out to explore the potential of energy-saving in four driving cycles defined with real-world vehicle operations. The results from 35 rounds of offline undisturbed learning show that the heuristic action execution policies can improve the learning performance of conventional double Q-learning by achieving at least 1.09% higher energy efficiency. The proposed methods achieve similar results obtained by dynamic programming, but they have the capability of real-time online application. Double Q-learnings are shown more robust to turbulence during the disturbed learning: they realise at least three times improvement in energy efficiency compared to the standard Q-learning. Random execution policy achieves 1.18% higher energy efficiency than the max-value-based policy for the same driving condition. Significant tests show that deciding factor in the random execution policy has little impact on learning performance. By implementing the control strategies for online learning, the proposed model-free control method can save energy by more than 4.55% in the predefined real-world driving conditions compared to the method using standard Q-learning.
Original language | English |
---|---|
Article number | 114900 |
Pages (from-to) | 1-11 |
Number of pages | 11 |
Journal | Applied Energy |
Volume | 267 |
Early online date | 6 Apr 2020 |
DOIs | |
Publication status | Published - 1 Jun 2020 |
Bibliographical note
Funding Information:The work is partially funded by the Innovate UK (Grant 102253 ) for the development of world-leading off-highway vehicular technologies. It is also sponsored by the State Key Laboratory of Automotive Safety and Energy ( Tsinghua University ) under Project No. KF2029 for the development of digital-twin enabled online reinforcement learning. The authors gratefully acknowledge the support from Textron Ground Support Equipment UK and Hyper-Drive Innovation.
Publisher Copyright:
© 2020
Keywords
- charge-sustaining control
- double Q-learning
- energy efficiency optimisation
- hybrid vehicle
- reinforcement learning
- Double Q-learning
- Hybrid vehicle
- Charge-sustaining control
- Energy efficiency optimisation
- Reinforcement learning
ASJC Scopus subject areas
- Mechanical Engineering
- General Energy
- Management, Monitoring, Policy and Law
- Building and Construction
Fingerprint
Dive into the research topics of 'Heuristic action execution for energy efficient charge-sustaining control of connected hybrid vehicles with model-free double Q-learning'. Together they form a unique fingerprint.Projects
- 1 Finished
-
Hybrid Electric Push-Back Tractor (HEIP-BT)
Xu, H. (Principal Investigator) & Olatunbosun, R. (Co-Investigator)
1/07/15 → 30/09/17
Project: Other Government Departments