Model-free reinforcement learning for integrated energy control of hybrid road vehicles

This chapter explores the application of model-free reinforcement learning (RL) for optimizing energy control in various types of hybrid road vehicles within the integrated energy systems (IESs). The need for model-free RL arises from the complexity and dynamic nature of hybrid road vehicle environments, where traditional model-based approaches struggle to handle uncertainties and rapid changes in driving conditions. Furthermore, model-free RL methods are particularly advantageous in scenarios where developing an accurate model of the system is challenging or impractical. These methods can adapt to diverse driving patterns and environmental conditions without the need for extensive modeling or prior knowledge of the system’s dynamics. The IES concept aims to maximize energy efficiency by integrating and coordinating diverse energy storage and conversion technologies in hybrid road vehicles. This is achieved through dynamic energy management, integration of multiple energy sources, and advanced control strategies. By dynamically managing the distribution of energy between electric motors and internal combustion engines, IES can enhance fuel efficiency and reduce emissions. To facilitate the exploration of model-free RL methods, a comprehensive experimental platform is established, incorporating a diverse range of driving cycles and a robust simulation and verification environment. The effectiveness and adaptability of model-free RL methods in addressing energy management challenges are explored through three case studies, demonstrating their potential in real-world scenarios across a range of road vehicles. These case studies include ensemble learning for optimal exploration-exploitation control, integration of RL with rule-based, conventional optimization methods, and multihorizon RL training mechanisms with strategy transfer. The chapter also discusses the future prospects of model-free RL in energy control for road vehicles within the IES framework, addressing the potential for energy saving, emission reduction, and challenges in real-world applications. As research continues to investigate the synergies between model-free RL and IES, it is anticipated that this integrated approach will play a pivotal role in advancing sustainable and efficient transportation solutions.