Reinforcement learning for electric vehicle applications in power systems: A critical review

Dawei Qiu, Yi Wang*, Weiqi Hua, Goran Strbac

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

57 Downloads (Pure)

Abstract

Electric vehicles (EVs) are playing an important role in power systems due to their significant mobility and flexibility features. Nowadays, the increasing penetration of renewable energy resources has been observed in modern power systems, which brings many benefits for improving climate change and accelerating the low-carbon transition. However, the intermittent and unstable nature of renewable energy sources introduces new challenges to both the planning and operation of power systems. To address these issues, vehicle-to-grid (V2G) technology has been gradually recognized as a valid solution to provide various ancillary service provisions for power systems. Many studies have developed model-based optimization methods for EV dispatch problems. Nevertheless, this type of method cannot effectively handle the highly dynamic and stochastic environment due to the complexity of power systems. Reinforcement learning (RL), a model-free and online learning method, can capture various uncertainties through numerous interactions with the environment and adapt to various state conditions in real-time. As a result, using advanced RL algorithms to solve various EV dispatch problems has attracted a surge of attention in recent years, leading to many outstanding research papers and important findings. This paper provides a comprehensive review of popular RL algorithms categorized by single-agent RL and multi-agent RL, and summarizes how these advanced algorithms can be applied to various EV dispatch problems, including grid-to-vehicle (G2V), vehicle-to-home (V2H), and V2G. Finally, key challenges and important future research directions are discussed, which involve five aspects: (a) data quality and availability; (b) environment setup; (c) safety and robustness; (d) training performance; and (e) real-world deployment.
Original languageEnglish
Article number113052
Number of pages22
JournalRenewable and Sustainable Energy Reviews
Volume173
Early online date29 Nov 2022
DOIs
Publication statusPublished - Mar 2023

Keywords

  • Electric vehicles
  • Vehicle-to-grid
  • Reinforcement learning
  • Power systems

Fingerprint

Dive into the research topics of 'Reinforcement learning for electric vehicle applications in power systems: A critical review'. Together they form a unique fingerprint.

Cite this