Graph-based safe reinforcement learning for dynamic optimal power flow with hybrid action space considering time-varying network topologies

Xihai, Zhang, Shaoyun, Ge, Zhou, Yue ORCID: https://orcid.org/0000-0002-6698-4714, Hong, Liu, Shida, Zhang and Changxu, Jiang 2025. Graph-based safe reinforcement learning for dynamic optimal power flow with hybrid action space considering time-varying network topologies. Journal of Modern Power Systems and Clean Energy 14 (1) , pp. 250-260. 10.35833/MPCE.2024.001198

PDF - Published Version Available under License Creative Commons Attribution. Download (1MB)

Abstract

The proliferation of distributed energy resources and time-varying network topologies in active distribution networks presents unprecedented challenges for network operators. While reinforcement learning (RL) has shown promise in addressing network-constrained energy scheduling, it faces difficulties in managing the complexities of dynamic topologies and discrete-continuous hybrid action spaces. To address these challenges, a graph-based safe RL approach is proposed to learn dynamic optimal power flow under time-varying network topologies. This proposed approach leverages graph convolution operators to handle network topology changes, while safe RL with parameterized action ensures policy development. Specifically, the graph convolution operator abstracts key characteristics of the network topology, enabling effective power flow management in non-stationary environments. Besides that, a parameterized action constrained Markov decision process is employed to handle the hybrid action space and ensure compliance with physical network constraints, thereby accelerating the deployment of safe policy for hybrid action spaces. Numerical results demonstrate that the proposed approach efficiently navigates the discrete-continuous decision space while accounting for the constraints imposed by the dynamic nature of power flow in time-varying network topologies.

Item Type:	Article
Date Type:	Published Online
Status:	Published
Schools:	Schools > Engineering
Publisher:	Institute of Electrical and Electronics Engineers
ISSN:	2196-5625
Date of First Compliant Deposit:	11 February 2026
Date of Acceptance:	27 March 2025
Last Modified:	11 Feb 2026 12:48
URI:	https://orca.cardiff.ac.uk/id/eprint/184582

Actions (repository staff only)

Edit Item