论文标题
最佳初级频率控制的强化学习：Lyapunov方法
Reinforcement Learning for Optimal Primary Frequency Control: A Lyapunov Approach
论文作者
论文摘要
随着更多与逆变器连接的可再生资源集成到网格中，由于机械惯性和阻尼的减少，频率稳定性可能会降低。缓解这种降级的一种常见方法是将可再生资源的功率电子接口用于主要频率控制。由于与逆变器连接的资源可以实现对频率变化的几乎任意响应，因此它们不仅限于再现线性下垂行为。为了充分利用其能力，增强学习（RL）已成为设计非线性控制器以优化一系列目标功能的一种流行方法。 由于逆变器连接的资源和同步发电机在近乎和中间的未来都将是网格的重要组成部分，因此，对于后者的非线性动力学，前者的学识渊博控制器应稳定。为了克服这一挑战，我们明确地设计了基于神经网络的控制器的结构，从而通过使用Lyapunov功能来保证通过构造的系统稳定性。复发性神经网络体系结构用于有效地训练控制器。最终的控制器仅使用本地信息和优于最佳线性下垂以及其他最先进的学习方法。
As more inverter-connected renewable resources are integrated into the grid, frequency stability may degrade because of the reduction in mechanical inertia and damping. A common approach to mitigate this degradation in performance is to use the power electronic interfaces of the renewable resources for primary frequency control. Since inverter-connected resources can realize almost arbitrary responses to frequency changes, they are not limited to reproducing the linear droop behaviors. To fully leverage their capabilities, reinforcement learning (RL) has emerged as a popular method to design nonlinear controllers to optimize a host of objective functions. Because both inverter-connected resources and synchronous generators would be a significant part of the grid in the near and intermediate future, the learned controller of the former should be stabilizing with respect to the nonlinear dynamics of the latter. To overcome this challenge, we explicitly engineer the structure of neural network-based controllers such that they guarantee system stability by construction, through the use of a Lyapunov function. A recurrent neural network architecture is used to efficiently train the controllers. The resulting controllers only use local information and outperform optimal linear droop as well as other state-of-the-art learning approaches.
