• Energy Research

The anomalous measurements pose significant challenges for the secure and economical operation of multiple microgrids (MMGs). However, existing works still cannot effectively address this problem. Therefore, this paper proposes a robust decentralized multi-agent deep reinforcement learning (MADRL) control approach by developing a novel actor network on the basis of the centralized training and decentralized execution framework (CTDEF). To achieve robust control of MMG systems against anomalous data, this approach extracts the variation patterns of measurements from both temporal and spatial perspectives. From the spatial perspective, the measurements are first cast to a graph, and a multi-head graph attention (MGAT) network is employed to extract the structural correlations among these measurements. From the temporal perspective, the measurements feature extracted by MGAT along with the state and historical actions are processed by two recurrent networks to obtain the trajectory history feature of each MG. The confederate image technology is developed therein for each agent to infer the intentions of other agents in order to better extract the trajectory history. To more fully express the structural correlations between nodes and decision intentions of other agents, a node cognition regularizer and a mutual information-based regularization term are designed for optimizing MGAT and the confederate image network, respectively. By integrating temporal and spatial perspectives, the proposed approach achieves greater robustness to outliers than approaches that consider only one perspective. The experimental results confirm the effectiveness of the proposed approach.

This paper develops a robust physics-informed state estimation method for the distribution network with inaccurate topology information. An aggregated k-nearest neighbor graph is first derived as the feature graph according to the inaccurate topology and measurement features. Then, graph propagation and aggregation are performed by an adaptive multi-channel graph attention model on both the feature graph and the graph constructed based on the inaccurate given topology. To fuse the different graph embeddings, an attention module is further employed to adaptively assign importance weights for them. This allows the proposed method to achieve robustness against anomalous measurements even when the given topology information is inaccurate. Comparative results with state-of-the-art distribution system state estimation methods demonstrate the accuracy and robustness of the proposed method.

This paper proposes a novel multi-agent deep reinforcement learning (MADRL) approach for the energy management of multiple microgrids considering the robust voltage control under the missing measurements. Missing measurement control poses challenges to the MADRL. To address the problem, we propose a trajectory history information-utilized opponent modeling-based distributed MADRL to avoid the collapse of control caused by the loss of current time measurement. Simulation results demonstrate that, whether the measurements are complete or not, the proposed approach achieves the ideal results.

An accelerated loss of life (LOL) of distribution transformers has been observed in recent years owing to the increasing penetration of electric vehicles (EVs). This paper proposes an evolutionary curriculum learning (ECL)-based multi-agent deep reinforcement learning (MADRL) approach for optimizing transformer LOL while considering various charging demands of different EV owners. Specifically, the problem of charging multiple EVs is cast as a Markov game. It is solved by the proposed MADRL algorithm by modeling each EV controller as an agent with a specific objective. During the centralized training stage, a novel centralized ECL mechanism is adopted to enhance the coordination of multiple EVs. It enables the proposed approach to address the management of large-scale EV charging. When the training procedure is completed, the proposed approach is deployed in a decentralized manner. Herein, all the agents make decisions based solely on local information. The decentralized manner of execution helps preserve the privacy of EV owners, reduce the related communication cost, and avoid single-point failure. Comparative tests utilizing real-world data demonstrate that the proposed approach can achieve coordinated charging of a large number of EVs while satisfying the various charging demands of different EV owners.