• Energy Research

Abstract We consider the multistage stochastic programming problem where uncertainty enters the right-hand sides of the problem. Stochastic Dual Dynamic Programming (SDDP) is a popular method to solve such problems under the assumption that the random data process is stagewise independent. There exist two approaches to incorporate dependence into SDDP. One approach is to model the data process as an autoregressive time series and to reformulate the problem in stagewise independent terms by adding state variables to the model (TS-SDDP). The other approach is to use Markov Chain discretization of the random data process (MC-SDDP). While MC-SDDP can handle any Markovian data process, some advantages of statistical analysis of the policy under the true process are lost. In this work, we compare both approaches based on a computational study using the long-term operational planning problem of the Brazilian interconnected power systems. We found that for the considered problem the optimality bounds computed by the MC-SDDP method close faster than its TS-SDDP counterpart, and the MC-SDDP policy dominates the TS-SDDP policy. When implementing the optimized policies on real data, we observe that not only the method but also the quality of the stochastic model has an impact on policy performance and that using an AV @ R formulation is effective in making the policy robust against a misspecified stochastic model.

We consider the stochastic dual dynamic programming (SDDP) algorithm - a widely employed algorithm applied to multistage stochastic programming - and propose a variant using experience replay - a batch learning technique from reinforcement learning. To connect SDDP with reinforcement learning, we cast SDDP as a Q-learning algorithm and describe its application in both risk-neutral and risk-averse settings. We demonstrate the superiority of the algorithm over conventional SDDP by benchmarking it against PSR's SDDP software using a large-scale instance of the long-term planning problem of inter-connected hydropower plants in Colombia. We find that SDDP with batch learning is able to produce tighter optimality gaps in a shorter amount of time than conventional SDDP. We also find that batch learning improves the parallel efficiency of SDDP backward passes.

Grid energy storage plays a key role in making carbon-free, renewable energy production a reality. Yet, when it comes to maximizing profit, owners of storage assets still struggle with coordinating their trading activities across time because of the complex nature of multisettlement electricity markets. In “Coordination of Multimarket Bidding of Grid-Energy Storage,” Nils Löhndorf and David Wozabal propose a multistage stochastic programming model for market-oriented optimization of energy storage. To calculate lower and upper bounds on optimal values, they develop novel methods for scenario-tree generation and information relaxation. They show that a coordinated policy that reserves capacity for the short-term markets is optimal and that the gap to a sequential policy increases with short-term price volatility and market liquidity. The authors find that coordination is beneficial for all considered asset types and that flexible storages with high price impact benefit most. Their findings inform storage owners which markets contribute most value, how to organize trading across time, and how to calculate optimal bidding strategies.