Deep reinforcement learning for optimal hydropower reservoir operation

Simple item page
dc.contributor.authorXu, Wei
dc.contributor.authorMeng, Fanlin
dc.contributor.authorGuo, Weisi
dc.contributor.authorLi, Xia
dc.date.accessioned2021-05-26T13:48:32Z
dc.date.available2021-05-26T13:48:32Z
dc.date.issued2021-05-21
dc.description.abstractOptimal operation of hydropower reservoir systems is a classical optimization problem of high dimensionality and stochastic nature. A key challenge lies in improving the interpretability of operation strategies, i.e., the cause–effect relationship between system outputs (or actions) and contributing variables such as states and inputs. This paper reports for the first time a new deep reinforcement learning (DRL) framework for optimal operation of reservoir systems based on deep Q-networks (DQNs), which provides a significant advance in understanding the performance of optimal operations. DQN combines Q-learning and two deep artificial neural networks (ANNs), and acts as the agent to interact with the reservoir system through learning its states and providing actions. Three knowledge forms of learning considering the states, actions, and rewards were constructed to improve the interpretability of operation strategies. The impacts of these knowledge forms and DRL learning parameters on operation performance were analyzed. The DRL framework was tested on the Huanren hydropower system in China, using 400-year synthetic flow data for training and 30-year observed flow data for verification. The discretization levels of reservoir water level and energy output yield contrasting effects: finer discretization of water level improved performance in terms of annual hydropower generated and hydropower production reliability; however, finer discretization of hydropower production can reduce search efficiency, and thus the resulting DRL performance. Compared with benchmark algorithms including dynamic programming, stochastic dynamic programming, and decision tree, the proposed DRL approach can effectively factor in future inflow uncertainties when determining optimal operations and can generate markedly higher hydropower. This study provides new knowledge of the performance of DRL in the context of hydropower system characteristics and data input features, and shows promise for potentially being implemented in practice to derive operation policies that can be updated automatically by learning from new data.en_UK
dc.identifier.citationXu W, Meng F, Guo W, et al., (2021) Deep reinforcement learning for optimal hydropower reservoir operation. Journal of Water Resources Planning and Management, Volume 147, Issue 8, August 2021, Article number 04021045en_UK
dc.identifier.issn0733-9496
dc.identifier.urihttps://doi.org/10.1061/(ASCE)WR.1943-5452.0001409
dc.identifier.urihttps://dspace.lib.cranfield.ac.uk/handle/1826/16718
dc.language.isoenen_UK
dc.publisherAmerican Society of Civil Engineersen_UK
dc.rightsAttribution-NonCommercial 4.0 International*
dc.rights.urihttp://creativecommons.org/licenses/by-nc/4.0/*
dc.subjectReservoir Operationen_UK
dc.subjectHydropower 40 Systemen_UK
dc.subjectDeep Reinforcement Learningen_UK
dc.subjectDeep Q-Networken_UK
dc.subjectArtificial Intelligenceen_UK
dc.titleDeep reinforcement learning for optimal hydropower reservoir operationen_UK
dc.typeArticleen_UK

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Deep_reinforcement_learning_optimal_hydropower_reservoir-2021.pdf
Size:
1.76 MB
Format:
Adobe Portable Document Format
Description:
Download
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
1.63 KB
Format:
Item-specific license agreed upon to submission
Description:
Download

Collections

Staff publications (SATM)