Solution of the linear quadratic regulator problem of black box linear systems using reinforcement learning
Date published
2022-03-05
Free to read from
Authors
Perrusquía, Adolfo
Supervisor/s
Journal Title
Journal ISSN
Volume Title
Publisher
Elsevier
Department
Type
Article
ISSN
0020-0255
Format
Citation
Perrusquia A. (2022) Solution of the linear quadratic regulator problem of black box linear systems using reinforcement learning, Information Sciences, Volume 595, May 2022, pp. 364-377
Abstract
In this paper, a Q-learning algorithm is proposed to solve the linear quadratic regulator problem of black box linear systems. The algorithm only has access to input and output measurements. A Luenberger observer parametrization is constructed using the control input and a new output obtained from a factorization of the utility function. An integral reinforcement learning approach is used to develop the Q-learning approximator structure. A gradient descent update rule is used to estimate on-line the parameters of the Q-function. Stability and convergence of the Q-learning algorithm under the Luenberger observer parametrization is assessed using Lyapunov stability theory. Simulation studies are carried out to verify the proposed approach.
Description
Software Description
Software Language
Github
Keywords
Linear quadratic regulator, State observer parametrization, Q-learning, Gradient descent, Output feedback, Persistency of excitation
DOI
Rights
Attribution 4.0 International