Solution of the linear quadratic regulator problem of black box linear systems using reinforcement learning

Date

2022-03-05

Free to read from

Supervisor/s

Journal Title

Journal ISSN

Volume Title

Publisher

Elsevier

Department

Type

Article

ISSN

0020-0255

Format

Citation

Perrusquia A. (2022) Solution of the linear quadratic regulator problem of black box linear systems using reinforcement learning, Information Sciences, Volume 595, May 2022, pp. 364-377

Abstract

In this paper, a Q-learning algorithm is proposed to solve the linear quadratic regulator problem of black box linear systems. The algorithm only has access to input and output measurements. A Luenberger observer parametrization is constructed using the control input and a new output obtained from a factorization of the utility function. An integral reinforcement learning approach is used to develop the Q-learning approximator structure. A gradient descent update rule is used to estimate on-line the parameters of the Q-function. Stability and convergence of the Q-learning algorithm under the Luenberger observer parametrization is assessed using Lyapunov stability theory. Simulation studies are carried out to verify the proposed approach.

Description

Software Description

Software Language

Github

Keywords

Linear quadratic regulator, State observer parametrization, Q-learning, Gradient descent, Output feedback, Persistency of excitation

DOI

Rights

Attribution 4.0 International

Relationships

Relationships

Supplements

Funder/s