Explainable data-driven Q-learning control for a class of discrete-time linear autonomous systems

Simple item page
dc.contributor.authorPerrusquía, Adolfo
dc.contributor.authorZou, Mengbang
dc.contributor.authorGuo, Weisi
dc.date.accessioned2024-08-12T08:54:19Z
dc.date.available2024-08-12T08:54:19Z
dc.date.freetoread2024-08-12
dc.date.issued2024-11-01
dc.date.pubOnline2024-08-06
dc.description.abstractExplaining what a reinforcement learning (RL) control agent learns play a crucial role in the safety critical control domain. Most of the approaches in the state-of-the-art focused on imitation learning methods that uncover the hidden reward function of a given control policy. However, these approaches do not uncover what the RL agent learns effectively from the agent-environment interaction. The policy learned by the RL agent depends in how good the state transition mapping is inferred from the data. When the state transition mapping is wrongly inferred implies that the RL agent is not learning properly. This can compromise the safety of the surrounding environment and the agent itself. In this paper, we aim to uncover the elements learned by data-driven RL control agents in a special class of discrete-time linear autonomous systems. Here, the approach aims to add a new explainable dimension to data-driven control approaches to increase their trust and safe deployment. We focus on the classical data-driven Q-learning algorithm and propose an explainable Q-learning (XQL) algorithm that can be further expanded to other data-driven RL control agents. Simulation experiments are conducted to observe the effectiveness of the proposed approach under different scenarios using several discrete-time models of autonomous platforms.
dc.description.journalNameInformation Sciences
dc.format.extentArticle number 121283
dc.identifier.citationPerrusquía A, Zou M, Guo W. (2024) Explainable data-driven Q-learning control for a class of discrete-time linear autonomous systems. Information Sciences, Volume 682, November 2024, Article number 121283
dc.identifier.issn0020-0255
dc.identifier.urihttps://doi.org/10.1016/j.ins.2024.121283
dc.identifier.urihttps://dspace.lib.cranfield.ac.uk/handle/1826/22771
dc.language.isoen
dc.publisherElsevier
dc.publisher.urihttps://www.sciencedirect.com/science/article/pii/S0020025524011976
dc.rightsAttribution 4.0 Internationalen
dc.rights.urihttp://creativecommons.org/licenses/by/4.0/
dc.subjectQ-learning
dc.subjectState-transition function
dc.subjectExplainable Q-learning (XQL)
dc.subjectControl policy
dc.titleExplainable data-driven Q-learning control for a class of discrete-time linear autonomous systems
dc.typeArticle
dcterms.dateAccepted2024-07-31

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Explainable_data-driven_Q-learning_control-2024.pdf
Size:
772.24 KB
Format:
Adobe Portable Document Format
Download
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
1.63 KB
Format:
Item-specific license agreed upon to submission
Description:
Download

Collections

Staff publications (SATM)