Explaining data-driven control in autonomous systems: a reinforcement learning case study

Explaining what does a data-driven control algorithm learns play a crucial role for safety critical control of autonomous platforms in transportation. This is more acute in reinforcement learning control algorithms, where the learned control policy depends on various factors that are hidden within the data. Explainable artificial intelligence methods have been used to explain the outcomes of machine learning methods by analysing input-output relations. However, data-driven control does not pose a simple input-output mapping and hence, the resulting explanations lack depth. To deal with this issue, this paper proposes a explainable data-driven control method that allows to understand what the data-driven method is learning from the data. The model is composed by a Q-learning algorithm enhanced by a dynamic mode decomposition with control (DMDc) algorithm for state-transition function estimation. Both the Q-learning and DMDc provides the elements that are learned from the data and allow the construction of counterfactual explanations. The proposed approach is robust and does not require hyperparameter tuning. Simulation experiments are conducted to observe the benefits and challenges of the method.