General multi-agent reinforcement learning integrating heuristic-based delay priority strategy for demand and capacity balancing
| dc.contributor.author | Chen, Yutong | |
| dc.contributor.author | Xu, Yan | |
| dc.contributor.author | Hu, Minghua | |
| dc.date.accessioned | 2023-08-08T14:30:16Z | |
| dc.date.available | 2023-08-08T14:30:16Z | |
| dc.date.issued | 2023-06-22 | |
| dc.description.abstract | Reinforcement learning (RL) techniques have been studied for solving the demand and capacity balancing (DCB) problem in air traffic management to exploit their full computational potential. Due to the lack of generalisation and the seemingly reduced optimisation performance affected by the training scenarios, it is challenging for existing RL-based DCB methods to be effectively applied in practice. This paper proposes a general multi-agent reinforcement learning (MARL) method that integrates a heuristic-based delay priority strategy to improve the efficiency of the solution and the generalisation of the model. The delay priority strategy is used to reduce the potential learning task and thus training difficulty. This study explores what features of the delay priority strategy are better suited to the MARL method. A long short-term memory (LSTM) network is integrated into a deep q-learning network (DQN) to ensure the model compatible with arbitrary DCB instances and to facilitate agents to identify key sectors. This study is conducted as a part of a large-scale European DCB research project, where real data from French and Spanish airspace are used for experimentation. Results suggest that the proposed method has advantages in generalisation, optimisation performance and computational performance over state-of-the-art RL-based DCB methods. | en_UK |
| dc.identifier.citation | Chen Y, Xu Y, Hu M. (2023) General multi-agent reinforcement learning integrating heuristic-based delay priority strategy for demand and capacity balancing. Transportation Research Part C: Emerging Technologies, Volume 153, August 2023, Article No. 104218 | en_UK |
| dc.identifier.issn | 0968-090X | |
| dc.identifier.uri | https://doi.org/10.1016/j.trc.2023.104218 | |
| dc.identifier.uri | https://dspace.lib.cranfield.ac.uk/handle/1826/20069 | |
| dc.language.iso | en | en_UK |
| dc.publisher | Elsevier | en_UK |
| dc.rights | Attribution-NonCommercial-NoDerivatives 4.0 International | * |
| dc.rights.uri | http://creativecommons.org/licenses/by-nc-nd/4.0/ | * |
| dc.subject | Demand and capacity balancing | en_UK |
| dc.subject | Air traffic flow management | en_UK |
| dc.subject | Multi-agent reinforcement learning | en_UK |
| dc.subject | Heuristic algorithm | en_UK |
| dc.subject | Deep q-learning network | en_UK |
| dc.subject | Long short-term memory | en_UK |
| dc.title | General multi-agent reinforcement learning integrating heuristic-based delay priority strategy for demand and capacity balancing | en_UK |
| dc.type | Article | en_UK |