General multi-agent reinforcement learning integrating heuristic-based delay priority strategy for demand and capacity balancing

dc.contributor.authorChen, Yutong
dc.contributor.authorXu, Yan
dc.contributor.authorHu, Minghua
dc.date.accessioned2023-08-08T14:30:16Z
dc.date.available2023-08-08T14:30:16Z
dc.date.issued2023-06-22
dc.description.abstractReinforcement learning (RL) techniques have been studied for solving the demand and capacity balancing (DCB) problem in air traffic management to exploit their full computational potential. Due to the lack of generalisation and the seemingly reduced optimisation performance affected by the training scenarios, it is challenging for existing RL-based DCB methods to be effectively applied in practice. This paper proposes a general multi-agent reinforcement learning (MARL) method that integrates a heuristic-based delay priority strategy to improve the efficiency of the solution and the generalisation of the model. The delay priority strategy is used to reduce the potential learning task and thus training difficulty. This study explores what features of the delay priority strategy are better suited to the MARL method. A long short-term memory (LSTM) network is integrated into a deep q-learning network (DQN) to ensure the model compatible with arbitrary DCB instances and to facilitate agents to identify key sectors. This study is conducted as a part of a large-scale European DCB research project, where real data from French and Spanish airspace are used for experimentation. Results suggest that the proposed method has advantages in generalisation, optimisation performance and computational performance over state-of-the-art RL-based DCB methods.en_UK
dc.identifier.citationChen Y, Xu Y, Hu M. (2023) General multi-agent reinforcement learning integrating heuristic-based delay priority strategy for demand and capacity balancing. Transportation Research Part C: Emerging Technologies, Volume 153, August 2023, Article No. 104218en_UK
dc.identifier.issn0968-090X
dc.identifier.urihttps://doi.org/10.1016/j.trc.2023.104218
dc.identifier.urihttps://dspace.lib.cranfield.ac.uk/handle/1826/20069
dc.language.isoenen_UK
dc.publisherElsevieren_UK
dc.rightsAttribution-NonCommercial-NoDerivatives 4.0 International*
dc.rights.urihttp://creativecommons.org/licenses/by-nc-nd/4.0/*
dc.subjectDemand and capacity balancingen_UK
dc.subjectAir traffic flow managementen_UK
dc.subjectMulti-agent reinforcement learningen_UK
dc.subjectHeuristic algorithmen_UK
dc.subjectDeep q-learning networken_UK
dc.subjectLong short-term memoryen_UK
dc.titleGeneral multi-agent reinforcement learning integrating heuristic-based delay priority strategy for demand and capacity balancingen_UK
dc.typeArticleen_UK

Files

Original bundle
Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
delay_priority_strategy-2023.pdf
Size:
5.71 MB
Format:
Adobe Portable Document Format
Description:
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
1.63 KB
Format:
Item-specific license agreed upon to submission
Description: