General multi-agent reinforcement learning integrating adaptive manoeuvre strategy for real-time multi-aircraft conflict resolution

Simple item page
dc.contributor.authorChen, Yutong
dc.contributor.authorHu, Minghua
dc.contributor.authorYang, Lei
dc.contributor.authorXu, Yan
dc.contributor.authorXie, Hua
dc.date.accessioned2023-06-21T11:43:00Z
dc.date.available2023-06-21T11:43:00Z
dc.date.issued2023-04-12
dc.description.abstractReinforcement learning (RL) techniques are under investigation for resolving conflict in air traffic management (ATM), exploiting their computational capabilities and ability to cope with flight uncertainty. However, the limitations of generalisation make it difficult for existing RL-based conflict resolution (CR) methods to be effective in practice. This paper proposes a general multi-agent reinforcement learning (MARL) method that integrates an adaptive manoeuvre strategy to enhance both the solution’s efficiency and the model’s generalisation in multi-aircraft conflict resolution (MACR). A partial observation approach based on the imminent threat detection sectors is used to gather critical environmental information, enabling the model to be applied in arbitrary scenarios. Agents are trained to provide the correct flight intention (such as increasing speed and yawing to the left), while an adaptive manoeuvre strategy generates the specific manoeuvre (speed and heading parameters) based on the flight intention. To address flight uncertainty and performance challenges caused by the intrinsic non-stationarity in MARL, a warning area for each aircraft is introduced. We employ a state-of-the-art Deep Q-learning Network (DQN) method, Rainbow DQN, to improve the efficiency of the RL algorithm. The multi-agent system is trained and deployed in a distributed manner to adapt to real-world scenarios. A sensitivity analysis of uncertainty levels and warning area sizes is conducted to explore their impact on the proposed method. Simulation experiments confirm the effectiveness of the training and generalisation of the proposed method.en_UK
dc.identifier.citationChen Y, Hu M, Yang L, et al., (2023) General multi-agent reinforcement learning integrating adaptive manoeuvre strategy for real-time multi-aircraft conflict resolution, Transportation Research Part C: Emerging Technologies, Volume 151, June 2023, Article Number 104125en_UK
dc.identifier.eissn1879-2359
dc.identifier.issn0968-090X
dc.identifier.urihttps://doi.org/10.1016/j.trc.2023.104125
dc.identifier.urihttps://dspace.lib.cranfield.ac.uk/handle/1826/19827
dc.language.isoenen_UK
dc.publisherElsevieren_UK
dc.rightsAttribution-NonCommercial-NoDerivatives 4.0 International*
dc.rights.urihttp://creativecommons.org/licenses/by-nc-nd/4.0/*
dc.subjectAir traffic managementen_UK
dc.subjectMulti-aircraft conflict resolutionen_UK
dc.subjectMulti-agent reinforcement learningen_UK
dc.subjectDeep q-learning networken_UK
dc.subjectGeneralisationen_UK
dc.subjectUncertaintyen_UK
dc.titleGeneral multi-agent reinforcement learning integrating adaptive manoeuvre strategy for real-time multi-aircraft conflict resolutionen_UK
dc.typeArticleen_UK

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Real-time_multi-aircraft_conflict_resolution-2023.pdf
Size:
28.42 MB
Format:
Adobe Portable Document Format
Description:
Download
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
1.63 KB
Format:
Item-specific license agreed upon to submission
Description:
Download

Collections

Staff publications (SATM)