General real-time three-dimensional multi-aircraft conflict resolution method using multi-agent reinforcement learning

Simple item page
dc.contributor.authorChen, Yutong
dc.contributor.authorXu, Yan
dc.contributor.authorYang, Lei
dc.contributor.authorHu, Minghua
dc.date.accessioned2023-10-27T09:02:28Z
dc.date.available2023-10-27T09:02:28Z
dc.date.issued2023-10-10
dc.description.abstractReinforcement learning (RL) techniques have been studied for solving the conflict resolution (CR) problem in air traffic management, leveraging their potential for computation and ability to handle uncertainty. However, challenges remain that impede the application of RL methods to CR in practice, including three-dimensional manoeuvres, generalisation, trajectory recovery, and success rate. This paper proposes a general multi-agent reinforcement learning approach for real-time three-dimensional multi-aircraft conflict resolution, in which agents share a neural network and are deployed on each aircraft to form a distributed decision-making system. To address the challenges, several technologies are introduced, including a partial observation model based on imminent threats for generalisation, a safety separation relaxation model for multiple flight levels for three-dimensional manoeuvres, an adaptive manoeuvre strategy for trajectory recovery, and a conflict buffer model for success rate. The Rainbow Deep Q-learning Network (DQN) is used to enhance the efficiency of the RL process. A simulation environment that considers flight uncertainty (resulting from mechanical and navigation errors and wind) is constructed to train and evaluate the proposed approach. The experimental results demonstrate that the proposed method can resolve conflicts in scenarios with much higher traffic density than in today’s real-world situations.en_UK
dc.identifier.citationChen Y, Xu Y, Yang L, Hu M. (2023) General real-time three-dimensional multi-aircraft conflict resolution method using multi-agent reinforcement learning. Transportation Research Part C: Emerging Technologies, Volume 157, December 2023, Article number 104367en_UK
dc.identifier.issn0968-090X
dc.identifier.urihttps://doi.org/10.1016/j.trc.2023.104367
dc.identifier.urihttps://dspace.lib.cranfield.ac.uk/handle/1826/20460
dc.language.isoenen_UK
dc.publisherElsevieren_UK
dc.rightsAttribution-NonCommercial-NoDerivatives 4.0 International*
dc.rights.urihttp://creativecommons.org/licenses/by-nc-nd/4.0/*
dc.subjectAir traffic managementen_UK
dc.subjectThree-dimensional multi-aircraft conflict resolutionen_UK
dc.subjectMulti-agent reinforcement learningen_UK
dc.subjectDeep q-learning networken_UK
dc.subjectGeneralisationen_UK
dc.subjectUncertaintyen_UK
dc.titleGeneral real-time three-dimensional multi-aircraft conflict resolution method using multi-agent reinforcement learningen_UK
dc.typeArticleen_UK

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
multi-aircraft_conflict_resolution_method-2023.pdf
Size:
4.21 MB
Format:
Adobe Portable Document Format
Description:
Download
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
1.63 KB
Format:
Item-specific license agreed upon to submission
Description:
Download

Collections

Staff publications (SATM)