Multi-agent deep reinforcement learning for solving large-scale air traffic flow management problem: a time-step sequential decision approach

Simple item page
dc.contributor.authorTang, Yifan
dc.contributor.authorXu, Yan
dc.date.accessioned2023-01-05T19:49:23Z
dc.date.available2023-01-05T19:49:23Z
dc.date.issued2022-11-15
dc.description.abstractIn this paper, we focus on the demand-capacity balancing (DCB) problem in air traffic flow management, which is considered as a fully cooperative multi-agent learning task. First, a rule-based time-step environment is designed to mimic the DCB process. In this environment, each agent ‘flight’ decides its action at valid time steps. Three different rules are defined, based on the remaining capacity and the number of cooperative flights in each sector, to ease the learning process. Second, a multi-agent reinforcement learning framework, built on the proximal policy optimization (MAPPO), is proposed by using the parameter sharing mechanism and the mean-field approximation method, where an inherent feature of all other agents is extracted to address the credit assignment problem. Moreover, a supervisor integrated MAPPO framework is proposed, where a supervisor is designed to generate supervised actions, in such a way to further improve the learning performance. In the experiments, two performance indices, Search Capability and Generalization Capability, are considered. Both indices are assessed with the evaluation of two toy cases and a real-world case study. Results suggest that, the supervisor integrated MAPPO with supervised actions achieves the best performance across the different cases; other proposed methods also show some promising Search Capability, but only prove an acceptable Generalization Capability in simpler cases than the training cases.en_UK
dc.identifier.citationTang Y, Xu Y. (2021) Multi-agent deep reinforcement learning for solving large-scale air traffic flow management problem: a time-step sequential decision approach. In: 2021 AIAA/IEEE 40th Digital Avionics Systems Conference (DASC), 3-7 October 2021, San Antonio, USAen_UK
dc.identifier.eisbn978-1-6654-3420-1
dc.identifier.isbn978-1-6654-3421-8
dc.identifier.urihttps://doi.org/10.1109/DASC52595.2021.9594329
dc.identifier.urihttps://dspace.lib.cranfield.ac.uk/handle/1826/18882
dc.language.isoenen_UK
dc.publisherIEEEen_UK
dc.rightsAttribution-NonCommercial 4.0 International*
dc.rights.urihttp://creativecommons.org/licenses/by-nc/4.0/*
dc.subjectair traffic flow managementen_UK
dc.subjectdemand-capacity balanceen_UK
dc.subjectmulti-agent reinforcement learningen_UK
dc.subjectproximal policy optimizationen_UK
dc.titleMulti-agent deep reinforcement learning for solving large-scale air traffic flow management problem: a time-step sequential decision approachen_UK
dc.typeConference paperen_UK

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
solving_large-scale_air_traffic_flow_management_problem-2022.pdf
Size:
777.38 KB
Format:
Adobe Portable Document Format
Description:
Download
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
1.63 KB
Format:
Item-specific license agreed upon to submission
Description:
Download

Collections

Staff publications (SATM)