Locally generalised multi-agent reinforcement learning for demand and capacity balancing with customised neural networks

Simple item page
dc.contributor.authorChen, Yutong
dc.contributor.authorHu, Minghua
dc.contributor.authorXu, Yan
dc.contributor.authorYang, Lei
dc.date.accessioned2023-04-26T10:39:43Z
dc.date.available2023-04-26T10:39:43Z
dc.date.issued2023-04-18
dc.description.abstractReinforcement Learning (RL) techniques are being studied to solve the Demand and Capacity Balancing (DCB) problems to fully exploit their computational performance. A locally generalised Multi-Agent Reinforcement Learning (MARL) for real-world DCB problems is proposed. The proposed method can deploy trained agents directly to unseen scenarios in a specific Air Traffic Flow Management (ATFM) region to quickly obtain a satisfactory solution. In this method, agents of all flights in a scenario form a multi-agent decision-making system based on partial observation. The trained agent with the customised neural network can be deployed directly on the corresponding flight, allowing it to solve the DCB problem jointly. A cooperation coefficient is introduced in the reward function, which is used to adjust the agent’s cooperation preference in a multi-agent system, thereby controlling the distribution of flight delay time allocation. A multi-iteration mechanism is designed for the DCB decision-making framework to deal with problems arising from non-stationarity in MARL and to ensure that all hotspots are eliminated. Experiments based on large-scale high-complexity real-world scenarios are conducted to verify the effectiveness and efficiency of the method. From a statistical point of view, it is proven that the proposed method is generalised within the scope of the flights and sectors of interest, and its optimisation performance outperforms the standard computer-assisted slot allocation and state-of-the-art RL-based DCB methods. The sensitivity analysis preliminarily reveals the effect of the cooperation coefficient on delay time allocation.en_UK
dc.identifier.citationChen Y, Hu M, Xu Y, Yang L. (2023) Locally generalised multi-agent reinforcement learning for demand and capacity balancing with customised neural networks. Chinese Journal of Aeronautics, Volume 36, Issue 4, April 2023, pp. 338-353en_UK
dc.identifier.issn1000-9361
dc.identifier.urihttps://doi.org/10.1016/j.cja.2023.01.010
dc.identifier.urihttps://dspace.lib.cranfield.ac.uk/handle/1826/19549
dc.language.isoenen_UK
dc.publisherElsevieren_UK
dc.rightsAttribution-NonCommercial-NoDerivatives 4.0 International*
dc.rights.urihttp://creativecommons.org/licenses/by-nc-nd/4.0/*
dc.subjectAir traffic flow managementen_UK
dc.subjectDemand and capacity balancingen_UK
dc.subjectDeep Q-learning networken_UK
dc.subjectFlight delaysen_UK
dc.subjectGeneralisationen_UK
dc.subjectGround delay programen_UK
dc.subjectMulti-agent reinforcement learningen_UK
dc.titleLocally generalised multi-agent reinforcement learning for demand and capacity balancing with customised neural networksen_UK
dc.typeArticleen_UK

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Locally_generalised_multi-agent_reinforcement_learning-2023.pdf
Size:
3.39 MB
Format:
Adobe Portable Document Format
Description:
Download
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
1.63 KB
Format:
Item-specific license agreed upon to submission
Description:
Download

Collections

Staff publications (SATM)