Nash double Q-based multi-agent deep reinforcement learning for interactive merging strategy in mixed traffic

Simple item page
dc.contributor.authorLi, Lin
dc.contributor.authorZhao, Wanzhong
dc.contributor.authorWang, Chunyan
dc.contributor.authorFotouhi, Abbas
dc.contributor.authorLiu, Xuze
dc.date.accessioned2023-10-03T12:47:06Z
dc.date.available2023-10-03T12:47:06Z
dc.date.issued2023-09-19
dc.description.abstractThe interaction between ramp and mainline vehicles plays a crucial role in merging areas, especially in the mixed-traffic environment. The driving behaviours of human drivers are uncertain and diverse, and the uncertainty makes it more complex for connected automated vehicles (CAV) to plan trajectories and merge into the mainline. To overcome this problem, a interactive merging strategy based on multi-agent deep reinforcement learning (MADRL) is designed, enabling the ramp vehicle (CAV) to consider the dynamic reaction of mainline vehicles. There are two agents in our interactive strategy, one of which is to predict and analyse the behaviour of mainline vehicles (human-driven vehicles, HDV, or non-connected vehicles). The other is created for exploring optimal merging actions of ramp vehicles. Firstly, game theory is used to model the competitive behaviours between ramp and mainline vehicles, and the Nash equilibrium of joint actions guides the ramp vehicle to learn best response to the mainline vehicle. Secondly, the Nash double Q algorithm is developed to ensure the outputs of Q networks are trained to efficiently converge to the Nash equilibrium point. The trained Q networks are then used for online control. Finally, our strategy is compared with single RL and existing MADRL algorithms in real on-ramp scenarios. Simulations show our strategy to be successful in coordinating both vehicles via analysis of human drivers, resulting in improved driving performance in terms of global safety, efficiency, and comfort.en_UK
dc.identifier.citationLi L, Zhao W, Wang C, et al., (2024) Nash double Q-based multi-agent deep reinforcement learning for interactive merging strategy in mixed traffic. Expert Systems with Applications, Volume 237, Part B, March 2024, Article number 121458en_UK
dc.identifier.issn0957-4174
dc.identifier.urihttps://doi.org/10.1016/j.eswa.2023.121458
dc.identifier.urihttps://dspace.lib.cranfield.ac.uk/handle/1826/20320
dc.language.isoenen_UK
dc.publisherElsevieren_UK
dc.rightsAttribution-NonCommercial-NoDerivatives 4.0 International*
dc.rights.urihttp://creativecommons.org/licenses/by-nc-nd/4.0/*
dc.subjectReinforcement learningen_UK
dc.subjectTrajectory planningen_UK
dc.subjectDeep reinforcement learningen_UK
dc.subjectAutonomous drivingen_UK
dc.titleNash double Q-based multi-agent deep reinforcement learning for interactive merging strategy in mixed trafficen_UK
dc.typeArticleen_UK

Files

Original bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
interactive_merging_strategy_in_mixed_traffic-2023.pdf
Size:
3.39 MB
Format:
Adobe Portable Document Format
Description:
Download
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
1.63 KB
Format:
Item-specific license agreed upon to submission
Description:
Download

Collections

Staff publications (SATM)