Privacy-preserving federated deep reinforcement learning for mobility-as-a-service

Date

2023-10-03

Free to read from

Supervisor/s

Journal Title

Journal ISSN

Volume Title

Publisher

IEEE

Department

Type

Article

ISSN

1524-9050

Format

Citation

Chu K-F, Guo W. (2024) Privacy-preserving federated deep reinforcement learning for mobility-as-a-service, IEEE Transactions on Intelligent Transportation Systems, Volume 25, Issue 2, February 2024, pp. 1882-1896

Abstract

Mobility-as-a-service (MaaS) is a new transport model that combines multiple transport modes in a single platform. Dynamic passenger behavior based on past experiences requires reinforcement-based optimization of MaaS services. Deep reinforcement learning (DRL) may improve passenger satisfaction by offering the most appropriate transport services based on individual passenger experiences and preferences. However, this produces a new privacy risk to the MaaS platform using the centralized DRL method. Information leakage will occur if the platform is not carefully designed with privacy-preserving mechanisms. In this paper, we propose a federated deep deterministic policy gradient (FDDPG) that maximizes passenger satisfaction and MaaS long-term profit while preserving privacy. We enforce an equally weighted experience sampling mechanism to prevent sampling bias such that the solution quality of FDDPG is statistically equivalent to the centralized algorithm. During the model training and inference, information is processed locally, and only the gradients are shared, which prevents information leakage to any semi-honest participants and eavesdroppers. Secure aggregation protocol in line with the dynamic property of the mobile agent is also used in the gradient sharing step to ensure that the algorithm is prevented from inference attacks. We perform experiments on New York City-based real-world and synthetic scenarios. The results show that the proposed FDDPG can improve the MaaS profit and passenger satisfaction by about 90% and 15%, respectively, and maintain stable training against agent dropout. Our approach and findings could enhance MaaS utility as well as facilitate passenger trust and participation in MaaS and other data-driven transportation systems.

Description

Software Description

Software Language

Github

Keywords

Privacy-preserving machine learning, federated reinforcement learning, mobility-as-a-service, passenger behavior, secret sharing

DOI

Rights

Attribution 4.0 International

Relationships

Relationships

Supplements

Funder/s

Engineering and Physical Sciences Research Council (EPSRC): EP/V039164/1