Partially explainable big data driven deep reinforcement learning for green 5G UAV

UAV enabled terrestrial wireless networks enables targeted user-centric service provisioning to en-richen both deep urban coverage and target various rural challenge areas. However, UAVs have to balance the energy consumption of flight with the benefits of wireless capacity delivery via a high dimensional optimisation problem. Classic reinforcement learning (RL) cannot meet this challenge and here, we propose to use deep reinforcement learning (DRL) to optimise both aggregate and minimum service provisioning. In order to achieve a trusted autonomy, the DRL agents have to be able to explain its actions for transparent human-machine interrogation. We design a Double Dueling Deep Q-learning Neural Network (DDDQN) with Prioritised Experience Replay (PER) and fixed Q-targets to achieve stable performance and avoid over-fitting, offering performance gains over naive DQN algorithms. We then use a big data driven case study and found that UAVs battery size determines the nature of its autonomous mission, ranging from an efficient exploiter of one hotspot (100% reward gain) to a stochastic explorer of many hotspots (60-150% reward gain). Using a variety of telecom and social media data, we infer driving Quality-of-Experience (QoE) and Quality-of-Service (QoS) metrics that are in contention with UAV power and communication constraints. Our greener UAVs (30-40% energy saved) address both quantitative QoS and qualitative QoE issues. Partial interpretability in the reinforcement learning is achieved using data features extracted in the hidden layers, offering an initial step for explainable AI (XAI) connecting machine intelligence with human expertise.