Decentralized mission planning for multiple unmanned aerial vehicles

The focus of this thesis is the mission planning challenge for multiple unmanned aerial vehicles (UAVs), with a particular emphasis on their stable operations in a stochastic and dynamic environment. Mission planning is a crucial module in automated multi-UAV systems, allowing for efficient resource allocation, conflict resolution, and reliable operation. However, the distributed nature of the system and physical and environmental constraints make it challenging to develop effective mission planning algorithms. The thesis begins with a review of the taxonomy, frameworks, and techniques in multiple-UAV mission planning. Following this, it identifies four critical research challenges, encompassing scalability, efficiency, adaptability and robustness, and energy management and renewable strategies. In response to these challenges, four objectives have been defined with the overall aim of developing a generic decentralized mission planning paradigm for multi-UAV systems. The thesis subsequently concentrates on accomplishing these objectives, with notable contributions in the development of 1) a decentralized task coordination algorithm, 2) an efficient route planner in consideration of recharging, and 3) an energy-aware planning framework. This research first proposes a decentralized auction-based coordination strategy for task-constrained multi-agent stochastic planning problems. Through casting the problem as task-constrained Markov decision processes (MDPs), the task dependency due to an exclusive constraint is despatched from Multi-agent Markov decision processes (M-MDPs) and then resolved by adopting an auction-based coordination method. For multi-agent stochastic planning problems, the suggested technique resolves the trade-off concern between computational tractability and solution quality. The proposed method ensures convergence, achieves at least 50% optimality under the assumption of a submodular reward function, and greatly reduces the computational complexity compared to multi-agent MDPs. Deep Auction is then proposed as an approximate modification of the suggested auction-based coordination method, where two neural network approximators are introduced to facilitate scaled-up implementations. By theoretical analysis, these two proposed algorithms achieve better robustness and feature less computing complexity compared to the state-of-the-art. Finally, a case study of drone delivery with time windows is implemented for validation. Simulation results demonstrate the theoretical benefits of the recommended methodologies. Then, an efficient route planner for individual UAVs accounting for recharging services is proposed. Despite extensive research in decision-making algorithms, existing models have limitations in accurately representing real-world scenarios in terms of UAV’s physical restrictions and stochastic operating environments. To address this, a drone delivery problem with recharging (DDP-R) is proposed. The problem is characterized by directional edges and stochastic edge costs affected by winds. To solve DDP-Rs, a novel edge-enhanced attention model (AM-E) is proposed and trained via the REINFOCE algorithm to map the optimal policy. AM-E consists of a series of edge-enhanced dot-product attention layers that capture the heterogeneous relationships between nodes in DDP-Rs by incorporating adjacent edge information. Simulation results show that the edge enhancement achieves better results with a simpler architecture and fewer trainable parameters, compared to other deep learning models. Extensive simulations demonstrate that the proposed DRL method outperforms state-of-the-art heuristics in solving the DDP-R problem, especially at large sizes, for both non-wind and windy scenarios. Finally, we integrate the above route planning algorithm into an online energy inference framework, namely, the Energy-aware Planning Framework (EaPF), with the aim of optimizing solution quality in consideration of possible time-window violation and battery depletion. The framework comprises a statistical energy predictive model, a risk assessment module, and a route optimizer, with functions of modelling energy costs, estimating risks, and optimizing the risk-sensitive objective. Concretely, a Mixture Density Network (MDN) is established for predicting the distribution of future energy consumption taking account of wind conditions. The MDN is trained by historical data and continuously updated as new data is collected. Then, a risk-sensitive criterion is formed based on the MDN model of energy consumption to assess the risk of task lateness and battery depletion. To minimize the risk-sensitive objective, the EaPF incorporates the proposed AM-E planner using a Model-based Multi-Sampling (MBMS) route construction strategy, to further improve solution quality and planning robustness. In the context of drone deliveries, simulations validate the effectiveness of the MDN energy model and the EaPF. Results show that the integration of EaPF achieves an average cost reduction of 25%, which implies a lower energy cost, a higher task accomplishment rate, and a smaller battery depletion risk compared to the stand-alone DRL planner.