CERES :: Browsing by Author "Shin, Hyo-Sang"

Browsing by Author "Shin, Hyo-Sang"

Now showing 1 - 20 of 26

Open Access
Adversarial proximal policy optimisation for robust reinforcement learning
(AIAA, 2024-01-04) Ince, Bilkan; Shin, Hyo-Sang; Tsourdos, Antonios
Robust reinforcement learning (RL) aims to develop algorithms that can effectively handle uncertainties and disturbances in the environment. Model-free methods play a crucial role in addressing these challenges by directly learning optimal policies without relying on a pre-existing model of the environment. This abstract provides an overview of model-free methods in robust RL, highlighting their key features, advantages, and recent advancements. Firstly, we discuss the fundamental concepts of RL and its challenges in uncertain environments. We then delve into model-free methods, which operate by interacting with the environment and collecting data to learn an optimal policy. These methods typically utilize value-based or policy-based approaches to estimate the optimal action-value function or the policy directly, respectively. To enhance robustness, model-free methods often incorporate techniques such as exploration-exploitation strategies, experience replay, and reward shaping. Exploration-exploitation strategies facilitate the exploration of uncertain regions of the environment, enabling the discovery of more robust policies. Experience replay helps improve sample efficiency by reusing past experiences, allowing the agent to learn from a diverse set of situations. Reward shaping techniques provide additional guidance to the RL agent, enabling it to focus on relevant features of the environment and mitigate potential uncertainties. In this paper, a robust reinforcement learning methodology is adapted utilising a novel Adversarial Proximal Policy Optimisation (A-PPO) method integrating an Adaptive KL penalty PPO. Comparison is made with DQN, DDQN and a conventional PPO algorithm.
Open Access
AFJPDA: a multiclass multi-object tracking with appearance feature-aided joint probabilistic data association
(AIAA, 2024-01-02) Kim, Sukkeun; Petrunin, Ivan; Shin, Hyo-Sang
This study addresses a multiclass multi-object tracking problem in consideration of clutters in the environment. To alleviate issues with clutters, we propose the appearance feature-aided joint probabilistic data association filter. We also implemented simple adaptive gating logic for the computational efficiency and track maintenance logic, which can save the lost track for re-association after occlusion or missed detection. The performance of the proposed algorithm was evaluated against a state-of-the-art multi-object tracking algorithm using both multiclass multi-object simulation and real-world aerial images. The evaluation results indicate significant performance improvement of the proposed method against the benchmark state-of-the-art algorithm, especially in terms of reduction in identity switches and fragmentation.
Open Access
Analysis of the traffic conflict situation for speed probability distributions
(Cambridge University Press, 2023-03-30) Öreg, Zs.; Shin, Hyo-Sang; Tsourdos, Antonios
The increasingly widespread application of drones and the emergence of urban air mobility leads to a challenging question in airspace modernisation: how to create a safe and scalable air traffic management system that can handle the expected density of operations. Increasing the number of vehicles in a given airspace volume and enabling routine operations are essential for these services to be economically viable. However, a higher density of operations increases risks, poses a great challenge for coordination and necessitates the development of a novel solution for traffic management. This paper contributes to the research towards such a strategy and the field of airspace management by calculating and analysing the conflict probability in an en-route, free-flight scenario for autonomous vehicles. Analytical methods are used to determine the directional dependence of conflict probabilities for exponential and normal prescribed speed probability distributions. The notions of geometric and speed conflict are introduced and distinguished throughout the calculations of the paper. The effect of truncating the set of available flight speeds is also investigated. The sensitivity of the calculated results to speed and heading perturbations is studied within the analytical framework and verified by numerical simulations. Results enable a fresh approach to conflict detection and resolution through distribution shaping and are intended to be used in an integrated, stochastic coordination framework.
Open Access
Behaviour monitoring: investigation of local and distributed approaches
(2017) Turchi, Dario; Shin, Hyo-Sang; Tsourdos, Antonios
Nowadays, the widespread availability of cheap and eﬃcient unmanned systems (either aerial, ground or surface) has led to signiﬁcant opportunities in the ﬁeld of remote sensing and automated monitoring. On the one hand, the deﬁnition of eﬃcient approaches to information collection, ﬁltering and fusion has been the focus of extremely relevant research streams over the last decades. On the other hand, far less attention has been given to the problem of ‘interpreting’ the data, thus implementing inference processes able to, e.g., spot anomalies and possible threats in the monitored scenario. It is easy to understand how the automation of the ‘target assessment’ process could bring a great impact on monitoring applications since it would allow sensibly alleviating the analysis burden for human operators. To this end, the research project proposed in this thesis addresses the problem of behaviour assessment leading to the identiﬁcation of targets that exhibit features “of interest”. Firstly, this thesis has addressed the problem of distributed target assessment based on behavioural and contextual features. The assessment problem is analysed making reference to a layered structure and a possible implementation approach for the middle-layer has been proposed. An extensive analysis of the ‘feature’ concept is provided, together with considerations about the target assessment process. A case study considering a road-traﬃc monitoring application is then introduced, suggesting a possible implementation for a set of features related to this particular scenario. The distributed approach has been implemented employing a consensus protocol, which allows achieving agreement about high-level, non-measurable, characteristics of the monitored vehicles. Two diﬀerent techniques, ‘Belief’ and ‘Average’ consensus, for distributed target assessment based on features are ﬁnally presented, enabling the comparison of consensus eﬀects when implemented at diﬀerent level of the considered conceptual hierarchy. Then, the problem of identifying targets concerning features is tackled using a diﬀerent approach: a probabilistic description is adopted for the target characteristics of interest and a hypothesis testing technique is applied to the feature probability density functions. Such approach is expected to allow discerning whether a given vehicle is a target of interest or not. The assessment process introduced is also able to account for information about the context of the vehicle, i.e. the environment where it moves or is operated. In so doing the target assessment process can be eﬀectively adapted to the contour conditions. Results from simulations involving a road monitoring scenario are presented, considering both synthetic and real-world data. Lastly, the thesis addresses the problem of manoeuvre recognition and behaviour anomalies detection for generic targets through pattern matching techniques. This problem is analysed considering motor vehicles in a multi-lane road scenario. The proposed approach, however, can be easily extended to signiﬁcantly diﬀerent monitoring contexts. The overall proposed solution consists in a trajectory analysis tool, which classiﬁes the target position over time into a sequence of ‘driving modes’, and a string-matching technique. This classiﬁcation allows, as result of two diﬀerent approaches, detecting both a priori deﬁned patterns of interest and general behaviours standing out from those regularly exhibited from the monitored targets. Regarding the pattern matching process, two techniques are introduced and compared: a basic approach based on simple strings and a newly proposed method based on ‘regular expressions’. About reference patterns, a technique for the automatic deﬁnition of a dictionary of regular expressions matching the commonly observed target manoeuvres is presented. Its assessment results are then compared to those of a classic multi-layered neural network. In conclusion, this thesis proposes some novel approaches, both local and distributed, for the identiﬁcation of the ‘targets of interest’ within a multi-target scenario. Such assessment is solely based on the behaviour actually exhibited by a target and does not involve any speciﬁc knowledge about the targets (analytic dynamic models, previous data, signatures of any type, etc.), being thus easily applicable to diﬀerent scenarios and target types. For all the novel approaches described in the thesis, numerical results from simulations are reported: these results, in all the cases, conﬁrm the eﬀectiveness of the proposed techniques, even if they appear to be open to interpretation because of the inherent subjectivity of the assessment process.
Open Access
Computational guidance using sparse Gauss-Hermite quadrature differential dynamic programming
(Elsevier, 2019-11-25) He, Shaoming; Shin, Hyo-Sang; Tsourdos, Antonios
This paper proposes a new computational guidance algorithm using differential dynamic programming and sparse Gauss-Hermite quadrature rule. By the application of sparse Gauss-Hermite quadrature rule, numerical differentiation in the calculation of Hessian matrices and gradients in differential dynamic programming is avoided. Based on the new differential dynamic programming approach developed, a three-dimensional computational algorithm is proposed to control the impact angle and impact time for an air-to-surface interceptor. Extensive numerical simulations are performed to show the effectiveness of the proposed approach.
Open Access
Cooperative control of multi-uavs under communication constraints.
(2018-10) Lee, Hae-In; Shin, Hyo-Sang; Tsourdos, Antonios
This research aims to develop an analysis and control methodology for the multiple un-manned aerial vehicles (UAVs), connected over a communication network. The wireless communication network between the UAVs is vulnerable to errors and time delays, which may lead to performance degradation or even instability. Analysis on the effects of the potential communication constraints in the multiple UAV control is a critical issue for successful operation of multiple UAVs. Therefore, this thesis proposes a systematic method by incorporating three steps: proposing the analysis method and metrics considering the wireless communication dynamics, designing the structure of the cooperative controller for UAVs, and applying the analysis method to the proposed control in representative applications. For simplicity and general insights on the effect of communication topology, a net-worked system is first analysed without considering the agent or communication dynamics. The network theory specifies important characteristics such as robustness, effectiveness, and synchronisability with respect to the network topology. This research not only reveals the trade-off relationship among the network properties, but also proposes a multi-objective optimisation (MOO) method to find the optimal network topology considering these trade-offs. Extending the analysis to the networked control system with agent and communication dynamics, the effect of the network topology with respect to system dynamics and time delays should be considered. To this end, the effect of communication dynamics is then analysed in the perspective of robustness and performance of the controller. The key philosophy behind this analysis is to approximate the networked control system as a transfer function, and to apply the concepts such as stability margin and sensitivity function in the control theory. Through the analysis, it is shown that the information sharing between the agents to determine their control input deteriorates the robustness of their stability against system uncertainties. In order to compensate the robustness and cancel out the effect of uncertainties, this thesis also develops two different adaptive control methods. The proposed adaptive control methods in this research aim to cope with unmatched uncertainty and time-varying parameter uncertainty, respectively. The effect of unmatched uncertainty is reduced on the nominal performance of the controller, using the parameter-robust linear quadratic Gaussian method and adaptive term. On the other hand, time-varying parameter uncertainty is estimated without requiring the persistent excitation using concurrent learning with the directional forgetting algorithm. The stability of the tracking and parameter estimation error is proved using Lyapunov analysis. The proposed analysis method and control design are demonstrated in two application examples of a formation control problem without any physical interconnection between the agents, and an interconnected slung-load transportation system. The performance of the proposed controllers and the effect of topology and delay on the system performance are evaluated either analytically or numerically.
Open Access
Decentralised multi-robot task allocation algorithms.
(2017) Segui-Gasco, Pau; Shin, Hyo-Sang; Tsourdos, Antonios
Multi Robot Systems (MRS) are gaining increasing popularity both in the research community and in industry. A fundamental problem that underpins the effective coordinated operation of these systems is Task Allocation. To solve this problem, the MRS should be able to find an answer quickly, reliably, and effectively to the question: "given the robots available in the system and the tasks that ought to be carried out, what is the best allocation of these tasks among us?". In this thesis we focus on solving this problem in the decentralised setting, that is, when each agent only has access to its own utility function and does not have any knowledge of the functions corresponding to other agents, i.e.the utility function is local or private. Our algorithms are based on improved versions of the measured continuous greedy algorithm for general matroid-constrained submodular maximisation. The first improvement is a new and smoother increment rule that enables us to reduce the number of steps required to solve the relaxation. The second improvement is to adapt the Decreasing-Threshold procedure for monotone submodular functions to work with non-monotone submodular functions. Then, we present the first decentralised algorithm with constant-factor approximation guarantees for general submodular task allocation. Our algorithm provides an approximation factor of 1-1/e-4ϵ (≈63%) for monotone submodular utilities, and a factor of (1/e-3ϵ) (≈37%) for non-monotone submodular functions. To illustrate the possibilities enabled by non-monotone submodular task allocation, we present a submodular task allocation model for a multi-UAV surveillance mission. Our model features the allocation of heterogeneous surveillance tasks to a heterogeneous multi-UAV team under risk of enemy detection. We develop the model and present proofs to show that it is non-monotone submodular. Then, we run numerical experiments to study the effect of different parameters of our algorithm and compare its performance against the state-of-the- art. To conclude the thesis, we take a completely different approach, the key idea is to trade constant-factor approximation guarantees in exchange for flexibility. We present a preliminary framework based on combinatorial auctions that can transfer centralised solution method to the decentralised Task Allocation domain while requiring a polynomial number of communication rounds. In other words, our framework provides a way to transfer successful methods to solve NP-Hard problems such as Metaheuristics, Mixed-Integer Programming, Constraint Programming, etc. to the decentralised setting.
Open Access
Development of model free flight control system using deep deterministic policy gradient (DDPG)
(Cranfield University, 2019-09) Budiarti, Dewi H.; Tsourdos, Antonios; Shin, Hyo-Sang
Developing a flight control system for a complete 6 degree-of-freedom for an air vehicle remains a huge task that requires time and effort to gather all the necessary data. This thesis proposes the use of reinforcement learning to develop a policy for a flight control system of an air vehicle. This method is designed to be independent of a model but it does require a set of samples for the reinforcement learning agent to learn from. A novel reinforcement learning method called Deep Deterministic Policy Gradient (DDPG) is applied to counter the problem with large and continuous space in a flight control. However, applying the DDPG for multiple action is often difficult. Too many possibilities can hinder the reinforcement learning agent from converging its learning process. This thesis proposes a learning strategy that helps shape the way the learning agent learns with multiple actions. It also shows that the final policy for flight control can be extracted and applied immediately for a flight control system.
Open Access
Distributed target tracking over a low-cost sensor network.
(2019-09) He, Shaoming; Shin, Hyo-Sang; Tsourdos, Antonios
Proliferation of low-cost, lightweight, and power efficient sensors and advances in networked systems enable the employment of multiple sensors. Distributed estimation provides a scalable and fault-robust fusion framework with a peer-to-peer communication architecture. This thesis investigates the problem of distributed target(s) tracking over a low-cost sensor network and proposes several efficient and reliable algorithms to address the aforementioned problem. The primary issue of using low-cost sensors in distributed target tracking is how to balance between communication cost and convergence performance. To overcome this difficulty, we develop a new sample greedy gossip distributed Kalman filter for distributed single-target tracking over a low-cost sensor network. The proposed algorithm leverages the information weighted fusion concept and a new sample greedy gossip process. Instead of finding the optimal path of each node in a greedy manner, the proposed approach utilises a suboptimal communication path by performing greedy selection among randomly selected active sensor nodes. Theoretical convergence analysis and uniform boundedness are also performed to support the proposed algorithm. The main feature of the new algorithm is that it provides fast convergence rate with relatively low communication overload. In distributed multi-target tracking, each sensor node runs a local multi-target tracking filter for multi-target state and identity estimation. The issue of current multi-target tracking algorithms is that they usually suffer from exponential complexity and require prior knowledge on the environment, e.g., target detection probability and clutter rate. This hinders the application of low-cost sensors in multi-target tracking as these sensors usually have limited computational capability and offline calibration is also not cost-effective. To address these sensor-related practical issues, we propose a polynomial-time joint probabilistic data association filter using stochastic Gibbs sampling technique and incorporate it with a multi-Bernoulli filter to accommodate the unknown environmental parameters. It is theoretically proved that the proposed solution provides a performance-guaranteed approximation and is demonstrated to show promising performance in a dynamic environment. We then consider the problem of accurate Gaussian mixture approximation for implementing joint probabilistic data association filter when targets are moving closely and propose a novel information-driven approach to tackle this problem. The proposed information-driven joint probabilistic data association algorithm is obtained from the minimisation of a weighted Kullback-Leibler divergence to approximate the posterior Gaussian mixture probability density function. The resulting approximation approach turns out to show similar structure as the generalised covariance intersection in sensor fusion. Theoretical analysis reveals that the proposed approach with ideal detection probability guarantees boundedness of the error covariance and yields unbiased estimation. Different from single-target tracking, local multi-target estimations contain data association uncertainty and therefore this issue requires careful adjustment in sensor fusion. Through an equivalent information form, we extend conventional fusion strategies, i.e., consensus on measurement and consensus on information, to multi-target tracking scenarios using joint probabilistic data association filter. This means that data association uncertainty and sensor fusion problems are solved simultaneously. Motivated by the complementary characteristics of these two different fusion approaches, a novel distributed multi-target tracking algorithm using a hybrid fusion strategy, e.g., a mix between consensus on measurement and consensus on information, is proposed. A distributed counting algorithm is incorporated into the tracker to provide the knowledge of the total number of sensor nodes. The new algorithm developed shows advantages in preserving boundedness of local estimates, guaranteeing global convergence to the optimal centralised version and being implemented without requiring global information, compared with other fusion approaches. To support the implementation of distributed multi-target tracking, we finally investigate the problem of practical implementation of multi-dimensional assignment. To address this problem, we propose two efficient implementation algorithms using Tabu search and Gibbs sampling. As the rst step, we formulate the problem of generating the best global hypothesis in multi-dimensional assignment as the problem of finding a maximum weighted independent set of a weighted undirected graph. Then, the meta-heuristic Tabu search with two basic movements is designed to find the global optimal solution of the problem formulated. To improve the computational efficiency, we also develop a sampling based algorithm using Gibbs sampling. The problem formulated for the Tabu search based algorithm is reformulated as a max product problem to enable implementation of Gibbs sampling. The detailed algorithm is then designed and the convergence is also theoretically analysed. The performance of the two algorithms proposed are verified through nu-merical simulations and compared with that of a mainstream Lagrangian relaxation implementation algorithm.
Open Access
Dynamic knowledge-based tracking and autonomous anomaly detection
(IEEE, 2023-11-28) Chai, Jianduo; He, Shaoming; Shin, Hyo-Sang; Tsourdos, Antonios
This paper presents a study on the problem of region surveillance in complex terrain using an unmanned aerial vehicle (UAV), and proposes a novel framework for on-road ground target tracking and detection of anomalous driving behavior with the assistance of domain-constrained information. In order to improve the accuracy of ground target tracking, terrain information is extracted and incorporated as constraints into the tracking process. To account for the dynamic changes in terrain-constrained information, a sliding window approach leveraging a dynamic programming algorithm is employed for domain-constrained knowledge inference. To improve the autonomy and intelligence of the monitoring UAV, a mechanism for recognizing suspicious driving patterns is seamlessly integrated into the target tracking process with the aid of domain knowledge. The effectiveness of proposed method is validated using extensive numerical simulations.
Open Access
Effective task allocation frameworks for large-scale multiple agent systems.
(2018-10) Jang, Inmo; Shin, Hyo-Sang; Tsourdos, Antonios
This research aims to develop innovative and transformative decision-making frameworks that enable a large-scale multi-robot system, called robotic swarm, to autonomously address multi-robot task allocation problem: given a set of complicated tasks, requiring cooperation, how to partition themselves into subgroups (or called coalitions) and assign the subgroups to each task while maximising the system performance. The frameworks should be executable based on local information in a decentralised manner, operable for a wide range of the system size (i.e., scalable), predictable in terms of collective behaviours, adaptable to dynamic environments, operable asynchronously, and preferably able to accommodate heterogeneous agents. Firstly, for homogeneous robots, this thesis proposes two frameworks based on biological inspiration and game theories, respectively. The former, called LICA-MC (Markov-Chan-based approach under Local Information Consistency Assumption), is inspired by fish in nature: despite insufficient awareness of the entire group, they are well-coordinated by sensing social distances from neighbours. Analogously, each agent in the framework relies only on local information and requires its local consistency over neighbouring agents to adaptively generate the stochastic policy. This feature offers various advantages such as less inter-agent communication, a shorter timescale for using new information, and the potential to accommodate asynchronous behaviours of agents. We prove that the agents can converge to a desired collective status without resorting to any global information, while maintaining scalability, flexibility, and long-term system efficiency. Numerical experiments show that the framework is robust in a realistic environment where information sharing over agents is partially and temporarily disconnected. Furthermore, we explicitly present the design requirements to have all these advantages, and implementation examples concerning travelling costs minimisation, over-congestion avoidance, and quorum models, respectively. The game-theoretical framework, called GRAPE (GRoup Agent Partitioning and placing Event), regards each robot as a self-interested player attempting to join the most preferred coalition according to its individual preferences regarding the size of each coalition. We prove that selfish agents who have social inhibition can always converge to a Nash stable partition (i.e., a social agreement) within polynomial time under the proposed framework. The framework is executable based on local interactions with neighbour agents under a strongly-connected communication network and even in asynchronous environments. This study analyses an outcome’s minimum-guaranteed suboptimality, and additionally shows that at least 50% is guaranteed if social utilities are non-decreasing functions with respect to the number of co-working agents. Numerical experiments confirm that the framework is scalable, fast adaptable against dynamical environments, and robust even in a realistic situation where some of the agents temporarily halt operation during a mission. The two proposed frameworks are compared in the domain of division of labour. Empirical results show that LICA-MC provides excellent scalability with respect to the number of agents, whereas GRAPE has polynomial complexity but is more efficient in terms of convergence time (especially when accommodating a moderate number of robots) and total travelling costs. It also turns out that GRAPE is sensitive to traffic congestion, meanwhile LICA-MC suffers from slower robot speed. We discuss other implicit advantages of the frameworks such as mission suitability and additionally-builtin decision-making functions. Importantly, it is found that GRAPE has the potential to accommodate heterogeneous agents to some extent, which is not the case for LICA-MC. Accordingly, this study attempts to extend GRAPE to incorporate the heterogeneity of agents. Particularly, we consider the case where each task has its minimum workload requirement to be fulfilled by multiple agents and the agents have different work capacities and costs depending on the tasks. The objective is to find an assignment that minimises the total cost of assigned agents while satisfying the requirements. GRAPE cannot be directly used because of the heterogeneity, so we adopt tabu-learning heuristics where an agent penalises its previously chosen coalition whenever it changes decision: this variant is called T-GRAPE. We prove that, by doing so, a Nash stable partition is always guaranteed to be determined in a decentralised manner. Experi-mental results present the properties of the proposed approach regarding suboptimality and algorithmic complexity. Finally, the thesis addresses a more complex decision-making problem involving team formation, team-to-task assignment, agent-to-working-position selection, fair resource allocation concerning tasks’ minimum requirements for completion, and trajectory optimisation with collision avoidance. We propose an integrated framework that decouples the original problem into three subproblems (i.e., coalition formation, position allocation, and path planning) and deals with them sequentially by three respective modules. The coalition formation module based on T-GRAPE deals with a max-min problem, balancing the work resources of agents in proportion to the task’s requirements. We show that, given reasonable assumptions, the position allocation subproblem can be solved efficiently in terms of computational complexity. For the path planning, we utilise an MPC-SCP (Model Predictive Control and Sequential Convex Programming) approach that enables the agents to produce collision-free trajectories. As a proof of concept, we implement the framework into a cooperative stand-in jamming mission scenario using multiple UAVs. Numerical experiments suggest that the framework could be computationally feasible, fault-tolerant, and near-optimal. Comparison of the proposed frameworks for multi-robot task allocation is discussed in the last chapter regarding the desired features described at first (i.e., decentralisation, scalability, predictability, flexibility, asynchronisation, heterogeneity), along with future work and possible applications in other domains.
Open Access
Efficient underwater acoustical localization method based on time difference and bearing measurements
(IEEE, 2020-12-16) Zhang, Liang; Zhang, Tao; Shin, Hyo-Sang; Xu, Xiang
This article addresses the underwater acoustical localization problem by using the time-difference-of-arrival (TDOA) and bearing-angle-of-arrival (BAOA) measurements. For the underwater acoustic equipment, such as the ultrashort baseline system (USBL), whose bearing measurements are different from the traditional angle-of-arrival (AOA) model, a closed-form solution for the hybrid TDOA/BAOA-based source localization problem is developed. However, the solution suffers from the measurement noise and cannot achieve the Cramer–Rao lower bound (CRLB) performance in the case of large measurement noise. Thus, an iterative constrained weighted least-squares method is presented to further minimize the error in the case of large noise. The CRLB for hybrid TDOA/BAOA source localization is analyzed, and the solution is proved to achieve the CRLB performance. Numerical simulations and field tests demonstrate that the proposed method outperforms the traditional methods in terms of estimation bias and accuracy. It can achieve the CRLB performance better.
Open Access
Game-theoretical approach to heterogeneous multi-robot task assignment problem with minimum workload requirements
(IEEE, 2017-11-09) Jang, Inmo; Shin, Hyo-Sang; Tsourdos, Antonios
This paper addresses a multi-robot task assignment problem with heterogeneous agents and tasks. Each task has a different type of minimum workload requirement to be accomplished by multiple agents, and the agents have different work capacities and costs depending on the tasks. The objective is to find an assignment that minimises the total cost of assigned agents while satisfying the requirements of the tasks. We formulate this problem as the minimisation version of the generalised assignment problem with minimum requirements (MinGAP-MR). We propose a distributed game-theoretical approach in which each selfish player (i.e., robot) wants to join a task-specific coalition that minimises its own cost as possible. We adopt tabu-learning heuristics where a player penalises its previously chosen coalition, and thereby a Nash-stable partition is always guaranteed to be determined. Experimental results present the properties of our proposed approach in terms of suboptimality and algorithmic complexity.
Open Access
Gaussian process adaptive incremental backstepping flight control
(AIAA, 2021-12-29) Ignatyev, Dmitry I.; Shin, Hyo-Sang; Tsourdos, Antonios
The presence of uncertainties caused by unforeseen malfunctions in the actuation system or changes in aircraft behaviour could lead to aircraft loss of control during flight. The paper proposes almost model-independent control law combining recent developments in nonlinear control theory, data-driven methods, and sensor technologies by considering Gaussian Processes Adaptive augmentation for Incremental Backstepping control (IBKS) algorithm. IBKS uses angular accelerations and current control deflections to reduce the dependency on the aircraft model. However, it requires knowledge of control effectiveness. Conducted research shows that if the input-affine property of the IBKS is violated, e.g., in severe conditions with a combination of multiple failures, the IBKS can lose stability. Meanwhile, the GP-based estimator provides fast identification and the resultant GP-adaptive IBKS algorithm demonstrates improved stability and tracking performance. The performance of the algorithm is validated using a large transport aircraft flight dynamics model.
Open Access
Integrated approaches to handle UAV actuator fault
(Cranfield University, 2015-08) Lo, Chang How; Shin, Hyo-Sang; Tsourdos, Antonios
Unmanned AerialVehicles (UAV) has historically shown to be unreliable when compared to their manned counterparts. Part of the reason is they may not be able to a ord the redundancies required to handle faults from system or cost constraints. This research explores instances when actuator fault handling may be improved with integrated approaches for small UAVs which have limited actuator redundancy. The research started with examining the possibility of handling the case where no actuator redundancy remains post fault. Two fault recovery schemes, combing control allocation and hardware means, for a Quad Rotor UAV with no redundancy upon fault event are developed to enable safe emergency landing. Inspired by the integrated approach, a proposed integrated actuator control scheme is developed, and shown to reduce the magnitude of the error dynamics when input saturation faults occur. Geometrical insights to the proposed actuator scheme are obtained. Simulations using an Aerosonde UAV model with the proposed scheme showed significant improvements to the fault tolerant stuck fault range and improved guidance tracking performance. While much research literature has previously been focused on the controller to handle actuator faults, fault tolerant guidance schemes may also be utilized to accommodate the fault. One possible advantage of using fault tolerant guidance is that it may consider the fault degradation e ects on the overall mission. A fault tolerant guidance reconfiguration method is developed for a path following mission. The method provides an additional degree of freedom in design, which allows more flexibility to the designer to meet mission requirements. This research has provided fresh insights into the handling UAV extremal actuator faults through integrated approaches. The impact of this work is to expand on the possibilities a practitioner may have for improving the fault handling capabilities of a UAV.
Open Access
An integrated decision-making framework of a heterogeneous aerial robotic swarm for cooperative tasks with minimum requirements
(SAGE, 2018-05-15) Jang, Inmo; Shin, Hyo-Sang; Tsourdos, Antonios; Jeong, Junho; Kim, Seungkeun; Suk, Jinoung
Given a cooperative mission consisting of multiple tasks spatially distributed, an aerial robotic swarm’s decision-making issues include team formation, team-to-task assignment, agent-to-work-position assignment and trajectory optimisation with collision avoidance. The problem becomes even more complicated when involving heterogeneous agents, tasks’ minimum requirements and fair allocation. This paper formulates all the combined issues as an optimisation problem and then proposes an integrated framework that addresses the problem in a decentralised fashion. We approximate and decouple the complex original problem into three subproblems (i.e. coalition formation, position allocation and path planning), which are sequentially addressed by three different proposed modules. The coalition formation module based on game theories deals with a max-min problem, the objective of which is to partition the agents into disjoint task-specific teams in a way that balances the agents’ work resources in proportion to the task’s minimum workload requirements. For agents assigned to the same task, given reasonable assumptions, the position allocation subproblem can be efficiently addressed in terms of computational complexity. For the trajectory optimisation, we utilise a Model Predictive Control and Sequential Convex Programming algorithm, which reduces the size of the problem so that the agents can generate collision-free trajectories on a real-time basis. As a proof of concept, we implement the framework into an unmanned aerial vehicle swarm’s cooperative stand-in jamming mission scenario and show its feasibility, fault tolerance and near-optimality based on numerical experiment.
Open Access
Multiple agents routing and scheduling algorithms for network-based transportation systems.
(2018-10) Bae, Sangjun; Shin, Hyo-Sang; Tsourdos, Antonios
This research attempts to develop effective and practical algorithms that enable multiple agents to address routing and scheduling problems simultaneously: given a set of initial points and final points for multiple agents in a route network, separation-compliant routes and speed profiles are to be found for every agent while maximising a performance index subject to satisfy operational constraints. The algorithms are applicable to many transportation systems that consider many operational factors such as flight planning problems in the Air Traffic Management (ATM) system, and analysing urban airspace structure for an Unmanned Aircraft System (UAS) Traffic Management (UTM) system. This thesis focuses on an investigation of a new horizontal Routing and Scheduling (R&S) algorithm for homogeneous multiple arrivals at a single airport. Importantly, this study is the first to investigate the routing problem and scheduling problem simultaneously in the ATM domain, and it is found that a time-based separation concept and a flight time weighting scheme applied in the proposed algorithm allows for horizontal separation-compliant routing and scheduling for each flight. Simulation results show that the current flight planning approach would benefit from the proposed R&S algorithm that provides detailed flight plans in a less computation time. Another part of this thesis focuses on the extension of the R&S algorithm to deal with multiple heterogeneous aircraft arriving at multiple airports, and also to cope with three-dimensional route network. With these extensions, the proposed R&S algorithm can be adopted to handle a wider range of operational conditions represented by various combinations of aircraft types in a fleet and neighbour-dependent separation requirements. Numerical simulation using a simple route network model shows that the R&S algorithm can find the near-optimal route and schedule within polynomial time. As a more realistic case study, we tested the algorithm into the London Terminal Manoeuvring Area (LTMA). The numerical experiment shows that the algorithm provides a separation-compliant route and schedule for multiple heterogeneous aircraft in the three-dimensional LTMA efficiently. By modifying the proposed algorithm, we address flight planning problems that arise in drone delivery, which is one of the most promising applications of the UTM system. As a preliminary study, we demonstrate two last-mile delivery cases (1-to-M) and one first-mile delivery case (M-to-1) within a route network over roads. The results of each case show that detailed flight plans could support analysis of the route network capacity and help to establish requirements for safe and efficient operations. On the basis of this observation, the analysis of the structured urban airspace capacity is performed for four different types of drone delivery operation (1-to-M, M-to-1, N - to-M, and M-to-N ) using the proposed algorithms, where we suggest four intuitive metrics calculated from the detailed flight plans. We apply two different sequencing algorithms (First Come First Served algorithm and Last Come First Served algorithm) - an outer loop of the R&S algorithms - for each operation type. Monte Carlo simulation results suggest to use either more efficient sequencing algorithm or both of the algorithms together in a timely manner for each operation type. From the simulation results, we could expect that the proposed algorithms provide the analysis and suggestions for designing urban airspace to support designers, regulators, and policymakers. Collectively, the algorithms proposed in this thesis may play a key role in many network-based transport planning problems regarding effective and safe operations, along with future works on extension of the algorithm to real-time planning algorithms and to other transportation systems.
Open Access
A new multiple flights routing and scheduling algorithm in terminal manoeuvring area
(IEEE, 2018-12-10) Bae, Sangjun; Shin, Hyo-Sang; Lee, Chang-Hun; Tsourdos, Antonios
We address multiple flights planning problems from its initial waypoint to its destination while satisfying the minimum separation requirement between each aircraft at all times in a Terminal Manoeuvring Area (TMA) to maintain or increase runway throughput. Due to operational constraints for safety, most of the current aircraft fly over or by waypoints, and along nominal routes in the airspace. Where the waypoints and routes in the airspace can be modelled as a weighted digraph, called airspace graph. We propose a problem that consists of determining a flight path (routing problem) and its speed profile (scheduling problem) in a given airspace graph in which a time-based weighting scheme of the airspace graph is proposed to reflect a speed-limitation-compliant schedule that satisfy the minimum separation requirement. For multiple flights cases, the flight paths and schedules are obtained by iteratively solving the problem for each flight by applying the First Come First Served (FCFS) algorithm to determine an arrival sequence. The main contributions of this paper are increasing a solution search space by solving two problems simultaneously, efficient computational time, and providing the separation-compliant flight path and speed profile within the speed limitation for each flight. We demonstrate the advantages of the proposed approach through a case study in which multiple flights arrive at a single airport, and we compare the results with Regulated Tactical Flight Model (RTFM) obtained from EUROCONTROL Demand Data Repository 2 (DDR2). Although, we consider only a single airport and make an assumption to simplify flight routes from holding stacks to a Final Approach Fix (FAF), the results show the potential usage of the proposed algorithm as a Decision Support Tool (DST) for Air Traffic Controllers (ATCOs) if the following considerations are taken into account: detailed routes-based flights after the holding stacks, multiple airports, departing aircraft, all possibe aircraft types, and uncertainties produced by external sources.
Open Access
Nonlinear acceleration controller for exo-atmospheric and endo-atmospheric interceptors with TVC
(IEEE, 2017-07-20) Lee, Chang-Hun; Jun, Byung-Eul; Shin, Hyo-Sang; Tsourdos, Antonios
In this paper, we propose a nonlinear acceleration controller that can be used for both the endo- and exo-atmospheric interceptors with thrust vector control (TVC) without changing the control configuration. The acceleration perpendicular to the velocity vector is selected as the output to be controlled. Then apply the feedback linearization and the specific form of the desired error dynamics to create the resulting controller which is given by the well-known three loop control structure with parameter-varying control gains. According to changes in altitude operating conditions, the proposed controller can adaptively allocate the aerodynamic force and the thrust to produce the required normal acceleration. Also, we can have confidence in the reliability of the proposed controller because it is given by a similar form of the well-known three loop controller. Numerical simulations are performed to show the validity of the proposed method.
Open Access
Nonlinear flight control with reduced model dependency
(2020-05) Byoung-Ju, Jeon; Shin, Hyo-Sang; Tsourdos, Antonios
This thesis aims to innovate knowledge on nonlinear flight control algorithms with reduced model dependency by resolving the research gaps for practical applications. Two control schemes with different principles on reducing model dependency are considered in this thesis; incremental control scheme and adaptive control scheme. In incremental control scheme, state derivative and control surface deflection angle measurements are additionally utilized to substitute required model information except control effectiveness. In adaptive control scheme, uncertain model parameters are estimated online via adaptation law and these estimates are utilized in control input command calculation. Discussions in this thesis are based on incremental backstepping control (IBKS) and composite adaptive backstepping control (C-ABKS) which are obtained by applying those control schemes to backstepping control (BKS). Contributions of this thesis with each algorithm are detailed as follows. This thesis provides critical understandings on IBKS in a systematic way via theoretical analysis under various defects. As a starting point, closed-loop analyses under the model uncertainties are conducted with IBKS and BKS for theoretical interpretations on reduced model dependency in IBKS. Stability and performance of the closed-loop system with IBKS are shown to be not affected by the model uncertainties, while they significantly influence the closed-loop characteristics with BKS. One interesting observation is that the uncertainty on control effectiveness information, which is still required to implement IBKS, does not have any impact on the closed-loop system with IBKS if a control input is calculated, transmitted and reflected fast enough to an actual control surface. The next two analyses are conducted to identify how the defects on the additional measurements together with the model uncertainties affect stability and performance of the closed-loop system with IBKS. First, the closed-loop characteristics with IBKS is analyzed under biases on the additional measurements and the model uncertainties. The measurement biases result in a steady state error while not affecting the closed-loop system stability with IBKS. Unlike the analysis results only with the model uncertainties, the uncertainty in control effectiveness information has an impact on the steady-state error of the closed-loop system. Second, the closed-loop system with IBKS under delays on the additional measurements and the model uncertainties is examined with the analysis framework proposed in this thesis. New analysis framework with optimization concept is proposed to systematically and efficiently test the closed-loop system stability under measurement delays. The key finding is that the delays on the additional measurements should satisfy a specific relationship for the closed-loop stability with IBKS. Besides, it is identified that this stability condition is affected by the uncertainty on control effectiveness information. A new C-ABKS is designed by resolving research gaps of the composite adaptive control for a practical application as follows. First, parameter convergence under finite excitation (FE) is guaranteed with a new paradigm for the information matrix design which is suggested by developing a modulation-based approach. It is proven that the new information matrix is positive definite for all the time from the beginning under FE, while the accumulation-based approach in previous studies requires uncertain amount of time to populate the information matrix to be full rank. The closed-loop system with the C-ABKS utilizing the new information matrix is guaranteed to be globally exponentially stable for all the time under FE. Comparing to the accumulation-based approach, the new modulation-based approach provides advantages in adaptation speed and system robustness since the information matrix is designed to have all eigenvalues with moderate level of magnitudes. Second, the adaptation speed is improved without excessive increase of the adaptation gains in the new logarithmic regression-based composite adaptive control system. The parameter convergence speed is enhanced by slowing down the adaptation speed degeneration at the later stage where the estimation error is small; a concave and monotonically increasing characteristics of a logarithmic function is utilized for the regression term design in this research. The closed-loop system with the proposed logarithmic regression-based C-ABKS is shown to be asymptotically stable under FE by applying Lyapunov theory. Within the system boundary, the new logarithmic regression-based algorithm is proven to be always faster than the well-known linear regression-based algorithm under the same adaptation gain if its design parameters satisfy the suggested condition. In order to make the linear regression-based approach to become faster than the logarithmic regression-based approach with the design parameters satisfying this condition, the adaptation gain of the linear regression term should be increased and this can result in reduced robustness. Important findings for IBKS and C-ABKS are suggested and verified with simulations throughout the thesis. A comparative study is additionally conducted to show different properties of IBKS and C-ABKS under model uncertainties and measurement delays via numerical simulations.