CERES :: Browsing by Author "Yuksek, Burak"

Browsing by Author "Yuksek, Burak"

Now showing 1 - 13 of 13

Open Access
Cooperative planning for an unmanned combat aerial vehicle fleet using reinforcement learning
(American Society of Mechanical Engineers, 2021-07-07) Yuksek, Burak; Demirezen, Mustafa Umut; Inalhan, Gokhan; Tsourdos, Antonios
In this study, reinforcement learning (RL)-based centralized path planning is performed for an unmanned combat aerial vehicle (UCAV) fleet in a human-made hostile environment. The proposed method provides a novel approach in which closing speed and approximate time-to-go terms are used in the reward function to obtain cooperative motion while ensuring no-fly-zones (NFZs) and time-of-arrival constraints. Proximal policy optimization (PPO) algorithm is used in the training phase of the RL agent. System performance is evaluated in two different cases. In case 1, the warfare environment contains only the target area, and simultaneous arrival is desired to obtain the saturated attack effect. In case 2, the warfare environment contains NFZs in addition to the target area and the standard saturated attack and collision avoidance requirements. Particle swarm optimization (PSO)-based cooperative path planning algorithm is implemented as the baseline method, and it is compared with the proposed algorithm in terms of execution time and developed performance metrics. Monte Carlo simulation studies are performed to evaluate the system performance. According to the simulation results, the proposed system is able to generate feasible flight paths in real-time while considering the physical and operational constraints such as acceleration limits, NFZ restrictions, simultaneous arrival, and collision avoidance requirements. In that respect, the approach provides a novel and computationally efficient method for solving the large-scale cooperative path planning for UCAV fleets.
Open Access
Data-driven synthetic air data estimation system development for a fighter aircraft
(AIAA, 2023-06-08) Karali, Hasan; Uzun, Mevlut; Yuksek, Burak; Inalhan, Gokhan
In this paper, we propose an AI-based methodology for estimating angle-of-attack and angle-of-sideslip without the need for traditional vanes and pitot-static systems. Our approach involves developing a custom neural-network model to represent the input-output relationship between air data and measurements from various sensors such as inertial measurement units. To generate the training data required for the neural network, we use a 6-degrees-of-freedom F-16 simulator, which is further modified to simulate more realistic flight data. The training data covers the full flight envelope, allowing the neural network to generate accurate predictions in all feasible flight conditions. Our methodology achieves high-accuracy estimations of angle-of-attack and angle-of-sideslip, with mean absolute errors of 0.534 deg and 0.247 deg, respectively, during the test phase. The results demonstrate the potential of the proposed methodology to accurately estimate important flight parameters without the need for complex and costly instrumentation systems. The proposed methodology could have significant practical applications in the aviation industry, particularly in next-generation aircraft instrumentation and control. Future research could focus on further refining the neural-network model and exploring its application in other aircraft systems to improve safety and reduce costs.
Open Access
Development of reinforcement learning based mission planning method for active off-board decoys on naval platforms
(AIAA, 2021-12-29) Bildik, Enver; Yuksek, Burak; Tsourdos, Antonios; Inalhan, Gokhan
In this paper, a reinforcement learning-based decoy deployment strategy is proposed to protect naval platforms against radar seeker-equipped anti-ship missiles. The decoy system consists of a rotary-wing unmanned aerial vehicle (UAV) and an integrated onboard jammer. This decoy concept enables agility which is quite critical for jamming operations against a high-speed anti-ship missile. There are two main purposes of the developed jamming strategy; a) flying in the field of view of the anti-ship missile to conceal the naval platform, and b) flying away from the target ship to increase the miss distance between the anti-ship missile and naval platform. Here, it is aimed to meet these requirements simultaneously. Kinematics models are used to represent missile, decoy UAV, and target motion. Jammer and seeker signal strengths are modeled and the radar-cross section of a frigate is utilized to increase the realism of the simulation environment. Deep Deterministic Policy Gradient (DDPG) algorithm is applied to train an actor-critic agent which maps the observation parameters to decoy’s lateral acceleration. A heuristic way is chosen to create an appropriate reward function to solve the decoy guidance problem. Finally, simulations studies are performed to evaluate the system performance.
Open Access
Development of UCAV fleet autonomy by reinforcement learning in a wargame simulation environment
(AIAA, 2021-01-04) Yuksek, Burak; Demirezen, Umut M.; Inalhan, Gokhan
In this study, we develop a machine learning based fleet autonomy for Unmanned Combat Aerial Vehicles (UCAVs) utilizing a synthetic simulation-based wargame environment. Aircraft survivability is modeled as Markov processes. Mission success metrics are developed to introduce collision avoidance and survival probability of the fleet. Flight path planning is performed utilizing the proximal policy optimization (PPO) based reinforcement learning method to obtain attack patterns with a multi-objective mission success criteria corresponding to the mission success metrics. Performance of the proposed system is evaluated by utilizing the Monte Carlo analysis in which a wider initial position interval is used when compared to the defined interval in the training phase. This provides a preliminary insight about the generalization ability of the RL agent
Open Access
Federated meta learning for visual navigation in GPS-denied urban airspace
(IEEE, 2023-11-10) Yuksek, Burak; Yu, Zhengxin; Suri, Neeraj; Inalhan, Gokhan
In this paper, we have proposed a novel FLVO framework which can improve pose estimation accuracy in terms of translational and rotational RMSE drift while reducing security and privacy risks. It also enables fast adaptation to new conditions thanks to the aggregation process of the local agents which operate in different environments. In addition, we have shown that it is possible to transfer an end-to-end visual odometry agent that is trained by using ground vehicle dataset (i.e. KITTI dataset) to an aerial vehicle pose estimation problem for low-altitude and low-speed operating conditions. Dataset size is an important topic that should be considered in both AI-based end-to-end visual odometry applications and federated learning approaches. Although it is demonstrated that federated learning could be applied for visual odometry applications to aggregate the agents that are trained in different environments, more data should be collected to improve the translational and rotational pose estimation performance of the aggregated agents. In our future work, we will evaluate cyber-attack detection performance of the proposed FLVO framework by utilizing multiple learning loops. In addition, dataset size will be expanded by utilizing real flight tests to increase the realm of the training data and to improve the robustness of the proposed federated learning based end-to-end visual odometry algorithm.
Open Access
High fidelity progressive reinforcement learning for agile maneuvering UAVs
(AIAA, 2020-01-05) Bekar, Can; Yuksek, Burak; Inalhan, Gokhan
In this work, we present a high fidelity model based progressive reinforcement learning method for control system design for an agile maneuvering UAV. Our work relies on a simulation-based training and testing environment for doing software-in-the-loop (SIL), hardware-in-the-loop (HIL) and integrated flight testing within photo-realistic virtual reality (VR) environment. Through progressive learning with the high fidelity agent and environment models, the guidance and control policies build agile maneuvering based on fundamental control laws. First, we provide insight on development of high fidelity mathematical models using frequency domain system identification. These models are later used to design reinforcement learning based adaptive flight control laws allowing the vehicle to be controlled over a wide range of operating conditions covering model changes on operating conditions such as payload, voltage and damage to actuators and electronic speed controllers (ESCs). We later design outer flight guidance and control laws. Our current work and progress is summarized in this work.
Open Access
Intelligent wargaming approach to increase course of action effectiveness in military operations
(AIAA, 2023-01-19) Yuksek, Burak; Guner, Guney; Karali, Hasan; Candan, Batu; Inalhan, Gokhan
In this study, an intelligent wargaming approach is proposed to evaluate the effectiveness of a military operation plan in terms of operational success and survivability of the assets. The proposed application is developed based on classical military decision making and planning (MDMP) workflow for ease of implementation into the real-world applications. Contributions of this study are threefold; a) developing an intelligent wargaming approach to accelerate the course of action (COA) analysis step in the MDMP which leads creating more candidate COAs for a military operation, b) generating effective tactics against the opposite forces to increase operational success, and c) developing an efficient, visual wargame-based MDMP framework for future systems that require a small team of operators to supervise a network of automated agents. Several example engagement scenarios are performed to evaluate the system capabilities and results are given. Moreover, fleet composition issue for automated agents is investigated and the fleet composer algorithm with hyperparameter tuning architecture is proposed.
Open Access
Real-time on-the-fly motion planning for urban air mobility via updating tree data of sampling-based algorithms using neural network inference
(MDPI, 2024-01-22) Lou, Junlin; Yuksek, Burak; Inalhan, Gokhan; Tsourdos, Antonios
In this study, we consider the problem of motion planning for urban air mobility applications to generate a minimal snap trajectory and trajectory that cost minimal time to reach a goal location in the presence of dynamic geo-fences and uncertainties in the urban airspace. We have developed two separate approaches for this problem because designing an algorithm individually for each objective yields better performance. The first approach that we propose is a decoupled method that includes designing a policy network based on a recurrent neural network for a reinforcement learning algorithm, and then combining an online trajectory generation algorithm to obtain the minimal snap trajectory for the vehicle. Additionally, in the second approach, we propose a coupled method using a generative adversarial imitation learning algorithm for training a recurrent-neural-network-based policy network and generating the time-optimized trajectory. The simulation results show that our approaches have a short computation time when compared to other algorithms with similar performance while guaranteeing sufficient exploration of the environment. In urban air mobility operations, our approaches are able to provide real-time on-the-fly motion re-planning for vehicles, and the re-planned trajectories maintain continuity for the executed trajectory. To the best of our knowledge, we propose one of the first approaches enabling one to perform an on-the-fly update of the final landing position and to optimize the path and trajectory in real-time while keeping explorations in the environment.
Open Access
Reinforcement learning based closed‐loop reference model adaptive flight control system design
(Wiley, 2020-10-07) Yuksek, Burak; Inalhan, Gokhan
In this study, we present a reinforcement learning (RL)‐based flight control system design method to improve the transient response performance of a closed‐loop reference model (CRM) adaptive control system. The methodology, known as RL‐CRM, relies on the generation of a dynamic adaption strategy by implementing RL on the variable factor in the feedback path gain matrix of the reference model. An actor‐critic RL agent is designed using the performance‐driven reward functions and tracking error observations from the environment. In the training phase, a deep deterministic policy gradient algorithm is utilized to learn the time‐varying adaptation strategy of the design parameter in the reference model feedback gain matrix. The proposed control structure provides the possibility to learn numerous adaptation strategies across a wide range of flight and vehicle conditions instead of being driven by high‐fidelity simulators or flight testing and real flight operations. The performance of the proposed system was evaluated on an identified and verified mathematical model of an agile quadrotor platform. Monte‐Carlo simulations and worst case analysis were also performed over a benchmark helicopter example model. In comparison to the classical model reference adaptive control and CRM‐adaptive control system designs, the proposed RL‐CRM adaptive flight control system design improves the transient response performance on all associated metrics and provides the capability to operate over a wide range of parametric uncertainties.
Open Access
An RRT* based method for dynamic mission balancing for urban air mobility under uncertain operational conditions
(IEEE, 2021-11-15) Lou, Junlin; Yuksek, Burak; Inalhan, Gokhan; Tsourdos, Antonios
Urban air mobility provides an enabling technology towards on-demand and flexible operations for passenger and cargo transportation in metropolitan areas. Electric vertical-takeoff and landing (eVTOL) concept is a potential candidate for urban air mobility platform because of its lower carbon emissions, lower noise generations and potentially lower operational costs. However, such a transportation model is subject to numerous complicated environmental and urban design factors including buildings, dynamic obstacles and micro-weather patterns. In addition, communication, navigation and surveillance quality-of-service and availability would be affected on the overall system performance and resilience. Some social factors such as privacy, noise and visual pollution should also be considered to provide a seamless integration of the urban air mobility applications into the daily life. This paper describes an integrated RRT* based approach for designing and executing flight trajectories for urban airspace subject to operating constraints, mission constraints, and environmental conditions. The generated path is energy-efficient and enables aerial vehicle to perform mid-flight landing for battery changing or emergency situations. Moreover, this paper proposes another approach that allows on-the-fly path re-planning under dynamic constraints such as geofences or micro-weather patterns. As such, the approach also provides a method toward contingency operations such as emergency landing on the fly.
Open Access
System identification and model-based flight control system design for an agile maneuvring quadrotor platform
(AIAA, 2020-01-05) Yuksek, Burak; Saldiran, Emre; Cetin, Aykut; Yeniceri, Ramazan; Inalhan, Gokhan
In this paper, we provide a system identification, model stitching and model-based flight control system design methodology for an agile maneuvering quadrotor micro aerial vehicle (MAV) technology demonstrator platform. The proposed MAV is designed to perform agile maneuvers in hover/low-speed and fast forward flight conditions in which significant changes in system dynamics are observed. As such, these significant changes result in considerable loss of performance and precision using classical hover or forward flight model based controller designs. To capture the changing dynamics, we consider an approach which is adapted from the full-scale manned aircraft and rotorcraft domain. Specifically, linear mathematical models of the MAV in hover and forward flight are obtained by using the frequency-domain system identification method and they are validated in time-domain. These point models are stitched with the trim data and quasi-nonlinear mathematical model is generated for simulation purposes. Identified linear models are used in a multi objective optimization based flight control system design approach in which several handling quality specifications are used to optimize the controller parameters. Lateral reposition and longitudinal depart/abort mission task elements from ADS-33E-PRF are scaled-down by using kinematic scaling to evaluate the proposed flight control systems. Position hold, trajectory tracking and aggressiveness analysis are performed, Monte-Carlo simulations and actual flight test results are compared. The results show that the proposed methodology provides high precision and predictable maneuvering control capability over an extensive speed envelope in comparison to classical control techniques. Our current work focuses on i) extension of the flight envelope of the mathematical model and ii) improvement of agile maneuvering capability of the MAV.
Open Access
Towards safe deep reinforcement learning for autonomous airborne collision avoidance systems
(AIAA, 2021-12-29) Panoutsakopoulos, Christos; Yuksek, Burak; Inalhan, Gokhan; Tsourdos, Antonios
In this paper we consider the application of Safe Deep Reinforcement Learning in the context of a trustworthy autonomous Airborne Collision Avoidance System. A simple 2D airspace model is defined, in which a hypothetical air vehicle attempts to fly to a given waypoint while autonomously avoiding Near Mid-Air collisions (NMACs) with non-cooperative traffic. We use Proximal Policy Optimisation for our learning agent, and we propose a reward engineering approach based on a combination of sparse terminal rewards at natural termination points and dense step rewards providing the agent with continuous feedback on its actions, based on relative geometry and motion attributes of its trajectory with respect to the traffic and the target waypoint. The performance of our trained agent is evaluated through Monte-Carlo simulations, and it is demonstrated that it achieves to master the collision avoidance task with respect to safety for a reasonable trade-off in mission performance.
Open Access
Transition flight control system design for fixed-wing VTOL UAV: a reinforcement learning approach
(AIAA, 2021-12-29) Yuksek, Burak; Inalhan, Gokhan
Tilt-rotor vertical takeoff and landing aerial vehicles have been gaining popularity in urban air mobility applications because of their ability in performing both hover and forward flight regimes. This hybrid concept leads energy efficiency which is quite important to obtain a profitable and sustainable operation. However, inherent dynamical nonlinearities of the aerial platform requires adaptation capability of the control systems. In addition, transition flight phase should be planned carefully not only for a profitable operation but also for a safe transition between flight regimes in the urban airspace. In this paper, transition flight phase of a tilt-rotor vertical-takeoff-and-landing unmanned aerial vehicle (UAV) is studied. Low-level flight control systems are designed based on adaptive dynamic inversion methodology to compensate aerodynamic effects during the transition phase. Reinforcement learning method is utilized to provide safety and energy efficiency during the transition flight phase. An actor-critic agent is utilized and trained by using deep deterministic policy gradient algorithm to augment the collective channel of the UAV. This augmentation on the collective input is used to adjust flight path angle of the UAV which results in adjusting the angle of attack when pitch angle is zero. By using this relationship, it is proposed to generate aerodynamic lift force and perform transition flight with minimum altitude change and energy usage. Simulation results show that the agent reduces the collective signal level as the aerodynamic lift force is created in the descent flight phase. This affects overall system efficiency, reduces operational costs and makes the enterprise more profitable.

Browsing by Author "Yuksek, Burak"

Results Per Page

Sort Options