CERES :: Browsing by Author "Perrusquía, Adolfo"

Browsing by Author "Perrusquía, Adolfo"

Now showing 1 - 20 of 40

Open Access
A COLREGs compliance reinforcement learning approach for USV manoeuvring in track-following and collision avoidance problems
(Elsevier, 2025-01-15) Sonntag, Valentin; Perrusquía, Adolfo; Tsourdos, Antonios; Guo, Weisi
The development of new technologies for autonomous platforms has allowed their integration into sea mine countermeasures. This has allowed to remove the personnel from the potential danger by having the mine search task performed by an unmanned surface vessel (USV). Traditional intelligent systems are built by agglomerating hand-coded behaviours that determine how a good manoeuvre looks like. This induces cognitive bias into the pre-defined behaviours that can violate safety and regulatory rules imposed by the COLREGs. To alleviate this issue, this paper proposes a COLREGs compliant reinforcement learning (RL) approach that gives a solution for the autonomous navigation of USVs. A custom simulation environment is developed. The RL agents are trained to deal with path-following problem with obstacle avoidance capabilities. A custom reward function is defined to consider the turning disks for the agent's decision process. A smoothing decision feature is used to smooth the transitions between consecutive actions. The results demonstrate good convergence and high performance under different scenarios. The collision avoidance with COLREGs compliances shows the effectiveness of the proposed approach under several scenarios with static and moving obstacles.
Open Access
A novel physics-informed recurrent neural network approach for state estimation of autonomous platforms
(IEEE, 2024-06-30) Perrusquía, Adolfo; Guo, Weisi
State estimation of autonomous platforms is a crucial element in the design and test of perception algorithms. The nonlinear nature of autonomous platforms makes hard to design accurate state estimation algorithms without using linearization techniques, large amount of data or knowledge of the physical parameters of the platform. This paper reports a novel state estimation algorithm of autonomous platforms. The proposed approach is based on a physics informed recurrent neural network (PIRNN) that combines the power of recurrent nets with an estimate structure of the autonomous platform model. This estimated model regularises the weights’ manifold of the network for the accurate estimation of the states. Boundedness of the proposed PIRNN is verified using Lyapunov stability theory as long as the physics-informed signals satisfy a persistent of excitation condition. Simulations are conducted to test the PIRNN model and show its benefits and challenges.
Open Access
An advanced path planning and UAV relay system: enhancing connectivity in rural environments
(MDPI, 2024-03-06) El Debeiki, Mostafa; Al-Rubaye, Saba; Perrusquía, Adolfo; Conrad, Christopher; Flores Campos, Juan Alejandro
The use of unmanned aerial vehicles (UAVs) is increasing in transportation applications due to their high versatility and maneuverability in complex environments. Search and rescue is one of the most challenging applications of UAVs due to the non-homogeneous nature of the environmental and communication landscapes. In particular, mountainous areas pose difficulties due to the loss of connectivity caused by large valleys and the volumes of hazardous weather. In this paper, the connectivity issue in mountainous areas is addressed using a path planning algorithm for UAV relay. The approach is based on two main phases: (1) the detection of areas of interest where the connectivity signal is poor, and (2) an energy-aware and resilient path planning algorithm that maximizes the coverage links. The approach uses a viewshed analysis to identify areas of visibility between the areas of interest and the cell-towers. This allows the construction of a blockage map that prevents the UAV from passing through areas with no coverage, whilst maximizing the coverage area under energy constraints and hazardous weather. The proposed approach is validated under open-access datasets of mountainous zones, and the obtained results confirm the benefits of the proposed approach for communication networks in remote and challenging environments.
Open Access
Advancing fault diagnosis in aircraft landing gear: an innovative two-tier machine learning approach with intelligent sensor data management
(AIAA, 2024-01-04) Kadripathi, K. N.; Ignatyev, Dmitry; Tsourdos, Antonios; Perrusquía, Adolfo
Revolutionizing aircraft safety, this study unveils a pioneering two-tier machine learning model specifically designed for advanced fault diagnosis in aircraft landing gear systems. Addressing the critical gap in traditional diagnostic methods, our approach deftly navigates the challenges of sensor data anomalies, ensuring robust and accurate real-time health assessments. This innovation not only promises to enhance the reliability and safety of aviation but also sets a new benchmark in the application of intelligent machine-learning solutions in high-stakes environments. Our method is adept at identifying and compensating for data anomalies caused by faulty or uncalibrated sensors, ensuring uninterrupted health assessment. The model employs a simulation-based dataset reflecting complex hydraulic failures to train robust machine learning classifiers for fault detection. The primary tier focuses on fault classification, whereas the secondary tier corrects sensor data irregularities, leveraging redundant sensor inputs to bolster diagnostic precision. Such integration markedly improves classification accuracy, with empirical evidence showing an increase from 95.88% to 98.76% post-imputation. Our findings also underscore the importance of specific sensors—particularly temperature and pump speed—in evaluating the health of landing gear, advocating for their prioritized usage in monitoring systems. This approach promises to revolutionize maintenance protocols, reduce operational costs, and significantly enhance the safety measures within the aviation industry, promoting a more resilient and data-informed safety infrastructure.
Open Access
A closed-loop output error approach for physics-informed trajectory inference using online data
(IEEE, 2022-09-21) Perrusquía, Adolfo; Guo, Weisi
While autonomous systems can be used for a variety of beneficial applications, they can also be used for malicious intentions and it is mandatory to disrupt them before they act. So, an accurate trajectory inference algorithm is required for monitoring purposes that allows to take appropriate countermeasures. This article presents a closed-loop output error approach for trajectory inference of a class of linear systems. The approach combines the main advantages of state estimation and parameter identification algorithms in a complementary fashion using online data and an estimated model, which is constructed by the state and parameter estimates, that inform about the physics of the system to infer the followed noise-free trajectory. Exact model matching and estimation error cases are analyzed. A composite update rule based on a least-squares rule is also proposed to improve robustness and parameter and state convergence. The stability and convergence of the proposed approaches are assessed via the Lyapunov stability theory under the fulfilment of a persistent excitation condition. Simulation studies are carried out to validate the proposed approaches.
Open Access
Closed-loop output error approaches for drone’s physics informed trajectory inference
(IEEE, 2023-02-22) Perrusquía, Adolfo; Guo, Weisi
The design of adequate countermeasures against drone's threats needs accurate trajectory estimation to avoid economic damage to the aerospace industry and national infrastructure. As trajectory estimation algorithms need highly accurate physics informed models or off-line learning algorithms, radical innovation in on-line trajectory inference is required. In this paper, a novel drone's physics informed trajectory inference algorithm is proposed. The algorithm constructs a physic informed model and infers the drone's trajectories simultaneously using a closed-loop output error architecture. Two different approaches are proposed based on a physics structure and an admittance filtering model which considers: i) full states measurements and ii) partial states measurements. Stability and convergence of the proposed schemes are assessed using Lyapunov stability theory. Simulations studies are carried out to demonstrate the scope and high inference capabilities of the proposed approach.
Open Access
A complementary learning approach for expertise transference of human-optimized controllers
(Elsevier, 2021-10-21) Perrusquía, Adolfo
In this paper, a complementary learning scheme for experience transference of unknown continuous-time linear systems is proposed. The algorithm is inspired in the complementary learning properties that exhibit the hippocampus and neocortex learning systems via the striatum. The hippocampus is modelled as pattern-separated data of a human optimized controller. The neocortex is modelled as a Q-reinforcement learning algorithm which improves the hippocampus control policy. The complementary learning (striatum) is designed as an inverse reinforcement learning algorithm which relates the hippocampus and neocortex learning models to seek and transfer the weights of the hidden expert’s utility function. Convergence of the proposed approach is analysed using Lyapunov recursions. Simulations are given to verify the proposed approach.
Open Access
Constant speed control of slider-crank mechanisms: a joint-task space hybrid control approach
(IEEE, 2021-04-15) Flores-Campos, Juan Alejandro; Perrusquía, Adolfo; Hernández-Gómez, Luis Héctor; González, Noé; Armenta-Molina, Alejandra
In this paper, a constant speed control of slider-crank mechanisms for machine tools is proposed. A joint-task space hybrid controller based on a second-order sliding mode control and time-base generator was used to guarantee a constant speed trajectory tracking and a complete turn of the mechanism crank. A switching criterion was implemented in order to avoid the singularities located at the two extreme positions of the slider stroke. A trapezoidal speed profile with parabolic blends was designed directly over task space slider trajectory considering a constant cutting speed, the workpiece dimensions and the slider stroke length. Stability of the second-order sliding mode control was validated with the Lyapunov stability theory. Simulations were carried out to verify this approach.
Open Access
Control layer security: a new security paradigm for cooperative autonomous systems
(IEEE, 2024-03) Guo, Weisi; Wei, Zhuangkun; González-Villarreal, Oscar J.; Perrusquía, Adolfo; Tsourdos, Antonios
Autonomous systems often cooperate to ensure safe navigation. Embedded within the centralised or distributed coordination mechanisms are a set of observations, unobservable states, and control variables. Security of data transfer between autonomous systems is crucial for safety, and both cryptography and physical layer security methods have been used to secure communication surfaces - each with its drawbacks and dependencies. Here, we show for the first time a new wireless Control Layer Security (CLS) mechanism. CLS exploits mutual physical states between cooperative autonomous systems to generate cipher keys. These mutual states are chosen to be observable to legitimate users and not sufficient to eavesdroppers, thereby enhancing the resulting secure capacity. The CLS cipher keys can encrypt data without key exchange or a common key pool, and offers very low information leakage. As such the security of digital data channels is now dependent on physical state estimation rather than wireless channel estimation. This protects the estimation process from wireless jamming and channel entropy dependency. We review for first time what kind of signal processing techniques are used for hidden state estimation and key generation, and the performance of CLS in different case studies.
Open Access
Cost inference of discrete-time linear quadratic control policies using human-behaviour learning
(IEEE, 2022-06-30) Perrusquía, Adolfo; Guo, Weisi
In this paper, a cost inference algorithm for discrete-time systems using human-behaviour learning is pro-posed. The approach is inspired in the complementary learning that exhibits the neocortex, hippocampus, and striatum learning systems to achieve complex decision making. The main objective is to infer the hidden cost function from expert's data associated to the hippocampus (off-policy data) and transfer it to the neocortex for policy generalization (on-policy data) in different systems and environments. The neocortex is modelled by a Q-learning and a least-squares identification algorithms for on-policy learning and system identification. The cost inference is obtained using a one-step gradient descent rule and an inverse optimal control algorithm. Convergence of the cost inference algorithm is discussed using Lyapunov recursions. Simulations verify the effectiveness of the approach.
Open Access
A deep mixture of experts network for drone trajectory intent classification and prediction using non-cooperative radar data
(IEEE, 2024-01-01) Fraser, Benjamin; Perrusquía, Adolfo; Panagiotakopoulos, Dimitrios; Guo, Weisi
The intent prediction of unmanned aerial vehicles (UAVs) also known as drones is a challenging task due to the different mission profiles and tasks that the drone can perform. To alleviate this issue, this paper proposes a deep mixture of experts network to classify and predict drones trajectories measured from non-cooperative radars. Telemetry data of open-access datasets are converted to simulated radar tracks to generate a pool of heterogeneous trajectories and construct three independent datasets to train, validate, and test the proposed architecture. The network is composed of two main components: i) a deep network that predicts the class associated to the input trajectories and ii) a set of deep experts models that learns the extreme bounds of the trajectories in different future time steps. The proposed approach is tested and compared with different deep models to verify its effectiveness under different flight profiles and time-windows.
Open Access
Drone’s objective inference using policy error inverse reinforcement learning
(IEEE, 2025-01) Perrusquía, Adolfo; Guo, Weisi
Drones are set to penetrate society across transport and smart living sectors. While many are amateur drones that pose no malicious intentions, some may carry deadly capability. It is crucial to infer the drone’s objective to prevent risk and guarantee safety. In this article, a policy error inverse reinforcement learning (PEIRL) algorithm is proposed to uncover the hidden objective of drones from online data trajectories obtained from cooperative sensors. A set of error-based polynomial features are used to approximate both the value and policy functions. This set of features is consistent with current onboard storage memories in flight controllers. The real objective function is inferred using an objective constraint and an integral inverse reinforcement learning (IRL) batch least-squares (LS) rule. The convergence of the proposed method is assessed using Lyapunov recursions. Simulation studies using a quadcopter model are provided to demonstrate the benefits of the proposed approach.
Open Access
Enhancing situational awareness of helicopter pilots in unmanned aerial vehicle-congested environments using an airborne visual artificial intelligence approach
(MDPI, 2024-12-04) Mugabe, John; Wisniewski, Mariusz; Perrusquía, Adolfo; Guo, Weisi
The use of drones or Unmanned Aerial Vehicles (UAVs) and other flying vehicles has increased exponentially in the last decade. These devices pose a serious threat to helicopter pilots who constantly seek to maintain situational awareness while flying to avoid objects that might lead to a collision. In this paper, an Airborne Visual Artificial Intelligence System is proposed that seeks to improve helicopter pilots’ situational awareness (SA) under UAV-congested environments. Specifically, the system is capable of detecting UAVs, estimating their distance, predicting the probability of collision, and sending an alert to the pilot accordingly. To this end, we aim to combine the strengths of both spatial and temporal deep learning models and classic computer stereo vision to (1) estimate the depth of UAVs, (2) predict potential collisions with other UAVs in the sky, and (3) provide alerts for the pilot with regards to the drone that is likely to collide. The feasibility of integrating artificial intelligence into a comprehensive SA system is herein illustrated and can potentially contribute to the future of autonomous aircraft applications.
Open Access
Explainable data-driven Q-learning control for a class of discrete-time linear autonomous systems
(Elsevier, 2024-11-01) Perrusquía, Adolfo; Zou, Mengbang; Guo, Weisi
Explaining what a reinforcement learning (RL) control agent learns play a crucial role in the safety critical control domain. Most of the approaches in the state-of-the-art focused on imitation learning methods that uncover the hidden reward function of a given control policy. However, these approaches do not uncover what the RL agent learns effectively from the agent-environment interaction. The policy learned by the RL agent depends in how good the state transition mapping is inferred from the data. When the state transition mapping is wrongly inferred implies that the RL agent is not learning properly. This can compromise the safety of the surrounding environment and the agent itself. In this paper, we aim to uncover the elements learned by data-driven RL control agents in a special class of discrete-time linear autonomous systems. Here, the approach aims to add a new explainable dimension to data-driven control approaches to increase their trust and safe deployment. We focus on the classical data-driven Q-learning algorithm and propose an explainable Q-learning (XQL) algorithm that can be further expanded to other data-driven RL control agents. Simulation experiments are conducted to observe the effectiveness of the proposed approach under different scenarios using several discrete-time models of autonomous platforms.
Open Access
Flight plan optimisation of unmanned aerial vehicles with minimised radar observability using action shaping proximal policy optimisation
(MDPI , 2024-10-01) Ali, Ahmed Moazzam; Perrusquía, Adolfo; Guo, Weisi; Tsourdos, Antonios
The increasing use of unmanned aerial vehicles (UAVs) is overwhelming air traffic controllers for the safe management of flights. There is a growing need for sophisticated path-planning techniques that can balance mission objectives with the imperative to minimise radar exposure and reduce the cognitive burden of air traffic controllers. This paper addresses this challenge by developing an innovative path-planning methodology based on an action-shaping Proximal Policy Optimisation (PPO) algorithm to enhance UAV navigation in radar-dense environments. The key idea is to equip UAVs, including future stealth variants, with the capability to navigate safely and effectively, ensuring their operational viability in congested radar environments. An action-shaping mechanism is proposed to optimise the path of the UAV and accelerate the convergence of the overall algorithm. Simulation studies are conducted in environments with different numbers of radars and detection capabilities. The results showcase the advantages of the proposed approach and key research directions in this field.
Open Access
Hippocampus experience inference for safety critical control of unknown multi-agent linear systems
(Elsevier, 2022-12-16) Perrusquía, Adolfo; Guo, Weisi
Risk mitigation is usually addressed in simulated environments for safety critical control. The migration of the final controller requires further adjustments due to the simulation assumptions and constraints. This paper presents the design of an experience inference algorithm for safety critical control of unknown multi-agent linear systems. The approach is inspired in the close relationship between three main areas of the brain cortex that enables transfer learning and decision making: the hippocampus, the neocortex, and the striatum. The hippocampus is modelled as a stable linear model that communicates to the striatum how the real-world system is expected to behave. The hippocampus model is controlled by an adaptive dynamic programming (ADP) algorithm to achieve an optimal desired performance. The neocortex and the striatum are designed simultaneously by an actor control policy algorithm that ensures experience inference to the real-world system. Experimental and simulations studies are carried out to verify the proposed approach.
Open Access
Human-behavior learning for infinite-horizon optimal tracking problems of robot manipulators
(IEEE, 2022-02-01) Perrusquía, Adolfo; Yu, Wen
In this paper, a human-behavior learning approach for optimal tracking control of robot manipulators is proposed. The approach is a generalization of the reinforcement learning control problem which merges the capabilities of different intelligent and control techniques in order to solve the tracking task. Three cognitive models are used: robot and reference dynamics and neural networks. The convergence of the algorithm is achieved under a persistent exciting and experience replay fulfillment. The algorithm learns online the optimal decision making controller according to the proposed cognitive models. Simulations were carry out to verify the approach using a 2-DOF planar robot.
Open Access
Human-behavior learning: a new complementary learning perspective for optimal decision making controllers
(Elsevier, 2022-03-17) Perrusquía, Adolfo
This paper reviews an almost new method for the design of optimal decision making controllers named as Human-Behavior learning. This new paradigm is inspired by the complementary learning that different areas of the human brain have to improve learning and experience transference. It is shown that independent and well identified sources of knowledge can enhance learning and facilitate the design of the optimal decision making controller. This interaction is modelled as a Markov Decision Process defined by a tuple of actions, cognitions, and emotions sets. Existing methods of both control and reinforcement learning theories are reviewed and connected to complete the behavior learning picture for a class of linear systems.
Open Access
Hybrid deep neural networks for drone high level intent classification using non-cooperative radar data
(IEEE, 2023-09-22) Fraser, Benjamin; Perrusquía, Adolfo; Panagiotakopoulos, Dimitrios; Guo, Weisi
The proliferation of drones has brought many benefits in different industrial and government sectors due to their low cost and potential applications. Nevertheless, the security and air space can be compromised due to anomalous performances derived to negligence or intentional malicious activities. Thus, identify the hidden intentions of drones’ mission profiles is paramount to execute adequate countermeasures. In this paper, an hybrid deep neural network architecture is proposed to classify the high level intent of drones’ mission profiles using non-cooperative radar. Radar measurements are created synthetically using open access telemetry data of flight trajectories. The proposed architecture exploits the classification and reconstruction capabilities of deep neural models to classify the drones hidden high-level intent. Several experiments and comparisons are carried out to verify the effectiveness of the proposed approach.
Open Access
An input error method for parameter identification of a class of Euler-Lagrange systems
(IEEE, 2021-12-14) Perrusquía, Adolfo; Garrido, Ruben; Yu, Wen
In this paper, an input error identification algorithm for a class of Euler-Lagrange systems is proposed. The algorithm has a state-observer structure which uses the input error between the real system and an estimated model instead of the output error. Both systems are controlled by two Proportional-Derivative (PD) controllers with the same gain values. An excitation signal is added to the PD controllers to guarantee parameter estimates convergence. Stability of the complete identification method and parameter estimates convergence are assessed via Lyapunov stability theory. Simulation studies are carried out to verify the approach.