Browsing by Author "Wisniewski, Mariusz"
Now showing 1 - 5 of 5
Results Per Page
Sort Options
Item Open Access Drone model classification using convolutional neural network trained on synthetic data(MDPI, 2022-08-12) Wisniewski, Mariusz; Rana, Zeeshan A.; Petrunin, IvanWe present a convolutional neural network (CNN) that identifies drone models in real-life videos. The neural network is trained on synthetic images and tested on a real-life dataset of drone videos. To create the training and validation datasets, we show a method of generating synthetic drone images. Domain randomization is used to vary the simulation parameters such as model textures, background images, and orientation. Three common drone models are classified: DJI Phantom, DJI Mavic, and DJI Inspire. To test the performance of the neural network model, Anti-UAV, a real-life dataset of flying drones is used. The proposed method reduces the time-cost associated with manually labelling drones, and we prove that it is transferable to real-life videos. The CNN achieves an overall accuracy of 92.4%, a precision of 88.8%, a recall of 88.6%, and an f1 score of 88.7% when tested on the real-life dataset.Item Open Access Drone model identification by convolutional neural network from video stream(IEEE, 2021-11-15) Wisniewski, Mariusz; Rana, Zeeshan A.; Petrunin, IvanWe present a convolutional neural network model that correctly identifies drone models in real-life video streams of flying drones. To achieve this, we show a method of generating synthetic drone images. To create a diverse dataset, the simulation parameters (such as drone textures, lighting, and orientation) are randomized. This synthetic dataset is used to train a convolutional neural network to identify the drone model: DJI Phantom, DJI Mavic, or DJI Inspire. The model is then tested on a real-life Anti-UAV dataset of flying drones. The benchmark results show that the DenseNet201 architecture performed the best. Adding Gaussian noise to the training dataset and performing full training (as opposed to freezing layers) shows the best results. The model shows an average accuracy of 92.4%, and an average precision of 88.6% on the test dataset.Item Open Access Reinforcement learning for pan-tilt-zoom camera control, with focus on drone tracking(AIAA, 2023-01-19) Wisniewski, Mariusz; Rana, Zeeshan A.; Petrunin, IvanReliable detection and tracking of objects using pan-tilt-zoom (PTZ) cameras is an unsolved problem. We attempt to answer whether the use of reinforcement learning (RL) is an appropriate tool for solving it. We present an environment for training RL agents to track a drone using a (PTZ) camera. We also present an agent trained using this environment, which learns to correctly pan, tilt, and zoom the camera to follow a randomly moving drone, using continuous actions. The input into the agent is the RGB image observed by the camera. The agent is rewarded for correctly tracking the drone, and penalized if it loses it from its viewport. We use the recurrent proximal policy optimization (PPO) algorithm with a long short-term memory (LSTM) layer. We find that the agent reliably learns ways of tracking the drone after around 1.4 million steps of training.Item Open Access Rock segmentation in the navigation vision of the planetary rovers(MDPI, 2021-11-24) Kuang, Boyu; Wisniewski, Mariusz; Rana, Zeeshan A.; Zhao, YifanVisual navigation is an essential part of planetary rover autonomy. Rock segmentation emerged as an important interdisciplinary topic among image processing, robotics, and mathematical modeling. Rock segmentation is a challenging topic for rover autonomy because of the high computational consumption, real-time requirement, and annotation difficulty. This research proposes a rock segmentation framework and a rock segmentation network (NI-U-Net++) to aid with the visual navigation of rovers. The framework consists of two stages: the pre-training process and the transfer-training process. The pre-training process applies the synthetic algorithm to generate the synthetic images; then, it uses the generated images to pre-train NI-U-Net++. The synthetic algorithm increases the size of the image dataset and provides pixel-level masks—both of which are challenges with machine learning tasks. The pre-training process accomplishes the state-of-the-art compared with the related studies, which achieved an accuracy, intersection over union (IoU), Dice score, and root mean squared error (RMSE) of 99.41%, 0.8991, 0.9459, and 0.0775, respectively. The transfer-training process fine-tunes the pre-trained NI-U-Net++ using the real-life images, which achieved an accuracy, IoU, Dice score, and RMSE of 99.58%, 0.7476, 0.8556, and 0.0557, respectively. Finally, the transfer-trained NI-U-Net++ is integrated into a planetary rover navigation vision and achieves a real-time performance of 32.57 frames per second (or the inference time is 0.0307 s per frame). The framework only manually annotates about 8% (183 images) of the 2250 images in the navigation vision, which is a labor-saving solution for rock segmentation tasks. The proposed rock segmentation framework and NI-U-Net++ improve the performance of the state-of-the-art models. The synthetic algorithm improves the process of creating valid data for the challenge of rock segmentation. All source codes, datasets, and trained models of this research are openly available in Cranfield Online Research Data (CORD).Item Open Access Towards fully autonomous drone tracking by a reinforcement learning agent controlling a pan–tilt–zoom camera(MDPI, 2024-05-30) Wisniewski, Mariusz; Rana, Zeeshan A.; Petrunin, Ivan; Holt, Alan; Harman, StephenPan–tilt–zoom cameras are commonly used for surveillance applications. Their automation could reduce the workload of human operators and increase the safety of airports by tracking anomalous objects such as drones. Reinforcement learning is an artificial intelligence method that outperforms humans on certain specific tasks. However, there exists a lack of data and benchmarks for pan–tilt–zoom control mechanisms in tracking airborne objects. Here, we show a simulated environment that contains a pan–tilt–zoom camera being used to train and evaluate a reinforcement learning agent. We found that the agent can learn to track the drone in our basic tracking scenario, outperforming a solved scenario benchmark value. The agent is also tested on more complex scenarios, where the drone is occluded behind obstacles. While the agent does not quantitatively outperform the optimal human model, it shows qualitative signs of learning to solve the complex, occluded non-linear trajectory scenario. Given further training, investigation, and different algorithms, we believe a reinforcement learning agent could be used to solve such scenarios consistently. Our results demonstrate how complex drone surveillance tracking scenarios may be solved and fully autonomized by reinforcement learning agents. We hope our environment becomes a starting point for more sophisticated autonomy in control of pan–tilt–zoom cameras tracking of drones and surveilling airspace for anomalous objects. For example, distributed, multi-agent systems of pan–tilt–zoom cameras combined with other sensors could lead towards fully autonomous surveillance, challenging experienced human operators.