Eyes-out airborne object detector for pilots situational awareness

With the exponential development of new flying objects, pilots need to pay even more attention to evaluate their environment, make decisions, and fly safely. Such situation awareness (SA) has multiple codified rules to guarantee the safety of pilots. This paper analyses the feasibility of a portable perception augmentation module (PAM) to help pilots improve their situational awareness based on two key actions on long-distance airborne objects, namely, object detection and distance and trajectory estimation. The developed object detection pipeline based on the state-of-the-art (SOTA) YOLOv8 architecture achieves high accuracy with a mAP50 of 0.835 for objects up to 3000 meters. The inference of the system is 1 second for a 360° scan of the aeroplane surroundings thanks to 4 wide FOV high-resolution cameras. The data used for the model is generated by Airsim in a completely automatized process. The potential implementation of stereo vision and the influence to the PAM are also evaluated. All of these tests are also performed on additional real-life data to evaluate generalization performances, which also show satisfactory results. Efforts in the development of the PAM were made to find the best balance between various constraints such as weight, energy consumption, and accuracy. Characteristic analysis of the PAM such as weight, energy consumption, and accuracy are proposed to seek the optimal balance between various real-world constraints. Real hardware considerations are made to estimate the hardware cost of the PAM based on the simulated results in this study. With further improvement in the trajectory estimation and model generalization, a prototype could be made, deployed, and sold to recreational pilots for safer flights. The code and data are available on: https://github.com/Alcharyx/IRP-Eye-out/