CNN-fusion architecture with visual and thermographic images for object detection

Date published

2023-06-12

Free to read from

Supervisor/s

Journal Title

Journal ISSN

Volume Title

Publisher

SPIE

Department

Type

Conference paper

ISSN

0277-786X

Format

Citation

Adiuku N, Avdelidis NP, Tang G, et al., (2023) CNN-fusion architecture with visual and thermographic images for object detection. In: SPIE Defense + Commercial Sensing, 30 April - 4 May 2023, Orlando, Florida, Volume 12536, Thermosense: Thermal Infrared Applications XLV

Abstract

Mobile robots performing aircraft visual inspection play a vital role in the future automated aircraft maintenance, repair and overhaul (MRO) operations. Autonomous navigation requires understanding the surroundings to automate and enhance the visual inspection process. The current state of neural network (NN) based obstacle detection and collision avoidance techniques are suitable for well-structured objects. However, their ability to distinguish between solid obstacles and low-density moving objects is limited, and their performance degrades in low-light scenarios. Thermal images can be used to complement the low-light visual image limitations in many applications, including inspections. This work proposes a Convolutional Neural Network (CNN) fusion architecture that enables the adaptive fusion of visual and thermographic images. The aim is to enhance autonomous robotic systems’ perception and collision avoidance in dynamic environments. The model has been tested with RGB and thermographic images acquired in Cranfield’s University hangar, which hosts a Boeing 737-400 and TUI hangar. The experimental results prove that the fusion-based CNN framework increases object detection accuracy compared to conventional models.

Description

Software Description

Software Language

Github

Keywords

DOI

Rights

Attribution-NonCommercial 4.0 International

Relationships

Relationships

Supplements

Funder/s