Self localization and mapping using optical and thermal imagery

Given a mobile robot starting from an unknown position in an unknown environment, with the task of explores the surroundings, it has to be able to build an environmental map and localize itself inside that map. Achieving a solution of this problem allows the exploration of area that can be dangerous or inaccessible for humans. In our implementation we decide to use two primary sensors for the environment exploration: an optical and a thermal camera. Prior work on the combined use of optical and thermal sensors for the Simultaneous Localization And Mapping (SLAM) problem is limited. The innovative aspect of this work is based on this combined use of a secondary thermal camera as an additional visual sensor for navigation under varying environmental conditions. A secondary innovative aspect is that we focus our attention on both cameras, using them as two separate and independent sensors and combine the information in the final stage of environmental mapping. During the mobile robot navigation the two cameras capture images on the environment and SURF feature points are extracted and matched between successive scenes. Using a prior work on bearing-only SLAM approach as a reference, a feature initialization method is implemented and allows each new good candidate feature (optical or thermal) to be initialized with a sum of Gaussians that represents a set of possible spatial positions of the detected feature. Using successive observations, is possible to estimate the environment coordinates of the feature and adding it to the Extended Kalman Filter (EKF) dynamic state vector. The EKF state vector contains the information about the position of the 6 degree of freedom mobile robot and the environmental landmark coordinates. The update of this information is managed by the EKF algorithm, a statistical method that allows a prediction of the state vector and it updates based on sensor information available. The final methodology is tested in indoor and outdoor environments with several different light conditions and robot trajectories producing results that are robust in terms of noise in the images and in other sensor data (i.e. encoders and GPS). The use of the thermal camera improves the number of landmarks detected during the navigation adding useful information about the explored area.