3D reconstruction of rail tracks based on fusion of RGB and infrared sensors

Rail tracks, an essential part of the rail system, have remarkably demanded thorough inspections amid rising passenger volumes and high-speed rail development. Non-destructive testing (NDT), without disrupting train operations, aims to mitigate risks by employing safe physical properties like sound, electromagnetic, and light. However, each NDT technique is sensitive to specific damage types, offering limited diagnostic perspectives and placing considerable requirements on operators, resulting in a high cognitive load. To improve the above situation, this study proposes an innovative approach for rail inspection by developing a 3D RGB-T model that combines Visual Testing (VT) and Thermal Inspection (TI) through image registration, 3D reconstruction, sensor fusion, and non-destructive testing (NDT). Their fusion facilities a complementary assessment of rail tracks by capturing both surface texture and thermal radiation to identify damages effectively. The introduction of a novel RGB and IR registration method enables the spatial alignment of images from both, reconstructing the 3D RGB-T model. This model broadens the detection scope beyond the limitations of singular NDT methods, utilizing complementary data to locate and assess the damage extent effectively and accurately. This integrated approach reduces training requirements, minimizes human errors, and provides a clear and interpretable visualization of track conditions.