Localisation and Navigation Framework for Autonomous Railway Robotic Inspection and Repair System

In the path towards the intelligent industrial 4.0, the railway industry is keen to develop intelligent asset management strategies for digitalization and smart management for rail infrastructure. It aims to both reduce the cost and exposure of human-labor, associated with track maintenance risk, as well as increase the autonomy and accuracy for the railway inspection and repair job. A Robotic Inspection and Repair System (RIRS) is proposed to undertake the automated railway maintenance consisting of the autonomous off-track travel between base workshop and track, road-rail conversion, autonomous on-track inspection, and repair as well as remote communicating to railway signaling system and infrastructure system. This paper presents a localization and navigation framework for this new autonomous system; applied to the mentioned railway maintenance job. This system comprises a commercial Unmanned Ground Vehicle (UGV, named Warthog) with a robotic manipulator (UR10e), and multiple onboard sensors including Lidar, camera, RTK GNSS, IMU, wheel odometry, and multiple types of cameras. An adaptive trolley is also designed for the purpose of road-rail conversion. This research also focuses on how to increase accuracy for the support of track defect detection and localization.


Introduction
Nowadays, one of the fastest expanding fields of scientific research is industrial mobile robotics [1]. These new smart mobile machines can replace humans in hazardous and out of hours tasks, including railway infrastructure maintenance. Railways across the world are getting busier with trains travelling at higher speeds and carrying more passengers and heavier axle loads than ever before [2]. Considering all these factors, they have put considerable pressure on the existing infrastructure, leading to increased demands in inspection and repair of rail assets. In the British railways, track inspections are primarily conducted by specialist trains called NMT (NEW Measurement Train), supported by localised inspections and maintenance tasks are typically scheduled at night to avoid service disruptions [3]. There is potential for the benefits brought about by mobile manipulators for both railway track inspection and repair. In this regard, there is a growing interest and demand to apply robotics and autonomous systems in the railway maintenance sector, which is largely due to the increased competition, rapid expansion, and ever-increasing expenses. Autonomous vehicles (AV) have provided lots of benefits to the railway network such as decreasing the track downtime and ensuring proper inspection and repair work. In [4], Vithanage et. al. completely investigates the importance and application of Robotics and Autonomous Systems (RAS) in railway maintenance sector. They analysed more than 70 research publication and found that the majority of RAS developed, are for rolling-stock maintenance, followed by railway track maintenance.
In terms of the locomotion of the mobile robot, they can be classified into different categories including stationary (arm/manipulator), land-based, water-based [5], and air-based [6]. Each of these robots are designed for a specific situation. For instance, starship robot [7] is a six-wheeled ground robot that can navigate streets and sidewalks autonomously. It is designed for offering on-demand package delivery for consumers and businesses. Or in another example, a mobile manipulator [8] is only designed for moving on track, and they are not able to do off-track navigation. For the design of this robot, they have applied a motorized inspection kart which is composed of trolley, robotic arm, laser scanner and ACFM. According to the conducted studies, an autonomous mobile manipulator has not designed yet which be able to do autonomous offtrack operations as well as on-track functions including autonomous inspection and repair.
The aim of this paper is to introduce a demonstration of Robotic Inspection and Repair System (RIRS) which is a specifically designed vehicle capable of travelling on the rail network without local operator intervention. The elements that make up the RIRS, are to be modular in design to enable future deployment in alternative maintenance and inspection operations. Based on its designed structure, it will be able to undertake offtrack operations including unmanned travel from base to track, road-rail conversion vehicle (RRV) and also on-track operations including navigation and conducting safe defect inspection and repair. For making this vehicle into an autonomous system, the first step is about designing a framework for localisation and navigation for both on-track and off-track scenario. Properly localisation of the vehicle in a global scale especially on the railway track will lead to specifying the location of the defect with high accuracy. This is the main aim and motivation of this paper, which makes distinction between this system and the available systems. More detail about the design and structure of RIRS can be found in [9]. This paper is organized as follows: section 2 describes the motivation of the proposed project. Section 3 explains the hardware and software structure of this system. In the next section, the main items for making this robot into an autonomous robot and the related challenges are explained. Finally, the conclusion of the work is mentioned in section 5.

Motivation
The main aim of network rail is to provide a safe and reliable network, by identifying the probable defects on the railway track which threaten the safety of train and passengers. But the inspection and maintenance of tracks is unlikely to be met unless by a sufficient increase in personnel to undertake these activities. So, IN2SMART2, a research project within the European railway innovation program Shift2Rail, is contributing to integrate intelligent and innovation strategy to the rail industry in order to reduce the exposure of personnel to trackside work and the associated risks. RIRS (Robotic inspection and Repair System) is proposed as a solution for the inspection and repair on the railway track.

Hardware & Software Structure of RIRS
The hardware demonstrator development employs a remote-control unmanned ground Vehicle (UGV) from Clearpath named Warthog, which can move in all-terrain. The payload mounting plates and accessible power and communication ports allow Warthog to be easily customized with different types of sensors. It can run with the max speed of 18km/hr. On the Warthog payload platform, several sub-systems are mounted including RTK GNSS, Inertial Measurement Unit (IMU), velodyne 3D LiDAR, wheel odometer, Axis network camera and a stereo camera that later will be added. In addition, a monocular camera is fixed at the end of the arm for imaging and simulating the inspection and repair work. [10] is utilized for estimating the absolute location of the vehicle. It provides two solutions, namely, float and fix. In this project, RTK-GPS with fix solution is applied, as it provides better accuracy within cm level of accuracy. LiDAR [11], [12] stands for Light Detection and Ranging. It can detect obstacles up to 200 meters and accurately estimate their positions. This sensor is mostly considered as the most accurate sensor for position estimation. It is employed for building point cloud modelling in both outdoor and indoor environments with the aim of determining the available direction, object detection, and collision avoidance. Velodyne VLP16, a powerful 16-beam 3D lidar is applied for this robot. It publishes data with a rate of 10 Hz. Camera [10] is considered for providing a highresolution image with the purpose of features and landmarks detection in the environment. A monocular wrist camera is also attached to the UR10e arm, is manufactured by Robotiq. This sensor is used for the defect detection on the railway track. IMU is another device that directly measures the three linear acceleration components as well as the three rotational rate components (6-DOF) of the vehicle. It is used to ensure the stability of the robot. Another sensor, which is applied in the robot is the wheel odometer. It is used to estimate wheeled vehicle position by counting the number of revolutions of the wheels that are in contact with the ground. It applies a relative positioning technique [13], [14]. IMU and wheel odometer are fused to ensure accurate localisation is gained during short periods of GPS obstruction. It is worth mentioning 3 that data fusion approaches offer a more accurate and robust solution for the localization issues [15].

RTK-GPS
Besides these on-board sensors, warthog is equipped with a manipulator from Universal Robotics which can move in 6 various forms (6 DOF) and has an arm head camera with two fingers end-effector. An on-board integration processor with multiple plugin interfaces is also installed onboard. The proposed software architecture for application programming interface is based on the Robotic Operating System (ROS) [16]. Because it provides the required tools to easily access sensors data, process that data, and generate an appropriate response for the motors and other actuators of the robot.

Discussion
As mentioned before, this robot is designed for both off-track and on-track navigation. To accomplish an autonomous system, four major steps in the operation of the vehicle should be considered: perception, localisation and mapping, path planning, and the control part. In perception, the vehicle applies several onboard sensors to detect, understand, and interpret the surrounding environment, including static and dynamic objects. Localisation and mapping tasks attempt to locate the vehicle globally with respect to world coordinates [17]. Subsequently, path planning exploits the output of the previous two tasks in order to adopt the optimal and safest feasible route for the RIRS to reach its destination while considering all other possible obstacles on the way [18]. Lastly, based on the selected path, the control element outputs the necessary values of acceleration, torque, braking and steering angle for the vehicle to follow the selected path [19]. ROS is used in order to make these elements work together and build a real autonomous system. Figure 2 depicts the related software structure.
The localisation system of RIRS is responsible for operating in three main job phases: off-track unfixed path navigation, on-track fixed path navigation, and tunnel local navigation. Considering from the operation scenario, the first two navigation conditions are about global localisation with absolute position information input. The tunnel condition is local navigation without absolute position information which is similar to the indoor navigation.
As this vehicle is designed to navigate off-track and also on the railway track, it will face a number of key challenges which are necessitated to be investigated. The first one is the unavailability of GPS in some parts of the environments such as tunnels, indoor places, dense vegetation and so on. In this situation, navigation will be based on SLAM (simultaneous localisation and mapping) which provides map of the environment as well as location of the vehicle in the related map.
Perception or understanding of the surrounding environment is another challenge as the robot should be able to respond to obstacles, people, and network assets. Perception part is responsible for analyzing the raw sensor data and extracting meaningful information about the scene. Obstacle detection, and scene understanding are the two main tasks of this module. Point cloud data from Lidar will be developed for interacting with the environment and obstacle avoidance. The images captured by the camera is to be analysed for recognizing the environment features and landmarks to help the location mapping.
Another key challenge is about autonomous navigation and path planning. For the off-track navigation, it is important to find the optimized route among the various routing options, to avoid obstacles and have a safe navigation. On the other side, for the on-track navigation, following the railway track has high importance.
The last and the important challenge is specifying high-accurate track defect positioning on the railway track by integrating multi-scale localization information coming from the global robot-base geolocation and relative positioning of track defect using NDT sensing (Non-destructive testing).

Off-track Phase
For the off-track operation, the vehicle should be able to move from its workshop to the road-rail conversion location under world mode autonomous navigation. Road-rail conversion trolley is deployed for carrying the mobile manipulator on the railway track. Autonomous navigation will be based on the fusion of GPS [20] and SLAM. So for merging GPS data with the navigation stack, Robot-localization package introduced by Tom moore, will be used [21]. This package works based on the Extended Kalman Filter (EKF), that from one side merge the continuous data including wheel odometer and IMU to produce locally accurate state estimation, and from the other side fuse continuous data with noncontinuous data (GPS data) to provide an accurate and complete global state estimation. SLAM will 4 provide map and do obstacles detection. For the time that the vehicle enters GPS-denied environment, the robot will rely on the SLAM technique. So, for this reason, SLAM will be fed into EKF in a feedback loop. Combining GPS and SLAM has a number of potential benefits including increasing robustness to cope with complex and unstructured environment, increasing scalability, and improving the accuracy of localization [22].
The SLAM technique for this work, will be a graph-based SLAM technique [23]. According to the studies, graph-based SLAM methods excel on filter-based method as they provide better accuracy and consistency in their estimating techniques and also, they are able to consume and process large area of mapping [24].
Safe path planning is another major part of this work, that when used in conjunction with object detection and obstacle avoidance techniques, gives a more robust result. The path planning can be divided into two categories: the global path planning where all the information of the environment is known to the robot before starting, and the local path planning where almost all the information of the environment is unknown to the robot before starting. For the object detection part, Lidar is the best candidate as it is the most accurate sensor for estimating location of the obstacles. The framework for the off-track navigation is shown in figure 3.

On-Track Phase
For the on-track operation, the vehicle will move from its road-rail conversion location to the specified location under track mode autonomous navigation. Localisation and Navigation on the railway track is different from off-track navigation in several ways. Here the robot should only follow the railway track. Furthermore, if the robot detects an object on the track, it should stop and send an alert to the third-party human.
As it is mentioned, the main purpose of movement of this robot on the railway track is for fault detection and repair tasks. So, for achieving high accuracy on-track localisation, sufficient distance or size reference input is essential for position optimisation and error reduction. In the track environment, the track-based or trackside objects are the only reference sources. For this case, the fusion of on-board sensor sources including Lidar and stereo camera can provide a type of reference. Because the point cloud provided by Lidar is not always dense enough for high-accuracy localisation in a fast-driving scenario, so it will be fused with the vision camera that provides high resolution image which is composed of adequate information.
In addition to these sensing sources, the object reference selected for localisation is also important. Poor selection would introduce error, uncertainty for localisation accuracy and decision robustness. The reference objects should have following features, which are suitable for accuracy improvement: • Be easily captured.
• Having consistent Interval distance • Fixed size Therefore, for the high-accuracy on-track localisation and navigation, the trackside objects like signal post or electrical boxes can be used as referencing objects. These objects which can be captured by both vision camera and LiDAR, are relatively low frequency objects but with consistent interval distance. To employ high frequency occurring objects, the track-base objects can be considered. The fishplate for track section connection is a middle frequency object with relative consistent interval distance and size. Besides, the balise or sleepers are high frequency occurring objects, but the size or distance between them are not always consistent. Now for the detection and tracking of these objects, feature detection and matching technique will be applied. In this method, by extracting sufficient and meaningful features from consecutive frames and applying feature matching method, track-base object in the consecutive frames can be tracked. The following figure depicts the objects which can be considered as a reference for the improvement of the localisation accuracy. In the following figure a simple framework for on-track navigation is proposed. For this case, the output of fusion of lidar and camera from one side will be used for capturing trackside object references which will be fed into the SLAM, and from the other side, it will be used for object detection on the railway track.
In terms of safety of job execution, the collision avoidance of the RIRS on the railway track should be taken into consideration as well. So, 3D Lidar sensor will detect the objects on the railway track. Only there is one deficiency with Lidar point of view, that it cannot see the very small objects which are situated in the ground blind area. To solve this issue, a depth camera which generate image-based point cloud, which is attached on the lower body of warthog, will monitor the blind ground area to achieve the full area collision avoidance. In figure 6, collision avoidance by the function of Lidar and stereo camera is shown.

Conclusion
The main aim of RIRS is to use an autonomous vehicle to undertake safe inspection and repair on the railway track. So, for this purpose, a unique RIRS system is proposed which is composed of unmanned ground vehicle with an arm manipulator. This system is considered for off-track operations that by using RRV technique, the vehicle will be situated on a trolley, and ready for on-track inspection and repair. So, for achieving an autonomous system, localisation and navigation are the most important parts, that here the basic and main ideas for implementing them are mentioned. The framework for the improvement of the localisation in severe and variant on-track environment, as well as a strategy for selecting localisation reference is proposed. By the completion of this project, this RIRS system aims to be the first autonomous prototype for the robotic rail maintenance demonstrator.