Motion prediction and object detection for image-based visual servoing systems using deep learning

This study primarily investigates advanced object detection and time series prediction methods in image-based visual servoing systems, aiming to capture targets better and predict the motion trajectory of robotic arms in advance, thereby enhancing the system’s performance and reliability. The research first implements object detection on the VOC2007 dataset using the Detection Transformer (DETR) and achieves ideal detection scores. The particle swarm optimization algorithm and 3-5-3 polynomial interpolation methods were utilized for trajectory planning, creating a unique dataset through simulation. This dataset contains randomly generated trajectories within the workspace, fully simulating actual working conditions. Significantly, the Bidirectional Long Short-Term Memory (BILSTM) model was improved by substituting its traditional Multilayer Perceptron (MLP) components with Kolmogorov–Arnold Networks (KANs). KANs, inspired by the K-A theorem, improve the network representation ability by placing learnable activation functions on fixed node activation functions. By implementing KANs, the model enhances parameter efficiency and interpretability, thus addressing the typical challenges of MLPs, such as the high parameter count and lack of transparency. The experiments achieved favorable predictive results, indicating that the KAN not only reduces the complexity of the model but also improves learning efficiency and prediction accuracy in dynamic visual servoing environments. Finally, Gazebo software was used in ROS to model and simulate the robotic arm, verify the effectiveness of the algorithm, and achieve visual servoing.