Reinforcement learning for pan-tilt-zoom camera control, with focus on drone tracking

Reliable detection and tracking of objects using pan-tilt-zoom (PTZ) cameras is an unsolved problem. We attempt to answer whether the use of reinforcement learning (RL) is an appropriate tool for solving it. We present an environment for training RL agents to track a drone using a (PTZ) camera. We also present an agent trained using this environment, which learns to correctly pan, tilt, and zoom the camera to follow a randomly moving drone, using continuous actions. The input into the agent is the RGB image observed by the camera. The agent is rewarded for correctly tracking the drone, and penalized if it loses it from its viewport. We use the recurrent proximal policy optimization (PPO) algorithm with a long short-term memory (LSTM) layer. We find that the agent reliably learns ways of tracking the drone after around 1.4 million steps of training.