A catadioptric and pan-tilt-zoom camera pair object tracking system for UAVs

Unmanned aerial vehicles (UAVs) are seeing widespread use in military, scientific, and civilian sectors in recent years. As the mission demands increase, these systems are becoming more complicated. Omnidirectional camera is a vision sensor that can captures 360° view in a single frame. In recent years omnidirectional camera usage has experienced a remarkable increase in many fields, where many innovative research has been done. Although, it is very promising, employment of omnidirectional cameras in UAVs is quite new. In this paper, an innovative sensory system is proposed, that has an omnidirectional imaging device and a pan tilt zoom (PTZ) camera. Such a system combines the advantages of both of the camera systems. The system can track any moving object within its 360° field of view and provide detailed images of it. The detection of the moving object has been accomplished by an adaptive background subtraction method implemented on the lowered resolution images of the catadioptric camera. A novel algorithm has also been developed to estimate the relative distance of the object with respect to the UAV, using tracking information of both of the cameras. The algorithms are implemented on an experimental system to validate the approach.