Data fusion of radar and image measurements for multi-object tracking via Kalman filtering

Data fusion is an important issue for object tracking in autonomous systems such as robotics and surveillance. In this paper, we present a multiple-object tracking system whose design is based on multiple Kalman filters dealing with observations from two different kinds of physical sensors. Hardware integration which combines a cheap radar module and a CCD camera has been developed and data fusion method has been proposed to process measurements from those modules for multi-object tracking. Due to the limited resolution of bearing angle measurements of the cheap radar module, CCD measurements are used to compensate for the low angle resolution. Conversely, the radar module provides radial distance information which cannot be measured easily by the CCD camera. The proposed data fusion enables the tracker to efficiently utilize the radial measurements of objects from the cheap radar module and 2D location measurements of objects in image space of the CCD camera. To achieve the multi-object tracking we combine the proposed data fusion method with the integrated probability data association (IPDA) technique underlying the multiple-Kalman filter framework. The proposed complementary system based on the radar and CCD camera is experimentally evaluated through a multi-person tracking scenario. The experimental results demonstrate that the implemented system with fused observations considerably enhances tracking performance over a single sensor system.