Robust multiple object tracking is the backbone of many higher-level applications such as people counting, behavioral analytics and biomedical imaging. We enhance multiple hypothesis tracker robustness to the problems of split, merge, occlusion and fragment through hierarchical approach. Foreground segmentation and clustered optical flow are used as the first-level tracker input. Only associated track of the first level is fed into the second level with the additional of two virtual measurements. Occlusion predictor is obtained by using the predicted data of each track to distinguish between merge and occlusion. Kalman filter is used to predict and smooth the track's state. Gaussian modelling is used to measure the quality of the hypotheses. Histogram intersection is applied to limit the size expansion of the track. The results show improvement both in terms of accuracy and precision compared to the benchmark trackers [1, 2].