Multi-frame co-saliency spatio-temporal regularization correlation filters for object tracking.

The spatial regularization weight of the correlation filter is not related to the object content and the model degradation in the tracking process. To solve this problem, a new multi-frame co-saliency spatio-temporal regularization correlation filters (MCSRCF) is proposed for visual object tracking. To the best our knowledge, this is the first application of co-saliency regularization to CF-based tracking. In MCSRCF, grayscale features, directional gradient histogram (HOG) features and CNN features are extracted to improve the tracking precision of the tracker. Secondly, the three-dimensional spatial saliency and semantic saliency are introduced to obtain the initial weight of the spatial regularization with object content information. Then, the heterogeneous saliency fusion method is exploited to add a co-saliency spatial regularization term to the objective function to make the spatial penalty weight learn the change of the object region. In additional, the temporal saliency regularization is introduced to learn the information between adjacent frames, which reduces the overfitting effect caused by inaccurate samples. A variety of evaluations are conducted on public benchmarks, and the experimental results show that the proposed tracker achieves good robustness against many state-of-the-art trackers in various complex scenarios. [Display omitted] • The three-dimensional spatial and semantic saliency regularization is exploited to obtain the regularized weights with the spatial saliency information of the object, solving the boundary effect and alleviating the influence of background interference. • The heterogeneous saliency fusion method is used to obtain the spatial co-saliency regularization weights, which can improve the discrimination of the classifier. • Temporal saliency regularization weights are exploited to obtain object change information between frames and limit the change rate of the response map. The Alternating Direction Method of the Multiplier (ADMM) saves computing resources and improves the tracking speed. [ABSTRACT FROM AUTHOR]