The previous target tracking algorithms based on correlation filters have excellent tracking performance. However, when encountering some challenging problems such as fast motion, occlusion, scale variations, motion blur, etc., tracking drift or even tracking failure occurs during the tracking process. Aiming at the above problems, we propose a novel tracking method. On the basis of kernelized correlation filter, a scale adaptive filter is added to adapt to the scale variations of the target during the tracking process. In addition, a feedback mechanism using the average peak-to-correlation energy (APCE) as the judgment criterion is introduced to enable the model to be updated under the premise of high-confidence and avoid tracking model corruption. Experimental results show that our algorithm performs better than traditional correlation filtering algorithms on challenging sequences.