AEVRNet: Adaptive exploration network with variance reduced optimization for visual tracking

For visual tracking methods based on reinforcement learning, action space determines the ability of exploration, which is crucial to model robustness. However, most trackers adopted simple strategies with action space, which will suffer local optima problem. To address this issue, a novel reinforcement learning based tracker called AEVRNet is proposed with non-convex optimization and effective action space exploration. Firstly, inspired by combinatorial upper confidence bound, we design an adaptive exploration strategy leveraging temporal and spatial knowledge to enhance effective action exploration and jump out of local optima. Secondly, we define the tracking problem as a non-convex problem and incorporate non-convex optimization in stochastic variance reduced gradient as backward propagation of our model, which can converge faster with lower loss. Thirdly, different from existing reinforcement learning based trackers using classification method to train model, we define a regression based action-reward loss function, which is more sensitive to aspects of the target states, e.g., the width and height of the target to further improve robustness. Extensive experiments on six benchmark datasets demonstrate that our proposed AEVRNet achieves favorable performance against the state-of-the-art reinforcement learning based methods.