Search

• It is necessary to reduce the training costs of current tracking frameworks. • Our model can realize the attention of equivalent linear complexity. • Our linearization scheme can be used for both self-attention and cross-attention. • Our model achieves stable tracking performance with high robustness and fast speed. Mainstream tracking approaches have achieved remarkable performance by adopting transformer structures. However, transformer structures' inherent design of dot-product with softmax normalization incurs quadratic computation complexity regarding sequence length. This issue is further complicated when vision tasks employ softmax attention, as sequence length scales with the square of images' sizes. Even though sparse attention and low-rank decomposition can alleviate over-inflated computation, it is still laborious to balance trackers' accuracy, computation cost, and inference speed. To tackle the above problems, we propose an Information-Diffused Graph tracking pipeline with linear complexity (IDGtrack). As the feature constraint relationship in the physical world is an important cue for vision tasks, graph modules are constructed with information-diffused adjacency matrices to substitute softmax attention, which is not only efficient for linear computations but also maintains the non-negativity and global distribution of the attention matrix. Distinct from traditional linear attention methods exclusive to self-attention, a self-integrated and cross-context graph module with linear complexity is explored where a complete bipartite graph is established between the target and search region, facilitating a comprehensive perception of local and background information. Extensive experiments are conducted on public tracking benchmarks, demonstrating that our method achieves state-of-the-art (SOTA) performance with 111 FPS on GPU RTX3090. [ABSTRACT FROM AUTHOR]