Leveraging Diverse Vectors in ViT for Image Super Resolution.

Vector serves as the fundamental element in the Attention calculation of the Vision Transformer (ViT). In the conventional ViT approach, spatial points within the feature map are treated as vectors, with each spatial point representing a point in h × w dimensions. We aim to broaden the spectrum of vectors and explore various forms, such as considering numerical values of feature map, channels, or windows as vectors for Attention calculation. We propose a hybrid ViT model named the High to Low Frequency Attention Network (HLFANet). Through Single Image Super-Resolution (SISR) experiments, it has been demonstrated that the different forms of vectors complement each other in learning and representing high and low-frequency features. Additionally, utilizing mixed attention helps mitigate the trade-off between computational cost and performance associated with window-based attention. [ABSTRACT FROM AUTHOR]