Search

Nowadays relational position embedding is widely used in many large multi‐modal models. It begins with relational captioning (a branch of image captioning) and contains two procedures: geometric modelling and prior attention. However, there are some problems that remain unsolved in the conventional procedures. This paper reviews the shortcomings of geometric modelling and prior attention. Then, a new framework called relational guided transformer (RGT) is proposed to verify the authors' conclusion from the origin of relational position embedding—relational captioning. Specifically, RGT has two simple but effective improvements in geometric modelling and prior attention: (1) A machine‐learned geometric modelling strategy called multi‐task geometric modelling (MTG) is used under multi‐task learning, replacing the original hand‐made geometric feature. (2) The effectiveness of multiple kinds of prior attention is discussed and preserved in a better form, which is called spatial guided attention (SGA) to integrate the geometric prior knowledge. Extensive experiments on MSCOCO and Flickr30k have been performed to investigate the effectiveness of each module and prove our argument. The superiority of the model comparing to the authors' baseline has also been proven on the offline evaluation with the "Karpathy" test split of both datasets. [ABSTRACT FROM AUTHOR]

Abstract Nowadays relational position embedding is widely used in many large multi‐modal models. It begins with relational captioning (a branch of image captioning) and contains two procedures: geometric modelling and prior attention. However, there are some problems that remain unsolved in the conventional procedures. This paper reviews the shortcomings of geometric modelling and prior attention. Then, a new framework called relational guided transformer (RGT) is proposed to verify the authors' conclusion from the origin of relational position embedding—relational captioning. Specifically, RGT has two simple but effective improvements in geometric modelling and prior attention: (1) A machine‐learned geometric modelling strategy called multi‐task geometric modelling (MTG) is used under multi‐task learning, replacing the original hand‐made geometric feature. (2) The effectiveness of multiple kinds of prior attention is discussed and preserved in a better form, which is called spatial guided attention (SGA) to integrate the geometric prior knowledge. Extensive experiments on MSCOCO and Flickr30k have been performed to investigate the effectiveness of each module and prove our argument. The superiority of the model comparing to the authors' baseline has also been proven on the offline evaluation with the “Karpathy” test split of both datasets.

Facial recognition systems are widely used in various daily applications due to their high accuracy rate and ease of use. Unfortunately, their performance drops drastically when encountering spoofing attacks. In this paper, a novel pipeline for face spoofing detection based on the extraction of Heterogeneous Auto-Similarities of Characteristics (HASC) descriptor from the HSV and YCbCr color spaces is proposed. To detect print and replay spoofing attacks, the HASC descriptor encodes linear and non-linear dependencies between low-level dense features of the color face image using covariance and information-theoretic measures. The combination of covariance and information-theoretic measures makes HASC an efficient descriptor to explore how the features are interrelated in real and fake faces. The HASC texture descriptions are extracted separately from each color channel of the HSV and YCbCr color spaces, and the resulting feature vectors are concatenated into an extended feature vector to better discriminate between real and fake faces. Extensive experiments are conducted on publicly available databases namely, Replay-Attack, CASIA FASD, and MSU MFSD. The obtained results demonstrate that while being of low dimension, the HASC descriptor when fed into a binary Support Vector Machine (SVM) classifier, achieves competitive results compared to the state-of-the-art. [ABSTRACT FROM AUTHOR]

Texture perception plays a vital role in various fields, from computer vision to geology, influencing object recognition, image segmentation, and rock classification. Despite advances in convolutional neural networks (CNNs), their effectiveness in texture-based classification tasks, particularly in rock classification, still needs exploration. This paper addresses this gap by evaluating different CNN architectures using diverse publicly available texture datasets and custom datasets tailored for rock classification. We investigated the performance of 38 distinct models pre-trained on the ImageNet dataset, employing both transfer learning and fine-tuning techniques. The study highlights the efficacy of transfer learning in texture classification tasks and offers valuable perspectives on the performance of different networks on different datasets. We observe that while CNNs trained on datasets like ImageNet prioritize texture-based features, they face challenges in nuanced texture-to-texture classification tasks. Our findings underscore the need for further research to enhance CNNs’ capabilities in texture analysis, particularly in the context of rock classification. Through this exploration, we contribute insights into the suitability of CNN architectures for rock texture classification, fostering advancements in both computer vision and geology.

Generative Adversarial Networks (GANs) emerged thanks to the development of deep neural networks. Forgery images generated by various variants of GANs are widely spread on the Internet, which may be damage personal credibility and cause huge property losses. Thus, numerous methods are proposed to detect forgery images, but most of them are designed to detect forgery faces. Therefore, a method to detect forgery images of various scenes is proposed. In this work, central difference convolution and vanilla convolution (CDC‐Mix) are mixed after considering the depth and width features of neural networks and analyzing the influence of attention on network performance. Based on CDC‐Mix, a separable convolution (SeparableCDC‐Mix) is proposed. The proposed method consists of three parts: (1) CDC‐Mix and SeparableCDC‐Mix are used to extract the gradient information and texture features; (2) CDCM is used to extract the multi‐scale information of the image; (3) multi‐scale fusion module (MS‐Fusion) is used to fuse the multi‐scale information from different locations of the network. A large number of experiments have been carried out on several datasets generated by GAN, and the experimental results show that the proposed method has a great improvement compared with the existing advanced methods. [ABSTRACT FROM AUTHOR]