Search

The few-shot learning paradigm aims to generalize to unseen tasks with limited samples. However, a focus solely on class-level discrimination may fall short of achieving robust generalization, especially when neglecting instance diversity and discriminability. This study introduces a metric-based few-shot approach, named Self-supervised Feature Fusion with Transformer (SFT), which integrates self-supervised learning with a transformer. SFT addresses the limitations of previous approaches by employing two distinct self-supervised tasks in separate models during pre-training, thus enhancing both instance diversity and discriminability in the feature space. The training process unfolds in two stages: pre-training and transfer learning. In pre-training, each model undergoes training with specific self-supervised tasks to harness the benefits of enhanced feature space. In the subsequent transfer learning stage, model weights are frozen, acting as feature extractors. The features from both models are amalgamated using a feature fusion technique and are transformed into task-specific features by a transformer, boosting discrimination on unseen tasks. The combined features enable the model to learn a well-generalized representation, effectively tackling the challenges posed by few-shot tasks. The proposed SFT method achieves state-of-the-art results on three benchmark datasets in few-shot image classification.

Text-to-image synthesis is a fascinating area of research that aims to generate images based on textual descriptions. The main goal of this field is to generate images that match the given textual description in terms of both semantic consistency and image realism. While text-to-image synthesis has shown remarkable progress in recent years, it still faces several challenges, mainly related to the level of image realism and semantic consistency. To address these challenges, various approaches have been proposed, which mainly rely on Generative Adversarial Networks (GANs) for optimal performance. This paper provides a review of the existing text-to-image synthesis approaches, which are categorized into four groups: image realism, multiple scene, semantic enhancement, and style transfer. In addition to discussing the existing approaches, this paper also reviews the widely used datasets for text-to-image synthesis, including Oxford-102, CUB-200-2011, and COCO. The evaluation metrics used in this field are also discussed, including Inception Score, Fréchet Inception Distance, Structural Similarity Index, R-precision, Visual-Semantic Similarity, and Semantic Object Accuracy. The paper also offers a compilation of the performance of existing works in the field.

The task of Text-to-Image synthesis is a difficult challenge, especially when dealing with low-data regimes, where the number of training samples is limited. In order to address this challenge, the Self-Supervision Text-to-Image Generative Adversarial Networks (SS-TiGAN) has been proposed. The method employs a bi-level architecture, which allows for the use of self-supervision to increase the number of training samples by generating rotation variants. This, in turn, maximizes the diversity of the model representation and enables the exploration of high-level object information for more detailed image construction. In addition to the use of self-supervision, SS-TiGAN also investigates various techniques to address the stability issues that arise in Generative Adversarial Networks. By implementing these techniques, the proposed SS-TiGAN has achieved a new state-of-the-art performance on two benchmark datasets, Oxford-102 and CUB. These results demonstrate the effectiveness of the SS-TiGAN method in synthesizing high-quality, realistic images from text descriptions under low-data regimes.