Search

Few-shot class-incremental learning (FSCIL) aims to continually learn new classes using a few samples while not forgetting the old classes. The scarcity of new training data will seriously destroy the model’s stability and plasticity. Continually Evolved Classifiers (CEC) (Zhang et al., 2021), a kind of framework, maintains the stability by freezing the encoder and achieves the plasticity by evolving the classifier along with a pseudo incremental learning scheme. However, the performance of CEC is limited due to 1) inequitable information gains between classifier weights and test features, and 2) inefficient learning task construction strategy. To address the first issue, we propose a Knowledge-guided Relation Refinement Module (KRRM) to update both the classifier weights and test features. The main function of KRRM is achieved through cross-attention to propagate the knowledge represented by old encoded data. To address the second issue, we design a Pseudo Incremental relation Refinement Learning (PIRL) that utilizes a novel hard concepts mining strategy to mine hard concept tasks globally and locally. By successfully addressing the two issues, our proposed method, named Improved Continually Evolved Classifiers (CEC+), extends the potential of CEC without introducing any additional parameters. More precisely, extensive experiments on CIFAR100, miniImageNet, and Caltech-UCSD Birds-200-2011, demonstrate that our proposed method surpasses prior state-of-the-art methods.

Recently, the emerging concept of “unmanned retail” has drawn more and more attention, and the unmanned retail based on the intelligent unmanned vending machines (UVMs) scene has great market demand. However, existing product recognition methods for intelligent UVMs cannot adapt to large-scale categories and have insufficient accuracy. In this article, we propose a method for large-scale categories product recognition based on intelligent UVMs. It can be divided into two parts: 1) first, we explore the similarities and differences between products through manifold learning, and then we build a hierarchical multigranularity label to constrain the learning of representation; and 2) second, we propose a hierarchical label object detection network, which mainly includes coarse-to-fine refine module (C2FRM) and multiple granularity hierarchical loss (MGHL), which are used to assist in capturing multigranularity features. The highlights of our method are mine potential similarity between large-scale category products and optimization through hierarchical multigranularity labels. Besides, we collected a large-scale product recognition dataset GOODS-85 based on the actual UVMs scenario. Experimental results and analysis demonstrate the effectiveness of the proposed product recognition methods.

AI-based methods are shining across a variety of industries, especially unmanned retail. Product recognition is the problem of recognizing the category and quantity of products (e.g., beverages and mineral water) in intelligent unmanned vending machines (UVMs) to automatic checkout during purchase. However, for similar products in hundreds of categories, the existing method is not accurate enough. Besides, they cannot be extended for new products without retraining. In this article, we propose a product recognition approach based on intelligent UVMs, called Split-Check, which first splits the region of interest of products by detection and then check product by instance-level retrieval. Split-Check is the combination of two important components. The preliminary detection distinguishes items that contain the different coarse-grained features, then locates items, and classifies them into coarse-grained categories as a candidate. The retrieval further distinguishes the candidate items that contain the different fine-grained features. Besides, we reconstruct a large-scale categories product dataset GOODS-85 based on actual UVMs scenarios, in which the number of categories of items is larger than the existing dataset. Experimental results demonstrate the effectiveness of the proposed approach. Our method significantly improves the recognition performance of hundreds of products and increases the scalability of products.

Most recommendation systems focus on predicting rating or finding aspect information in reviews to understand user preferences and item properties. However, these methods ignore the effectiveness and persuasiveness of recommendation results. Consequently, explainable recommendation, namely providing recommendation results with recommendation reasons at the same time, has attracted increasing attention of researchers due to its ability in fostering transparency and trust. It is lucky that some E-commerce websites provide a kind of new interaction box called Tips and users can express their comments on items with a simple sentence. This brings us an opportunity to realize explainable recommendation. Under the supervision of two explicit feedbacks, namely rating and textual tips, we can implement a multi-task learning model which can provide recommendation results and generate recommendation reasons at the same time. In this paper, we propose an Encoder-Decoder and Multi-Layer Perception (MLP) based Explainable Recommendation model named EMER to simultaneously implement reason generation and rating prediction. Item’s title contains significant product-related information and plays an important role in grabbing user’s attention, so we fuse it in our model to generate recommendation reasons. Numerous experiments on benchmark datasets demonstrate that our model is superior to the state-of-the-art models.

Dating recommendation becomes a critical task since the rapidly development of the online dating sites and it is beneficial for users to find their ideal relationships from a large number of registered members. Different users usually have different tastes when choosing their dating partners. Therefore, it is necessary to distinguish the user’s personal features and preferences in dating recommendation methods. However, present approaches don’t capture enough user preferences from social graph and attribute data. They also ignore user attributes, which is the complementary and consistent side information of user social graphs. In this paper, we propose a Matching Individual Preferences with Features (MIPF) model to recommend dating partners jointly using user attributes and social graphs. Our specific design in the model is that user preferences and user features are completely different. User features are users’ personal characteristics, while user preferences indicate which kind of other users they are seeking. We aim to model user features and preferences to identify what the user is and what the user likes. We also distinguish user preferences into explicit preference and implicit preferences. The implicit preferences are mined from social graphs, while the explicit preferences are captured from the social links. Additionally, convolutional neural networks are used to extract the latent non-linear information in user attributes. Experiments on real-world online dating datasets demonstrate our MIPF model is superior to existing methods.

Short-to-medium term temperature prediction in high resolution is a very challenging task, involving meteorology, physics, mathematics, geography, and many other subjects. Its purpose is to fit a complex function from historical meteorological data to predict the future 1–5 days temperature, which is a typical spatio-temporal prediction problem. Meteorological data show complex correlations in local space. Most of the existing machine learning methods are based on image pixel-level tasks or spatio-temporal prediction tasks, which model meteorological data without considering the characteristics of meteorological data and use rough global patterns to model local space which would lose many details. To address the above issues, our work fine-grained conditional convolution network (FCCN) proposes a novel grid-level conditional convolution module, including a local geographic adaptive weight (GAW) and a local data adaptive weight (DAW). These two components are integrated into a multiscale meteorological fusion gated recurrent unit (GRU) architecture for the end-to-end temperature prediction. Experiments in real-world datasets from ERA-5 show our FCCN model has a better performance than all other baseline methods.

In image-text matching fields, one of the keys to improving performance is to extract features with more semantic information. Existing works demonstrate that semantic enrichment through knowledge expansion can improve performance. Most of them expand image features, however, the shortage of semantic information in text modality and the unilateral character of the view are often bottlenecks that limit the performance of image-text matching models. To solve the two problems, we aggregate knowledge from multiple views and propose a word imagination graph (WIG). A WIG can be used to expand textual semantic information by imagination based on input images. Then, utilizing WIG, we construct a novel multiview text imagination network (MTIN). A MTIN enables latent alignment of images and texts on tags, which can assist matching on a semantic level. Results from the Flickr30K and MS-COCO datasets demonstrate the effectiveness of our method. The source code has been released on GitHub https://github.com/smileslabsh/Multiview-Text-Imagination-Network.