Search

Click-through rate (CTR) prediction, whose goal is to predict the probability of the user to click on an item, has become increasingly significant in the recommender systems. Recently, some deep learning models with the ability to automatically extract the user interest from his/her behaviors have achieved great success. In these work, the attention mechanism is used to select the user interested items in historical behaviors, improving the performance of the CTR predictor. Normally, these attentive modules can be jointly trained with the base predictor by using gradient descents. In this paper, we regard user interest modeling as a feature selection problem, which we call user interest selection. For such a problem, we propose a novel approach under the framework of the wrapper method, which is named Meta-Wrapper. More specifically, we use a differentiable module as our wrapping operator and then recast its learning problem as a continuous bilevel optimization. Moreover, we use a meta-learning algorithm to solve the optimization and theoretically prove its convergence. Meanwhile, we also provide theoretical analysis to show that our proposed method 1) efficiencies the wrapper-based feature selection, and 2) achieves better resistance to overfitting. Finally, extensive experiments on three public datasets manifest the superiority of our method in boosting the performance of CTR prediction. [ABSTRACT FROM AUTHOR]

Human behaviors in recommendation systems are driven by many high-level, complex, and evolving intentions behind their decision making processes. In order to achieve better performance, it is important for recommendation systems to be aware of user intentions besides considering the historical interaction behaviors. However, user intentions are seldom fully or easily observed in practice, so that the existing works are incapable of fully tracking and modeling user intentions, not to mention using them effectively into recommendation. In this paper, we present the Intention-Aware Sequential Recommendation (ISRec) method, for capturing the underlying intentions of each user that may lead to her next consumption behavior and improving recommendation performance. Specifically, we first extract the intentions of the target user from sequential contexts, then take complex intent transition into account through the message-passing mechanism on an intention graph, and finally obtain the future intentions of this target user from inference on the intention graph. The sequential recommendation for a user will be made based on the predicted user intentions, offering more transparent and explainable intermediate results for each recommendation. Extensive experiments on various real-world datasets demonstrate the superiority of our method against several state-of-the-art baselines in sequential recommendation in terms of different metrics. [ABSTRACT FROM AUTHOR]

Change in users’ preferences over time is a challenging issue in collaborative filtering models. Recent temporal recommender systems (RSs) use latent temporal features or probabilistic latent transition models. The models designed using latent temporal features remain confined to point estimation only, i.e., considering single predicted values (e.g., rating) and completely ignore the likelihood of other possible items being generated due to change in users’ preferences. The latent transitive models represent users’ preference over different states using distribution estimation but ignore user–item features. This article proposes an integrated approach for designing collaborative filtering models using both point and distribution estimation techniques together to address this issue. The matrix factorization-based point estimation strategy is applied to integrate user–item correlations and feature information, whereas the hidden Markov model is used for users’ preference distribution over time. Three benchmark datasets were used to evaluate and compare the proposed model’s performance with related state-of-the-art temporal RS models. The proposed model makes the system more robust in handling dynamic users’ preferences. [ABSTRACT FROM AUTHOR]

Link prediction aims at inferring missing links or predicting future ones based on the currently observed network. This topic is important for many applications such as social media, bioinformatics and recommendation systems. Most existing methods focus on homogeneous settings and consider only low-order pairwise relations while ignoring either the heterogeneity or high-order complex relations among different types of nodes, which tends to lead to a sub-optimal embedding result. This paper presents a method named Heterogeneous Hypergraph Variational Autoencoder (HeteHG-VAE) for link prediction in heterogeneous information networks (HINs). It first maps a conventional HIN to a heterogeneous hypergraph with a certain kind of semantics to capture both the high-order semantics and complex relations among nodes, while preserving the low-order pairwise topology information of the original HIN. Then, deep latent representations of nodes and hyperedges are learned by a Bayesian deep generative framework from the heterogeneous hypergraph in an unsupervised manner. Moreover, a hyperedge attention module is designed to learn the importance of different types of nodes in each hyperedge. The major merit of HeteHG-VAE lies in its ability of modeling multi-level relations in heterogeneous settings. Extensive experiments on real-world datasets demonstrate the effectiveness and efficiency of the proposed method. [ABSTRACT FROM AUTHOR]

In this paper, we propose a hierarchical representation of latent contextual information that captures contextual situations in which users are recommended particular items. We also introduce an algorithm that converts unstructured latent contextual information into structured hierarchical representations. In addition, we present two general context-aware recommendation algorithms that extend collaborative filtering (CF) approaches and utilize structured and unstructured latent contextual information. In particular, the first algorithm utilizes structured latent contexts and the second one combines the structured and the unstructured latent contextual representations. By using latent contextual information in a recommendation model, we capture and represent both the structure of the latent context in the form of a hierarchy and the values of contextual variables in the form of an unstructured vector. We tested the two proposed methods with two CF-based methods on several context-rich datasets under different experimental settings. We show that using hierarchical latent contextual representations leads to significantly better recommendations than the baselines for the datasets having high- and medium-dimensional contexts. Although this is not the case for the low-dimensional contextual data, the hybrid approach, combining structured and unstructured latent contextual information, significantly outperforms other baselines across all the experimental settings and dimensions of contextual data. [ABSTRACT FROM AUTHOR]

Predictive filtering methods are widely used in industry to remove random noise, owing to their stability and efficiency. Traditional predictive filtering methods use a fixed autoregressive order (filtering factor) in the spatial direction. However, denoising spatially varying seismic data have thus far been ineffective. In this study, the effect of nonstationary seismic data variations is overcome using local windowing or nonstationary denoising methods. We propose a nonstationary predictive filtering method in which an autoregressive model is constructed using spatially varying filtering factors. First, we evaluate the structural complexity of the global data based on a local window selection using plane-wave destruction. Then, adaptive filtering factors are proposed depending on the structural complexity of the seismic data. Finally, the autoregressive model containing the adaptive filtering factors is solved. We use three quality measures to evaluate the denoising performance: the signal-to-noise ratio ($S/N$), the peak signal-to-noise ratio (PSNR), and local similarity (LS). The proposed method acts on both synthetic and field data, including both pre- and post-stack seismic data. Compared with prior nonstationary filtering methods, experimental results demonstrate that structural complexity-guided (CG) predictive filtering enables efficient random noise attenuation and reduces signal leakage. [ABSTRACT FROM AUTHOR]

Machine learning can provide deep insights into data, allowing machines to make high-quality predictions and having been widely used in real-world applications, such as text mining, visual classification, and recommender systems. However, most sophisticated machine learning approaches suffer from huge time costs when operating on large-scale data. This issue calls for the need of Large-scale Machine Learning (LML), which aims to learn patterns from big data with comparable performance efficiently. In this paper, we offer a systematic survey on existing LML methods to provide a blueprint for the future developments of this area. We first divide these LML methods according to the ways of improving the scalability: 1) model simplification on computational complexities, 2) optimization approximation on computational efficiency, and 3) computation parallelism on computational capabilities. Then we categorize the methods in each perspective according to their targeted scenarios and introduce representative methods in line with intrinsic strategies. Lastly, we analyze their limitations and discuss potential directions as well as open issues that are promising to address in the future. [ABSTRACT FROM AUTHOR]

In recent years, the establishment of a substantial amount of academic groups on scientific social network has brought new opportunities for the collaboration among researchers. In this situation, conducting paper recommendation to these academic groups is of terrific necessity in that it can further facilitate group activities. However, when producing group recommendation, existing methods fail to make full use of the abundant group information, from which a great deal of valuable information can be inferred to facilitate the recommendation performance. In addition, those methods tend to assign an equal weight to each group member when aggregating their recommendations, which is unreasonable in practice. Although some improvements have been made to remedy this problem by assigning different weights to group members, they fail to take into account the reliabilities of group members. Therefore, a group-oriented paper recommendation method based on probabilistic matrix factorization and evidential reasoning (GPMF_ER) is proposed in this article to tackle these problems. More specifically, the group and paper content information are integrated into the probabilistic matrix factorization model to enhance the accuracy of individual recommendation. Afterward, evidential reasoning rule is introduced in the aggregation step to consider both the weights and reliabilities of group members. Extensive experiments have been conducted on the real world CiteULike dataset and the results demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Many studies focusing on integrating reviews with ratings to improve recommendation performance have been quite successful. However, these works still face several shortcomings: (1) The importance of dynamically integrating review and interaction data features is typically ignored, yet treating these fusion features equally may lead to an incomplete understanding of user preferences. (2) Some forms of soft attention methods are adopted to model the local semantic information of words. As features thus captured may contain irrelevant information, the generated attention map is neither discriminatory nor detailed. In this paper, we propose a novel Adaptive Hierarchical Attention-enhanced Gated network integrating reviews for item recommendation, named AHAG. AHAG is a unified framework to capture the hidden intentions of users by adaptively incorporating reviews. Specifically, we design a gated network to dynamically fuse the extracted features and select the features that are most relevant to user preferences. To capture distinguishing fine-grained features, we introduce a hierarchical attention mechanism to learn important semantic information features and the dynamic interaction of these features. Besides, the high-order non-linear interaction of neural factorization machines is utilized to derive the rating prediction. Experiments on seven real-world datasets show that the proposed AHAG significantly outperforms state-of-the-art methods. Furthermore, the attention mechanism can highlight the relevant information in reviews to increase the interpretability of the recommendation task. Source codes are available in https://github.com/luojia527/AHAG. [ABSTRACT FROM AUTHOR]

Data in many real-world applications such as social networks, users shopping behaviors, and inter-item relationships can be represented as graphs. Graph Neural Networks (GNNs) have shown great success in learning meaningful representations for graphs by inherently integrating node information and topological structure. Data in social recommendations can also be denotes as graph data in the form of user-user social graphs and user-item graphs. In addition, the relationships between items can be denoted as item-item graphs. GNNs provide an unprecedented opportunity to advance social recommendations. However, there are tremendous challenges in building GNNs-based social recommendations where (1) users (items) are simultaneously involved in the user-item graph and user-user social graph (item-item graph); (2) user-item graphs not only contain user-item interactions but also include users’ opinions on items; and (3) the nature of social relations are heterogeneous among users. In this paper, we propose a novel graph neural network framework (GraphRec+) for social recommendations, which is able to coherently model graph data in order to learn better user and item representations. Specifically, we introduce a principled approach for jointly capturing interactions and opinions in the user-item graph and also propose an attention mechanism to differentiate the heterogeneous strengths of social relations. Comprehensive experiments on three real-world datasets show the effectiveness of the proposed framework. [ABSTRACT FROM AUTHOR]

Traditional recommender systems (RS) only consider homogeneous data and cannot fully model heterogeneous information of complex objects and relations. Recent advances in the study of Heterogeneous Information Network (HIN) have shed some light on how to leverage heterogeneous information in RS. However, existing HIN-based recommendation models assume HIN is invariable and merely use HIN as a data source for assisting recommendation, which limits their performance. In this paper, we propose a multi-task learning framework, called MTRec, for recommendation over HIN. MTRec relies on self-attention mechanism to learn the semantics of meta-paths in HIN and jointly optimizes the tasks of both recommendation and link prediction. Using a Bayesian task weight learner, MTRec is able to achieve the balance of two tasks during optimization automatically. Moreover, MTRec provides good interpretabilities of recommendation through a “translation” mechanism which is used to model the three-way interactions among users, items and the meta-paths connecting them. Experimental results demonstrate the superiority of MTRec over state-of-the-art HIN-based recommendation models, and the case studies we provide illustrate that MTRec enhances the explainability of RS. [ABSTRACT FROM AUTHOR]

A non-negative latent factor analysis (NLFA)-based recommender can make precise recommendations by correctly representing the non-negative characteristic of industrial data. It commonly relies on a nonconvex and bilinear optimization process, where the effects of first-order solvers maybe significantly reduced. Higher order solvers like a Newton-type method are expected to make a breakthrough; however, its computation efficiency and scalability are greatly limited due to the numerous parameters involved in a Hessian matrix. To address this issue, this article proposes an approach for assimilating second-order information for building NLFA-based recommenders. The key idea is an inner second-order solver that employs a Hessian-free method for avoiding the highly expensive manipulations of a Hessian matrix. Empirical studies on eight data cases emerging from real industrial applications indicate that the proposed approach outperforms state-of-the-art models in prediction accuracy with affordable computational burden. [ABSTRACT FROM AUTHOR]

Ranking on graphs is a centerpiece in many high-impact application domains, such as information retrieval, recommender systems, team management, neuroscience and many more. PageRank, along with many of its variants, is widely used across these application domains thanks to its mathematical elegance and the superior performance. Although PageRank and its variants are effective in ranking nodes on graphs, they often lack an efficient and effective way to audit the ranking results in terms of the input graph structure, e.g., which node or edge in the graph contributes most to the top-1 ranked node; which subgraph plays a crucial role in generating the overall ranking result? In this paper, we propose to audit graph ranking by finding the influential graph elements (e.g., edges, nodes, attributes, and subgraphs) regarding their impact on the ranking results. First, we formulate graph ranking auditing problem as quantifying the influence of graph elements on the ranking results. Second, we show that our formulation can be applied to a variety of graph structures. Third, we propose effective and efficient algorithms to find the top- $k$ k influential edges/nodes/subgraph. Finally, we perform extensive empirical evaluations on real-world datasets to demonstrate that the proposed methods (Aurora) provide intuitive auditing results with linear scalability. [ABSTRACT FROM AUTHOR]

Tremendous success of machine learning (ML) and the unabated growth in model complexity motivated many ML-specific designs in hardware architectures to speed up the model inference. While these architectures are diverse, highly optimized low-precision arithmetic is a component shared by most. Nevertheless, recommender systems important to Facebook's personalization services are demanding and complex: They must serve billions of users per month responsively with low latency while maintaining high prediction accuracy. Do these low-precision architectures work well with our production recommendation systems? They do. But not without significant effort. In this article, we share our search strategies to adapt reference recommendation models to low-precision hardware, our optimization of low-precision compute kernels, and the tool chain to maintain our models' accuracy throughout their lifespan. We believe our lessons from the trenches can promote better codesign between hardware architecture and software engineering, and advance the state of the art of ML in industry. [ABSTRACT FROM AUTHOR]