Search

Click-Through Rate (CTR) prediction, which estimates the probability of a user clicking on a particular item, constitutes a pivotal element in the realms of both online advertising and recommender systems. However, issues surrounding sparse and imbalanced data have yet to be resolved. To cope with these challenges, this paper proposes a semi-supervised framework called SS4CTR. Two distinctive features characterise the proposed SS4CTR model. Firstly, it employs an interpretable approach to select negative samples based on the global popularity of items, ensuring a balanced ratio of positive and negative samples within the input dataset. Secondly, by integrating both labeled and unlabeled data into the training process, the model effectively tackles the challenge of data sparsity and significantly enhances the accuracy of user click-through rate predictions. And the confidence threshold mechanism for pseudo-labelling also ensures that unlabeled data can be used in a secure manner. To the best of our knowledge, this is the first study to address the key challenges posed by sparse and imbalanced data simultaneously in the context of CTR prediction. Extensive experiments conducted on four real-world sparse datasets confirm the effectiveness and applicability of the SS4CTR model in scenarios characterized by sparse and imbalanced data. [ABSTRACT FROM AUTHOR]

Gradient boosting decision tree (GBDT) is a mainstream model for advertisement click-through rate (CTR) prediction. Since the complex working mechanism of GBDT, advertising analysts often fail to analyze the decision-making and the iterative evolution process of a large number of decision trees, as well as to understand the impact of different features on the prediction results, which makes the model tuning quite challenging. To address these challenges, we propose a visual analytics system, GBDT4CTRVis, which helps advertising analysts understand the working mechanism of GBDT and facilitate model tuning through intuitive and interactive views. Specifically, we propose instance-level views to hierarchically explore the prediction results of advertising data, feature-level views to analyze the importance of features and their correlations from various perspectives, and model-level views to investigate the structure of representative decision trees and the temporal evolution of information gain during model prediction. We also provide multi-view interactions and panel control for flexible exploration. Finally, we evaluate GBDT4CTRVis through three case studies and expert evaluations. Feedback from experts indicated the usefulness and effectiveness of GBDT4CTRVis in helping to understand the model mechanism and tune the model. [ABSTRACT FROM AUTHOR]

Click‐through rate (CTR) prediction is a pivotal challenge in recommendation systems. Existing models are prone to disturbances from noise and redundant features, hindering their ability to fully capture implicit and higher‐order feature interactions present in sparse feature data. Moreover, conventional dual‐tower models overlook the significance of layer‐level feature interactions. To address these limitations, this article introduces Gate‐enhanced Multi‐space Interactive Neural Networks (GMINN), a novel model for CTR prediction. GMINN adopts a dual‐tower architecture in which a multi‐space interaction layer is introduced after each layer in the dual‐tower deep neural network. This layer allocates features into multiple subspaces and employs matrix multiplication to establish layer‐level interactions between the dual towers. Simultaneously, a field‐aware gate mechanism is proposed to extract crucial latent information from the original features. Experimental validation on publicly available datasets, Criteo and Avazu, demonstrates the superiority of the proposed GMINN model. Comparative analyses against baseline models reveal that GMINN substantially improves up to 4.09% in AUC and a maximum reduction of 7.21% in Logloss. Additionally, ablation experiments provide further validation of the effectiveness of GMINN. [ABSTRACT FROM AUTHOR]

In the era of intelligent economy, the click-through rate (CTR) prediction system can evaluate massive service information based on user historical information, and screen out the products that are most likely to be favored by users, thus realizing customized push of information and achieve the ultimate goal of improving economic benefits. Sequence modeling is one of the main research directions of CTR prediction models based on deep learning. The user's general interest hidden in the entire click history and the short-term interest hidden in the recent click behaviors have different influences on the CTR prediction results, which are highly important. In terms of capturing the user's general interest, existing models paid more attention to the relationships between item embedding vectors (point-level), while ignoring the relationships between elements in item embedding vectors (union-level). The Lambda layer-based Convolutional Sequence Embedding (LCSE) model proposed in this paper uses the Lambda layer to capture features from click history through weight distribution, and uses horizontal and vertical filters on this basis to learn the user's general preferences from union-level and point-level. In addition, we also incorporate the user's short-term preferences captured by the embedding-based convolutional model to further improve the prediction results. The AUC (Area Under Curve) values of the LCSE model on the datasets Electronic, Movie & TV and MovieLens are 0.870 7, 0.903 6 and 0.946 7, improving 0.45%, 0.36% and 0.07% over the Caser model, proving the effectiveness of our proposed model. [ABSTRACT FROM AUTHOR]

Alongside natural language processing and computer vision, large learning models have found their way into e-commerce. Especially, for recommender systems and click-through rate prediction, these models have shown great predictive power. In this work, we aim to predict the probability that a customer will click on a given recommendation, given only its current session. Therefore, we propose a two-stage approach consisting of a customer behavior-embedding representation and a recurrent neural network. In the first stage, we train a self-supervised skip-gram embedding on customer activity data. The resulting embedding representation is used in the second stage to encode the customer sequences which are then used as input to the learning model. Our proposed approach diverges from the prevailing trend of utilizing extensive end-to-end models for click-through rate prediction. The experiments, which incorporate a real-world industrial use case and a widely used as well as openly available benchmark dataset, demonstrate that our approach outperforms the current state-of-the-art models. Our approach predicts customers' click intention with an average F1 accuracy of 94% for the industrial use case which is one percentage point higher than the state-of-the-art baseline and an average F1 accuracy of 79% for the benchmark dataset, which outperforms the best tested state-of-the-art baseline by more than seven percentage points. The results show that, contrary to current trends in that field, large end-to-end models are not always needed. The analysis of our experiments suggests that the reason for the performance of our approach is the self-supervised pre-trained embedding of customer behavior that we use as the customer representation. [ABSTRACT FROM AUTHOR]

Alongside natural language processing and computer vision, large learning models have found their way into e-commerce. Especially, for recommender systems and click-through rate prediction, these models have shown great predictive power. In this work, we aim to predict the probability that a customer will click on a given recommendation, given only its current session. Therefore, we propose a two-stage approach consisting of a customer behavior-embedding representation and a recurrent neural network. In the first stage, we train a self-supervised skip-gram embedding on customer activity data. The resulting embedding representation is used in the second stage to encode the customer sequences which are then used as input to the learning model. Our proposed approach diverges from the prevailing trend of utilizing extensive end-to-end models for click-through rate prediction. The experiments, which incorporate a real-world industrial use case and a widely used as well as openly available benchmark dataset, demonstrate that our approach outperforms the current state-of-the-art models. Our approach predicts customers’ click intention with an average F1 accuracy of 94% for the industrial use case which is one percentage point higher than the state-of-the-art baseline and an average F1 accuracy of 79% for the benchmark dataset, which outperforms the best tested state-of-the-art baseline by more than seven percentage points. The results show that, contrary to current trends in that field, large end-to-end models are not always needed. The analysis of our experiments suggests that the reason for the performance of our approach is the self-supervised pre-trained embedding of customer behavior that we use as the customer representation.

Extracting more information from feature interactions is essential to improve click-through rate (CTR) prediction accuracy. Although deep learning technology can help capture high-order feature interactions, the combination of features lacks interpretability. In this paper, we propose a multi-semantic feature interaction learning network (MeFiNet), which utilizes convolution operations to map feature interactions to multi-semantic spaces to improve their expressive ability and uses an improved Squeeze & Excitation method based on SENet to learn the importance of these interactions in different semantic spaces. The Squeeze operation helps to obtain the global importance distribution of semantic spaces, and the Excitation operation helps to dynamically re-assign the weights of semantic features so that both semantic diversity and feature diversity are considered in the model. The generated multi-semantic feature interactions are concatenated with the original feature embeddings and input into a deep learning network. Experiments on three public datasets demonstrate the effectiveness of the proposed model. Compared with state-of-the-art methods, the model achieves excellent performance (+ 0.18% in AUC and - 0.34% in LogLoss VS DeepFM; + 0.19% in AUC and - 0.33% in LogLoss VS FiBiNet). [ABSTRACT FROM AUTHOR]

With the exponential growth of network resources, recommendation systems have become successful at combating information overload. In intelligent recommendation systems, the prediction of click-through rates (CTR) plays a crucial role. Most CTR models employ a parallel network architecture to successfully capture explicit and implicit feature interactions. However, the existing models ignore two aspects. One limitation observed in most models is that they focus only on the interaction of paired term features, with no emphasis on modeling unary terms. The second issue is that most models input characteristics indiscriminately into parallel networks, resulting in network input oversharing. We propose a disentangled self-attention neural network based on information sharing (DSAN) for CTR prediction to simulate complex feature interactions. Firstly, an embedding layer transforms high-dimensional sparse features into low-dimensional dense matrices. Then, the disentangled multi-head self-attention learns the relationship between different features and is fed into a parallel network architecture. Finally, we set up a shared interaction layer to solve the problem of insufficient information sharing in parallel networks. Results from experiments conducted on two real-world datasets demonstrate that our proposed method surpasses existing methods in predictive accuracy. [ABSTRACT FROM AUTHOR]

Efficiently extracting user interest from user behavior sequences is the key to improving the click-through rate, and learning sophisticated feature interaction information is also critical in maximizing CTR. However, in terms of interest extraction, the problem of sequence dependence in most existing methods renders low training efficiency. Meanwhile, when exploring high-order feature interactions, the existing method fails to exploit information from all layers of the model. In this study, we propose an interest evolution network (TGRIEN) based on transformer and a gated residual. First, a transformer network supervised by an auxiliary loss function is proposed to extract users' interests from behavioral sequences in parallel to enhance the training efficiency. Second, a minimal gated unit with an attention forget gate is constructed to detect interests related to target ads and capture the evolution of users' interests. A gating mechanism is also employed in the residual module to construct a skip gated residual network, which can realize more abundant and effective feature interaction information in several ways. We evaluate the performance of TGRIEN on two real-world datasets. Experimental results demonstrate that our model significantly outperforms state-of-the-art baselines in terms of both prediction and training efficiency. [ABSTRACT FROM AUTHOR]

The development of online social networks enables users to generate a large amount of historical behavior data. How to effectively extract user interests from these data is crucial to improve the performance of click-through rate (CTR) prediction. However, existing research ignores the importance of collaborative personalized interest extraction based on the dynamic relationship in CTR prediction, which will cause the model to lose some important information to improve performance. Therefore, this paper designs an adaptive dynamic extraction (ADE) Unit to obtain user interest representations. First, the ADE unit designs a time weighting function to consider the dynamic evolution of user interests; then, the user's personalized interest representation is generated for a specific item. Finally, collaborative users are used to obtain the interest intensity of newly interested users. Based on the ADE unit, a new CTR prediction model is proposed, called personalized collaborative interest extraction network (PC-IEN). The model design feature embedding layer obtains a low-dimensional representation vector. Mini-batch aware regularization and the Dice activation function are employed to train many parameters of the network. The experimental results on three real datasets show that compared with other algorithms, the model can improve AUC by at least 0.11% and GAUC by 0.2%, indicating that the proposed algorithm outperforms other models in performance. [ABSTRACT FROM AUTHOR]

Click-through rate (CTR) prediction is a critical task in recommender systems and online advertising systems. The extensive collection of behavior data has become popular for building prediction models by capturing user interests from behavior sequences. There are two types of entities involved in behavior sequences, users and items, which form three kinds of relationships: user-to-user, user-to-item, and item-to-item. Most related work focuses on only one or two of these relationships, often ignoring the association between users, which also helps discover potential user interests. In this paper, we consider all three relationships useful and propose a Multi-dimensional Interest Network (MIN) to focus on their impact on CTR prediction simultaneously. It consists of three sub-networks that capture users' preferences regarding group interests and individual interests. Specifically, the u-u sub-network models the relationship between the target user and those who have clicked on the target item. It takes user representations learned from behavior sequences via transformer as input. Two other sub-networks capture the individual interest of the target user. The u-i sub-network models the relationship between the target user and the target item. The i-i sub-network models the relationship between the target item and the items the target user has interacted with in the past time. Extensive evaluations on the real datasets show that our MIN model outperforms the state-of-the-art solutions in prediction accuracy (+ 5.0% in AUC and − 17.2% in Logloss, averagely). The ablation experiments also validate that each sub-network in MIN helps with improving the CTR prediction performance by using the u-u sub-network playing a more critical role. The source code is available at https://github.com/cocolixiao/MIN. [ABSTRACT FROM AUTHOR]

Click-through rate (CTR) prediction plays a vital role in recommendation systems. Most models pay little attention to the relationship between target items in the user behavior sequence. The attention units used in these models cannot fully capture the context information, which can be used to reflect the variations of user interests. To address these problems, we propose a new model named interest extraction method based on multi-head attention mechanism (IEN) for CTR prediction. Specifically, we design an interest extraction module, which consists of two sub-modules: the item representation module (IRM) and the context–item interaction module (CIM). In IRM, we learn the relationship between target items in the user behavior sequence by a multi-head attention mechanism. Then, the user representation is gained by integrating the refined item representation and position information. At last, the correlation between the user and the target item is used to reflect user interests. In CIM, the context information has valuable temporal features which can reflect the variations of user interests. Therefore, user interests can be further acquired through the feature interaction between the context and the target item. After that, the learned relevance and the feature interaction are fed to the multi-layer perceptron (MLP) for prediction. Besides, experiments on four Amazon datasets were conducted to evaluate the effectiveness of our method in capturing user interests. The experimental results show that our proposed method outperforms state-of-the-art methods in terms of AUC and RI in the CTR prediction task. [ABSTRACT FROM AUTHOR]

Click-Through Rate (CTR) prediction is important in many industrial applications, such as E-commerce, news, and information. Understanding sophisticated feature interactions behind users’ behaviors is essential for CTR prediction. Although existing methods have made significant improvements, there exist some problems: (1) only concentrate on modeling implicit information from the user side, while ignoring the interest hidden in the historical interactive behaviors; (2) insufficient feature extraction, only focusing on high-order feature interactions or low-order feature interactions. To overcome these limitations, we propose a Deep Adaptive Interest Network (DAIN) for CTR Prediction in the local and global views, respectively. Specifically, to extract user’s interest, we first develop a local attention mechanism applied to the user behaviors and candidate ads, which can adaptively calculate users’ interest representation given a candidate ad in the local views. To capture feature interactions, we propose a feature interaction extractor containing Multi-layer Perceptrons (MLP) and Factorization Machines (FM) components to capture high-order and low-order feature interactions. To adaptively learn the influences of high-order and low-order feature interactions on the target item, we finally employ a linear-based global attention mechanism in the feature interaction extractor. The effectiveness of DAIN is verified by comprehensive experiments on three datasets.

Click-through rate prediction(CTR) is a critical task in an online advertising system. Recently, deep learning based architectures have brought great attention in Click-through rate prediction by learning the nonlinear interaction between feature embedding of users and items. However, these methods have the following issues: (1) The collaborative information between users and items could not be fully explored due to the static embedding with lookup-table technique. (2) The learning procedure lacks cognitive reasoning about what the users want to do and what they may need. To address the above challenges, we propose a graph aware collaborative reasoning method for CTR prediction which explores the collaborative information with graph and then predicts the users' behaviors with logical reasoning. Specifically, the graph is built by the common behaviors between users, and the embedding of users and items can be learned by propagating the collaborative information in the graph. Then with the collaborative embedding of users and items, two logical operations NOT and OR are adopted to integrate the embedding for logical reasoning with the neural networks. By learning the proposed architecture in an end-to-end manner, the logical behaviors of users in the behavior sequences can be learned efficiently. Extensive experiments on five real-world datasets show that the proposed method outperforms several state-of-the-art methods in CTR prediction. [ABSTRACT FROM AUTHOR]

We tackle the challenge of feature embedding for the purposes of improving the click-through rate prediction process. We select three models: logistic regression, factorization machines and deep factorization machines, as our baselines and propose five different feature embedding modules: embedding scaling, FM embedding, embedding encoding, NN embedding and the embedding reweighting module. The embedding modules act as a way to improve baseline model feature embeddings and are trained alongside the rest of the model parameters in an end-to-end manner. Each module is individually added to a baseline model to obtain a new augmented model. We test the predictive performance of our augmented models on a publicly accessible dataset used for benchmarking click-through rate prediction models. Our results show that several proposed embedding modules provide an important increase in predictive performance without a drastic increase in training time. [ABSTRACT FROM AUTHOR]

Click-through rate(CTR) prediction is an important task in the recommendation system.Its goal is to predict the pro-bability of a user clicking on an advertisement or item.Feature embedding and feature interacting are critical for prediction performance.Therefore,the ideas of many click-through rate prediction models are optimized based on these two aspects.However,most of the work only focus on one of the aspects,and almost all models do not distinguish features in feature interacting.The same embedding and interacting method are used when crossing the same feature with other features,which hinders the improvement of model performance.In order to solve this problem,the automatic super-field-aware feature embedding and interacting(Auto-SEI) model is proposed.Firstly,it assigns each sub-field to a super-field,and obtains the feature embedding according to the grouping,then selects appropriate interacting method for the feature pair to obtain the cross feature,and finally makes prediction.In Auto-SEI model,the division of sub-field and the selection of interacting methods are parameterized as an architecture search problem,and the neural architecture search(NAS) algorithm is used to compress the search space and make selections.A large number of experiments are conducted on three real large-scale data sets and the results show the excellent performance of the Auto-SEI model on the task of click-through rate prediction.

When modeling user-item interaction sequences to extract sequential patterns, current recommender systems face the dual issues of a) long-distance dependencies in conjunction with b) high levels of noise. In addition, with the complexity of current recommendation model architectures there is a significant increase in computation time. Therefore, these models cannot meet the requirement of fast response needed in application scenarios such as online advertising. To deal with these issues, we propose a Knowledge Distilled Attention-based Latent Information Extraction Network for Sequential user behavior (KD-ALIENS). In this model structure, user and item attributes and history are utilized to model the latent information from high-order feature interactions in conjunction with user sequential historical behavior. With regard to the issues of long-distance dependency and noise, we have adopted the self-attention mechanism to learn the sequential patterns between items in a user-item interaction history. With regard to the issue of a complex model architecture which cannot meet the requirement of fast response, the use of model compression and acceleration is realized by: (a) use of a knowledge-distilled teacher and student module, wherein the complex teacher module extracts a user's general preference from high-order feature interactions and sequential patterns of long history sequences; and (b) a sampling method to sample both the relatively long-term and short-term item histories. Experimental studies on two real-world datasets demonstrate considerable improvements for click-through rate (CTR) prediction accuracy relative to strong baseline models and also show the effectiveness of the student-model compression and acceleration for speed. [ABSTRACT FROM AUTHOR]

Click-through rate prediction is a critical task for computational advertising and recommendation systems, where the key challenge is to model feature interactions between different feature domains. At present, the main click-through rate prediction models model feature interactions in an implicit way, which leads to poor interpretation of the model, and the interaction between each pair of features may introduce noise into the model, thus limiting the predictive ability of the model. In response to the above problems, this paper proposes a click-through rate prediction model (GAIAN) based on the graph attention interactive aggregation network, which explicitly obtains cross features on the graph structure. Our specific method is to design a feature interactive selection mechanism to select cross features that are beneficial to model prediction, reducing model noise and reducing the risk of model overfitting. On this basis, the bilinear interaction function is integrated into the aggregation strategy of the graph neural network, and the fine-grained intersection features are extracted in a flexible and explicit way, which makes graph neural networks more suitable for modeling feature interactions and enhances the interpretability of the model. Compared with several other state-of-the-art models on the Criteo and Avazu datasets, the experimental results show the superiority of the model. [ABSTRACT FROM AUTHOR]

Click-through rate prediction is crucial in network applications such as recommendation systems and online networks. Designing feature extraction schemes to obtain features and modeling users’ click behavior are used to estimate the probability of users clicking on recommended items. The AutoInt model is a recent and effective research finding. It constructs combined features by referencing the multi-head attention mechanism but does not fully mine meaningful high-order cross-features and ignores user privacy protection. To address this problem, this study proposes the differential privacy bidirectional long short-term memory network (DP-Bi-LSTM-AutoInt) model, which is an improved AutoInt model. A bidirectional long short-term memory network is added after the embedding layer to deeply mine the nonlinear relationship between user click behaviors and construct high-order features. Further, differential privacy technology is adopted for user privacy protection, and the Gaussian mechanism is used to randomly perturb the gradient descent algorithm of the model. Using the Criteo dataset to conduct experiments, the experimental results show that the accuracy of the Bi-LSTM-AutoInt model proposed herein is improved by 0.65 % compared to the original AutoInt model. When the privacy budget is greater than 3.0, the accuracies of the DP-Bi-LSTM-AutoInt and Bi-LSTM-AutoInt models are nearly equivalent. However, the DP-Bi-LSTM-AutoInt model algorithm is more secure and reliable than the AutoInt model.

CTR prediction is an important task in recommender systems, which is used to estimate the likelihood of a user clicking on an advertisement.In the past, the CTR prediction model based on the deep neural network mainly obtains the implicit feature combination of the model at the bit-wise level, and the interpretability and generalization of the model are poor. At the same time, the prediction accuracy of the model is poor. For the above problems, We propose a click-through rate prediction model (DTM) with double matrix-level cross-features. The model integrates various components such as multi-head self-attention, residual network and interaction network into an end-to-end model, and automatically obtains explicit feature combinations at the vector-wise level and bit-wise level, which not only has better interpretability, generalization and memory, and reduce the inherent flaws and engineering complexity of multi-modules. The experimental results show that on the datasets Criteo and Avazu, compared with other state-of-the-art CTR prediction models, the AUC values of the DTM model are increased by 4% and 3% on average, and the loss values are decreased by 3.5% and 2.8% on average, respectively.

Click-through rate (CTR) prediction, whose goal is to predict the probability of the user to click on an item, has become increasingly significant in the recommender systems. Recently, some deep learning models with the ability to automatically extract the user interest from his/her behaviors have achieved great success. In these work, the attention mechanism is used to select the user interested items in historical behaviors, improving the performance of the CTR predictor. Normally, these attentive modules can be jointly trained with the base predictor by using gradient descents. In this paper, we regard user interest modeling as a feature selection problem, which we call user interest selection. For such a problem, we propose a novel approach under the framework of the wrapper method, which is named Meta-Wrapper. More specifically, we use a differentiable module as our wrapping operator and then recast its learning problem as a continuous bilevel optimization. Moreover, we use a meta-learning algorithm to solve the optimization and theoretically prove its convergence. Meanwhile, we also provide theoretical analysis to show that our proposed method 1) efficiencies the wrapper-based feature selection, and 2) achieves better resistance to overfitting. Finally, extensive experiments on three public datasets manifest the superiority of our method in boosting the performance of CTR prediction. [ABSTRACT FROM AUTHOR]

Click-through rate (CTR) prediction can provide considerable economic and social benefits. Few studies have considered the importance of low-order features, usually employing a simple feature interaction method. To address these issues, we propose a novel model called Senet and extreme deep field-embedded factorization machine (Se-xDFEFM) for more effective CTR prediction. We first embed the squeeze-excitation network (Senet) module into Se-xDFEFM to complete low-order feature refinement, which can better filter noisy information. Then, we implement our field-embedded factorization machine (FEFM) to learn the symmetric matrix embeddings for each field pair, along with the single-vector embeddings for each feature, which builds a firm foundation for the subsequent feature interaction. Finally, we design a compressed interaction network (CIN) to realize feature construction with definite order through a vector-wise interaction. We use a deep neural network (DNN) with the CIN to simultaneously implement effective but complementary explicit and implicit feature interactions. Experimental results demonstrate that the Se-xDFEFM model outperforms other state-of-the-art baselines. Our model is effective and robust for CTR prediction. Importantly, our model variants also achieve competitive recommendation performance, demonstrating their scalability. [ABSTRACT FROM AUTHOR]

Graph Neural Networks (GNNs) have led to state-of-the-art performance on a variety of machine learning tasks such as recommendation, node classification and link prediction. Graph neural network models generate node embeddings by merging nodes features with the aggregated neighboring nodes information. Most existing GNN models exploit a single type of aggregator (e.g., mean-pooling) to aggregate neighboring nodes information, and then add or concatenate the output of aggregator to the current representation vector of the center node. However, using only a single type of aggregator is difficult to capture the different aspects of neighboring information and the simple addition or concatenation update methods limit the expressive capability of GNNs. Not only that, existing supervised or semi-supervised GNN models are trained based on the loss function of the node label, which leads to the neglect of graph structure information. In this paper, we propose a novel graph neural network architecture, Graph Attention & Interaction Network (GAIN), for inductive learning on graphs. Unlike the previous GNN models that only utilize a single type of aggregation method, we use multiple types of aggregators to gather neighboring information in different aspects and integrate the outputs of these aggregators through the aggregator-level attention mechanism. Furthermore, we design a graph regularized loss to better capture the topological relationship of the nodes in the graph. Additionally, we first present the concept of graph feature interaction and propose a vector-wise explicit feature interaction mechanism to update the node embeddings. We conduct comprehensive experiments on two node-classification benchmarks and a real-world financial news dataset. The experiments demonstrate our GAIN model outperforms current state-of-the-art performances on all the tasks. [ABSTRACT FROM AUTHOR]

Many recent user behavior based click‐through rate models adopt a similar item‐level paradigm: learn the user representation from a list of item representations via a sequence model and/or a pooling mechanism. However, sequence models are usually sensitive to the exact order of the behavior sequence, while item‐level pooling mechanisms simply neglect the chronological information. In this paper, we balance the two approaches by decomposing the long item sequence into a group of extremely short sequences (item pairs) and conducting relational reasoning on them. More specifically, the relational reasoning mechanism consists of two parts, which are designed for capturing various transitional patterns in the behavior sequences. An attentive pooling layer is employed to emphasize those relation‐level signals that are highly related to the target item. Therefore, our approach is essentially a middle ground between the previous two approaches. To verify the effectiveness of our method, we conduct extensive experiments on three public datasets. Experimental results show that our methods achieve better performance than others. Besides, we explore the properties of our model and verify the effectiveness of each component by controlled experiments. © 2021 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC. [ABSTRACT FROM AUTHOR]

Click-through rate (CTR) is a positive feedback of user preferences or product purchases, and its small increase can bring huge benefits. Therefore, CTR prediction plays a key role in computing advertising and recommendation systems. Research shows that the accuracy of CTR prediction models is closely related to the input features. Existing related models usually focus on certain aspects of features, such as second-order interactions or temporal changes, and ignore the diversity of features. In this paper, a modular click-through rate prediction framework JointCTR is proposed. The framework integrates four types of prediction models, each of which learns different types of features, including original features, embedded features, interactive features, and sequential features. According to actual application scenarios, these models can be assembled or removed flexibly, and models of the same type can be replaced by each other. In order to avoid the impact of data diversity, sparsity, and high-dimensional features, an embedding layer is added to the framework to achieve unified embedding processing for different data types. To learn sequential behaviors, we propose a model SeqCTR based on the attention mechanism to capture the dynamics of user interests in the framework. To better learn high-order features, we apply HorderCTR proposed in our previous work to the framework to automatically identify high-value feature combinations. Extensive experiments on four public datasets show the effectiveness of the proposed framework, which yields competitive performance compared to state-of-the-art models (the AUC increases by + 0.24% on MovieLens-1 M and 0.85% on Criteo). Almost all different types of popular models can be assembled into it, that shows the flexibility and scalability of the framework. [ABSTRACT FROM AUTHOR]

Online advertising is becoming an important direction in the advertising industry with its strengths in diverse users, strong interactions, real-time feedback, and expandability. Online advertisement (Ads) can show great marketing ability by processing data from multiple channels to convey information, understanding what users want, and approaching them easily. Moreover, predicting the click-through rate (CTR) can increase advertisement revenue and user satisfaction. However, advertising data contains many features, and the amount is growing rapidly. This can be alleviated through the segmentation of users with similar interests. We assumed that the change of interest of a user could be predicted by other users' change of interest. More specifically, similar users will change their interest in a similar direction. On the basis of this idea, we proposed a novel model, the Deep User Segment Interest Network, to improve CTR prediction. We suggested three novel layers for improving performance: i) an individual interest extractor, ii) a segment interest extractor, and iii) a segment interest activation. These layers captures the latent interest of each user and creates the expressive interest representation of the segment by aggregating each user's interest. We conducted experiments using TaoBao data, which are a kind of real commercial data from an advertising platform, to confirm the CTR prediction improvement by reflecting the segment interest. The proposed algorithm obtained an AUC gain of 0.0029 with a behavior sequence length of 100. This performance exhibited the greatest improvement over other baselines, indicating the proposed method's potential contribution to business improvement.

Recommendation system has drawn growing attention in the academia and industry because it can solve the problem of information overload. Among a variety of methods, the click-through rate prediction model plays an important role in predicting user’s attention to a specific item. To predict click-through rate, high-dimensional and sparse features are usually adopted, and the accuracy of the prediction result depends on the combination of high-order features to a great extent. Therefore, many methods have been proposed to find the low-dimensional representation from sparse high-dimensional original features, and the meaningful way of feature combination has also been mined to improve the accuracy of the model. However, the click-through rate prediction models generally have two problems. One is that they can’t extract the feature interaction of non-Euclidean features very well. Another one is that it is hard to explain the inward meaning of feature interaction. In this paper, a GCN-int model based on the interaction of Graph Convolutional Network is proposed to solve the above problems. The proposed model simplifies the complex interaction among multiple features, gets a better representation of the interaction between high-order features, and improves the interpretability of feature interaction. The experimental results on the public movie recommendation dataset and our own IPTV movie recommendation dataset show that the proposed GCN-int model gets higher accuracy and efficiency compared with the state-of-the-art models.

Click-through rate (CTR) prediction is one of the key areas in industrial bidding advertising. Recently, to improve prediction performance, researchers have proposed the interest-based deep models that learn the user's latent interest from historical click behaviors. However, the interest-based deep models lack the ability to trace the dynamics of interest from both the positive and negative samples, which leading to prediction errors. In particular, user's interest in candidate advertisements could be positive (i.e. interested) or negative (i.e. uninterested), and changing over time. To solve this problem, we propose a deep-based dynamic interest perception network (DIPN) that can trace both positive interest and negative interest. The proposed DIPN model introduces three new parts to the interest-based deep model: a gated recurrent unit (GRU) learns the implied interest vector from historical click sequences; an attention mechanism improves the expressiveness of interest vector with the weight vector; an interest degree feature scales the weight vector by the d-Softmax function to improve the expressiveness of interest vector. We evaluate the effectiveness of the DIPN model by conducting extensive comparative experiments using real datasets. The experimental results demonstrate that compared with state-of-the-art models, DIPN achieves the highest prediction performance.

Click-through rate prediction (CTR) is an essential task in recommender system. The existing methods of CTR prediction are generally divided into two classes. The first class is focused on modeling feature interactions, the second class is focused on solving time-series problems. However, the existing models of the second class are not able to handle time-series problems with user feedback information, so we propose PMN to solve this kind of problem. To be able to take full advantage of historical user behavior along with the user feedback, PMN uses the attention mechanism to get the user historical behavior representation and the user preference representation from the original input. Specially, user preference representation is derived from the user feedback information and it explicitly shows the user's attitude towards the candidate, which greatly improve the model performance. Finally, we introduced user preference baselines to solve the problem of inconsistent scoring standards for different users. In this paper, we focus on the CTR prediction modeling in the scenario of video recommendation in Video On Demand (VOD) service. Experimental results on multiple data sets have shown that our PMN model is effective.

Click-through rate (CTR) prediction is an important issue in online advertising and recommendation systems. It is used to estimate the likelihood that a user will click on ads. The method used in the traditional CTR prediction task is to improve the prediction effect through a large number of feature engineering. However, these methods are time-consuming and laborious, and the construction process is not universal. Due to the sparse and high-dimensional characteristics of data features, it is necessary to measure the importance of sparse features and obtain efficient feature interactions. In this paper, a novel CTR model based on Mutual Information and Feature Interaction (MiFiNN) is proposed. First, the mutual information of each sparse feature and the click result is calculated as the weight of each sparse feature. Subsequently, an interactive method combining the outer product and inner product is constructed to carry out the feature interaction. Then, the resulting feature interactions and the dense features set of the original input are taken as DNN inputs. We verify the proposed model on four datasets. In addition, several widely known models are introduced for comparison. The experimental results indicate the superiority of the model.

Click-through rate prediction is a critical task for computational advertising and recommendation systems, where the key challenge is to model feature interactions between different feature domains. At present, the main click-through rate prediction models model feature interactions in an implicit way, which leads to poor interpretation of the model, and the interaction between each pair of features may introduce noise into the model, thus limiting the predictive ability of the model. In response to the above problems, this paper proposes a click-through rate prediction model (GAIAN) based on the graph attention interactive aggregation network, which explicitly obtains cross features on the graph structure. Our specific method is to design a feature interactive selection mechanism to select cross features that are beneficial to model prediction, reducing model noise and reducing the risk of model overfitting. On this basis, the bilinear interaction function is integrated into the aggregation strategy of the graph neural network, and the fine-grained intersection features are extracted in a flexible and explicit way, which makes graph neural networks more suitable for modeling feature interactions and enhances the interpretability of the model. Compared with several other state-of-the-art models on the Criteo and Avazu datasets, the experimental results show the superiority of the model.