Search

In light of the emergence of deep reinforcement learning (DRL) in recommender systems research and several fruitful results in recent years, this survey aims to provide a timely and comprehensive overview of recent trends of deep reinforcement learning in recommender systems. We start by motivating the application of DRL in recommender systems, followed by a taxonomy of current DRL-based recommender systems and a summary of existing methods. We discuss emerging topics, open issues, and provide our perspective on advancing the domain. The survey serves as introductory material for readers from academia and industry to the topic and identifies notable opportunities for further research. [ABSTRACT FROM AUTHOR]

Recommender systems (RSs) have been employed for many real-world applications including search engines, social networks, and information retrieval systems as powerful intelligent techniques for creating personalized recommendations. RSs mainly utilize different data resources such as user-item ratings, tag data, and social relationships in their recommendation process to overcome the data sparsity issue. One of the main challenges is how to integrate the results obtained by these data resources. Moreover, most of the existing RSs focus on the recommendation accuracy, and do not consider the reliability of the recommendations. To address these challenges, in this study, a reliable recommendation method is developed, which employs deep neural networks and reinforcement learning. The proposed method firstly uses a stacked denoising autoencoder model to obtain the latent features from the rating, tag, and trust data. The similarity value between each pair of users is then calculated in three different aspects according to the extracted latent features. An ensemble approach is developed by integrating the three similarities using reinforcement learning to determine the final similarity value. The ensemble similarity values are used in the rating prediction process, and the reliability of the predicted ratings is evaluated using an effective measure. Finally, a recommendation strategy is developed based on the integration of the predicted ratings and their reliability values. We evaluated the effectiveness of the proposed method by conducting numerous experiments on three datasets where the results show that the proposed method is far superior to other recommendation models in making more accurate recommendations. [ABSTRACT FROM AUTHOR]

Precise prescription of medication dosing is crucial to patients, especially among Intensive Care Unit (ICU) patients. However, improper administration of some sensitive therapeutic medications (e.g., heparin) might place patients at unneeded risk, even life-threatening. Numerous factors such as a patient's clinical phenotype, genotype, and environmental factors will affect the heparin dose response. As a result, it is challenging to prescribe the optimal initial dose of heparin. In this paper, an individualized dosing policy is proposed to determine the optimal initial dose and minimize the risk of mis-dosing, as well as preventing the patients from late complications associated with medications dosing. A latent batch-constrained deep reinforcement learning (RL) algorithm is proposed to guarantee the safety of the medication recommendation system. The agent can observe a latent representation of patents and generate medication dosing solutions in successive and limited action spaces. The individualized dosing policy aims to reduce the extrapolation errors in the off-policy algorithms, by evaluating over-dosing and under-dosing of heparin in patients. Our results evaluated on Medical Information Mart for Intensive Care III (MIMIC-III) database demonstrate that the latent batch-constrained RL algorithm can work effectively from the retrospective data, showing promise to be used in future medication dosing policies. [ABSTRACT FROM AUTHOR]

Knowledge graphs (KGs) can provide rich, structured information for recommendation systems as well as increase accuracy and perform explicit reasoning. Deep reinforcement learning (RL) has also sparked great interest in personalized recommendations. The combination of the two holds promise in carrying out interpretable causal inference procedures and improving the performance of graph-structured recommendation. However, most KG-based recommendation focus on rich semantic relationships between entities in a heterogeneous knowledge graph, and thus fail to fully make use of the image information corresponding to an entity. In order to address these issues, we proposed a novel Multi-modal Knowledge-aware Reinforcement Learning Network (MKRLN), which couples recommendation and interpretability by providing actual paths in multi-modal KG (MKG). The MKRLN can generate path representation by composing the structural and visual information of entities, and infers the underlying rational of agent-MKG interactions by leveraging the sequential dependencies within a path from the MKG. In addition, as KGs have too many attributes and entities, their combination with RL leads to too many action spaces and states in the reinforcement learning space, which complicates the search of action spaces. Furthermore, in order to solve this problem, we proposed a new hierarchical attention-path, which makes users focus their attention on the items they are interested in. This reduces the relations and entities in the KGs, which in turn reduces the action space and state in RL, shortens the path to the target entity, and improves the accuracy of recommendation. Our model has explicit explanation ability in knowledge and images. Finally, we extensively evaluated our model on several large-scale real-world benchmark datasets, and it yielded favorable results compared with state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Recommender systems aim to maximize the overall accuracy for long-term recommendations. However, most of the existing recommendation models adopt a static view, and ignore the fact that recommendation is a dynamic sequential decision-making process. As a result, they fail to adapt to new situations and suffer from the cold-start problem. Although sequential recommendation methods have been gaining attention recently, the objective of long-term recommendation still has not been explicitly addressed since these methods are developed for short-term prediction situations. To overcome these problems, we propose a novel top-N interactive recommender system based on deep reinforcement learning. In our model, the processes of recommendation are viewed as Markov decision processes (MDP), wherein the interactions between agent (recommender system) and environment (user) are simulated by the recurrent neural network (RNN). In addition, reinforcement learning is employed to optimize the proposed model for the purpose of maximizing long-term recommendation accuracy. Experimental results based on several benchmarks show that our model significantly outperforms previous top-N methods in terms of Hit-Rate and NDCG for the long-term recommendation, and can be applied to both cold-start and warm-start scenarios. • A novel top-N interactive recommender system based on deep reinforcement learning is proposed. • The interactions between recommender system and users are simulated by recurrent neural networks. • The proposed model can deal with both cold-start and warm-start scenarios. • Reinforcement learning and supervised learning are employed to optimize the proposed model for long-term recommendation accuracy. • Experiments and comparisons are conducted to show the merits of the proposed model. [ABSTRACT FROM AUTHOR]

Reinforcement learning techniques have recently been introduced to interactive recommender systems to capture the dynamic patterns of user behavior during the interaction with recommender systems and perform planning to optimize long-term performance. Most existing research work focuses on designing policy and learning algorithms of the recommender agent but seldom cares about the state representation of the environment, which is indeed essential for the recommendation decision making. In this paper, we first formulate the interactive recommender system problem with a deep reinforcement learning recommendation framework. Within this framework, we then carefully design four state representation schemes for learning the recommendation policy. Inspired by recent advances in feature interaction modeling in user response prediction, we discover that explicitly modeling user–item interactions in state representation can largely help the recommendation policy perform effective reinforcement learning. Extensive experiments on four real-world datasets are conducted under both the offline and simulated online evaluation settings. The experimental results demonstrate the proposed state representation schemes lead to better performance over the state-of-the-art methods. [ABSTRACT FROM AUTHOR]